I have a mongoose connect function in which I try to wait for reconnecting if the first attempt fails:
async connect() {
const options = {...}
try {
console.log("starting")
await this._connectWithRetry(options)
console.log("finished")
} catch (err) {
winston.error(`Could not connect to Mongo with error: ${err}`)
}
}
private async _connectWithRetry(options) {
return new Promise( async (resolve, reject) => {
try {
winston.info("Connecting to mongo...")
await mongoose.connect(this.dbURI, options)
winston.info("Connection successful.")
resolve()
} catch (err) {
winston.info("Failed to connect to mongo. Retrying in 5 seconds...")
setTimeout( async () => {
await this._connectWithRetry(options)
}, 5000)
}
})
}
It successfully waits until I'm connected. But once I connect, the second console line is not hit ("finished"). so I figure that my promise resolution is buggy. What am I doing wrong?
Your code "works" if the connection to the DB is established first time around.
If the retry mechanism is used, you will see the error you describe.
The Promise instantiated by the first call to mongoDBConnect is never resolved in the retry execution path.
This is because subsequent invocations of mongoDBConnect are made in a totally separate execution context on a future tick of the event loop, controlled by the setTimeout - and each invocation instantiates a new Promise totally disconnected from your connect function.
This refactoring should fix the issue:
const delay = (interval) => new Promise(resolve => setTimeout(resolve, interval))
async connect() {
const options = {...}
try {
console.log("starting")
await this._connectWithRetry(options)
console.log("finished")
} catch (err) {
winston.error(`Could not connect to Mongo with error: ${err}`)
}
}
private async _connectWithRetry(options) {
try {
winston.info("Connecting to mongo...")
await mongoose.connect(this.dbURI, options)
winston.info("Connection successful.")
} catch (err) {
winston.info("Failed to connect to mongo. Retrying in 5 seconds...")
await delay(5000)
await this._connectWithRetry(options)
}
}
Test harness:
let retryCount = 0
const mongoose = {
connect: ()=>retryCount++ === 2 ? Promise.resolve() : Promise.reject('fake error')
}
async function connect() {
try {
console.log("starting")
await connectWithRetry()
console.log("finished")
} catch (err) {
console.error(`connect error`, err)
}
}
async function connectWithRetry() {
try {
console.log("Connecting to mongo...")
await mongoose.connect()
console.log("Connection successful.")
} catch (err) {
console.log("Retrying in 1 second...", err)
await delay(1000)
await connectWithRetry()
}
}
const delay = (interval) => new Promise(resolve => setTimeout(resolve, interval))
connect()
Related
I am getting UnhandledPromiseRejection error even I wrapped the code in try catch block
I using await Prmomise.all together here
const express = require('express');
const app = express();
const port = 3003;
function testPromise(n) {
return new Promise(async (res, rej) => {
console.log(n);
if (n > 10) {
res(true);
} else {
setTimeout(() => {
rej(n);;
}, 1000)
}
});
}
function test2(n) {
return new Promise(async (res, rej) => {
console.log(n);
if (n > 10) {
res(true);
} else {
setTimeout(() => {
rej(n);;
}, 10000)
}
});
}
async function allCall(p) {
await Promise.all(p);
}
app.get('/', async (req, res) => {
try {
let a = [];
let b = [];
a.push(testPromise(1));
await test2(1);
a.push(testPromise(12));
// await Promise.all(a.map(m => m.then(() => { }).catch(err => { })));
await Promise.all(a);
res.send('Hello World!');
} catch (err) {
console.log('err');
console.log(err);
res.status(400).send('xxxxxxxxxx!')
}
})
app.listen(port, () => {
console.log(`Example app listening on port ${port}`)
})
I am not sure why it is throwing the error
[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise
which was not handled with .catch().
Please explain why and how to resolve this ?
You are getting this error because 2 promises are getting rejected but try/catch only handles one. The second promise is rejected but not handled.
1st rejection: a.push(testPromise(1));
2nd rejection: await test2(1);
NOTE:
Both the promises are started parallel.
try/catch only works with async/await, if you write promise inside try/catch it'll not be handled by the catch block.
try {
Promise.reject("Something")
} catch (error) {
console.log('Not here');
}
// Unhandled promise rejection
Explanation:
When you push a promise to an array a.push(testPromise(1));, it starts execution and rejects after 1 second. It goes to catch.
same time the second promise also started await test2(1); because you are not waiting for the first promise to resolve/reject. It'll get rejected after 1 second and not handled by the catch block. it'll go to catch only if you use with await. If you want to handle first rejection you have to use .catch. after 10 seconds second promise get rejected and handled by catch block.
Solution
const r = await testPromise(1);
await test2(1);
Another solution:
async (req, res) => {
try {
// a.push(testPromise(1));
await test2(1);
let a = [testPromise(1), testPromise(12)];
await Promise.all(a);
console.log("done");
} catch (err) {
console.log("err");
console.log(err);
}
};
let me explain what I mean using an example
async function async_function(){
await new Promise(r=>setTimeout(r,3000));
throw 'task completed'
}
async function do_something_meanwhile() {
await new Promise(r => setTimeout(r, 500));
console.log(Math.floor(Math.random()*10));
}
(async ()=>{
try {
async_function(); //this returns an error after a while
while (...)
await do_something_meanwhile();
} catch (err) { console.log('exited with error:',err) }
console.log('moving on');
})();
I'm trying to run an async function and after it is complete immediately terminate the loop,
the best way I could think of (without any time delay) was to send an error
but it gives this error instead of moving on after it's done:
node:internal/process/promises:246
triggerUncaughtException(err, true /* fromPromise */);
^
[UnhandledPromiseRejection: This error originated either by throwing
inside of an async function without a catch block,
or by rejecting a promise which was not handled with
.catch(). The promise rejected with the reason "task
completed".] {
code: 'ERR_UNHANDLED_REJECTION'
}
is there a way around this or a better to achieve the desired effect?
You can handle rejection by setting an error variable that you can check in the loop:
try {
let error;
async_function()
.catch(err => error = err);
while (...) {
if (error) {
throw error;
}
await do_something_meanwhile();
}
} catch (err) {
console.log('exited with error:',err)
}
If you need to proactively tell do_something_meanwhile to terminate as well, you could use an AbortController and pass its signal to do_something_meanwhile.
try {
let error;
const controller = new AbortController();
const { signal } = controller;
async_function()
.catch(err => {
error = err;
controller.abort();
});
while (...) {
if (error) {
throw error;
}
await do_something_meanwhile(signal);
}
} catch (err) {
console.log('exited with error:',err)
}
I think if I were doing that, I might subclass AbortController so I can put the error in it:
class AbortContollerWithError extends AbortController {
abort(error) {
this.error = error;
super.abort();
}
}
then:
try {
const controller = new AbortController();
const { signal } = controller;
async_function()
.catch(err => {
controller.abort(err);
});
while (...) {
if (signal.aborted) {
throw controller.error;
}
await do_something_meanwhile(signal);
}
} catch (err) {
console.log('exited with error:',err)
}
...or something along those lines.
You asked how you'd use the signal in do_something_meanwhile, and suggested in a comment that you're really using a timer in it. That's where the signal's abort event comes in handy, you can use that to settle the promise early:
async function do_something_meanwhile(signal) {
let cancelError = {};
try {
await new Promise((resolve, reject) => {
const timer = setTimeout(resolve, 500);
signal.addEventListener("abort", () => {
clearTimeout(timer);
cancelError = new Error();
reject(cancelError);
});
});
console.log(Math.floor(Math.random() * 10));
} catch (error) {
if (error === cancelError) {
// Probably do nothing
} else {
// Something else went wrong, re-throw
throw error;
}
}
}
Promise.all can run async_function and do_something_meanwhile in parallel mode.
While Promise/A doesn't have a cancel method, you can define a stopFlag, and check it in do_something_meanwhile function and the while loop.
let stopFlag = false
async function async_function() {
await new Promise(r=>setTimeout(r, 3000));
throw 'task completed'
}
async function do_something_meanwhile() {
await new Promise(r => setTimeout(r, 500));
if (!stopFlag) {
console.log(Math.floor(Math.random() * 10));
}
}
(async()=>{
try {
await Promise.all([
async_function().catch((err) => {
stopFlag = true
throw err
}), // this returns an error after a while
(async () => {
while (!stopFlag)
await do_something_meanwhile();
})()
])
} catch (err) {
console.log('exited with error:', err)
}
console.log('moving on');
})();
This question already has answers here:
Timeout in async/await
(3 answers)
Closed 1 year ago.
In my express application, I am making call to 2 APIs. The 2nd API is managed by 3rd party and sometimes can take more than 5 seconds to respond. Hence, I want to just wait for 1 second for the API to respond. If it does not, just proceed with data from 1st API.
Below is the mock-up of the functions being called.
I am thinking to use setTimeout to throw error if the API takes more than 1 second. If the API responds within 1 second then I just cancel the setTimeout and no error is ever thrown.
But there is problem with this approach:
setTimeout errors cannot be catched using try...catch block.
I cannot use axios's timeout option, as I still need to wait for the 2nd API to finish the processing and save the data in the DB. This will ofcourse, can happen later, when the 2nd API call finishes.
// Function to simulate it's taking time.
async function cWait(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
// Track whether it took time.
let isTimeOut = false
async function test() {
console.log('starting')
try {
const one = await apiCall1()
const myt = setTimeout(() => {
console.log('Its taking time, skip the 2nd API Call')
isTimeOut = true
throw new Error('Its taking time')
})
const two = await apiCall2(myt)
} catch (error) {
console.log(error)
}
saveInDB({ ...one, ...two })
}
async function apiCall2(timeOutInstance) {
console.log('start-apiCall')
await cWait(1800)
clearTimeout(timeOutInstance)
if (isTimeOut) saveInDB()
console.log('done-apiCall')
}
async function apiCall1() {
await cWait(5)
}
async function saveInDB(data) {
console.log('saveInDB')
}
test()
please note, this is not the answer as it was when it was accepted
as I misread the question and failed to call saveInDB in a timed out
situation
Promise.race seems perfect for the job
Also, you'd actually use your cWait function, not for mock-up, but to actually do something useful ... win the race :p
const api2delay = 800;
async function cWait(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
const TIMEDOUT = Symbol('TIMEDOUT');
async function cReject(ms) {
return new Promise((_, reject) => setTimeout(reject, ms, TIMEDOUT));
}
function apiCall2timeout(timeoutCallback) {
const resultApi2 = apiCall2();
const timeout = cReject(1000);
return Promise.race([resultApi2, timeout])
.catch(e => {
if (e === TIMEDOUT) {
resultApi2.then(timeoutCallback);
} else {
throw e;
}
});
}
async function test() {
console.log('starting')
let one, two;
try {
one = await apiCall1();
two = await apiCall2timeout(saveInDB);
} catch (error) {
console.log('error', error)
}
saveInDB({
...one,
...two
})
}
async function apiCall2() {
console.log('start-apiCall2')
await cWait(api2delay)
console.log('done-apiCall2')
return {
api2: 'done'
}
}
async function apiCall1() {
await cWait(5)
return {
api1: 'done'
}
}
async function saveInDB(data) {
console.log('saveInDB', data)
}
test()
Note: I changed where one and two were declared since const is block scoped
I you run with await cWait(800) in apiCall2, the saveInDB will run with both data.
But if you run await cWait(1800), the saveInDB will run 2 times.
// Function to simulate it's taking time.
async function cWait(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
// https://italonascimento.github.io/applying-a-timeout-to-your-promises/
const promiseTimeout = function (ms, promise) {
// Create a promise that rejects in <ms> milliseconds
let timeout = new Promise((resolve, reject) => {
let id = setTimeout(() => {
clearTimeout(id);
reject('Timed out in ' + ms + 'ms.')
}, ms)
})
// Returns a race between our timeout and the passed in promise
return Promise.race([
promise,
timeout
])
}
// Track whether it took time.
let isTimeOut = false
async function test() {
console.log('starting')
const one = await apiCall1() // get data from 1st API
let two = {};
try {
two = await promiseTimeout(1000, apiCall2())
} catch (error) {
isTimeOut = true;
console.log(error)
}
saveInDB({ ...one, ...two })
}
async function apiCall2() {
console.log('start-apiCall')
await cWait(800)
console.log('done-apiCall', isTimeOut)
if (isTimeOut) {
saveInDB({ 2: 'two' })
}
return { 2: 'two' }
}
async function apiCall1() {
await cWait(5)
return { 1: 'one' }
}
async function saveInDB(data) {
console.log('saveInDB', data)
}
test()
I would like to have my microservice waiting until the database becomes available.
(I have a sidecar Cloud SQL proxy that needs some time for the database connection).
So I was thinking of writing a for loop that attempts a connect and retries after a defined time.
Currently, the code looks as follows, but it doesn't seem to wait before reconnecting.
private class Database {
static async connectDatabase() {
try {
const retries = 20;
const tryTimeout = async (currentTry, retries) => {
return new Promise((resolve) =>
setTimeout(function () {
logger.info(`Try: ${currentTry}, ${retries} retries left.`);
}, 1000));
};
for (let i = 1; i <= retries; i++) {
try {
// Establish database connection
await SequelizeConnection.authenticate()
.then(() => {
logger.info(
"*** Database connection has been established successfully."
);
})
.catch(async (err) => {
logger.info("Error in connect database function: ", err);
throw err;
});
await SeqzelizeConnectionHealthcheck.authenticate()
.then(() => {
logger.info(
"*** Database connection for healthcheck has been established successfully."
);
})
.catch(async (err) => {
logger.error(
"Error in connect database function for healthcheck: ",
err
);
throw err;
});
} catch (error) {
logger.error("Error in connectDB retry function");
await tryTimeout(i, retries - i);
}
}
} catch (error) {
logger.error("Error in connect database function: ", error);
}
}
}
I was thinking of creating a retry wrapper function and tried some retry libraries but without success.
As mentioned in comments, there are a few issues in the code. Mainly that you're mixing await with .then()/.catch(), not resolving the promise from the wait and not breaking out of the for loop when the connections are successful.
To illustrate this, I've reformatted your method a bit and extracted some functions so it's hopefully clearer. In the snippet below I'm simulating the database connection succeeding after 3 tries and the healthcheck database connection succeeding after 5 tries.
I've also changed the logic a bit so that it makes the main database and healthcheck database connections concurrently and retries them independently - you don't want to be retrying the main database connection again just because the healthcheck one failed, so for this I created a retryUntilResolved function that will retry the given function until it resolves, or until the maximum number of retries has been reached.
// Mocks purely for snippet
const resolveNthTime = nth => {
let callCount = 0;
return () => ++callCount >= nth
? Promise.resolve()
: Promise.reject('failed!');
}
const SequelizeConnection = { authenticate: resolveNthTime(3) };
const SequelizeConnectionHealthcheck = { authenticate: resolveNthTime(5) };
// Example
const pause = ms =>
new Promise(resolve => setTimeout(resolve, ms));
const retryUntilResolved = (waitMs, maxRetries) => async func => {
let tries = 0;
while (true) {
try {
return await func();
} catch(err) {
if (tries++ < maxRetries) await pause(waitMs);
else return err;
}
}
};
const authenticateDatabase = async () => {
try {
await SequelizeConnection.authenticate();
console.info("Database connection established");
} catch (err) {
console.warn("Error connecting to database: ", err);
throw err;
}
};
const authenticateHealthcheck = async () => {
try {
await SequelizeConnectionHealthcheck.authenticate();
console.info("Database connection for healthcheck established");
} catch (err) {
console.warn("Error connecting to healthcheck database: ", err);
throw err;
}
};
class Database {
static async connectDatabase() {
const maxRetries = 20;
const msBeforeRetry = 1000;
const retry = retryUntilResolved(msBeforeRetry, maxRetries);
try {
await Promise.all([
retry(authenticateDatabase),
retry(authenticateHealthcheck),
]);
console.info('Both connections established');
} catch (error) {
console.error('Could not establish both connections');
}
}
}
Database.connectDatabase();
You should call resolve() in setTimeout().
Please see below example. It is the code that assumes that authenticate() always fails.
async function connectDatabase() {
try {
const retries = 20;
const tryTimeout = async (currentTry, retries) => {
return new Promise((resolve) =>
setTimeout(function () {
console.log(`Try: ${currentTry}, ${retries} retries left.`);
resolve();
}, 1000));
};
for (let i = 1; i <= retries; i++) {
try {
// Establish database connection
await sequelizeConnection_authenticate()
.then(() => {
console.log(
"*** Database connection has been established successfully."
);
})
.catch(async (err) => {
console.log("Error in connect database function: ", err);
throw err;
});
await seqzelizeConnectionHealthcheck_authenticate()
.then(() => {
console.log(
"*** Database connection for healthcheck has been established successfully."
);
})
.catch(async (err) => {
console.log(
"Error in connect database function for healthcheck: ",
err
);
throw err;
});
} catch (error) {
console.log("Error in connectDB retry function");
await tryTimeout(i, retries - i);
}
}
} catch (error) {
console.log("Error in connect database function: ", error);
}
}
async function sequelizeConnection_authenticate() {
return new Promise((_, reject) => {
setTimeout(function () {
reject();
}, 1000);
});
}
async function seqzelizeConnectionHealthcheck_authenticate() {
return new Promise((_, reject) => {
setTimeout(function () {
reject();
}, 1000);
});
}
connectDatabase();
I'm trying to retrieve values from mongoDB and its giving me the
UnhandledPromiseRejectionWarning: MongoError: topology was destroyed
UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch().
[DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Following is the scaled down code version
CLASS 1
connectToMongoDatabase() {
try {
return new Promise((resolve, reject) => {
mongoclient.connect('mongodb://************************', (err, db) => {
if (err) {
reject(err);
}
else {
resolve(db);
}
});
});
}
catch (err) {
console.log(err);
}
}
fetchIssuesFromMongo(dbName, collectionName, query, db) {
try {
let dbo = db.db(dbName);
return new Promise((resolve, reject) => {
let collection = dbo.collection(collectionName);
collection.find(query, (err, result) => {
if (err) {
reject(err);
}
else {
resolve(result);
dbo.close();
}
});
});
}
catch (err) {
console.log(err);
}
}
CLASS 2
executeQuery(issueCount){
this.CLASS1.connectToMongoDatabase().then((db) => {
this.CLASS1.fetchIssuesFromMongo(dbName, collectionName, query, db).then((result: any) => {
expect(result.count()).toEqual(issueCount);
});
});
}
SPEC FILE
it('verify result', (done) => {
CLASS2.executeQuery(6).then(() => {
done();
});
});
What I think is the test fails after this.CLASS1.connectToMongoDatabase().
Is there any issue with how I'm using promises ? I'm resolving all the promises and have reject statements also in place.
Any suggestions ?
Updating your Class 1
Remove the try catch since it will never catch on a returned promise. Here's the change for fetchIssuesFromMongo. You should do something similar for connectToMongoDatabase
fetchIssuesFromMongo(dbName, collectionName, query, db) {
const dbo = db.db(dbName);
return new Promise((resolve, reject) => {
const collection = dbo.collection(collectionName);
collection.find(query, (err, result) => {
if (err) {
reject(err); // at this point you should call a .catch
} else {
dbo.close(); // switching the order so the close actually happens.
// if you want this to close at the exit, you should
// probably not do it like this.
resolve(result);
}
});
});
}
Fixing the executeQuery in Class 2
In your executQuery:
executeQuery(issueCount){
// if connectToMongoDatabase is thenable, then you should also call .catch
// you should also return a promise here so your Protractor code can actually
// call .then in `CLASS2.executeQuery(6).then`
return this.CLASS1.connectToMongoDatabase().then((db) => {
this.CLASS1.fetchIssuesFromMongo(dbName, collectionName, query, db).then((result: any) => {
expect(result.count()).toEqual(issueCount);
}).catch(e => {
console.log(e);
});
}).catch(e => {
console.log(e);
});
}
Think about using async / await.
This usually helps clear up the nested chain of promises. I prefer this.
// this returns implicitly returns a Promise<void>
async executeQuery(issueCount) {
// valid to use try catch
try {
const db = await this.CLASS1.connectToMongoDatabase();
const result = await this.CLASS1.fetchIssuesFromMongo(dbName, collectionName, query, db);
expect(result.count()).toEqual(issueCount);
} catch(e) {
console.log(e);
}
}
Use async / await in your Protractor test
Finally in your Protractor test you should turn off the selenium promise manager. This is something you'll do in your configuration file. SELENIUM_PROMISE_MANAGER: false,
Next you can use async / wait in your test.
it('verify result', async () => {
await CLASS2.executeQuery(6);
});
I'm not a fan of expecting a condition in your class and it might be better to return the value from class 2. So I would maybe return a Promise from executeQuery.
const issueCount = 6;
const queryResult = await CLASS2.executeQuery(issueCount);
expect(queryResult).toEqual(issueCount);
Hope that helps.