Continue .map loop after util.promisify request fails in iteration (async await) - javascript

I have the below async function taking in an array of 4 objects. The first one is processed in a .map loop that executes a query and results in an error (in the await executeQuery(queryString)). When this first object is completed processing, the loop ends and the other 3 objects are not processed.
async function myFunction(arrayOfObjects, param1) {
const promises = arrayOfObjects.map(async function (currElement, i) {
var queryString = 'www.querystring.com?APIparameter=param1';
const executeQuery = util.promisify(request);
await executeQuery(queryString).then(data => {
//process data here using currElement, i
}).catch(err => console.log('error: ', err));
});
await Promise.all(promises).catch();
}
I originally only had await Promise.all(promises) and thought revising it to await Promise.all(promises).catch(); would do the trick but it is still failing out on the first object iterated.
I was wondering how best to achieve continuing the .map loop after an error hits the catch in the executeQuery.
Please and thank you!

Not 100% sure what's happening with this code but there are a few things I would change. Firstly it's not good practice to use an await inside a loop and secondly, your promises array is probably not what you expect as the await is yielding until each promise is resolved inside the loop. Return the Promise from the map function to generate an array of Promises that will each resolve and process the data in their own time. You can then await all the promises and catch any errors using try/catch. Something similar to the following should do what you are expecting...
async function myFunction(arrayOfObjects, param1) {
const executeQuery = util.promisify(request);
const promises = arrayOfObjects.map((currElement, i) => {
const queryString = 'www.querystring.com?APIparameter=param1';
return executeQuery(queryString).then(data => {
// process data here using currElement, i
});
});
try {
await Promise.all(promises);
// all promises will be resolved here
} catch (err) {
console.log('error: ', err);
}
}

Related

Async Mysql Query in Node.js

I want to get a specific value from my mysql database and then pass it to an API. So I have to wait for the query to finish. I found this async/await example on stackoverflow. But it's not working for me.
async function getUTC() {
try {
let result = await db.query(`SELECT utcEndSeconds FROM mp_games ORDER BY utcEndSeconds ASC LIMIT 1`)
return result
} catch (err) {
console.log(err)
}
}
let newUTC = getUTC()
console.log(newUTC)
newUTC.then(data => {
console.log(data)
})
The first console.log(newUTC) prints Promise { pending }. I expected to find my data here, because await should already resolve the promise???
Then I added the .then() block. console.log(data) prints a big object (mysql I guess) but my data is nowhere to be found in the object.
What am I missing here? Thanks in advance.
You have problems with understanding of async/await pattern.
newUTC.then(data => {
console.log(data)
})
the above code resolves promise and return you data object. but console.log(newUTC) gives you promise pending because it is not awaited nor encapsulated with promise for resolution.
if you change
let newUTC = getUTC()
console.log(newUTC)
to
let newUTC = getUTC()
console.log(await newUTC)
then first the code awaited and the result is passed to the console.log.
EDIT: await only works within async function. IF your function is not async then you need to use promise approach.
let newUTC = getUTC()
newUTC.then(data => console.log(data));
I think it's easier to use mysql.query callback function and run the rest of my code inside. It's not as pretty but works
db.query(`SELECT utcEndSeconds FROM mp_games ORDER BY utcEndSeconds ASC LIMIT 1`, (err, result) => {
let newUTC = result[0].utcEndSeconds
... rest of my code (api calls etc...)
}

Promise.all results are as expected, but individual items showing undefined

First of all, there are some issues with console.log in Google Chrome not functioning as expected. This is not the case as I am working in VSCode.
We begin with two async calls to the server.
promise_a = fetch(url)
promise_b = fetch(url)
Since fetch results are also promises, .json() will needed to be called on each item. The helper function process will be used, as suggested by a Stackoverflow user -- sorry lost the link.
let promiseResults = []
let process = prom => {
prom.then(data => {
promiseResults.push(data);
});
};
Promise.all is called. The resulting array is passed to .then where forEach calls process on item.json() each iteration and fulfilled promises are pushed to promiseResults.
Promise.all([promise_a, promise_b])
.then(responseArr => {
responseArr.forEach(item => {
process(item.json());
});
})
No argument is given to the final .then block because promiseResults are in the outer scope. console.log show confusing results.
.then(() => {
console.log(promiseResults); // correct results
console.log(promiseResults[0]); // undefined ?!?
})
Any help will be greatly appreciated.
If you are familiar with async/await syntax, I would suggest you not to use an external variable promiseResults, but return the results on the fly with this function:
async function getJsonResults(promisesArr) {
// Get fetch promises response
const results = await Promise.all(promisesArr);
// Get JSON from each response promise
const jsonResults = await Promise.all(results.map(r => r.json()));
return jsonResults
}
This is usage example:
promise_a = fetch(url1)
promise_b = fetch(url2)
getJsonResults([promise_a, promise_b])
.then(theResults => console.log('All results:', theResults))
Use theResults variable to extract necessary results.
You can try this, it looks the array loop is not going properly in the promise env.
Specifically: the promiseResults is filled after you are logging.
var resultAll = Promise.all([promise_a, promise_b])
.then(responseArr => {
return Promise.all(responseArr.map(item => return item.json()));
});
resultAll.then(promiseResults => {
console.log(promiseResults);
});

WHy am I getting a SyntaxError: Unexpected identifier in forEach with await?

I have an object with a series of named SQL query templates that I'm trying to execute.
The query() method is async, so I should be able to run it with await, but I get an Unexpected identifier error. If I take off the await, then I don't get an error, but I'm stuck with a Promise object...
Object.entries(sqlquerries).forEach(([queryName, queryTpl]) => {
try {
const querystr = replVars(queryTpl, vars);
report[queryName] = await query(querystr);
} catch (err) {
console.error('An error occured running the query : ', err );
}
});
return report;
Why would that be? What can I do to construct my report object and get around this error?
Any help/pointers would be greatly appreciated.
Because the function await appears in (the forEach callback) isn't an async function. If you make it one, beware that forEach does nothing with the callback's return value, which will be a promise if it's an async function, and so you need to be sure to handle errors inline (which you're doing).
But, your return report suggests you're expecting to wait for this process to complete. Beware that forEach will not wait for the previous iteration's promise to resolve before proceeding with the next. If you want to do that, use the promise reduce trick instead and await the result (I assume this is all in an async function):
await Object.entries(sqlqueries).reduce((p, [queryName, queryTpl]) => {
const querystr = replVars(queryTpl, vars);
return p.then(async() => {
report[queryName] = await query(querystr);
});
}, Promise.resolve());
return report;
Or if you can run the queries in parallel, use map and Promise.all instead, and again await the result:
await Promise.all(Object.entries(sqlqueries).map(async ([queryName, queryTpl]) => {
const querystr = replVars(queryTpl, vars);
report[queryName] = await query(querystr);
}));
return report;

Promises and cycles

There's a chain of promises, the first takes user's id from data base after i need to get his groups and names of constructor from data base
And these can be multiple copies
As a result cycle into cycle. eventually I receive object only of one user
async(takeIds()
.then(ids => {
// console.log("ids", ids);
for (var i = 0; i < ids.length; i++) {
data.id = ids[i];
await (takeIdvk(ids[i])
.then(idm => {
data.idvk = idm;
takeIdg(data.id)
.then(res => {
takeNamesCycle(res, data)
.then(data => {
console.log("data", data);
})
.catch(err => {
console.log(err);
});
})
.catch(err => {
console.log(err);
});
})
.catch(err => {
console.log(err);
}));
}
})
.catch(function(err) {
console.log(err);
}));
And function takeNamesCycle cos it differs from other
( in the rest simple requests to data base)
var takeNamesCycle = async(function(arr, obj) {
return new Promise((resolve, reject) => {
for (var i = 0; i < arr.length; i++) {
var idg = arr[i];
await (takeNames(arr[i])
.then(names => {
obj.idg[idg] = names;
})
.catch(err => {
console.log(err);
}));
}
resolve(obj);
});
});
Is it possible to simplify this?
I'd be grateful
Consider using real async/await syntax instead of a library with async() and await() functions.
const ids = await takeIds();
// console.log("ids", ids);
try {
for (var i = 0; i < ids.length; i++) {
data.id = ids[i];
const idem = await takeIdvk(ids[i]);
data.idvk = idm;
const res = await takeIdg(data.id);
console.log("data", await takeNamesCycle(res, data));
}
} catch(err) {
console.log(err);
}
and make sure to avoid the Promise constructor antipattern
async function takeNamesCycle(arr, obj) {
for (const idg of arr) {
const names = await takeNames(idg);
obj.idg[idg] = names;
}
return obj;
}
The posted code is mixing promise chain style of code within await and async functions within bad code.
First await does not permit .then or .catch methods to be called on the value it returns. The reason is that the value returned by await is never a promise. await only resumes execution after a successful promise operation and returns the result of the operation. So for example
await (takeIdvk(ids[i])
.then(idm => { // incorrect
data.idvk = idm;
// ... more code
becomes
let idm = await takeIdvk(ids[i]) // await returns fulfilled value
data.idvk = idm;
// ... more code
In other words, code to handle data for a successful operation is written inline after await instead of in a separate onfulfilled handler passed as a parameter to .then.
Promise Rejections
If the promise being waited on becomes rejected, the await operator throws the rejection reason without resuming function execution.
The rejection can be caught in a try/catch block in order to ignore the error, repeat the operation, or replace it with a different operation as determined by application requirements. E.G.
let idm;
try {
idm = await takeIdvk(ids[i]) // await returns fulfilled value
data.idvk = idm;
// ... more code
} catch( err) {
console.log( "ignoring error + err);
// the value of idm is undefined
}
On the other hand, if rejection is to be considered fatal don't put await inside a try/catch block - the thrown value will be caught and used to reject the promise returned by the async function, much like throwing an error inside a then or catch handler.
If calls to async functions are nested, each called using await without any try/catch blocks, rejection errors will bubble up from deeply nested functions and reject the promise returned by the outermost async function call. A single .catch clause on the outermost promise can be used to prevent uncaught promise rejections and notify that a failure occurred.
Conclusion
Try writing the code using plain promise chains if you are more familiar with that syntax. By all means continue with converting it to async/await syntax but be aware that async/await is an alternative syntax and they don't mix well in the same function. As already stated by #Bergi , returning a new promise from an async function is an anti pattern - async functions return a promise already.

Node JS : Array loop and function Not understanding how callbacks and promises work

New to concept of callbacks and promises. I'm trying to read contents of a directory which is correctly returning the addresses but this code is only printing console.log(value) and the console.log in the function getAccount "gettin info..." but nowhere printing the balance the api is getting closed and the process completes, I dont understand this because the address is being passed and the first console.log is printing inside the function but not doing further job. when i remove the fs.readdir and files.foreach and pass a single value to getAccount its working perfectly fine. No errors or bugs , its a runtime error and im guessing related to callbacks
'use strict';
const RippleAPI = require('ripple-lib').RippleAPI;
const testFolder = '/home/ripple/.keystore';
const fs = require('fs');
const api = new RippleAPI({server: 'wss://s1.ripple.com'});
function getAccount(address) {
var balance = 0.0;
console.log("getting info for :"+address);
return api.getAccountInfo(address).then(info => {
var balance = info.xrpBalance;
console.log("balance of "+address+" is "+balance);
return balance;
});
}
api.connect().then(() => {
fs.readdir(testFolder, function(err, files) {
// console.log("Total no of wallets : "+files.length);
// for (var i =3000, len=3200; i < len; i++) {
// var ad = files[i];
// console.log(ad + "\t" + typeof(ad));
// if(!xwallets.includes(ad))
files.forEach(function(value) {
getAccount(value).then(balance => {console.log(balance); });
console.log(value);
});
// console.log("running for index : "+i+" filedata :"+files[i]);
// }
});
// console.log(balance);
}).then(() => {
return api.disconnect();
}).then(() => {
console.log("done and disconnected");
}).catch(console.error);
You really don't want to mix asynchronous operations that return a promise with those that take a direct callback. It's hard to write good robust code when mixing like that. So, since you are already using with promises in a number of operations, it's best to take the remaining async operations and "promisify" them so you can use them with promises. Then, you can take advantage of all the clean error propagation and chaining of promises.
Then, secondly, you have to make sure all your asynchronous operations in a loop are properly chained to the parent promise. To do that here, we will accumulate the promises from getAccount() in the loop into an array of promises. Then, after the loop, we can tell when they're all done using Promise.all() and then return that promise from readdir() so that they will all be properly chained to the parent promise.
This will then make sure everything inside the loop finishes before the parent resolves and before you call api.disconnect().
Here's what that would look like:
'use strict';
const RippleAPI = require('ripple-lib').RippleAPI;
const testFolder = '/home/ripple/.keystore';
const fs = require('fs');
const api = new RippleAPI({server: 'wss://s1.ripple.com'});
function getAccount(address) {
var balance = 0.0;
console.log("getting info for :"+address);
return api.getAccountInfo(address).then(info => {
var balance = info.xrpBalance;
console.log("balance of "+address+" is "+balance);
return balance;
});
}
// promisify readdir
const readdir = util.promisify(fs.readdir);
api.connect().then(() => {
return readdir(testFolder).then(function(files) {
const promises = [];
files.forEach(function(file) {
console.log(file);
promises.push(getAccount(file).then(balance => {
console.log(balance);
// make balance be the resolved value for this promise
return balance;
}));
});
// make sure we are waiting for all these promises to be done
// by chaining them into the parent promise
return Promise.all(promises);
});
}).then(balances => {
// process balances array here
console.log(balances);
return api.disconnect();
}, err => {
// disconnect, even in the error case, but preserve the error
console.error(err);
return api.disconnect().then(() => {throw err;})
}).then(() => {
console.log("done and disconnected");
});
Summary of changes made:
Promisify fs.readdir() so we can use promise with it
Change how we call readdir() to use promises
Collect getAccount() promises inside of .forEach() into an array of promises
Add return Promise.all(promises) so that we wait for all those promises to be done and so that it is chained into the parent promise
Make balance be the resolved value of each promise in the .forEach() loop, even after logging its value
Make sure we're calling api.disconnect() even in the error cases
After logging error, preserve the error value as the reject reason

Categories