I have to call a promise for some async task for each item in an array but I would like to execute these serially.
Promise.all is useful only to have a new promise that merges a list of promise but it does not invoke them in a sequence.
How can I achieve this using standard promise api without third-party libraries like Q, bluebird....
You chain promises using .then() with a callback that returns another promise. So, let's say you have three functions a, b and c that all return a promise. You can chain them (execute in sequence) like this:
a().then(b).then(c).then(function(result) {
// all are done here
});
If you are processing an array and you have a promise-returning function myFunc that you want to call for each item in the array, you can use a standard design pattern for arrays and promises with .reduce() to walk through the array one item at a time like this:
var items = [...];
items.reduce(function(p, item) {
return p.then(function() {
return myFunc(item);
});
}, Promise.resolve());
As it turns out this is really just chaining a bunch of .then() handlers like in the first example, but using the structure of .reduce() to walk the array for you.
Starting with ES2017, you can also use async/await to serially process an array like this:
async function processArray(arr) {
for (let item of arr) {
let result = await somePromiseReturningFunc(item);
// do something with result here before
// going on to next array item
}
return someFinalResult;
}
processArray(someArray).then(result => {
// done processing array here
}).catch(err => {
// handle error here
});
Related
I think this is a fairly simple question, but I primarily program in ruby and ma having trouble wrapping my head around asynchronous functions in javascript. Specifically, I have a method that is supposed to populate an array with results fetched asynchronously from an API. I am able to get the results fine, but I can't seem to return the array after it has been populated. Rather, the return statement executes before the promise is resolved, so an empty array is returned. Simplified code example below:
async function getAnimalSounds(animals){
var sounds = []
for (const a of animals){
asynchronously_fetch_sound(a).then((result) => {
sounds.push(result)
})
}
return sounds // sounds returns as an empty array
}
Thank you in advance!
The problem here is that you are using a plain for loop to traverse the animals array. Loops in NodeJS won't wait for the promise of the current iteration to resolve before moving onto the next, so the loop will finish before you have resolved your promises.
The best thing to do is to construct an array of promises you want to resolve, and then call Promise.all on that array.
async function getAnimalSounds(animals){
const promises = animals.map(a => asynchronously_fetch_sound(a))
const sounds = await Promise.all(promises)
return sounds
}
Or if you're happy to use the Bluebird library (docs here), you can do this:
const Bluebird = require('bluebird') // import Bluebird library
async function getAnimalSounds(animals){
const sounds = await Bluebird.map(animals, (a) => asynchronously_fetch_sound(a))
return sounds
}
Remember that since you have written an async function, you will need to wait for it to resolve before doing anything with its output; either by awaiting it or by calling .then(...).
You should add await for the asynchronous method call.
async function getAnimalSounds(animals){
var sounds = []
for (const a of animals){
const result = await asynchronously_fetch_sound(a);
sounds.push(result)
}
return sounds // sounds returns as an empty array
}
However, the best practice is to use Promise.all()
async function getAnimalSounds(animals){
var sounds = []
const promises = []
for (const a of animals){
promises.push(asynchronously_fetch_sound(a));
}
sounds = await Promise.all(promises);
return sounds;
}
You can use Promise.all() to achieve that:
function getAnimalSounds(animals){
return Promise.all(animals.map(a => asynchronously_fetch_sound(a)));
}
If I have an array of elements and I want to do parallel operations on them.
I would use promise.all().
I knew promise.all() accepts array of promises. Correct me if I am wrong, I don't think so.
Here, it clearly says.
The Promise.all() method returns a single Promise that fulfills when all of the promises passed as an iterable have been fulfilled or when the iterable contains no promises or when the iterable contains promises that have been fulfilled and non-promises that have been returned. It rejects with the reason of the first promise that rejects, or with the error caught by the first argument if that argument has caught an error inside it using try/catch/throw blocks.
So, yes we can pass simple functions to promise.all(), and it resolves if they return and rejects if they throw an error.
Now look at the below code.
const promises = todayAssignedJobs.map(async todayAssigned => {
const [leaderboard, created] = await Leaderboard.findOrCreate({...});
if (!created) {
const rating = (todayAssigned.rating + leaderboard.rating * leaderboard.jobs_completed) / (leaderboard.jobs_completed + 1);
const commission = todayAssigned.commission + leaderboard.commission;
const jobsCompleted = leaderboard.jobs_completed + 1;
await Leaderboard.update({
rating,
commission,
jobs_completed: jobsCompleted,
updated_by: 'system',
}, {
where: {
id: leaderboard.id,
},
});
}
await AssignedJob.update({
is_leaderboard_generated: true,
}, {
where: {
id: todayAssigned.id,
},
});
});
await Promise.all(promises);
Here, I have a doubt.
We are iterating on each element of the array and doing operation on them asyncrhonously. They don't return anything explicitly.
So, I think map is doing parallel operations here too.
Why shuold then use promise.all() here?
.map() is not promise-aware. So, when you pass it an async callback like you are, it does not pay any attention to that returned promise. So, it just runs the loop one after another, not waiting for any of the returning promises. Thus, all the asynchronous operations started in the .map() loop will be in-flight at the same time.
If that's what you want and you want to collect all the returned promises so you can later see when they are all done with Promise.all(), then this pattern works well:
Promise.all(someArray.map(callbackThatReturnsAPromiseHere))
And, this is a common design pattern for it. In fact, the Bluebird promise library has a special function that combined these two called Promise.map(). It also offered another nice feature that let you control how many concurrent async operations could run at once (because it's map() operation is promise-aware).
It sounds like you're trying to figure out if you should only use .map() and not use Promise.all(). If you do that, you will run your asynchronous operations in parallel, but you will not have any idea when they are all done or any ability to collect all the results. You would use Promise.all() on the array of returned promises to know when they are all done and/or to collect their resolved results.
FYI, .map() is JUST a plain loop. It doesn't have any special asynchronous features or any special run-in-parallel features. You can do the same thing with a for loop if you want. It does NOT pause for your async callback to wait for it to be done so a side effect of running it is that you launch a bunch of parallel asynchronous operations.
The purpose of Promise.all here is to be able to await all the promises before continuing.
If you added a console.log immediately before the await Promise.all(promises); it would run synchronously before any promises have resolved, whereas a console.log immediately after that line will only appear after all the promises have resolved.
You only need Promise.all if you want to do something when all operations are complete. For example:
const promises = todayAssignedJobs.map(async todayAssigned => {
// lots of async stuff
});
await Promise.all(promises);
// now, all Promises have resolved
// alert the user that the leaderboard is completely updated
If you don't need anything to occur once you've determined that all Promises have completed, then there's no point to the Promise.all - you may as well just create the Promises in a loop and leave them as-is. In that case, since you wouldn't be using the resulting array of Promises, it would be more appropriate to use forEach (which is the array method to use for side-effects).
There's one issue, though - you aren't handling errors, but errors should be handled, otherwise they'll give warnings or exit the Node process:
const processJob = async (todayAssigned) => {
const [leaderboard, created] = await Leaderboard.findOrCreate({...});
if (!created) {
// ...
// ...
todayAssignedJobs.forEach(async todayAssigned => {
try {
await processJob(todayAssigned);
} catch(e) {
// handle errors
}
});
I'm trying to make multiple API calls concurrently using the Promise.all method with Axios, based on this example:
getUsers() {
return axios.get('/users');
}
getSessions() {
return axios.get('/sessions');
}
Promise.all([getUsers(), getSessions()])
.then(results => {
// Use the data
})
.catch(error => {
// Catch the error
});
However, as I will only know what concurrent API calls I need to make at this stage based on the result of a previous API call in the promise chain, I'm attempting to pass in an array of anonymous functions to the function in the form of:
var array = [];
array.push(() => {return axios.get('/users')});
array.push(() => {return axios.get('/sessions')});
Promise.all(array).then....
This doesn't work, and I understand that this is because I'm passing in function objects instead of referencing actual Promise objects as the method expects. However, pushing just axios.get(...) methods to the array results in them being called immediately, instead of later when the Promise.all method is executed.
I'm not sure how to do this properly, or whether there's a better approach to achieve what I'm after...
I am not familiar with Axios, but if I understand correctly axios.get returns a Promise, which you need for Promise.all. How about:
Promise.all(array.map(f => f()).then....
That way your functions are called when you actually want, and the map will give you an array of their results, so an array of Promises.
Note that this is essentially the same as the example you have cited with [getUsers(), getSessions()] – with the difference that your functions are anonymous and called implicitly using the map, instead of explicitly by their names. This way you have more flexibility in which functions actually get called.
Basically, in siplest scenario, to use anonymous function as a wrapper, you need to call it before you are put it to Promise.all.
So just try to :
const f = async () => {return service.myFunc();}
Promise.all([f()])
so, your wrapper (f) calling before, and you are providing array with one Promise.
or
Promise.all([(async () => {const result = await service.myFunc(); return result;})()]);
or (equal) :
const myPromise = (async () => {const result = await service.myFunc(); return result;})();
Promise.all([myPromise]);
So Promise will be provided to Promise.all, not just raw function. Also, be carefully and do not forget to use await in case of "big anonym function body".
i've been scratching my head about this for the longest time. in the first example you call the functions here:
Promise.all([getUsers(), getSessions()])
but in the second the anonymous functions are just stated, never called:
array.push( ()=>{return axios.get('/users')} );
so you would need to do this and call them as you pushed them:
array.push( (()=>{return axios.get('/users')}) () );
I would like to do a synchronous loop in a part of my code.
The function saveInDatabase checks if the item title (string) already exists in the database. That's why it can't be resolved in parallel, otherwise the condition will never apply (and would create duplicates).
Promise.all(arr.map(item => {
saveInDatabase(item).then((myResult) => ... );
}));
I tried to encapsulate this function into separate promises, also tried with npm packages (synchronous.js, sync), but it seems that it does not fit with my code.
Maybe this solution is completely silly.
Do you think it's a better idea to replace promise.all by a synchronous loop (foreach for example) ? The problem is that I need the results of each iteration...
I'm using Node 6.11.2. Could you give me some tips to handle that ?
Thank you in advance.
Without using await (which is not in node.js v6.11.2, but would make this simpler), a classic pattern for serializing a bunch of async operations that return a promise is to use a reduce() loop like this:
arr.reduce(function(p, item) {
return p.then(function() {
return saveInDatabase(item).then((myResult) => ... );
});
}, Promise.resolve()).then(function() {
// all done here
}).catch(function(err) {
// error here
});
If you want to save all the results, you can use your .then(myResult => ...) handler to .push() the result into an array which you can access when done.
This will serialize all the calls to saveInDatabase(item) to it waits for the first one to be done before calling the second one, waits for the second one to be done before calling the third one, etc...
The default implementation here will stop if saveInDatabase(item) rejects. If you want to keep going (you don't say in your question), even when it gives an error, then you can add a .catch() to it to turn the rejected promise into a fulfilled promise.
In node.js v7+, you can use await in a regular for loop:
async function myFunc() {
let results = [];
for (let item of arr) {
let r = await saveInDatabase(item).then((myResult) => ... );
results.push(r);
}
return results;
}
myFunc().then(function(results) {
// all done here
}).catch(function(err) {
// error here
});
If you could run all the requests in parallel, then you could do that like this:
Promise.all(arr.map(item => {
return saveInDatabase(item).then((myResult) => ... );
})).then(function(results) {
// all done here
}).catch(function(err) {
// error here
});
In any of these, if you don't want it to stop upon a rejection, then add a .catch() to your saveInDatabase() promise chain to turn the rejection into a resolved promise with some known value or error value you can detect.
I am trying to iterate through an array which pushes a new Thing to a list, inside the Thing it does some async calls of its own. How would I iterate through an array in a synchronous way, as the callback requires the the data from the list to work. Since my for loop is synchronous and does some asynchronous calls, the callback is called before the the list if finished being made.
I do not know how I would iterate through the array and do all the work before doing the callback
load(file, callback) {
fs.readFile(file, (err, fd) => {
var data = JSON.parse(fd);
for(var i of data.array){
this.list.push(new Thing(i.id)); // new Thing is Asynchronous
}
callback(); // Needs a finished list
return;
});
}
Solved it:
By converting my Thing class to synchronous, by removing the asynchronous calls to a function inside the class, and first instantiating all the Things in the loop then calling Promise.all calling the function I solved the issue:
load(file, callback) {
fs.readFile(file, (err, fd) => {
var data = JSON.parse(fd);
for(var i of data.array){
this.list.push(new Thing(i.id));
}
Promise.all(this.list.map(i => i.load()).then(callback);
});
}
You'd have to have some state inside of Thing to track it's doneness for example you could have an instance variable that's a promise. So given this hacked together example of Thing
class Thing {
constructor(id) {
this.id = id;
this.done = new Promise((resolve, reject) => {
asyncThingWithCallback((err, data) {
if (err) {
this.asyncVal = null;
reject(err);
} else {
this.asyncVal = data;
resolve()
}
})
});
}
}
You can use the done property inside of your callback like this:
load(file, callback) {
fs.readFile(file, (err, fd) => {
var data = JSON.parse(fd);
for(var i of data.array){
this.list.push(new Thing(i.id)); // new Thing is Asynchronous
}
Promise.all(this.list.map((thing) => thing.done))
.then(callback)
});
}
First off, it's generally not advisable to have a constructor that needs some asynchronous operation to finish creating a valid object. That just doesn't lead to easily writable or maintainable code because the constructor has to return the object reference and because the operation is asynchronous, it has to return that object reference before you're done creating a valid object. That just leads to messy, partially created objects. You can make it work by requiring a completion callback be passed to the constructor and making sure the calling code does not attempt to use the object until after the completion callback has been called, but this is just not a clean way to do things. It also makes it impossible to have your async operation return a promise (which is the future of async design) because the constructor has to return the object reference so it can't return a promise.
You could embed the promise in the object, but that's messy too because the promise is only really useful during the initial async operation.
What is often done instead is to make the constructor be only synchronous and then have a .init() method that does the async parts. That makes for cleaner code and is compatible with implementations using promises.
Or, you can create a factory function that returns a promise that resolves to the object reference.
Second off, as you already seem to know, your for loop runs synchronously. It doesn't "wait" for any async operations inside it to complete before going onto the next part of the loop. As long as each invocation of the loop is separate and doesn't depend upon the prior iteration, that's all fine. All you need to know is when all the async operations in the loop are done and making your async operations return promises and using Promise.all() is generally the best tool for that.
So, let's supposed you use the .init() method scheme where .init() does the async part of the initialization and the constructor is synchronous and .init() returns a promise. Then, you could do this:
// create all the things objects
let things = data.array.map(i => new Thing(i.id));
// initialize them all asynchronously
Promise.all(things.map(item => {
return item.init();
})).then(function() {
// all things are asynchronously initialized here
});
Or, using the concept of a factory function that returns a promise that resolves to the object:
function newThing(i) {
let o = new Thing(i.id);
return o.init().then(function() {
// resolve to the object itself
return o;
});
}
Promise.all(data.array.map(i => newThing(i))).then(things => {
// all things in the array ready to be used here
});
If you need to sequence your array iteration so the 2nd iteration did not start until the async part of the first iteration was done and 3rd waited until the 2nd iteration was done and so on, then you can't use a for loop because it simply doesn't work that way. There are several different ways to do such a serialized async iteration. You can see several different schemes in these other posts:
How to synchronize a sequence of promises?
JavaScript: Perform a chain of promises synchronously
ES6 Promises - something like async.each?
How can I execute shell commands in sequence?
You can use primise.all to run all the promises after the for loop .Then you can resolve the promise.all .
load(file) {
fs.readFile(file).Then(function (fd){
var data = JSON.parse(fd);
var EachPromise = [ ];
for(var i of data.array){
EachPromise.push(new Thing(i.id)); // new Thing is Asynchronous
}
Promise.all(EachPromise) .then(function (result){
console.log('this is result',result);
}).Catch(function (error){
console.log('this is error', error);
});
}