I know this isn't in the scope of a Array.map but I'd like to wait until the previous item has finished its promise before starting the next one. It just happens that I need to wait for the previous entry to be saved in the db before moving forwards.
const statsPromise = stats.map((item) => {
return playersApi.getOrAddPlayer(item, clubInfo, year); //I need these to wait until previous has finished its promise.
});
Promise.all(statsPromise)
.then((teamData) => {
..//
});
playersApi.getOrAddPlayer returns a new Promise
Edit
Reading more on it, it seems its important to show playersApi.getOrAddPlayer
getOrAddPlayer: function (item, clubInfo, year) {
return new Promise((resolve, reject) => {
var playerName = item.name.split(' '),
fname = playerName[0].caps(),
sname = playerName[1].caps();
Players.find({
fname: fname,
sname: sname,
}).exec()
.then(function(playerDetails, err){
if(err) reject(err);
var savePlayer = new Players();
//stuff
savePlayer.save()
.then(function(data, err){
if(err) reject(err);
item._id = data._id;
resolve(item);
});
});
});
}
You can use reduction instead of mapping to achieve this:
stats.reduce(
(chain, item) =>
// append the promise creating function to the chain
chain.then(() => playersApi.getOrAddPlayer(item, clubInfo, year)),
// start the promise chain from a resolved promise
Promise.resolve()
).then(() =>
// all finished, one after the other
);
Demonstration:
const timeoutPromise = x => {
console.log(`starting ${x}`);
return new Promise(resolve => setTimeout(() => {
console.log(`resolving ${x}`);
resolve(x);
}, Math.random() * 2000));
};
[1, 2, 3].reduce(
(chain, item) => chain.then(() => timeoutPromise(item)),
Promise.resolve()
).then(() =>
console.log('all finished, one after the other')
);
If you need to accumulate the values, you can propagate the result through the reduction:
stats
.reduce(
(chain, item) =>
// append the promise creating function to the chain
chain.then(results =>
playersApi.getOrAddPlayer(item, clubInfo, year).then(data =>
// concat each result from the api call into an array
results.concat(data)
)
),
// start the promise chain from a resolved promise and results array
Promise.resolve([])
)
.then(results => {
// all finished, one after the other
// results array contains the resolved value from each promise
});
Demonstration:
const timeoutPromise = x => {
console.log(`starting ${x}`);
return new Promise(resolve =>
setTimeout(() => {
console.log(`resolving result for ${x}`);
resolve(`result for ${x}`);
}, Math.random() * 2000)
);
};
function getStuffInOrder(initialStuff) {
return initialStuff
.reduce(
(chain, item) =>
chain.then(results =>
timeoutPromise(item).then(data => results.concat(data))
),
Promise.resolve([])
)
}
getStuffInOrder([1, 2, 3]).then(console.log);
Variation #1: Array.prototype.concat looks more elegant but will create a new array on each concatenation. For efficiency purpose, you can use Array.prototype.push with a bit more boilerplate:
stats
.reduce(
(chain, item) =>
chain.then(results =>
playersApi.getOrAddPlayer(item, clubInfo, year).then(data => {
// push each result from the api call into an array and return the array
results.push(data);
return results;
})
),
Promise.resolve([])
)
.then(results => {
});
Demonstration:
const timeoutPromise = x => {
console.log(`starting ${x}`);
return new Promise(resolve =>
setTimeout(() => {
console.log(`resolving result for ${x}`);
resolve(`result for ${x}`);
}, Math.random() * 2000)
);
};
function getStuffInOrder(initialStuff) {
return initialStuff
.reduce(
(chain, item) =>
chain.then(results =>
timeoutPromise(item).then(data => {
results.push(data);
return results;
})
),
Promise.resolve([])
);
}
getStuffInOrder([1, 2, 3]).then(console.log);
Variation #2: You can lift the results variable to the upper scope. This would remove the need to nest the functions to make results available via the nearest closure when accumulating data and instead make it globally available to the whole chain.
const results = [];
stats
.reduce(
(chain, item) =>
chain
.then(() => playersApi.getOrAddPlayer(item, clubInfo, year))
.then(data => {
// push each result from the api call into the globally available results array
results.push(data);
}),
Promise.resolve()
)
.then(() => {
// use results here
});
Demonstration:
const timeoutPromise = x => {
console.log(`starting ${x}`);
return new Promise(resolve =>
setTimeout(() => {
console.log(`resolving result for ${x}`);
resolve(`result for ${x}`);
}, Math.random() * 2000)
);
};
function getStuffInOrder(initialStuff) {
const results = [];
return initialStuff.reduce(
(chain, item) =>
chain
.then(() => timeoutPromise(item))
.then(data => {
results.push(data);
return results;
}),
Promise.resolve()
);
}
getStuffInOrder([1, 2, 3]).then(console.log);
If you are fine with using promise library, you can use Promise.mapSeries by Bluebird for this case.
Example:
const Promise = require("bluebird");
//iterate over the array serially, in-order
Promise.mapSeries(stats, (item) => {
return playersApi.getOrAddPlayer(item, clubInfo, year));
}).then((teamData) => {
..//
});
You can use a recursion solution
const statsPromise = (function s(p, results) {
return p.length ? playersApi.getOrAddPlayer(p.shift(), clubInfo, year) : results;
})(stats.slice(0), []);
statsPromise
.then((teamData) => {
//do stuff
});
let n = 0;
let promise = () => new Promise(resolve =>
setTimeout(resolve.bind(null, n++), 1000 * 1 + Math.random()));
let stats = [promise, promise, promise];
const statsPromise = (function s(p, results) {
return p.length ? p.shift().call().then(result => {
console.log(result);
return s(p, [...results, result])
}) : results;
})(stats.slice(0), []);
statsPromise.then(res => console.log(res))
You could use a kind of recursion:
function doStats([head, ...tail]) {
return !head ? Promise.resolve() :
playersApi.getOrAddPlayer(head, clubInfo, year)
.then(() => doStats(tail));
}
doStats(stats)
.then(() => console.log("all done"), e => console.log("something failed", e));
Another classic approach is to use reduce:
function doStats(items) {
return items.reduce(
(promise, item) =>
promise.then(() => playersApi.getOrAddPlayer(item, clubInfo, year)),
Promise.resolve());
By the way, you could clean up your getOrAddPlayer function quite a bit, and avoid the promise constructor anti-pattern, with:
getOrAddPlayer: function (item, clubInfo, year) {
var playerName = item.name.split(' '),
fname = playerName[0].caps(),
sname = playerName[1].caps();
return Players.find({fname, sname}).exec()
.then(playerDetails => new Players().save())
.then({_id} => Object.assign(item, {_id}));
}
I gave it a thought but I didn't find a better method than the reduce one.
Adapted to your case it would be something like this:
const players = [];
const lastPromise = stats.reduce((promise, item) => {
return promise.then(playerInfo => {
// first iteration will be undefined
if (playerInfo) {
players.push(playerInfo)
}
return playersApi.getOrAddPlayer(item, clubInfo, year);
});
}, Promise.resolve());
// assigned last promise to a variable in order to make it easier to understand
lastPromise.then(lastPlayer => players.push(lastPlayer));
You can see some explanation about this here.
Related
I have written a function that is being called in a loop (map) and that function is using promises. Now, I want that function to run synchronously and exit before its next instance is called.
function t1(){
let arr1 = [1,2,3,4,5];
return Promise.map(arr1, (val) =>{
const params = {
"param1" : val1
};
return t2(params);
});
}
function t2(event){
return Promise.resolve()
.then({
//do something
//code doesn't reach here in sync manner. all five instance are invoked and then code reaches here for first instance and so on
})
.then({
//promise chaining. do something more
})
}
t2 is beingcalled five times, but I want each instance to be called only after the instance before it has returned the value.
Currently its not behaving like that but invoking the function five times in parallel.
I can't use async/await due to project limitations.
The problem with your current code is that Promise.prototype.map, like forEach, does not wait for asynchronous functions called inside it to complete. (No asynchronous call will ever be waited for unless you tell the interpreter to do so explicitly with await or .then)
Have t1 await each call of t2:
async function t1(){
let arr1 = [1,2,3,4,5];
const results = [];
for (const val of arr1) {
results.push(await t2(val));
}
return results;
}
Or if you want to use reduce instead of async/await:
const delay = () => new Promise(res => setTimeout(res, 500));
function t1(){
let arr1 = [1,2,3,4,5];
return arr1.reduce((lastProm, val) => lastProm.then(
(resultArrSoFar) => t2(val)
.then(result => [...resultArrSoFar, result])
), Promise.resolve([]));
}
function t2(event){
return delay().then(() => {
console.log('iter');
return event;
});
}
t1()
.then(results => console.log('end t1', results));
Or, if you need the sequential functionality to be encapsulated in t2, then have t2 have a semi-persistent variable of the previous Promise it generated:
const delay = () => new Promise(res => setTimeout(res, 500));
const t1 = () => {
return Promise.all([1, 2, 3, 4].map(t2));
};
const t2 = (() => {
let lastProm = Promise.resolve();
return (event) => {
const nextProm = lastProm
.catch(() => null) // you may or may not want to catch here
.then(() => {
// do something with event
console.log('processing event');
return delay().then(() => event);
});
lastProm = nextProm;
return nextProm;
};
})();
t1().then(results => console.log('t1 done', results));
(function loop(index) {
const next = promiseArray[index];
if (!next) {
return;
}
next.then((response) => {
// do Something before next
loop(index + 1);
}).catch(e => {
console.error(e);
loop(index + 1);
});
})(0 /* startIndex */)
Here is what running Promises sequentially would look like when using .reduce() in combination with async/await:
async function main() {
const t2 = (v) => Promise.resolve(v*2)
const arr1 = [1,2,3,4,5];
const arr1_mapped = await arr1.reduce(async (allAsync, val) => {
const all = await allAsync
all.push(await t2(val) /* <-- your async transformation */)
return all
}, [])
console.log(arr1_mapped)
}
main()
var p1 = new Promise((resolve, reject) => {
setTimeout(resolve, 1000, 'one');
});
var p2 = new Promise((resolve, reject) => {
setTimeout(resolve, 2000, 'two');
});
var p3 = new Promise((resolve, reject) => {
setTimeout(resolve, 3000, 'three');
});
Promise.all([p1,p2,p3]).then(values => {
console.log(values);
}, reason => {
console.log(reason)
});
How can i wait for 2 promises to get completed ? Promise.race() wait for one promise to get completed.
Edit
I have n number of promises, what i want to achieve is wait for first k number of promises to get resolved and than trigger some event. assume k < n
Edit - 2
I am sure that k number of promise will be successfully resolved out of n numbers of promise given
(Note: Bluebird has a built-in helper function that serves precisely this purpose and has more or less the same behavior as my waitForN method below, so that option is always available)
I don't believe ES6 promises have an elegant built-in way to do this, but you could define the following relatively short now somewhat long helper function to take care of this.
Edit: Added an additional helper function that includes the indices of the successful promises in the result.
Edit: Updated to reject if the number of rejections reaches the point where successful resolution would be impossible.
Edit: Updated to more gracefully work with promises if it is iterable and check edge cases where the result should immediately resolve (e.g. if n === 0)
function waitForN(n, promises) {
let resolved = [];
let failCount = 0;
let pCount = 0;
return new Promise((resolve, reject) => {
const checkSuccess = () => {
if (resolved.length >= n) { resolve(resolved); }
};
const checkFailure = () => {
if (failCount + n > pCount) {
reject(new Error(`Impossible to resolve successfully. n = ${n}, promise count = ${pCount}, failure count = ${failCount}`));
}
};
for (let p of promises) {
pCount += 1;
Promise.resolve(p).then(r => {
if (resolved.length < n) { resolved.push(r); }
checkSuccess();
}, () => {
failCount += 1;
checkFailure();
});
}
checkFailure();
checkSuccess();
});
}
const waitForNWithIndices = (n, promises) =>
waitForN(n, promises.map((p, index) =>
Promise.resolve(p).then(result => ({ index, result }))
));
var p1 = new Promise((resolve, reject) => {
setTimeout(resolve, 1000, 'one');
});
var p2 = new Promise((resolve, reject) => {
setTimeout(resolve, 2000, 'two');
});
var p3 = new Promise((resolve, reject) => {
setTimeout(resolve, 3000, 'three');
});
var p4 = new Promise((resolve, reject) => {
setTimeout(resolve, 1500, 'four');
});
waitForN(2, [p1,p2,p3,p4]).then(values => {
console.log(values);
}, reason => {
console.log(reason)
});
waitForNWithIndices(2, [p1,p2,p3,p4]).then(values => {
console.log(values);
}, reason => {
console.log(reason)
});
You could use a higher order function to bake in the n then run all promises waiting for n to finish
var p1 = new Promise((resolve, reject) => {
setTimeout(resolve, 1000, 'one');
});
var p2 = new Promise((resolve, reject) => {
setTimeout(resolve, 2000, 'two');
});
var p3 = new Promise((resolve, reject) => {
setTimeout(resolve, 3000, 'three');
});
const raceN = n => (...promises) => {
const resolved = []
return new Promise((res, rej) =>
promises.forEach(promise =>
promise.then(x => {
resolved.push(x)
if (resolved.length === n)
res(resolved)
})
.catch(rej)
)
)
}
const race2 = raceN(2)
race2(p1, p2, p3)
.then(results => console.log(results))
race2(Promise.resolve('resolved'), Promise.reject('rejected'))
.then(results => console.log(results))
.catch(reason => console.log(reason))
Given my many comments on JLRishe's answer, I'll also post my subtly different version of the function:
function any(k, iterable) {
const results = [];
let fullfilled = 0, rejected = 0, n = 0;
return new Promise((resolve, reject) => {
k = Math.max(0, Math.floor(k));
function check() {
if (fulfilled == k)
resolve(results);
else if (rejected + k == n + 1)
reject(new Error(`No ${k} of ${n} got fulfilled`));
}
for (const thenable of iterable) {
const i = n++;
Promise.resolve(thenable).then(res => {
if (fulfilled++ < k) results[i] = res;
check();
}, () => {
rejected++;
check();
});
}
check();
});
}
A bit late but maybe this will do it for you:
const first = howMany => (promises,resolved=[],rejected=[]) => {
if(promises.length-rejected.length<howMany){
return Promise.reject([resolved,rejected]);
}
if(resolved.length===howMany){
return Promise.resolve(resolved);
}
if(resolved.length===0&&rejected.length===0){
promises=promises.map(
(p,index)=>Promise.resolve(p)
.then(resolve=>[resolve,index])
.catch(err=>Promise.reject([err,index]))
);
}
return Promise.race(promises)
.then(
([resolve,index])=>
first(howMany)(
promises.filter(
(p,i)=>i!==index
),
resolved.concat([resolve]),
rejected
)
)
.catch(
([err,index])=>
first(howMany)(
promises.filter(
(p,i)=>i!==index
),
resolved,
rejected.concat([err])
)
);
};
const first2 = first(2);
first2([p1,p2,p3])
.then(
result=>//you will have an array of 2 with resolve value(s)
)
.catch(
err=>//you will have 2 arrays, resolved ones and errors
)
EDIT: Updated to match question edits.
Pretty easy to roll your own when the abstraction limits your expressiveness.
This code is short and can easily be reworked to run in legacy ES3 environments.
You can also easily expand its usefulness by adding code to cancel unresolved timeouts. This is what you get when you don't hand everything over to a library; greater simplicity and more capability.
function raceN(n, fns, resolve, reject) {
const res = [], rej = [];
let halt = false;
for (const [idx, fn] of fns.entries()) {
fn(data => update(res, data, idx), data => update(rej, data, idx));
}
function update(arr, data, idx) {
if (halt) return;
arr.push({idx, data});
if ((halt=res.length >= n)) resolve(res);
else if ((halt=rej.length > fns.length - n)) reject(rej);
}
}
DEMO:
function raceN(n, fns, resolve, reject) {
const res = [], rej = [];
let halt = false;
for (const [idx, fn] of fns.entries()) {
fn(data => update(res, data, idx), data => update(rej, data, idx));
}
function update(arr, data, idx) {
if (halt) return;
arr.push({idx, data});
if ((halt=res.length >= n)) resolve(res);
else if ((halt=rej.length > fns.length - n)) reject(rej);
}
}
function rand() { return Math.ceil(Math.random() * 5000) }
var fns = [(resolve, reject) => {
setTimeout(resolve, rand(), 'one');
},
(resolve, reject) => {
setTimeout(resolve, rand(), 'two');
},
(resolve, reject) => {
setTimeout(reject, rand(), 'three');
}];
raceN(2, fns, values => {
console.log("SUCCESS:", values);
}, reason => {
console.log("REJECT:", reason)
});
If you're guaranteed that at least n elements will succeed, it then becomes even shorter and simpler.
function raceN(n, fns, resolve, reject) {
const res = [];
for (const [idx, fn] of fns.entries()) {
fn(data => res.length < n && res.push({idx, data}) == n) && resolve(res),
data => reject({idx, data}));
}
}
DEMO:
function raceN(n, fns, resolve, reject) {
const res = [];
for (const [idx, fn] of fns.entries()) {
fn(data => res.length < n && res.push({idx, data}) == n) && resolve(res),
data => reject({idx, data}));
}
}
function rand() { return Math.ceil(Math.random() * 4000) }
var fns = [(resolve, reject) => {
setTimeout(resolve, rand(), 'one');
},
(resolve, reject) => {
setTimeout(resolve, rand(), 'two');
},
(resolve, reject) => {
setTimeout(reject, rand(), 'three');
}];
raceN(2, fns, values => {
console.log("SUCCESS:", values);
}, reason => {
console.log("REJECTING:", reason)
});
One other idea could be using Promise.reject() to cut it short when the condition is met while catching possible errors thrown, could be as follows;
var p1 = new Promise((v, x) => setTimeout(v, 1000, 'one')),
p2 = new Promise((v, x) => setTimeout(v, 5000, 'two')),
p3 = new Promise((v, x) => setTimeout(v, 1500, 'three')),
pn = (n, ps, k = 0, r = []) => Promise.all(ps.map(p => p.then(v => (k === n - 1 ? Promise.reject(r.concat(v))
: (++k, r.push(v))))))
.catch(r => Array.isArray(r) ? r : Promise.reject(r));
pn(2,[p1,p2,p3]).then(rs => console.log(rs))
.catch(e => console.log(e));
A more modern approach, in 2022, using iter-ops library:
import {pipeAsync, take, waitRace} from 'iter-ops';
// easy-to-read way to define a job function:
const job = (delay, value) => new Promise(resolve => {
setTimeout(() => resolve(value), delay);
});
// list of jobs:
const input = [
job(1000, 'one'),
job(2000, 'two'),
job(3000, 'three')
];
const i = pipeAsync(
input, // our input
waitRace(10), // race-resolve with cache up to 10 items
take(2) // take only the first 2 items
).catch(err => {
console.log(err); // report any iteration-time error
});
// iterating through our list of jobs:
(async function () {
for await(const a of i) {
console.log(a);
}
})();
This will output one and two. You can twist the delays to see the result change accordingly.
CREDITS
I am the author of the library, while operator waitRace was implemented by #Bergi.
I'm using es6 and have the following promises. What i want is the next Promise.all to wait for previous Promise.all to be completed before execute the next one. I have tried with the below codes but it's not working, only Promise 1 is resolved.
var deletePromises = [];
arr.menuItems.forEach((item, idx) => {
if (item.replace) {
deletePromises.push(deleteFromFirebase(user.uid, item));
}
});
// Promise 1
Promise.all(deletePromises).then(res1 => {
var uploadPromises = [], updateRecordPromises = [];
arr.menuItems.forEach((item, idx) => {
uploadPromises.push(uploadToFirebase(user.uid, item));
});
// Promise 2
Promise.all(uploadPromises).then(res2 => {
arr.menuItems.forEach((item, idx) => {
item.replace = false;
updateRecordPromises.push(updateRecord(user.uid, item));
});
// Promise 3
Promise.all(updateRecordPromises).then(res3 => {
console.log('All promise execute with successfully');
});
});
});
MarkM Answer
Try to use chaining as Mark suggest but the problem still there. I had found where the problem was, it is uploadPromises that never get resolved and then is never get called.
uploadToFirebase function
Stuck here, but the file is successfully uploaded. I can see all the files.
const uploadToFirebase = (userid, item) => {
return new Promise((resolve, reject) => {
const uploadUri = Platform.OS === "ios"
? RNFetchBlob.wrap(item.pic_url.replace("file://", ""))
: RNFetchBlob.wrap(item.pic_url);
Blob.build(uploadUri, {
type: "image/png;"
}).then(blob => {
// upload image using Firebase SDK
firebase
.storage()
.ref("menu_items")
.child(userid)
.child(item.new_filename)
.put(blob, { contentType: "image/png" })
.then(snapshot => {
console.log("Promise resolve: ", snapshot);
resolve(snapshot);
blob.close();
})
.catch(error => {
reject(error.message);
});
});
});
};
Updated code
console.log('Print res2') is not printed
var deletePromises = [],
uploadPromises = [],
updateRecordPromises = [];
arr.menuItems.forEach((item, idx) => {
if (item.replace) {
deletePromises.push(deleteFromFirebase(user.uid, item));
}
});
Promise.all(deletePromises)
.then(res1 => {
console.log("Print res1:", res1);
arr.menuItems.forEach((item, idx) => {
uploadPromises.push(uploadToFirebase(user.uid, item));
});
return Promise.all(uploadPromises);
})
.then(res2 => {
console.log("Print res2:", res2);
dispatch({ type: MENU_UPDATE_SUCCESS, payload: arr });
dispatch(reset("menuItem"));
})
.catch(error => {
console.log("Print error:", error);
});
You don't need to nest the Promises, you can return your new Promise.all from then(), which will let you chain them. Easier with an example:
var arr = [1, 2, 3, 4, 5, 6]
function slowF(i) {
return new Promise((resolve, reject) => {
setTimeout(() => resolve(i), i*200)
})
}
var p = arr.map((i) => slowF(i))
Promise.all(p)
.then((p) => {
console.log("In first Promise.all resolving: ", p)
var newP = p.map(i => slowF(i) )
return Promise.all(newP)
})
.then((p)=>{
console.log("In second Promise.all resolving: ", p)
})
.catch((e) => console.log("error: ", e))
I found the solution. It is because one of menuItems array prop -> new_filename is empty, thats why promise never get resolved. So, i just need to add empty checking for each of item prop and then the promise was resolved properly. Thanks to #MarkM for the answer. Now the code much more cleaner and easier to read compare to the nested promise.
Promise.all(deletePromises)
.then(res1 => {
console.log("Print res1:", res1);
arr.menuItems.forEach((item, idx) => {
if (!isEmpty(item.new_filename)) {
uploadPromises.push(uploadToFirebase(user.uid, item));
}
});
return Promise.all(uploadPromises);
})
.then(res2 => {
console.log("Print res2:", res2);
dispatch({ type: MENU_UPDATE_SUCCESS, payload: arr });
dispatch(reset("menuItem"));
})
.catch(error => {
console.log("Print error:", error);
});
I need to chain promises which are using request promises, so it is kinda chaining nested promises.
Imagine the code:
const rp = require('request-promise');
function doStuff(){
for ( let i = 0; i <= 10; i++ ){
methodA();
}
};
function methodA(){
let options = {...};
rp(options)
.then(result => methodB(result))
.catch(err => console.log(err));
};
function methodB(resultA){
let options = {uri: resultA};
rp(options)
.then(result => methodC(resultA, result))
.catch(err => console.log(err));
};
function methodC(resultA, resultB){
//some calculations
};
In doStuff I need to wait for result of all ten executions of methodC and collect them into array. I have tried to chain it like that:
function doStuff(){
for ( let i = 0; i <= 10; i++ ){
let promiseA = new Promise((resolve, reject) => {
resolve(methodA());
});
let promiseB = promiseA.then(result => methodB(result));
let promiseC = promiseB.then(result => methodC(promiseA.result, result));
Promise.all([promiseA, promiseB, promiseC]);
}
};
But for sure it won't work, because in methodA and methodB we have HTTP requests which are asynchronous. Therefore, result in promiseB is undefined.
It means, the question is: how to chain promises, if they have nested promises? (and how to collect result in the end?)
Thanks!
UPDATE: Chaining promises also not much of the help, as 1 is returned prior array of AB's, but desired result is vice versa:
function methodA(){
let promise = new Promise((resolve, reject) => {
setTimeout(() => {
console.log('Resolved A');
resolve('A');
}, Math.random() * 2000);
});
return promise
.then(result => methodB(result))
.catch(err => console.log(err));
}
function methodB(resultA){
let promise = new Promise((resolve, reject) => {
setTimeout(() => {
console.log('Resolved B');
resolve('B');
}, Math.random() * 2000);
});
return promise
.then(result => methodC(resultA, result))
.catch(err => console.log(err));
}
function methodC(resultA, resultB){
return resultA + resultB;
}
function doStuff() {
let promises = [];
for (let i = 0; i <= 10; i++){
promises.push(methodA());
}
Promise.all(promises).then(results => {
console.log(results);
});
return 1;
}
console.log(doStuff());
Each of your functions needs to return their promise:
function methodA(){
let options = {...};
return rp(options)
.then(result => methodB(result))
.catch(err => console.log(err));
}
function methodB(resultA){
let options = {uri: resultA};
return rp(options)
.then(result => methodC(resultA, result))
.catch(err => console.log(err));
}
function methodC(resultA, resultB){
//some calculations
}
function doStuff() {
let promises = [];
for ( let i = 0; i <= 10; i++ ){
promises.push(methodA());
}
Promise.all(promises).then(...)
}
Edit: I created a test example, which creates promises in methodA and methodB. Each promise lasts some amount of time from 0 to 2 seconds. It seems to be working:
function methodA(){
let promise = new Promise((resolve, reject) => {
setTimeout(() => {
console.log('Resolved A');
resolve('A');
}, Math.random() * 2000);
});
return promise
.then(result => methodB(result))
.catch(err => console.log(err));
}
function methodB(resultA){
let promise = new Promise((resolve, reject) => {
setTimeout(() => {
console.log('Resolved B');
resolve('B');
}, Math.random() * 2000);
});
return promise
.then(result => methodC(resultA, result))
.catch(err => console.log(err));
}
function methodC(resultA, resultB){
return resultA + resultB;
}
function doStuff() {
let promises = [];
for (let i = 0; i <= 10; i++){
promises.push(methodA());
}
return Promise.all(promises).then(results => {
console.log(results);
return 1;
});
}
doStuff().then(result => {
console.log(result);
});
Answer by #Frank_Modica is the way to go.
Just want to add that if you enable async/await you could do it like this. But it does require Babel or Typescript:
async function myMethod() {
const options = { }
try {
const result_a = await rp(options)
const result_b = await rp({ uri: result_a })
const result_c = ...
} catch(err) {
console.log(err)
}
}
for (let i = 0; i <= 10; i++) {
await myMethod();
}
It has to be lightly changed to:
Promise.all([promiseA, promiseB, promiseC]).then([promiseD]);
Also in the functions itself it has to be a return statement to make them chained. For the request itself, just add: {async: false}
Also it can be used:
var runSequence = require('run-sequence');
function methodA () {
return runSequence(
'promiseA',
'promiseB',
['promiseC1', 'promiseC2'],
'promiseD',
callback
);
}
Given
let arr = [1,2,3];
function filter(num) {
return new Promise((res, rej) => {
setTimeout(() => {
if( num === 3 ) {
res(num);
} else {
rej();
}
}, 1);
});
}
function filterNums() {
return Promise.all(arr.filter(filter));
}
filterNums().then(results => {
let l = results.length;
// length should be 1, but is 3
});
The length is 3 because Promises are returned, not values. Is there a way to filter the array with a function that returns a Promise?
Note: For this example, fs.stat has been replaced with setTimeout, see https://github.com/silenceisgolden/learn-esnext/blob/array-filter-async-function/tutorials/array-filter-with-async-function.js for the specific code.
Here is a 2017 elegant solution using async/await :
Very straightforward usage:
const results = await filter(myArray, async num => {
await doAsyncStuff()
return num > 2
})
The helper function (copy this into your web page):
async function filter(arr, callback) {
const fail = Symbol()
return (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i=>i!==fail)
}
Demo:
// Async IIFE
(async function() {
const myArray = [1, 2, 3, 4, 5]
// This is exactly what you'd expect to write
const results = await filter(myArray, async num => {
await doAsyncStuff()
return num > 2
})
console.log(results)
})()
// Arbitrary asynchronous function
function doAsyncStuff() {
return Promise.resolve()
}
// The helper function
async function filter(arr, callback) {
const fail = Symbol()
return (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i=>i!==fail)
}
I'll even throw in a CodePen.
As mentioned in the comments, Array.prototype.filter is synchronous and therefore does not support Promises.
Since you can now (theoretically) subclass built-in types with ES6, you should be able to add your own asynchronous method which wraps the existing filter function:
Note: I've commented out the subclassing, because it's not supported by Babel just yet for Arrays
class AsyncArray /*extends Array*/ {
constructor(arr) {
this.data = arr; // In place of Array subclassing
}
filterAsync(predicate) {
// Take a copy of the array, it might mutate by the time we've finished
const data = Array.from(this.data);
// Transform all the elements into an array of promises using the predicate
// as the promise
return Promise.all(data.map((element, index) => predicate(element, index, data)))
// Use the result of the promises to call the underlying sync filter function
.then(result => {
return data.filter((element, index) => {
return result[index];
});
});
}
}
// Create an instance of your subclass instead
let arr = new AsyncArray([1,2,3,4,5]);
// Pass in your own predicate
arr.filterAsync(async (element) => {
return new Promise(res => {
setTimeout(() => {
res(element > 3);
}, 1);
});
}).then(result => {
console.log(result)
});
Babel REPL Demo
For typescript folk (or es6 just remove type syntax)
function mapAsync<T, U>(array: T[], callbackfn: (value: T, index: number, array: T[]) => Promise<U>): Promise<U[]> {
return Promise.all(array.map(callbackfn));
}
async function filterAsync<T>(array: T[], callbackfn: (value: T, index: number, array: T[]) => Promise<boolean>): Promise<T[]> {
const filterMap = await mapAsync(array, callbackfn);
return array.filter((value, index) => filterMap[index]);
}
es6
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
async function filterAsync(array, callbackfn) {
const filterMap = await mapAsync(array, callbackfn);
return array.filter((value, index) => filterMap[index]);
}
es5
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
function filterAsync(array, callbackfn) {
return mapAsync(array, callbackfn).then(filterMap => {
return array.filter((value, index) => filterMap[index]);
});
}
edit: demo
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
function filterAsync(array, callbackfn) {
return mapAsync(array, callbackfn).then(filterMap => {
return array.filter((value, index) => filterMap[index]);
});
}
var arr = [1, 2, 3, 4];
function isThreeAsync(number) {
return new Promise((res, rej) => {
setTimeout(() => {
res(number === 3);
}, 1);
});
}
mapAsync(arr, isThreeAsync).then(result => {
console.log(result); // [ false, false, true, false ]
});
filterAsync(arr, isThreeAsync).then(result => {
console.log(result); // [ 3 ]
});
Here's a way:
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
var filter = num => wait(1).then(() => num == 3);
var filterAsync = (array, filter) =>
Promise.all(array.map(entry => filter(entry)))
.then(bits => array.filter(entry => bits.shift()));
filterAsync([1,2,3], filter)
.then(results => console.log(results.length))
.catch(e => console.error(e));
The filterAsync function takes an array and a function that must either return true or false or return a promise that resolves to true or false, what you asked for (almost, I didn't overload promise rejection because I think that's a bad idea). Let me know if you have any questions about it.
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
var filter = num => wait(1).then(() => num == 3);
var filterAsync = (array, filter) =>
Promise.all(array.map(entry => filter(entry)))
.then(bits => array.filter(entry => bits.shift()));
filterAsync([1,2,3], filter)
.then(results => console.log(results.length))
.catch(e => console.error(e));
var console = { log: msg => div.innerHTML += msg + "<br>",
error: e => console.log(e +", "+ (e.lineNumber-25)) };
<div id="div"></div>
Promise Reducer to the rescue!
[1, 2, 3, 4].reduce((op, n) => {
return op.then(filteredNs => {
return new Promise(resolve => {
setTimeout(() => {
if (n >= 3) {
console.log("Keeping", n);
resolve(filteredNs.concat(n))
} else {
console.log("Dropping", n);
resolve(filteredNs);
}
}, 1000);
});
});
}, Promise.resolve([]))
.then(filteredNs => console.log(filteredNs));
Reducers are awesome. "Reduce my problem to my goal" seems to be a pretty good strategy for anything more complex than what the simple tools will solve for you, i.e. filtering an array of things that aren't all available immediately.
asyncFilter method:
Array.prototype.asyncFilter = async function(f){
var array = this;
var booleans = await Promise.all(array.map(f));
return array.filter((x,i)=>booleans[i])
}
Late to the game but since no one else mentioned it, Bluebird supports Promise.map which is my go-to for filters requiring aysnc processing for the condition,
function filterAsync(arr) {
return Promise.map(arr, num => {
if (num === 3) return num;
})
.filter(num => num !== undefined)
}
Two lines, completely typesafe
export const asyncFilter = async <T>(list: T[], predicate: (t: T) => Promise<boolean>) => {
const resolvedPredicates = await Promise.all(list.map(predicate));
return list.filter((item, idx) => resolvedPredicates[idx]);
};
In case someone is interested in modern typescript solution (with fail symbol used for filtering):
const failSymbol = Symbol();
export async function filterAsync<T>(
itemsToFilter: T[],
filterFunction: (item: T) => Promise<boolean>,
): Promise<T[]> {
const itemsOrFailFlags = await Promise.all(
itemsToFilter.map(async (item) => {
const hasPassed = await filterFunction(item);
return hasPassed ? item : failSymbol;
}),
);
return itemsOrFailFlags.filter(
(itemOrFailFlag) => itemOrFailFlag !== failSymbol,
) as T[];
}
There is a one liner to to do that.
const filterPromise = (values, fn) =>
Promise.all(values.map(fn)).then(booleans => values.filter((_, i) => booleans[i]));
Pass the array into values and the function into fn.
More description on how this one liner works is available here.
For production purposes you probably want to use a lib like lodasync:
import { filterAsync } from 'lodasync'
const result = await filterAsync(async(element) => {
await doSomething()
return element > 3
}, array)
Under the hood, it maps your array by invoking the callback on each element and filters the array using the result. But you should not reinvent the wheel.
You can do something like this...
theArrayYouWantToFilter = await new Promise(async (resolve) => {
const tempArray = [];
theArrayYouWantToFilter.filter(async (element, index) => {
const someAsyncValue = await someAsyncFunction();
if (someAsyncValue) {
tempArray.push(someAsyncValue);
}
if (index === theArrayYouWantToFilter.length - 1) {
resolve(tempArray);
}
});
});
Wrapped within an async function...
async function filter(theArrayYouWantToFilter) {
theArrayYouWantToFilter = await new Promise(async (resolve) => {
const tempArray = [];
theArrayYouWantToFilter.filter(async (element, index) => {
const someAsyncValue = await someAsyncFunction();
if (someAsyncValue) {
tempArray.push(someAsyncValue);
}
if (index === theArrayYouWantToFilter.length - 1) {
resolve(tempArray);
}
});
});
return theArrayYouWantToFilter;
}
A valid way to do this (but it seems too messy):
let arr = [1,2,3];
function filter(num) {
return new Promise((res, rej) => {
setTimeout(() => {
if( num === 3 ) {
res(num);
} else {
rej();
}
}, 1);
});
}
async function check(num) {
try {
await filter(num);
return true;
} catch(err) {
return false;
}
}
(async function() {
for( let num of arr ) {
let res = await check(num);
if(!res) {
let index = arr.indexOf(num);
arr.splice(index, 1);
}
}
})();
Again, seems way too messy.
A variant of #DanRoss's:
async function filterNums(arr) {
return await arr.reduce(async (res, val) => {
res = await res
if (await filter(val)) {
res.push(val)
}
return res
}, Promise.resolve([]))
}
Note that if (as in current case) you don't have to worry about filter() having
side effects that need to be serialized, you can also do:
async function filterNums(arr) {
return await arr.reduce(async (res, val) => {
if (await filter(val)) {
(await res).push(val)
}
return res
}, Promise.resolve([]))
}
Late to the party, and I know that my answer is similar to other already posted answers, but the function I'm going to share is ready for be dropped into any code and be used.
As usual, when you have to do complex operations on arrays, reduce is king:
const filterAsync = (asyncPred) => arr =>
arr.reduce(async (acc,item) => {
const pass = await asyncPred(item);
if(pass) (await acc).push(item);
return acc;
},[]);
It uses modern syntax so make sure your target supports it. To be 100% correct you should use Promise.resolve([]) as the initial value, but JS just doesn't care and this way it is way shorter.
Then you can use it like this:
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
const isOdd = x => wait(1).then(()=>x%2);
(filterAsync(isOdd)([1,2,3,4,4])).then(console.log) // => [1,3]
Here's a shorter version of #pie6k's Typescript version:
async function filter<T>(arr: T[], callback: (val: T) => Promise<Boolean>) {
const fail = Symbol()
const result = (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i => i !== fail)
return result as T[] // the "fail" entries are all filtered out so this is OK
}
An efficient way of approaching this is by processing arrays as iterables, so you can apply any number of required operations in a single iteration.
The example below uses library iter-ops for that:
import {pipe, filter, toAsync} from 'iter-ops';
const arr = [1, 2, 3]; // synchronous iterable
const i = pipe(
toAsync(arr), // make our iterable asynchronous
filter(async (value, index) => {
// returns Promise<boolean>
})
);
(async function() {
for await (const a of i) {
console.log(a); // print values
}
})();
All operators within the library support asynchronous predicates when inside an asynchronous pipeline (why we use toAsync), and you can add other operators, in the same way.
Use of Promise.all for this is quite inefficient, because you block the entire array from any further processing that can be done concurrently, which the above approach allows.