Related
I have an array of promise objects that must be resolved in the same sequence in which they are listed in the array, i.e. we cannot attempt resolving an element till the previous one has been resolved (as method Promise.all([...]) does).
And if one element is rejected, I need the chain to reject at once, without attempting to resolve the following element.
How can I implement this, or is there an existing implementation for such sequence pattern?
function sequence(arr) {
return new Promise(function (resolve, reject) {
// try resolving all elements in 'arr',
// but strictly one after another;
});
}
EDIT
The initial answers suggest we can only sequence results of such array elements, not their execution, because it is predefined in such example.
But then how to generate an array of promises in such a way as to avoid early execution?
Here's a modified example:
function sequence(nextPromise) {
// while nextPromise() creates and returns another promise,
// continue resolving it;
}
I wouldn't want to make it into a separate question, because I believe it is part of the same problem.
SOLUTION
Some answers below and discussions that followed went a bit astray, but the eventual solution that did exactly what I was looking for was implemented within spex library, as method sequence. The method can iterate through a sequence of dynamic length, and create promises as required by the business logic of your application.
Later on I turned it into a shared library for everyone to use.
Here are some simple examples for how you sequence through an array executing each async operation serially (one after the other).
Let's suppose you have an array of items:
var arr = [...];
And, you want to carry out a specific async operation on each item in the array, one at a time serially such that the next operation does not start until the previous one has finished.
And, let's suppose you have a promise returning function for processing one of the items in the array fn(item):
Manual Iteration
function processItem(item) {
// do async operation and process the result
// return a promise
}
Then, you can do something like this:
function processArray(array, fn) {
var index = 0;
function next() {
if (index < array.length) {
fn(array[index++]).then(next);
}
}
return next();
}
processArray(arr, processItem);
Manual Iteration Returning Promise
If you wanted a promise returned from processArray() so you'd know when it was done, you could add this to it:
function processArray(array, fn) {
var index = 0;
function next() {
if (index < array.length) {
return fn(array[index++]).then(function(value) {
// apply some logic to value
// you have three options here:
// 1) Call next() to continue processing the result of the array
// 2) throw err to stop processing and result in a rejected promise being returned
// 3) return value to stop processing and result in a resolved promise being returned
return next();
});
}
} else {
// return whatever you want to return when all processing is done
// this returne value will be the ersolved value of the returned promise.
return "all done";
}
return next();
}
processArray(arr, processItem).then(function(result) {
// all done here
console.log(result);
}, function(err) {
// rejection happened
console.log(err);
});
Note: this will stop the chain on the first rejection and pass that reason back to the processArray returned promise.
Iteration with .reduce()
If you wanted to do more of the work with promises, you could chain all the promises:
function processArray(array, fn) {
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item);
});
}, Promise.resolve());
}
processArray(arr, processItem).then(function(result) {
// all done here
}, function(reason) {
// rejection happened
});
Note: this will stop the chain on the first rejection and pass that reason back to the promise returned from processArray().
For a success scenario, the promise returned from processArray() will be resolved with the last resolved value of your fn callback. If you wanted to accumulate a list of results and resolve with that, you could collect the results in a closure array from fn and continue to return that array each time so the final resolve would be an array of results.
Iteration with .reduce() that Resolves With Array
And, since it now seems apparent that you want the final promise result to be an array of data (in order), here's a revision of the previous solution that produces that:
function processArray(array, fn) {
var results = [];
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item).then(function(data) {
results.push(data);
return results;
});
});
}, Promise.resolve());
}
processArray(arr, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
Working demo: http://jsfiddle.net/jfriend00/h3zaw8u8/
And a working demo that shows a rejection: http://jsfiddle.net/jfriend00/p0ffbpoc/
Iteration with .reduce() that Resolves With Array with delay
And, if you want to insert a small delay between operations:
function delay(t, v) {
return new Promise(function(resolve) {
setTimeout(resolve.bind(null, v), t);
});
}
function processArrayWithDelay(array, t, fn) {
var results = [];
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item).then(function(data) {
results.push(data);
return delay(t, results);
});
});
}, Promise.resolve());
}
processArray(arr, 200, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
Iteration with Bluebird Promise Library
The Bluebird promise library has a lot of concurrency controlling features built right in. For example, to sequence iteration through an array, you can use Promise.mapSeries().
Promise.mapSeries(arr, function(item) {
// process each individual item here, return a promise
return processItem(item);
}).then(function(results) {
// process final results here
}).catch(function(err) {
// process array here
});
Or to insert a delay between iterations:
Promise.mapSeries(arr, function(item) {
// process each individual item here, return a promise
return processItem(item).delay(100);
}).then(function(results) {
// process final results here
}).catch(function(err) {
// process array here
});
Using ES7 async/await
If you're coding in an environment that supports async/await, you can also just use a regular for loop and then await a promise in the loop and it will cause the for loop to pause until a promise is resolved before proceeding. This will effectively sequence your async operations so the next one doesn't start until the previous one is done.
async function processArray(array, fn) {
let results = [];
for (let i = 0; i < array.length; i++) {
let r = await fn(array[i]);
results.push(r);
}
return results; // will be resolved value of promise
}
// sample usage
processArray(arr, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
FYI, I think my processArray() function here is very similar to Promise.map() in the Bluebird promise library which takes an array and a promise producing function and returns a promise that resolves with an array of resolved results.
#vitaly-t - Here some some more detailed comments on your approach. You are welcome to whatever code seems best to you. When I first started using promises, I tended to use promises only for the simplest things they did and write a lot of the logic myself when a more advanced use of promises could do much more of it for me. You use only what you are fully comfortable with and beyond that, you'd rather see your own code that you intimately know. That's probably human nature.
I will suggest that as I understood more and more of what promises can do for me, I now like to write code that uses more of the advanced features of promises and it seems perfectly natural to me and I feel like I'm building on well tested infrastructure that has lots of useful features. I'd only ask that you keep your mind open as you learn more and more to potentially go that direction. It's my opinion that it's a useful and productive direction to migrate as your understanding improves.
Here are some specific points of feedback on your approach:
You create promises in seven places
As a contrast in styles, my code has only two places where I explicitly create a new promise - once in the factory function and once to initialize the .reduce() loop. Everywhere else, I'm just building on the promises already created by chaining to them or returning values within them or just returning them directly. Your code has seven unique places where you're creating a promise. Now, good coding isn't a contest to see how few places you can create a promise, but that might point out the difference in leverage the promises that are already created versus testing conditions and creating new promises.
Throw-safety is a very useful feature
Promises are throw-safe. That means that an exception thrown within a promise handler will automatically reject that promise. If you just want the exception to become a rejection, then this is a very useful feature to take advantage of. In fact, you will find that just throwing yourself is a useful way to reject from within a handler without creating yet another promise.
Lots of Promise.resolve() or Promise.reject() is probably an opportunity for simplification
If you see code with lots of Promise.resolve() or Promise.reject() statements, then there are probably opportunities to leverage the existing promises better rather than creating all these new promises.
Cast to a Promise
If you don't know if something returned a promise, then you can cast it to a promise. The promise library will then do it's own checks whether it is a promise or not and even whether it's the kind of promise that matches the promise library you're using and, if not, wrap it into one. This can save rewriting a lot of this logic yourself.
Contract to Return a Promise
In many cases these days, it's completely viable to have a contract for a function that may do something asynchronous to return a promise. If the function just wants to do something synchronous, then it can just return a resolved promise. You seem to feel like this is onerous, but it's definitely the way the wind is blowing and I already write lots of code that requires that and it feels very natural once you get familiar with promises. It abstracts away whether the operation is sync or async and the caller doesn't have to know or do anything special either way. This is a nice use of promises.
The factory function can be written to create one promise only
The factory function can be written to create one promise only and then resolve or reject it. This style also makes it throw safe so any exception occuring in the factory function automatically becomes a reject. It also makes the contract to always return a promise automatic.
While I realize this factory function is a placeholder function (it doesn't even do anything async), hopefully you can see the style to consider it:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve("one");
break;
case 1:
resolve("two");
break;
case 2:
resolve("three");
break;
default:
resolve(null);
break;
}
});
}
If any of these operations were async, then they could just return their own promises which would automatically chain to the one central promise like this:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve($.ajax(...));
case 1:
resole($.ajax(...));
case 2:
resolve("two");
break;
default:
resolve(null);
break;
}
});
}
Using a reject handler to just return promise.reject(reason) is not needed
When you have this body of code:
return obj.then(function (data) {
result.push(data);
return loop(++idx, result);
}, function (reason) {
return promise.reject(reason);
});
The reject handler is not adding any value. You can instead just do this:
return obj.then(function (data) {
result.push(data);
return loop(++idx, result);
});
You are already returning the result of obj.then(). If either obj rejects or if anything chained to obj or returned from then .then() handler rejects, then obj will reject. So you don't need to create a new promise with the reject. The simpler code without the reject handler does the same thing with less code.
Here's a version in the general architecture of your code that tries to incorporate most of these ideas:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve("zero");
break;
case 1:
resolve("one");
break;
case 2:
resolve("two");
break;
default:
// stop further processing
resolve(null);
break;
}
});
}
// Sequentially resolves dynamic promises returned by a factory;
function sequence(factory) {
function loop(idx, result) {
return Promise.resolve(factory(idx)).then(function(val) {
// if resolved value is not null, then store result and keep going
if (val !== null) {
result.push(val);
// return promise from next call to loop() which will automatically chain
return loop(++idx, result);
} else {
// if we got null, then we're done so return results
return result;
}
});
}
return loop(0, []);
}
sequence(factory).then(function(results) {
log("results: ", results);
}, function(reason) {
log("rejected: ", reason);
});
Working demo: http://jsfiddle.net/jfriend00/h3zaw8u8/
Some comments about this implementation:
Promise.resolve(factory(idx)) essentially casts the result of factory(idx) to a promise. If it was just a value, then it becomes a resolved promise with that return value as the resolve value. If it was already a promise, then it just chains to that promise. So, it replaces all your type checking code on the return value of the factory() function.
The factory function signals that it is done by returning either null or a promise whose resolved value ends up being null. The above cast maps those two conditions to the same resulting code.
The factory function catches exceptions automatically and turns them into rejects which are then handled automatically by the sequence() function. This is one significant advantage of letting promises do a lot of your error handling if you just want to abort processing and feed the error back on the first exception or rejection.
The factory function in this implementation can return either a promise or a static value (for a synchronous operation) and it will work just fine (per your design request).
I've tested it with a thrown exception in the promise callback in the factory function and it does indeed just reject and propagate that exception back to reject the sequence promise with the exception as the reason.
This uses a similar method as you (on purpose, trying to stay with your general architecture) for chaining multiple calls to loop().
Promises represent values of operations and not the operations themselves. The operations are already started so you can't make them wait for one another.
Instead, you can synchronize functions that return promises invoking them in order (through a loop with promise chaining for instance), or using the .each method in bluebird.
You can't simply run X async operations and then want them to be resolved in an order.
The correct way to do something like this is to run the new async operation only after the one before was resolved:
doSomethingAsync().then(function(){
doSomethingAsync2().then(function(){
doSomethingAsync3();
.......
});
});
Edit Seems like you want to wait for all promises and then invoke their callbacks in a specific order. Something like this:
var callbackArr = [];
var promiseArr = [];
promiseArr.push(doSomethingAsync());
callbackArr.push(doSomethingAsyncCallback);
promiseArr.push(doSomethingAsync1());
callbackArr.push(doSomethingAsync1Callback);
.........
promiseArr.push(doSomethingAsyncN());
callbackArr.push(doSomethingAsyncNCallback);
and then:
$.when(promiseArr).done(function(promise){
while(callbackArr.length > 0)
{
callbackArr.pop()(promise);
}
});
The problems that can occur with this is when one or more promises fail.
Although quite dense, here's another solution that will iterate a promise-returning function over an array of values and resolve with an array of results:
function processArray(arr, fn) {
return arr.reduce(
(p, v) => p.then((a) => fn(v).then(r => a.concat([r]))),
Promise.resolve([])
);
}
Usage:
const numbers = [0, 4, 20, 100];
const multiplyBy3 = (x) => new Promise(res => res(x * 3));
// Prints [ 0, 12, 60, 300 ]
processArray(numbers, multiplyBy3).then(console.log);
Note that, because we're reducing from one promise to the next, each item is processed in series.
It's functionally equivalent to the "Iteration with .reduce() that Resolves With Array" solution from #jfriend00 but a bit neater.
I suppose two approaches for handling this question:
Create multiple promises and use the allWithAsync function as follow:
let allPromiseAsync = (...PromisesList) => {
return new Promise(async resolve => {
let output = []
for (let promise of PromisesList) {
output.push(await promise.then(async resolvedData => await resolvedData))
if (output.length === PromisesList.length) resolve(output)
}
}) }
const prm1= Promise.resolve('first');
const prm2= new Promise((resolve, reject) => setTimeout(resolve, 2000, 'second'));
const prm3= Promise.resolve('third');
allPromiseAsync(prm1, prm2, prm3)
.then(resolvedData => {
console.log(resolvedData) // ['first', 'second', 'third']
});
Use the Promise.all function instead:
(async () => {
const promise1 = new Promise(resolve => {
setTimeout(() => { console.log('first');console.log(new Date());resolve() }, 1000)
})
const promise2 = new Promise(resolve => {
setTimeout(() => {console.log('second');console.log(new Date()); resolve() }, 3000)
})
const promise3 = new Promise(resolve => {
setTimeout(() => { console.log('third');console.log(new Date()); resolve() }, 7000)
})
const promises = [promise1, promise2, promise3]
await Promise.all(promises)
console.log('This line is shown after 7000ms')
})()
In my opinion, you should be using a for loop(yes the only time I would recommend a for loop). The reason is that when you are using a for loop it allows you to await on each of the iterations of your loop where using reduce, map or forEach with run all your promise iterations concurrently. Which by the sounds of it is not what you want, you want each promise to wait until the previous promise has resolved. So to do this you would do something like the following.
const ids = [0, 1, 2]
const accounts = ids.map(id => getId(id))
const accountData = async() => {
for await (const account of accounts) {
// account will equal the current iteration of the loop
// and each promise are now waiting on the previous promise to resolve!
}
}
// then invoke your function where ever needed
accountData()
And obviously, if you wanted to get really extreme you could do something like this:
const accountData = async(accounts) => {
for await (const account of accounts) {
// do something
}
}
accountData([0, 1, 2].map(id => getId(id)))
This is so much more readable than any of the other examples, it is much less code, reduced the number of lines needed for this functionality, follows a more functional programming way of doing things and is using ES7 to its full potential!!!!
Also depending on your set up or when you are reading this you may need to add the plugin-proposal-async-generator-functions polyfill or you may see the following error
#babel/plugin-proposal-async-generator-functions (https://git.io/vb4yp) to the 'plugins' section of your Babel config to enable transformation.
I have many asynchronous methods to execute and my program flows can change a lot depending on each method return. The logic below is one example. I could not write it in a easy-to-read way using Promises. How would you write it?
Ps: more complex flows are welcome.
Ps2: is_business is a predefined flag where we say whether we are writing a "business user" or a "person user".
begin transaction
update users
if updated
if is_business
update_business
if not updated
insert business
end if
else
delete business
end if
else
if upsert
insert user
if is_business
insert business
end if
end if
end if
commit transaction
The nice thing about promises is that they make a simple analogy between synchronous code and asynchronous code. To illustrate (using the Q library):
Synchronous:
var thisReturnsAValue = function() {
var result = mySynchronousFunction();
if(result) {
return getOneValue();
} else {
return getAnotherValue();
}
};
try {
var value = thisReturnsAValue();
console.log(value);
} catch(err) {
console.error(err);
}
Asynchronous:
var Q = require('q');
var thisReturnsAPromiseForAValue = function() {
return Q.Promise(function() {
return myAsynchronousFunction().then(function(result) {
if(result) {
// Even getOneValue() would work here, because a non-promise
// value is automatically cast to a pre-resolved promise
return getOneValueAsynchronously();
} else {
return getAnotherValueAsynchronously();
}
});
});
};
thisReturnsAPromiseForAValue().then(function(value) {
console.log(value);
}, function(err) {
console.error(err);
});
You just need to get used to the idea that return values are always accessed as arguments to then-callbacks, and that chaining promises equates to composing function calls (f(g(h(x)))) or otherwise executing functions in sequence (var x2 = h(x); var x3 = g(x2);). That's essentially it! Things get a little tricky when you introduce branches, but you can figure out what to do from these first principles. Because then-callbacks accept promises as return values, you can mutate a value you got asynchronously by returning another promise for an asynchronous operation which resolves to a new value based on the old one, and the parent promise will not resolve until the new one resolves! And, of course, you can return these promises from within if-else branches.
The other really nice thing illustrated in the example above is that promises (at least ones that are compliant with Promises/A+) handle exceptions in an equally analogous way. The first error "raised" bypasses the non-error callbacks and bubbles up to the first available error callback, much like a try-catch block.
For what it's worth, I think trying to mimic this behavior using hand-crafted Node.js-style callbacks and the async library is its own special kind of hell :).
Following these guidelines your code would become (assuming all functions are async and return promises):
beginTransaction().then(function() {
// beginTransaction() has run
return updateUsers(); // resolves the boolean value `updated`
}).then(function(updated) {
// updateUsers() has "returned" `updated`
if(updated) {
if(isBusiness) {
return updateBusiness().then(function(updated) {
if(!updated) {
return insertBusiness();
}
// It's okay if we don't return anything -- it will
// result in a promise which immediately resolves to
// `undefined`, which is a no-op, just like a missing
// else-branch
});
} else {
return deleteBusiness();
}
} else {
if(upsert) {
return insertUser().then(function() {
if(isBusiness) {
return insertBusiness();
}
});
}
}
}).then(function() {
return commitTransaction();
}).done(function() {
console.log('all done!');
}, function(err) {
console.error(err);
});
The solution is a mix of #mooiamaduck answer and #Kevin comment.
Using promises, ES6 generators and co library makes the code much clearer. I found a good example when reading a postgresql node library example (pg). In the example below pool.connect and client.query are asynchronous operations that returns Promises. We can easily add an if/else after geting result and then make more async operations keeping code looking like synchronous.
co(function * () {
var client = yield pool.connect()
try {
yield client.query('BEGIN')
var result = yield client.query('SELECT $1::text as name', ['foo'])
yield client.query('INSERT INTO something(name) VALUES($1)', [result.rows[0].name])
yield client.query('COMMIT')
client.release()
} catch(e) {
// pass truthy value to release to destroy the client
// instead of returning it to the pool
// the pool will create a new client next time
// this will also roll back the transaction within postgres
client.release(true)
}
})
This question already has answers here:
Why does the Promise constructor need an executor?
(2 answers)
Closed 6 years ago.
I wish to return a Promise which is self-resolved at a later time, but it seems that my syntax is invalid, and I'm curious what a better implementation would be. Why is an executor required for the Promise constructor?
promise = new Promise() is invalid because I need to supply a function
function getDetails (someHash) {
var url = "http://somewebsite.com/" + someHash,
promise = new Promise();
makeAjaxRequest(url, function (response) {
promise.resolve(response.data);
});
setTimeout(function () {
promise.reject("timeout");
}, 500);
return promise;
}
function makeAjaxRequest (url, callback) {
var someResponse = {data: "some data"};
setTimeout(function () {
callback(someResponse);
}, 200);
}
Is there a better way to implement this functionality?
Note: If you want to convert a callback API to promises see this question.
Let's start with something that should be said from the get go. An executor is not required in order to design promises. It is entirely possible to design a promises implementation that does something like:
let {promise, resolve, reject} = Promise.get();
If you promise not to tell anyone, I'll even let you in on a little secret - this API even exists from a previous iteration of the spec and in fact even still works in Chrome:
let {promise, resolve, reject} = Promise.defer();
However, it is being removed.
So, why do I need to pass an executor?
I just answered your other question about why an executor is an interesting design. It's throw-safe and it lets you take care of interesting things.
Can I still get the resolve, reject functions? I need those
In my experience, most of the times I needed resolve/reject I didn't actually need them. That said - I did in fact actually need them a couple of times.
The specifiers recognized this and for this reason the executor function is always run synchronously. You can get the .defer interface it just isn't the default:
function defer() {
let resolve, reject, promise = new Promise((res, rej) => {
[resolve, reject] = [res, rej];
});
return {resolve, reject, promise};
}
Again, this is typically not something you want to do but it is entirely possible that you have a justifiable use case which is why it's not the default but completely possible.
Your actual code
I would start with implementing things like timeout and a request as primitives and then compose functions and chain promises:
function delay(ms) {
return new Promise(r => setTimeout(r, ms));
}
function timeout(promise, ms) {
return Promise.race([
promise,
delay(ms).then(x => { throw new Error("timeout"); })
]);
}
function ajax(url) { // note browsers do this natively with `fetch` today
return new Promise((resolve, reject) => { // handle errors!
makeAjaxRequest(url, (result) => {
// if(result.isFailure) reject(...);
if(result.status >= 400) reject(new Error(result.status));
else resolve(result.data);
});
});
}
Now when we promisified the lowest level API surface we can write the above code quite declaratively:
function getDetails (someHash) {
var ajax = makeAjaxRequest("http://somewebsite.com/" + someHash);
return timeout(ajax, 500);
}
You need to pass a function to the Promise constructor (more info), which will get called to provide the resolve and reject functions:
function getDetails (someHash) {
var url = "http://somewebsite.com/" + someHash;
return new Promise(function(resolve, reject) {
makeAjaxRequest(url, function(err, response) {
if (err)
reject(err);
else
resolve(response.data);
});
setTimeout(function () {
reject("timeout");
}, 500);
});
}
function makeAjaxRequest (url, callback) {
var someResponse = {data: "some data"};
setTimeout(function () {
callback(null, someResponse);
}, 200);
}
I've also taken the liberty to make makeAjaxRequest use the standard convention of passing errors as first argument (because I assume that at some point you want to replace the setTimeout() with an actual AJAX request).
I have an array of promise objects that must be resolved in the same sequence in which they are listed in the array, i.e. we cannot attempt resolving an element till the previous one has been resolved (as method Promise.all([...]) does).
And if one element is rejected, I need the chain to reject at once, without attempting to resolve the following element.
How can I implement this, or is there an existing implementation for such sequence pattern?
function sequence(arr) {
return new Promise(function (resolve, reject) {
// try resolving all elements in 'arr',
// but strictly one after another;
});
}
EDIT
The initial answers suggest we can only sequence results of such array elements, not their execution, because it is predefined in such example.
But then how to generate an array of promises in such a way as to avoid early execution?
Here's a modified example:
function sequence(nextPromise) {
// while nextPromise() creates and returns another promise,
// continue resolving it;
}
I wouldn't want to make it into a separate question, because I believe it is part of the same problem.
SOLUTION
Some answers below and discussions that followed went a bit astray, but the eventual solution that did exactly what I was looking for was implemented within spex library, as method sequence. The method can iterate through a sequence of dynamic length, and create promises as required by the business logic of your application.
Later on I turned it into a shared library for everyone to use.
Here are some simple examples for how you sequence through an array executing each async operation serially (one after the other).
Let's suppose you have an array of items:
var arr = [...];
And, you want to carry out a specific async operation on each item in the array, one at a time serially such that the next operation does not start until the previous one has finished.
And, let's suppose you have a promise returning function for processing one of the items in the array fn(item):
Manual Iteration
function processItem(item) {
// do async operation and process the result
// return a promise
}
Then, you can do something like this:
function processArray(array, fn) {
var index = 0;
function next() {
if (index < array.length) {
fn(array[index++]).then(next);
}
}
return next();
}
processArray(arr, processItem);
Manual Iteration Returning Promise
If you wanted a promise returned from processArray() so you'd know when it was done, you could add this to it:
function processArray(array, fn) {
var index = 0;
function next() {
if (index < array.length) {
return fn(array[index++]).then(function(value) {
// apply some logic to value
// you have three options here:
// 1) Call next() to continue processing the result of the array
// 2) throw err to stop processing and result in a rejected promise being returned
// 3) return value to stop processing and result in a resolved promise being returned
return next();
});
}
} else {
// return whatever you want to return when all processing is done
// this returne value will be the ersolved value of the returned promise.
return "all done";
}
return next();
}
processArray(arr, processItem).then(function(result) {
// all done here
console.log(result);
}, function(err) {
// rejection happened
console.log(err);
});
Note: this will stop the chain on the first rejection and pass that reason back to the processArray returned promise.
Iteration with .reduce()
If you wanted to do more of the work with promises, you could chain all the promises:
function processArray(array, fn) {
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item);
});
}, Promise.resolve());
}
processArray(arr, processItem).then(function(result) {
// all done here
}, function(reason) {
// rejection happened
});
Note: this will stop the chain on the first rejection and pass that reason back to the promise returned from processArray().
For a success scenario, the promise returned from processArray() will be resolved with the last resolved value of your fn callback. If you wanted to accumulate a list of results and resolve with that, you could collect the results in a closure array from fn and continue to return that array each time so the final resolve would be an array of results.
Iteration with .reduce() that Resolves With Array
And, since it now seems apparent that you want the final promise result to be an array of data (in order), here's a revision of the previous solution that produces that:
function processArray(array, fn) {
var results = [];
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item).then(function(data) {
results.push(data);
return results;
});
});
}, Promise.resolve());
}
processArray(arr, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
Working demo: http://jsfiddle.net/jfriend00/h3zaw8u8/
And a working demo that shows a rejection: http://jsfiddle.net/jfriend00/p0ffbpoc/
Iteration with .reduce() that Resolves With Array with delay
And, if you want to insert a small delay between operations:
function delay(t, v) {
return new Promise(function(resolve) {
setTimeout(resolve.bind(null, v), t);
});
}
function processArrayWithDelay(array, t, fn) {
var results = [];
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item).then(function(data) {
results.push(data);
return delay(t, results);
});
});
}, Promise.resolve());
}
processArray(arr, 200, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
Iteration with Bluebird Promise Library
The Bluebird promise library has a lot of concurrency controlling features built right in. For example, to sequence iteration through an array, you can use Promise.mapSeries().
Promise.mapSeries(arr, function(item) {
// process each individual item here, return a promise
return processItem(item);
}).then(function(results) {
// process final results here
}).catch(function(err) {
// process array here
});
Or to insert a delay between iterations:
Promise.mapSeries(arr, function(item) {
// process each individual item here, return a promise
return processItem(item).delay(100);
}).then(function(results) {
// process final results here
}).catch(function(err) {
// process array here
});
Using ES7 async/await
If you're coding in an environment that supports async/await, you can also just use a regular for loop and then await a promise in the loop and it will cause the for loop to pause until a promise is resolved before proceeding. This will effectively sequence your async operations so the next one doesn't start until the previous one is done.
async function processArray(array, fn) {
let results = [];
for (let i = 0; i < array.length; i++) {
let r = await fn(array[i]);
results.push(r);
}
return results; // will be resolved value of promise
}
// sample usage
processArray(arr, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
FYI, I think my processArray() function here is very similar to Promise.map() in the Bluebird promise library which takes an array and a promise producing function and returns a promise that resolves with an array of resolved results.
#vitaly-t - Here some some more detailed comments on your approach. You are welcome to whatever code seems best to you. When I first started using promises, I tended to use promises only for the simplest things they did and write a lot of the logic myself when a more advanced use of promises could do much more of it for me. You use only what you are fully comfortable with and beyond that, you'd rather see your own code that you intimately know. That's probably human nature.
I will suggest that as I understood more and more of what promises can do for me, I now like to write code that uses more of the advanced features of promises and it seems perfectly natural to me and I feel like I'm building on well tested infrastructure that has lots of useful features. I'd only ask that you keep your mind open as you learn more and more to potentially go that direction. It's my opinion that it's a useful and productive direction to migrate as your understanding improves.
Here are some specific points of feedback on your approach:
You create promises in seven places
As a contrast in styles, my code has only two places where I explicitly create a new promise - once in the factory function and once to initialize the .reduce() loop. Everywhere else, I'm just building on the promises already created by chaining to them or returning values within them or just returning them directly. Your code has seven unique places where you're creating a promise. Now, good coding isn't a contest to see how few places you can create a promise, but that might point out the difference in leverage the promises that are already created versus testing conditions and creating new promises.
Throw-safety is a very useful feature
Promises are throw-safe. That means that an exception thrown within a promise handler will automatically reject that promise. If you just want the exception to become a rejection, then this is a very useful feature to take advantage of. In fact, you will find that just throwing yourself is a useful way to reject from within a handler without creating yet another promise.
Lots of Promise.resolve() or Promise.reject() is probably an opportunity for simplification
If you see code with lots of Promise.resolve() or Promise.reject() statements, then there are probably opportunities to leverage the existing promises better rather than creating all these new promises.
Cast to a Promise
If you don't know if something returned a promise, then you can cast it to a promise. The promise library will then do it's own checks whether it is a promise or not and even whether it's the kind of promise that matches the promise library you're using and, if not, wrap it into one. This can save rewriting a lot of this logic yourself.
Contract to Return a Promise
In many cases these days, it's completely viable to have a contract for a function that may do something asynchronous to return a promise. If the function just wants to do something synchronous, then it can just return a resolved promise. You seem to feel like this is onerous, but it's definitely the way the wind is blowing and I already write lots of code that requires that and it feels very natural once you get familiar with promises. It abstracts away whether the operation is sync or async and the caller doesn't have to know or do anything special either way. This is a nice use of promises.
The factory function can be written to create one promise only
The factory function can be written to create one promise only and then resolve or reject it. This style also makes it throw safe so any exception occuring in the factory function automatically becomes a reject. It also makes the contract to always return a promise automatic.
While I realize this factory function is a placeholder function (it doesn't even do anything async), hopefully you can see the style to consider it:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve("one");
break;
case 1:
resolve("two");
break;
case 2:
resolve("three");
break;
default:
resolve(null);
break;
}
});
}
If any of these operations were async, then they could just return their own promises which would automatically chain to the one central promise like this:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve($.ajax(...));
case 1:
resole($.ajax(...));
case 2:
resolve("two");
break;
default:
resolve(null);
break;
}
});
}
Using a reject handler to just return promise.reject(reason) is not needed
When you have this body of code:
return obj.then(function (data) {
result.push(data);
return loop(++idx, result);
}, function (reason) {
return promise.reject(reason);
});
The reject handler is not adding any value. You can instead just do this:
return obj.then(function (data) {
result.push(data);
return loop(++idx, result);
});
You are already returning the result of obj.then(). If either obj rejects or if anything chained to obj or returned from then .then() handler rejects, then obj will reject. So you don't need to create a new promise with the reject. The simpler code without the reject handler does the same thing with less code.
Here's a version in the general architecture of your code that tries to incorporate most of these ideas:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve("zero");
break;
case 1:
resolve("one");
break;
case 2:
resolve("two");
break;
default:
// stop further processing
resolve(null);
break;
}
});
}
// Sequentially resolves dynamic promises returned by a factory;
function sequence(factory) {
function loop(idx, result) {
return Promise.resolve(factory(idx)).then(function(val) {
// if resolved value is not null, then store result and keep going
if (val !== null) {
result.push(val);
// return promise from next call to loop() which will automatically chain
return loop(++idx, result);
} else {
// if we got null, then we're done so return results
return result;
}
});
}
return loop(0, []);
}
sequence(factory).then(function(results) {
log("results: ", results);
}, function(reason) {
log("rejected: ", reason);
});
Working demo: http://jsfiddle.net/jfriend00/h3zaw8u8/
Some comments about this implementation:
Promise.resolve(factory(idx)) essentially casts the result of factory(idx) to a promise. If it was just a value, then it becomes a resolved promise with that return value as the resolve value. If it was already a promise, then it just chains to that promise. So, it replaces all your type checking code on the return value of the factory() function.
The factory function signals that it is done by returning either null or a promise whose resolved value ends up being null. The above cast maps those two conditions to the same resulting code.
The factory function catches exceptions automatically and turns them into rejects which are then handled automatically by the sequence() function. This is one significant advantage of letting promises do a lot of your error handling if you just want to abort processing and feed the error back on the first exception or rejection.
The factory function in this implementation can return either a promise or a static value (for a synchronous operation) and it will work just fine (per your design request).
I've tested it with a thrown exception in the promise callback in the factory function and it does indeed just reject and propagate that exception back to reject the sequence promise with the exception as the reason.
This uses a similar method as you (on purpose, trying to stay with your general architecture) for chaining multiple calls to loop().
Promises represent values of operations and not the operations themselves. The operations are already started so you can't make them wait for one another.
Instead, you can synchronize functions that return promises invoking them in order (through a loop with promise chaining for instance), or using the .each method in bluebird.
You can't simply run X async operations and then want them to be resolved in an order.
The correct way to do something like this is to run the new async operation only after the one before was resolved:
doSomethingAsync().then(function(){
doSomethingAsync2().then(function(){
doSomethingAsync3();
.......
});
});
Edit Seems like you want to wait for all promises and then invoke their callbacks in a specific order. Something like this:
var callbackArr = [];
var promiseArr = [];
promiseArr.push(doSomethingAsync());
callbackArr.push(doSomethingAsyncCallback);
promiseArr.push(doSomethingAsync1());
callbackArr.push(doSomethingAsync1Callback);
.........
promiseArr.push(doSomethingAsyncN());
callbackArr.push(doSomethingAsyncNCallback);
and then:
$.when(promiseArr).done(function(promise){
while(callbackArr.length > 0)
{
callbackArr.pop()(promise);
}
});
The problems that can occur with this is when one or more promises fail.
Although quite dense, here's another solution that will iterate a promise-returning function over an array of values and resolve with an array of results:
function processArray(arr, fn) {
return arr.reduce(
(p, v) => p.then((a) => fn(v).then(r => a.concat([r]))),
Promise.resolve([])
);
}
Usage:
const numbers = [0, 4, 20, 100];
const multiplyBy3 = (x) => new Promise(res => res(x * 3));
// Prints [ 0, 12, 60, 300 ]
processArray(numbers, multiplyBy3).then(console.log);
Note that, because we're reducing from one promise to the next, each item is processed in series.
It's functionally equivalent to the "Iteration with .reduce() that Resolves With Array" solution from #jfriend00 but a bit neater.
I suppose two approaches for handling this question:
Create multiple promises and use the allWithAsync function as follow:
let allPromiseAsync = (...PromisesList) => {
return new Promise(async resolve => {
let output = []
for (let promise of PromisesList) {
output.push(await promise.then(async resolvedData => await resolvedData))
if (output.length === PromisesList.length) resolve(output)
}
}) }
const prm1= Promise.resolve('first');
const prm2= new Promise((resolve, reject) => setTimeout(resolve, 2000, 'second'));
const prm3= Promise.resolve('third');
allPromiseAsync(prm1, prm2, prm3)
.then(resolvedData => {
console.log(resolvedData) // ['first', 'second', 'third']
});
Use the Promise.all function instead:
(async () => {
const promise1 = new Promise(resolve => {
setTimeout(() => { console.log('first');console.log(new Date());resolve() }, 1000)
})
const promise2 = new Promise(resolve => {
setTimeout(() => {console.log('second');console.log(new Date()); resolve() }, 3000)
})
const promise3 = new Promise(resolve => {
setTimeout(() => { console.log('third');console.log(new Date()); resolve() }, 7000)
})
const promises = [promise1, promise2, promise3]
await Promise.all(promises)
console.log('This line is shown after 7000ms')
})()
In my opinion, you should be using a for loop(yes the only time I would recommend a for loop). The reason is that when you are using a for loop it allows you to await on each of the iterations of your loop where using reduce, map or forEach with run all your promise iterations concurrently. Which by the sounds of it is not what you want, you want each promise to wait until the previous promise has resolved. So to do this you would do something like the following.
const ids = [0, 1, 2]
const accounts = ids.map(id => getId(id))
const accountData = async() => {
for await (const account of accounts) {
// account will equal the current iteration of the loop
// and each promise are now waiting on the previous promise to resolve!
}
}
// then invoke your function where ever needed
accountData()
And obviously, if you wanted to get really extreme you could do something like this:
const accountData = async(accounts) => {
for await (const account of accounts) {
// do something
}
}
accountData([0, 1, 2].map(id => getId(id)))
This is so much more readable than any of the other examples, it is much less code, reduced the number of lines needed for this functionality, follows a more functional programming way of doing things and is using ES7 to its full potential!!!!
Also depending on your set up or when you are reading this you may need to add the plugin-proposal-async-generator-functions polyfill or you may see the following error
#babel/plugin-proposal-async-generator-functions (https://git.io/vb4yp) to the 'plugins' section of your Babel config to enable transformation.
I have used kriskowal's Q library for a project (web scraper / human-activity simulator) and have become acquainted with promises, returning them and resolving/rejecting them, and the library's basic asynchronous control flow methods and error-throwing/catching mechanisms have proven essential.
I have encountered some issues though. My promise.then calls and my callbacks have the uncanny tendency to form pyramids. Sometimes it's for scoping reasons, other times it's to guarantee a certain order of events. (I suppose I might be able to fix some of these problems by refactoring, but going forward I want to avoid "callback hell" altogether.)
Also, debugging is very frustrating. I spend a lot of time console.log-ing my way to the source of errors and bugs; after I finally find them I will start throwing errors there and catching them somewhere else with promise.finally, but the process of locating the errors in the first place is arduous.
Also, in my project, order matters. I need to do pretty much everything sequentially. Oftentimes I find myself generating arrays of functions that return promises and then chaining them to each other using Array.prototype.reduce, which I don't think I should have to do.
Here is an example of one of my methods that uses this reduction technique:
removeItem: function (itemId) {
var removeRegexp = new RegExp('\\/stock\\.php\\?remove=' + itemId);
return this.getPage('/stock.php')
.then(function (webpage) {
var
pageCount = 5,
promiseFunctions = [],
promiseSequence;
// Create an array of promise-yielding functions that can run sequentially.
_.times(pageCount, function (i) {
var promiseFunction = function () {
var
promise,
path;
if (i === 0) {
promise = Q(webpage);
} else {
path = '/stock.php?p=' + i;
promise = this.getPage(path);
}
return promise.then(function (webpage) {
var
removeMatch = webpage.match(removeRegexp),
removePath;
if (removeMatch !== null) {
removePath = removeitemMatch[0];
return this.getPage(removePath)
.delay(1000)
// Stop calling subsequent promises.
.thenResolve(true);
}
// Don't stop calling subsequent promises.
return false;
}.bind(this));
}.bind(this);
promiseFunctions.push(promiseFunction);
}, this);
// Resolve the promises sequentially but stop early if the item is found.
promiseSequence = promiseFunctions.reduce(function (soFar, promiseFunction, index) {
return soFar.then(function (stop) {
if (stop) {
return true;
} else {
return Q.delay(1000).then(promiseFunction);
}
});
}, Q());
return promiseSequence;
}.bind(this))
.fail(function (onRejected) {
console.log(onRejected);
});
},
I have other methods that do basically the same thing but which are suffering from much worse indentation woes.
I'm considering refactoring my project using coalan's async library. It seems similar to Q, but I want to know exactly how they differ. The impression I am getting is that async more "callback-centric" while Q is "promise-centric".
Question: Given my problems and project requirements, what would I gain and/or lose by using async over Q? How do the libraries compare? (Particularly in terms of executing series of tasks sequentially and debugging/error-handling?)
Both libraries are good. I have discovered that they serve separate purposes and can be used in tandem.
Q provides the developer with promise objects, which are future representations of values. Useful for time travelling.
Async provides the developer with asynchronous versions of control structures and aggregate operations.
An example from one attempt at a linter implementation demonstrates a potential unity among libraries:
function lint(files, callback) {
// Function which returns a promise.
var getMerged = merger('.jslintrc'),
// Result objects to invoke callback with.
results = [];
async.each(files, function (file, callback) {
fs.exists(file, function (exists) {
// Future representation of the file's contents.
var contentsPromise,
// Future representation of JSLINT options from .jslintrc files.
optionPromise;
if (!exists) {
callback();
return;
}
contentsPromise = q.nfcall(fs.readFile, file, 'utf8');
optionPromise = getMerged(path.dirname(file));
// Parallelize IO operations.
q.all([contentsPromise, optionPromise])
.spread(function (contents, option) {
var success = JSLINT(contents, option),
errors,
fileResults;
if (!success) {
errors = JSLINT.data().errors;
fileResults = errors.reduce(function (soFar, error) {
if (error === null) {
return soFar;
}
return soFar.concat({
file: file,
error: error
});
}, []);
results = results.concat(fileResults);
}
process.nextTick(callback);
})
.catch(function (error) {
process.nextTick(function () {
callback(error);
});
})
.done();
});
}, function (error) {
results = results.sort(function (a, b) {
return a.file.charCodeAt(0) - b.file.charCodeAt(0);
});
callback(error, results);
});
}
I want to do something potentially-blocking for each file. So async.each is the obvious choice. I can parallelize related operations per-iteration with q.all and reuse my option values if they apply to 2 or more files.
Here, Async and Q each influence the control flow of the program, and Q represents values resolving to file contents sometime in the future. The libraries work well together. One does not need to "choose one over the other".
Callback pyramids in your code can be simplified using promise composition and javascript lexical scoping.
removeItem: function (itemId) {
var removeRegexp = new RegExp('\\/stock\\.php\\?remove=' + itemId);
var found = false
var promise = getPage('/sock.php')
_.times(5, (i) => {
promise = promise.then((webpage) => {
if (found) return true
var removeMatch = webpage.match(removeRegexp)
var found = removeMath !== null
var nextPage = found ? removeMatch[0] : '/stock.php?p='+i+1
return Q.delay(1000).then(() => this.getPage(nextPage))
})
})
return promise.fail(console.log.bind(console))
},
IMHO async should not be used in new javascript code. Promises are more composable, and allow for a lot more intutive code.
The primary reason why node did not use promises was because of performance concerns which have largely been addressed very well by libraries like Bluebird and Q.
As async/await syntax becomes more mainstream, promises will pave the way for code that looks very similar with synchronous code.
While this is still not an actual answer to my question (Q vs async), regarding my problem, I've found Selenium / WebDriverJs to be a viable solution.
driver.get('http://www.google.com');
driver.findElement(webdriver.By.name('q')).sendKeys('webdriver');
driver.findElement(webdriver.By.name('btnG')).click();
driver.wait(function() {
return driver.getTitle().then(function(title) {
return title === 'webdriver - Google Search';
});
}, 1000);
WebDriver uses a queue to execute promises sequentially, which helps immensely with controlling indentation. Its promises are also compatible with Q's.
Creating a sequence of promises is no longer an issue. A simple for-loop will do.
As for stopping early in a sequence, don't do this. Instead of using a sequence, use an asynchronous-while design and branch.