I've gotten lost inside my own head about how to best structure this function.
This code below is mostly correct, I just stubbed out a few of the functions at the bottom, not important.
What is supposed to happen:
My node app runs at a certain time and calls refreshDevices()
The function fetches a JSON payload from an API
The function loops over elements in that payload and checks if there is a record already
If there is no record, one is created, if there is then it is updated
After this is run a log entry is created logging the number of changes.
My issue is the log entry happens prior to any of the record checks taking place... my log shows 0 added & 0 updated.
I've tried re-writing the code to use an inline async/await function but I think that is effectively the same thing as below.
How am I supposed to structure this function to do what I've said.
I am finding the mix of async and sync very confusing to wrap my head around.
I even tried adding the log entry bit to happen in a .finally()
block at the end, but that was executing before the checks happened.
I have tried this using both forEach and map.
refreshDevices(_timespan) {
const status = {
added: 0,
updated: 0,
};
// Standard API fetch & json() conversion
fetch(`https://api.example.com/test×pan=${timespan}`)
.then((response) => response.json())
.then((JSON) => {
JSON.forEach((device) => {
findDevice(device._id)
.then(record => {
if (!record) {
addDeviceRecord(device);
status.added++;
} else {
deviceSeen(device._id);
status.updated++;
}
})
.catch(error => {
console.log('Error!');
});
});
systemlog.log({
message: 'Refresh',
data: {
success: true,
note: `Added: ${status.added} & updated: ${status.updated}`,
},
});
})
.catch((error) => {
systemlog.log({
message: 'Refresh',
type: systemlog.logType.ERROR,
data: {
success: false,
note: error,
},
});
});
}
function findDevice(id) {
return DB('device').findOne({ id }); // this returns is a promise
}
function addDeviceRecord(record) {
return DB('device').insertOne(record); // this returns is a promise
}
function deviceSeen(id) {
return DB('device').updateOne({ id }, { $set: { lastSeen: new Date() } }); // this returns is a promise
}
This question
As you have identified, your system.log call happens immediately after launching a litany of promises which haven't resolved.
Inside your refreshDevices(_timespan) function add an empty array to add each new promise in the forEach.
Then modify your code like
JSON.forEach((device) => {
promises.push( // Add this promise to the array
findDevice(device._id)
.then(record => {
if (!record) {
addDeviceRecord(device);
status.added++;
} else {
deviceSeen(device._id);
status.updated++;
}
})
.catch(error => {
console.log('Error!');
});
) // end of .push() arg
});
Then use Promise.all() to wait for all promises to be resolved before calling systemLog.log, sort of like (pseudo-pseudo code :)
Promise.all(promises).then(
systemlog.log({
message: 'Refresh',
data: {
success: true,
note: `Added: ${status.added} & updated: ${status.updated}`,
},
});
)
You could use Async/await to make the code more readable and use for of to iterate over data.
for of loop will wait for the first iteration to complete and then go for the second one and so on.
On the other hand forEach is based on callback and so it will execute and register a callback for each iteration in the callback queue and exit and don't wait for the callbacks to complete.
Try this
async refreshDevices(_timespan) {
const status = {
added: 0,
updated: 0,
};
// Standard API fetch & json() conversion
try {
let response = await fetch(`https://api.example.com/test×pan=${timespan}`)
let data= await response.json();
for (const device of data) {
try {
let record = await findDevice(device._id);
if (!record) {
addDeviceRecord(device);
status.added++;
} else {
deviceSeen(device._id);
status.updated++;
}
} catch (error) {
console.log('Error!');
}
}
systemlog.log({
message: 'Refresh',
data: {
success: true,
note: `Added: ${status.added} & updated: ${status.updated}`,
},
});
} catch (error) {
systemlog.log({
message: 'Refresh',
type: systemlog.logType.ERROR,
data: {
success: false,
note: error,
},
});
}
}
function findDevice(id) {
return DB('device').findOne({ id }); // this returns is a promise
}
function addDeviceRecord(record) {
return DB('device').insertOne(record); // this returns is a promise
}
function deviceSeen(id) {
return DB('device').updateOne({ id }, { $set: { lastSeen: new Date() } }); // this returns is a promise
}
Alternatively you could also use promise.all to wait for all promises to be resolved.
async refreshDevices(_timespan) {
const status = {
added: 0,
updated: 0,
};
// Standard API fetch & json() conversion
try {
let response = await fetch(`https://api.example.com/test×pan=${timespan}`)
let data = await response.json();
let promises = data.map(device => {
return new Promise( async (resolve, reject) => {
try {
let record = await findDevice(device._id);
if (!record) {
addDeviceRecord(device);
status.added++;
} else {
deviceSeen(device._id);
status.updated++;
}
resolve();
} catch (error) {
console.log('Error!');
reject(error);
}
});
});
let result = await Promise.all(promises);
systemlog.log({
message: 'Refresh',
data: {
success: true,
note: `Added: ${status.added} & updated: ${status.updated}`,
},
});
} catch (error) {
systemlog.log({
message: 'Refresh',
type: systemlog.logType.ERROR,
data: {
success: false,
note: error,
},
});
}
}
function findDevice(id) {
return DB('device').findOne({ id }); // this returns is a promise
}
function addDeviceRecord(record) {
return DB('device').insertOne(record); // this returns is a promise
}
function deviceSeen(id) {
return DB('device').updateOne({ id }, { $set: { lastSeen: new Date() } }); // this returns is a promise
}
Let's say I have a set of Promises that are making network requests, of which one will fail:
// http://does-not-exist will throw a TypeError
var arr = [ fetch('index.html'), fetch('http://does-not-exist') ]
Promise.all(arr)
.then(res => console.log('success', res))
.catch(err => console.log('error', err)) // This is executed
Let's say I want to wait until all of these have finished, regardless of if one has failed. There might be a network error for a resource that I can live without, but which if I can get, I want before I proceed. I want to handle network failures gracefully.
Since Promise.all doesn't leave any room for this, what is the recommended pattern for handling this, without using a promises library?
Update, you probably want to use the built-in native Promise.allSettled:
Promise.allSettled([promise]).then(([result]) => {
//reach here regardless
// {status: "fulfilled", value: 33}
});
As a fun fact, this answer below was prior art in adding that method to the language :]
Sure, you just need a reflect:
const reflect = p => p.then(v => ({v, status: "fulfilled" }),
e => ({e, status: "rejected" }));
reflect(promise).then((v) => {
console.log(v.status);
});
Or with ES5:
function reflect(promise){
return promise.then(function(v){ return {v:v, status: "fulfilled" }},
function(e){ return {e:e, status: "rejected" }});
}
reflect(promise).then(function(v){
console.log(v.status);
});
Or in your example:
var arr = [ fetch('index.html'), fetch('http://does-not-exist') ]
Promise.all(arr.map(reflect)).then(function(results){
var success = results.filter(x => x.status === "fulfilled");
});
Similar answer, but more idiomatic for ES6 perhaps:
const a = Promise.resolve(1);
const b = Promise.reject(new Error(2));
const c = Promise.resolve(3);
Promise.all([a, b, c].map(p => p.catch(e => e)))
.then(results => console.log(results)) // 1,Error: 2,3
.catch(e => console.log(e));
const console = { log: msg => div.innerHTML += msg + "<br>"};
<div id="div"></div>
Depending on the type(s) of values returned, errors can often be distinguished easily enough (e.g. use undefined for "don't care", typeof for plain non-object values, result.message, result.toString().startsWith("Error:") etc.)
Benjamin's answer offers a great abstraction for solving this issue, but I was hoping for a less abstracted solution. The explicit way to to resolve this issue is to simply call .catch on the internal promises, and return the error from their callback.
let a = new Promise((res, rej) => res('Resolved!')),
b = new Promise((res, rej) => rej('Rejected!')),
c = a.catch(e => { console.log('"a" failed.'); return e; }),
d = b.catch(e => { console.log('"b" failed.'); return e; });
Promise.all([c, d])
.then(result => console.log('Then', result)) // Then ["Resolved!", "Rejected!"]
.catch(err => console.log('Catch', err));
Promise.all([a.catch(e => e), b.catch(e => e)])
.then(result => console.log('Then', result)) // Then ["Resolved!", "Rejected!"]
.catch(err => console.log('Catch', err));
Taking this one step further, you could write a generic catch handler that looks like this:
const catchHandler = error => ({ payload: error, resolved: false });
then you can do
> Promise.all([a, b].map(promise => promise.catch(catchHandler))
.then(results => console.log(results))
.catch(() => console.log('Promise.all failed'))
< [ 'Resolved!', { payload: Promise, resolved: false } ]
The problem with this is that the caught values will have a different interface than the non-caught values, so to clean this up you might do something like:
const successHandler = result => ({ payload: result, resolved: true });
So now you can do this:
> Promise.all([a, b].map(result => result.then(successHandler).catch(catchHandler))
.then(results => console.log(results.filter(result => result.resolved))
.catch(() => console.log('Promise.all failed'))
< [ 'Resolved!' ]
Then to keep it DRY, you get to Benjamin's answer:
const reflect = promise => promise
.then(successHandler)
.catch(catchHander)
where it now looks like
> Promise.all([a, b].map(result => result.then(successHandler).catch(catchHandler))
.then(results => console.log(results.filter(result => result.resolved))
.catch(() => console.log('Promise.all failed'))
< [ 'Resolved!' ]
The benefits of the second solution are that its abstracted and DRY. The downside is you have more code, and you have to remember to reflect all your promises to make things consistent.
I would characterize my solution as explicit and KISS, but indeed less robust. The interface doesn't guarantee that you know exactly whether the promise succeeded or failed.
For example you might have this:
const a = Promise.resolve(new Error('Not beaking, just bad'));
const b = Promise.reject(new Error('This actually didnt work'));
This won't get caught by a.catch, so
> Promise.all([a, b].map(promise => promise.catch(e => e))
.then(results => console.log(results))
< [ Error, Error ]
There's no way to tell which one was fatal and which was wasn't. If that's important then you're going to want to enforce and interface that tracks whether it was successful or not (which reflect does).
If you just want to handle errors gracefully, then you can just treat errors as undefined values:
> Promise.all([a.catch(() => undefined), b.catch(() => undefined)])
.then((results) => console.log('Known values: ', results.filter(x => typeof x !== 'undefined')))
< [ 'Resolved!' ]
In my case, I don't need to know the error or how it failed--I just care whether I have the value or not. I'll let the function that generates the promise worry about logging the specific error.
const apiMethod = () => fetch()
.catch(error => {
console.log(error.message);
throw error;
});
That way, the rest of the application can ignore its error if it wants, and treat it as an undefined value if it wants.
I want my high level functions to fail safely and not worry about the details on why its dependencies failed, and I also prefer KISS to DRY when I have to make that tradeoff--which is ultimately why I opted to not use reflect.
There is a finished proposal for a function which can accomplish this natively, in vanilla Javascript: Promise.allSettled, which has made it to stage 4, is officialized in ES2020, and is implemented in all modern environments. It is very similar to the reflect function in this other answer. Here's an example, from the proposal page. Before, you would have had to do:
function reflect(promise) {
return promise.then(
(v) => {
return { status: 'fulfilled', value: v };
},
(error) => {
return { status: 'rejected', reason: error };
}
);
}
const promises = [ fetch('index.html'), fetch('https://does-not-exist/') ];
const results = await Promise.all(promises.map(reflect));
const successfulPromises = results.filter(p => p.status === 'fulfilled');
Using Promise.allSettled instead, the above will be equivalent to:
const promises = [ fetch('index.html'), fetch('https://does-not-exist/') ];
const results = await Promise.allSettled(promises);
const successfulPromises = results.filter(p => p.status === 'fulfilled');
Those using modern environments will be able to use this method without any libraries. In those, the following snippet should run without problems:
Promise.allSettled([
Promise.resolve('a'),
Promise.reject('b')
])
.then(console.log);
Output:
[
{
"status": "fulfilled",
"value": "a"
},
{
"status": "rejected",
"reason": "b"
}
]
For older browsers, there is a spec-compliant polyfill here.
I really like Benjamin's answer, and how he basically turns all promises into always-resolving-but-sometimes-with-error-as-a-result ones. :)
Here's my attempt at your request just in case you were looking for alternatives. This method simply treats errors as valid results, and is coded similar to Promise.all otherwise:
Promise.settle = function(promises) {
var results = [];
var done = promises.length;
return new Promise(function(resolve) {
function tryResolve(i, v) {
results[i] = v;
done = done - 1;
if (done == 0)
resolve(results);
}
for (var i=0; i<promises.length; i++)
promises[i].then(tryResolve.bind(null, i), tryResolve.bind(null, i));
if (done == 0)
resolve(results);
});
}
var err;
Promise.all([
promiseOne().catch(function(error) { err = error;}),
promiseTwo().catch(function(error) { err = error;})
]).then(function() {
if (err) {
throw err;
}
});
The Promise.all will swallow any rejected promise and store the error in a variable, so it will return when all of the promises have resolved. Then you can re-throw the error out, or do whatever. In this way, I guess you would get out the last rejection instead of the first one.
I had the same problem and have solved it in the following way:
const fetch = (url) => {
return node-fetch(url)
.then(result => result.json())
.catch((e) => {
return new Promise((resolve) => setTimeout(() => resolve(fetch(url)), timeout));
});
};
tasks = [fetch(url1), fetch(url2) ....];
Promise.all(tasks).then(......)
In that case Promise.all will wait for every Promise will come into resolved or rejected state.
And having this solution we are "stopping catch execution" in a non-blocking way. In fact, we're not stopping anything, we just returning back the Promise in a pending state which returns another Promise when it's resolved after the timeout.
This should be consistent with how Q does it:
if(!Promise.allSettled) {
Promise.allSettled = function (promises) {
return Promise.all(promises.map(p => Promise.resolve(p).then(v => ({
state: 'fulfilled',
value: v,
}), r => ({
state: 'rejected',
reason: r,
}))));
};
}
Instead of rejecting, resolve it with a object.
You could do something like this when you are implementing promise
const promise = arg => {
return new Promise((resolve, reject) => {
setTimeout(() => {
try{
if(arg != 2)
return resolve({success: true, data: arg});
else
throw new Error(arg)
}catch(e){
return resolve({success: false, error: e, data: arg})
}
}, 1000);
})
}
Promise.all([1,2,3,4,5].map(e => promise(e))).then(d => console.log(d))
Benjamin Gruenbaum answer is of course great,. But I can also see were Nathan Hagen point of view with the level of abstraction seem vague. Having short object properties like e & v don't help either, but of course that could be changed.
In Javascript there is standard Error object, called Error,. Ideally you always throw an instance / descendant of this. The advantage is that you can do instanceof Error, and you know something is an error.
So using this idea, here is my take on the problem.
Basically catch the error, if the error is not of type Error, wrap the error inside an Error object. The resulting array will have either resolved values, or Error objects you can check on.
The instanceof inside the catch, is in case you use some external library that maybe did reject("error"), instead of reject(new Error("error")).
Of course you could have promises were you resolve an error, but in that case it would most likely make sense to treat as an error anyway, like the last example shows.
Another advantage of doing it this, array destructing is kept simple.
const [value1, value2] = PromiseAllCatch(promises);
if (!(value1 instanceof Error)) console.log(value1);
Instead of
const [{v: value1, e: error1}, {v: value2, e: error2}] = Promise.all(reflect..
if (!error1) { console.log(value1); }
You could argue that the !error1 check is simpler than an instanceof, but your also having to destruct both v & e.
function PromiseAllCatch(promises) {
return Promise.all(promises.map(async m => {
try {
return await m;
} catch(e) {
if (e instanceof Error) return e;
return new Error(e);
}
}));
}
async function test() {
const ret = await PromiseAllCatch([
(async () => "this is fine")(),
(async () => {throw new Error("oops")})(),
(async () => "this is ok")(),
(async () => {throw "Still an error";})(),
(async () => new Error("resolved Error"))(),
]);
console.log(ret);
console.log(ret.map(r =>
r instanceof Error ? "error" : "ok"
).join(" : "));
}
test();
I think the following offers a slightly different approach... compare fn_fast_fail() with fn_slow_fail()... though the latter doesn't fail as such... you can check if one or both of a and b is an instance of Error and throw that Error if you want it to reach the catch block (e.g. if (b instanceof Error) { throw b; }) . See the jsfiddle.
var p1 = new Promise((resolve, reject) => {
setTimeout(() => resolve('p1_delayed_resolvement'), 2000);
});
var p2 = new Promise((resolve, reject) => {
reject(new Error('p2_immediate_rejection'));
});
var fn_fast_fail = async function () {
try {
var [a, b] = await Promise.all([p1, p2]);
console.log(a); // "p1_delayed_resolvement"
console.log(b); // "Error: p2_immediate_rejection"
} catch (err) {
console.log('ERROR:', err);
}
}
var fn_slow_fail = async function () {
try {
var [a, b] = await Promise.all([
p1.catch(error => { return error }),
p2.catch(error => { return error })
]);
console.log(a); // "p1_delayed_resolvement"
console.log(b); // "Error: p2_immediate_rejection"
} catch (err) {
// we don't reach here unless you throw the error from the `try` block
console.log('ERROR:', err);
}
}
fn_fast_fail(); // fails immediately
fn_slow_fail(); // waits for delayed promise to resolve
I just wanted a polyfill that exactly replicated ES2020 behaviour since I'm locked into node versions a lot earlier than 12.9 (when Promise.allSettled appeared), unfortunately. So for what it's worth, this is my version:
const settle = (promise) => (promise instanceof Promise) ?
promise.then(val => ({ value: val, status: "fulfilled" }),
err => ({ reason: err, status: "rejected" })) :
{ value: promise, status: 'fulfilled' };
const allSettled = async (parr) => Promise.all(parr.map(settle));
This handles a mixed array of promise and non-promise values, as does the ES version. It hands back the same array of { status, value/reason } objects as the native version.
Here's my custom settledPromiseAll()
const settledPromiseAll = function(promisesArray) {
var savedError;
const saveFirstError = function(error) {
if (!savedError) savedError = error;
};
const handleErrors = function(value) {
return Promise.resolve(value).catch(saveFirstError);
};
const allSettled = Promise.all(promisesArray.map(handleErrors));
return allSettled.then(function(resolvedPromises) {
if (savedError) throw savedError;
return resolvedPromises;
});
};
Compared to Promise.all
If all promises are resolved, it performs exactly as the standard one.
If one of more promises are rejected, it returns the first one rejected much the same as the standard one but unlike it waits for all promises to resolve/reject.
For the brave we could change Promise.all():
(function() {
var stdAll = Promise.all;
Promise.all = function(values, wait) {
if(!wait)
return stdAll.call(Promise, values);
return settledPromiseAll(values);
}
})();
CAREFUL. In general we never change built-ins, as it might break other unrelated JS libraries or clash with future changes to JS standards.
My settledPromiseall is backward compatible with Promise.all and extends its functionality.
People who are developing standards -- why not include this to a new Promise standard?
I recently built a library that allows what you need. it executes promises in parallel, and if one fails, the process continues, at the end it returns an array with all the results, including errors.
https://www.npmjs.com/package/promise-ax
I hope and it is helpful for someone.
const { createPromise } = require('promise-ax');
const promiseAx = createPromise();
const promise1 = Promise.resolve(4);
const promise2 = new Promise((resolve, reject) => setTimeout(reject, 100, new Error("error")));
const promise3 = Promise.reject("error");
const promise4 = promiseAx.resolve(8);
const promise5 = promiseAx.reject("errorAx");
const asyncOperation = (time) => {
return new Promise((resolve, reject) => {
if (time < 0) {
reject("reject");
}
setTimeout(() => {
resolve(time);
}, time);
});
};
const promisesToMake = [promise1, promise2, promise3, promise4, promise5, asyncOperation(100)];
promiseAx.allSettled(promisesToMake).then((results) => results.forEach((result) => console.log(result)));
// Salida esperada:
// 4
// Error: error
// error
// 8
// errorAx
// 100
I would do:
var err = [fetch('index.html').then((success) => { return Promise.resolve(success); }).catch((e) => { return Promise.resolve(e); }),
fetch('http://does-not-exist').then((success) => { return Promise.resolve(success); }).catch((e) => { return Promise.resolve(e); })];
Promise.all(err)
.then(function (res) { console.log('success', res) })
.catch(function (err) { console.log('error', err) }) //never executed
I've been using following codes since ES5.
Promise.wait = function(promiseQueue){
if( !Array.isArray(promiseQueue) ){
return Promise.reject('Given parameter is not an array!');
}
if( promiseQueue.length === 0 ){
return Promise.resolve([]);
}
return new Promise((resolve, reject) =>{
let _pQueue=[], _rQueue=[], _readyCount=false;
promiseQueue.forEach((_promise, idx) =>{
// Create a status info object
_rQueue.push({rejected:false, seq:idx, result:null});
_pQueue.push(Promise.resolve(_promise));
});
_pQueue.forEach((_promise, idx)=>{
let item = _rQueue[idx];
_promise.then(
(result)=>{
item.resolved = true;
item.result = result;
},
(error)=>{
item.resolved = false;
item.result = error;
}
).then(()=>{
_readyCount++;
if ( _rQueue.length === _readyCount ) {
let result = true;
_rQueue.forEach((item)=>{result=result&&item.resolved;});
(result?resolve:reject)(_rQueue);
}
});
});
});
};
The usage signature is just like Promise.all. The major difference is that Promise.wait will wait for all the promises to finish their jobs.
I know that this question has a lot of answers, and I'm sure must (if not all) are correct.
However it was very hard for me to understand the logic/flow of these answers.
So I looked at the Original Implementation on Promise.all(), and I tried to imitate that logic - with the exception of not stopping the execution if one Promise failed.
public promiseExecuteAll(promisesList: Promise<any>[] = []): Promise<{ data: any, isSuccess: boolean }[]>
{
let promise: Promise<{ data: any, isSuccess: boolean }[]>;
if (promisesList.length)
{
const result: { data: any, isSuccess: boolean }[] = [];
let count: number = 0;
promise = new Promise<{ data: any, isSuccess: boolean }[]>((resolve, reject) =>
{
promisesList.forEach((currentPromise: Promise<any>, index: number) =>
{
currentPromise.then(
(data) => // Success
{
result[index] = { data, isSuccess: true };
if (promisesList.length <= ++count) { resolve(result); }
},
(data) => // Error
{
result[index] = { data, isSuccess: false };
if (promisesList.length <= ++count) { resolve(result); }
});
});
});
}
else
{
promise = Promise.resolve([]);
}
return promise;
}
Explanation:
- Loop over the input promisesList and execute each Promise.
- No matter if the Promise resolved or rejected: save the Promise's result in a result array according to the index. Save also the resolve/reject status (isSuccess).
- Once all Promises completed, return one Promise with the result of all others.
Example of use:
const p1 = Promise.resolve("OK");
const p2 = Promise.reject(new Error(":-("));
const p3 = Promise.resolve(1000);
promiseExecuteAll([p1, p2, p3]).then((data) => {
data.forEach(value => console.log(`${ value.isSuccess ? 'Resolve' : 'Reject' } >> ${ value.data }`));
});
/* Output:
Resolve >> OK
Reject >> :-(
Resolve >> 1000
*/
You can execute your logic sequentially via synchronous executor nsynjs. It will pause on each promise, wait for resolution/rejection, and either assign resolve's result to data property, or throw an exception (for handling that you will need try/catch block). Here is an example:
function synchronousCode() {
function myFetch(url) {
try {
return window.fetch(url).data;
}
catch (e) {
return {status: 'failed:'+e};
};
};
var arr=[
myFetch("https://ajax.googleapis.com/ajax/libs/jquery/2.0.0/jquery.min.js"),
myFetch("https://ajax.googleapis.com/ajax/libs/jquery/2.0.0/NONEXISTANT.js"),
myFetch("https://ajax.NONEXISTANT123.com/ajax/libs/jquery/2.0.0/NONEXISTANT.js")
];
console.log('array is ready:',arr[0].status,arr[1].status,arr[2].status);
};
nsynjs.run(synchronousCode,{},function(){
console.log('done');
});
<script src="https://rawgit.com/amaksr/nsynjs/master/nsynjs.js"></script>
Promise.all with using modern async/await approach
const promise1 = //...
const promise2 = //...
const data = await Promise.all([promise1, promise2])
const dataFromPromise1 = data[0]
const dataFromPromise2 = data[1]
I don't know which promise library you are using, but most have something like allSettled.
Edit: Ok since you want to use plain ES6 without external libraries, there is no such method.
In other words: You have to loop over your promises manually and resolve a new combined promise as soon as all promises are settled.
How do I wrap this routine inside a Promise so that I only resolve when I get all the data?
var accounts = [];
getAccounts(userId, accs => {
accs.forEach(acc => {
getAccountTx(acc.id, tx => {
accounts.push({
'id': acc.id,
'tx': tx
});
});
})
});
EDIT: Any issues if I do it like this?
function getAccountsAllAtOnce() {
var accounts = [];
var required = 0;
var done = 0;
getAccounts(userId, accs => {
required = accs.length;
accs.forEach(acc => {
getAccountTx(acc.id, tx => {
accounts.push({
'id': acc.id,
'tx': tx
});
done = done + 1;
});
})
});
while(done < required) {
// wait
}
return accounts;
}
Let's put this routine into a separate function, so it is easier to re-use it later. This function should return a promise, which will be resolved with array of accounts (also I'll modify your code as small as possible):
function getAccountsWithTx(userId) {
return new Promise((resolve, reject) => {
var accounts = [];
getAccounts(userId, accs => {
accs.forEach(acc => {
getAccountTx(acc.id, tx => {
accounts.push({
'id': acc.id,
'tx': tx
});
// resolve after we fetched all accounts
if (accs.length === accounts.length) {
resolve(accounts);
}
});
});
});
});
}
The single difference is just returning a promise and resolving after all accounts were fetched. However, callbacks tend your codebase to have this "callback hell" style, when you have a lot of nested callbacks, and it makes it hard to reason about it. You can workaround it using good discipline, but you can simplify it greatly switching to returning promises from all async functions. For example your func will look like the following:
function getAccountsWithTx(userId) {
getAccounts(userId)
.then(accs => {
const transformTx = acc => getAccountTx(acc.id)
.then(tx => ({ tx, id: acc.id }));
return Promise.all(accs.map(transformTx));
});
}
Both of them are absolutely equivalent, and there are plently of libraries to "promisify" your current callback-style functions (for example, bluebird or even native Node util.promisify). Also, with new async/await syntax it becomes even easier, because it allows to think in sync flow:
async function getAccountsWithTx(userId) {
const accs = await getUserAccounts(userId);
const transformTx = async (acc) => {
const tx = getAccountTx(acc.id);
return { tx, id: acc.id };
};
return Promise.all(accs.map(transformTx));
}
As you can see, we eliminate any nesting! It makes reasoning about code much easier, because you can read code as it will be actually executed. However, all these three options are equivalent, so it is up to you, what makes the most sense in your project and environment.
I'd split every step into its own function, and return a promise or promise array from each one. For example, getAccounts becomes:
function getAccountsAndReturnPromise(userId) {
return new Promise((resolve, reject) => {
getAccounts(userId, accounts => {
return resolve(accounts);
});
});
};
And getAccountTx resolves to an array of { id, tx } objects:
function getAccountTransactionsAndReturnPromise(accountId) {
return new Promise((resolve, reject) => {
getAccountTx(account.id, (transactions) => {
var accountWithTransactions = {
id: account.id,
transactions
};
return resolve(accountWithTransactions);
});
});
};
Then you can use Promise.all() and map() to resolve the last step to an array of values in the format you desire:
function getDataForUser(userId) {
return getAccountsAndReturnPromise(userId)
.then(accounts=>{
var accountTransactionPromises = accounts.map(account =>
getAccountTransactionsAndReturnPromise(account.id)
);
return Promise.all(accountTransactionPromises);
})
.then(allAccountsWithTransactions => {
return allAccountsWithTransactions.map(account =>{
return {
id: account.id,
tx: tx
}
});
});
}
I wonder if there is a way that I can execute an array of promises waterfally? i.e. I want the next promise not to start until the current promise is resolved.
My test:
import Promise from 'bluebird'
describe('promises waterfall', () => {
const result = []
function doItLater(val, time = 50) => {
return new Promise(resolve => {
setTimeout(() => {
result.push(val)
resolve(val)
}, time)
})
}
it('execute promises one after one resolved', done => {
result.push(1)
const promiseList = [doItLater('a', 100),doItLater('b', 1),doItLater('c')]
result.push(2)
Promise.each(
promiseList,
(output) => {
result.push(output + '-outputted')
}
)
.then(
() => {
result.push(3)
console.log(result)
expect(result).to.eql([ 1, 2, 'a', 'b', 'c', 'a-outputted', 'b-outputted', 'c-outputted', 3 ])
done()
}
)
})
})
Update
Sorry if I have made it too confusing. I'm asking how I can make my test pass. Currently my test fails - the actual result is in wrong order:
[ 1,
2,
'b',
'c',
'a',
'a-outputted',
'b-outputted',
'c-outputted',
3 ]
When you create promises like this
[doItLater('a', 100),doItLater('b', 1),doItLater('c')]
they are already executing asynchronously. There is no way of controlling their order of execution.
You can use Promise.reduce for your case, like this
Promise.reduce([['a', 100], ['b', 1], ['c']], function(res, args) {
return doItLater.apply(null, args).then(function(output) {
res.push(output + '-outputted');
return res;
});
}, result)
.then(function(accumulatedResult) {
....
});
Now, we are creating promises one by one, after the previous promise is resolved and we are accumulating the result in res (which is actually result). When all the promises are resolved, the accumulatedResult will have all the values.
Note: It is strongly recommended NOT to share data between functions. Please make sure that you have compelling reasons to do so, if you have to do.
// Promisse waterfall pattern
var promises = [1,2,3].map((guid)=>{
return (param)=> {
console.log("param", param);
var id = guid;
return new Promise(resolve => {
// resolve in a random amount of time
setTimeout(function () {
resolve(id);
}, (Math.random() * 1.5 | 0) * 1000);
});
}
}).reduce(function (acc, curr, index) {
return acc.then(function (res) {
return curr(res[index-1]).then(function (result) {
console.log("result", result);
res.push(result);
return res;
});
});
}, Promise.resolve([]));
promises.then(console.log);
I used to develop a lot with promise and now I am moving to RxJS. The doc of RxJS doesn't provide a very clear example on how to move from promise chain to observer sequence.
For example, I usually write promise chain with multiple steps, like
// a function that returns a promise
getPromise()
.then(function(result) {
// do something
})
.then(function(result) {
// do something
})
.then(function(result) {
// do something
})
.catch(function(err) {
// handle error
});
How should I rewrite this promise chain in the RxJS style?
For data flow (equivalent to then):
Rx.Observable.fromPromise(...)
.flatMap(function(result) {
// do something
})
.flatMap(function(result) {
// do something
})
.subscribe(function onNext(result) {
// end of chain
}, function onError(error) {
// process the error
});
A promise can be converted into an observable with Rx.Observable.fromPromise.
Some promise operators have a direct translation. For instance RSVP.all, or jQuery.when can be replaced by Rx.Observable.forkJoin.
Keep in mind that you have a bunch of operators that allows to transform data asynchronously, and to perform tasks that you cannot or would be very hard to do with promises. Rxjs reveals all its powers with asynchronous sequences of data (sequence i.e. more than 1 asynchronous value).
For error management, the subject is a little bit more complex.
there are catch and finally operators too
retryWhen can also help to repeat a sequence in case of error
you can also deal with errors in the subscriber itself with the onError function.
For precise semantics, have a deeper look at the documentation and examples you can find on the web, or ask specific questions here.
This would definitely be a good starting point for going deeper in error management with Rxjs : https://xgrommx.github.io/rx-book/content/getting_started_with_rxjs/creating_and_querying_observable_sequences/error_handling.html
A more modern alternative:
import {from as fromPromise} from 'rxjs';
import {catchError, flatMap} from 'rxjs/operators';
fromPromise(...).pipe(
flatMap(result => {
// do something
}),
flatMap(result => {
// do something
}),
flatMap(result => {
// do something
}),
catchError(error => {
// handle error
})
)
Also note that for all this to work, you need to subscribe to this piped Observable somewhere, but I assume it's handled in some other part of the application.
Update May 2019, using RxJs 6
Agree with the provided answers above, wished to add a concrete example with some toy data & simple promises (with setTimeout) using RxJs v6 to add clarity.
Just update the passed id (currently hard-coded as 1) to something that does not exist to execute the error handling logic too. Importantly, also note the use of of with catchError message.
import { from as fromPromise, of } from "rxjs";
import { catchError, flatMap, tap } from "rxjs/operators";
const posts = [
{ title: "I love JavaScript", author: "Wes Bos", id: 1 },
{ title: "CSS!", author: "Chris Coyier", id: 2 },
{ title: "Dev tools tricks", author: "Addy Osmani", id: 3 }
];
const authors = [
{ name: "Wes Bos", twitter: "#wesbos", bio: "Canadian Developer" },
{
name: "Chris Coyier",
twitter: "#chriscoyier",
bio: "CSS Tricks and CodePen"
},
{ name: "Addy Osmani", twitter: "#addyosmani", bio: "Googler" }
];
function getPostById(id) {
return new Promise((resolve, reject) => {
setTimeout(() => {
const post = posts.find(post => post.id === id);
if (post) {
console.log("ok, post found!");
resolve(post);
} else {
reject(Error("Post not found!"));
}
}, 200);
});
}
function hydrateAuthor(post) {
return new Promise((resolve, reject) => {
setTimeout(() => {
const authorDetails = authors.find(person => person.name === post.author);
if (authorDetails) {
post.author = authorDetails;
console.log("ok, post hydrated with author info");
resolve(post);
} else {
reject(Error("Author not Found!"));
}
}, 200);
});
}
function dehydratePostTitle(post) {
return new Promise((resolve, reject) => {
setTimeout(() => {
delete post.title;
console.log("ok, applied transformation to remove title");
resolve(post);
}, 200);
});
}
// ok, here is how it looks regarding this question..
let source$ = fromPromise(getPostById(1)).pipe(
flatMap(post => {
return hydrateAuthor(post);
}),
flatMap(post => {
return dehydratePostTitle(post);
}),
catchError(error => of(`Caught error: ${error}`))
);
source$.subscribe(console.log);
Output Data:
ok, post found!
ok, post hydrated with author info
ok, applied transformation to remove title
{ author:
{ name: 'Wes Bos',
twitter: '#wesbos',
bio: 'Canadian Developer' },
id: 1 }
The key part, is equivalent to the following using plain promise control flow:
getPostById(1)
.then(post => {
return hydrateAuthor(post);
})
.then(post => {
return dehydratePostTitle(post);
})
.then(author => {
console.log(author);
})
.catch(err => {
console.error(err);
});
If I understood correctly, you mean consuming the values, in which case you use sbuscribe i.e.
const arrObservable = from([1,2,3,4,5,6,7,8]);
arrObservable.subscribe(number => console.log(num) );
Additionally, you can just turn the observable to a promise using toPromise() as shown:
arrObservable.toPromise().then()
if getPromise function is in a middle of a stream pipe you should simple wrap it into one of functions mergeMap, switchMap or concatMap (usually mergeMap):
stream$.pipe(
mergeMap(data => getPromise(data)),
filter(...),
map(...)
).subscribe(...);
if you want to start your stream with getPromise() then wrap it into from function:
import {from} from 'rxjs';
from(getPromise()).pipe(
filter(...)
map(...)
).subscribe(...);
As far as i just found out, if you return a result in a flatMap, it converts it to an Array, even if you returned a string.
But if you return an Observable, that observable can return a string;
This is how I did it.
Previously
public fetchContacts(onCompleteFn: (response: gapi.client.Response<gapi.client.people.ListConnectionsResponse>) => void) {
const request = gapi.client.people.people.connections.list({
resourceName: 'people/me',
pageSize: 100,
personFields: 'phoneNumbers,organizations,emailAddresses,names'
}).then(response => {
onCompleteFn(response as gapi.client.Response<gapi.client.people.ListConnectionsResponse>);
});
}
// caller:
this.gapi.fetchContacts((rsp: gapi.client.Response<gapi.client.people.ListConnectionsResponse>) => {
// handle rsp;
});
After(ly?)
public fetchContacts(): Observable<gapi.client.Response<gapi.client.people.ListConnectionsResponse>> {
return from(
new Promise((resolve, reject) => {
gapi.client.people.people.connections.list({
resourceName: 'people/me',
pageSize: 100,
personFields: 'phoneNumbers,organizations,emailAddresses,names'
}).then(result => {
resolve(result);
});
})
).pipe(map((result: gapi.client.Response<gapi.client.people.ListConnectionsResponse>) => {
return result; //map is not really required if you not changing anything in the response. you can just return the from() and caller would subscribe to it.
}));
}
// caller
this.gapi.fetchContacts().subscribe(((rsp: gapi.client.Response<gapi.client.people.ListConnectionsResponse>) => {
// handle rsp
}), (error) => {
// handle error
});
RxJS sequence equivalent to promise.then()?
For example
function getdata1 (argument) {
return this.http.get(url)
.map((res: Response) => res.json());
}
function getdata2 (argument) {
return this.http.get(url)
.map((res: Response) => res.json());
}
getdata1.subscribe((data1: any) => {
console.log("got data one. get data 2 now");
getdata2.subscribe((data2: any) => {
console.log("got data one and two here");
});
});