I'm using Node.js and Bluebird to create some fairly complicated logic involving uncompressing a structured file, parsing JSON, creating and making changes to several MongoDB documents, and writing related files in multiple locations. I also have fairly complicated error handling for all of this depending on the state of the system when an error occurs.
I am having difficulty thinking of a good way to manage dependencies through the flow of promises.
My existing code basically looks like this:
var doStuff = function () {
var dependency1 = null;
var dependency2 = null;
promise1()
.then(function (value) {
dependency1 = value;
return promise2()
.then(function (value) {
dependency2 = value;
return promise3(dependency1)
.then(successFunction);
});
})
.catch(function (err) {
cleanupDependingOnSystemState(err, dependency1, dependency2);
});
};
Note that dependency1 isn't needed until promise3, and that the error handler needs to know about the dependencies.
To me this seems like spaghetti code (and my actual code is far worse with a lot of parallel control flow). I've also read that returning another promise inside of a .then callback is an antipattern. Is there a better/cleaner way of accomplishing what I'm trying to do?
I find both answers currently provided nice but clumsy. They're both good but contain overhead I don't think you need to have. If you instead use promises as proxies you get a lot of things for free.
var doStuff = function () {
var p1 = promise1();
var p2 = p1.then(promise2);
var p3 = p1.then(promise3); // if you actually need to wait for p2 here, do.
return Promise.all([p1, p2, p3]).catch(function(err){
// clean up based on err and state, can unwrap promises here
});
};
Please do not use successFunction and such it is an anti-pattern and loses information.
If you feel like you have to use successFunction you can write:
var doStuff = function () {
var p1 = promise1();
var p2 = p1.then(promise2);
var p3 = p1.then(promise3); // if you actually need to wait for p2 here, do.
Promise.join(p1, p2, p3, successFunction).catch(function(err){
// clean up based on err and state, can unwrap promises here
});
};
However, it is infinitely worse since it won't let the consumer handle errors they may be able to handle.
This question might be more appropriate for code review but here is how I'd approach it given this example:
var doStuff = function () {
// Set up your promises based on their dependencies. In your example
// promise2 does not use dependency1 so I left them unrelated.
var dep1Promise = promise1();
var dep2Promise = promise2();
var dep3Promise = dependency1Promise.then(function(value){
return promise3(value);
});
// Wait for all the promises the either succeed or error.
allResolved([dep1Promise, dep2Promise, dep3Promise])
.spread(function(dep1, dep2, dep3){
var err = dep1.error || dep2.error || dep3.error;
if (err){
// If any errored, call the function you prescribed
cleanupDependingOnSystemState(err, dep1.value, dep2.value);
} else {
// Call the success handler.
successFunction(dep3.value);
}
};
// Promise.all by default just fails on the first error, but since
// you want to pass any partial results to cleanupDependingOnSystemState,
// I added this helper.
function allResolved(promises){
return Promise.all(promises.map(function(promise){
return promise.then(function(value){
return {value: value};
}, function(err){
return {error: err};
});
});
}
The use of allResolved is only because of your callback specifics, if you had a more general error handler, you could simply resolve using Promise.all directly, or even:
var doStuff = function () {
// Set up your promises based on their dependencies. In your example
// promise2 does not use dependency1 so I left them unrelated.
var dep1Promise = promise1();
var dep2Promise = promise2();
var dep3Promise = dependency1Promise.then(function(value){
return promise3(value);
});
dep3Promise.then(successFunction, cleanupDependingOnSystemState);
};
It is certainly not an antipattern to return promises within thens, flattening nested promises is a feature of the promise spec.
Here's a possible rewrite, though I'm not sure it's cleaner:
var doStuff = function () {
promise1()
.then(function (value1) {
return promise2()
.then(function (value2) {
return promise3(value1)
.then(successFunction)
.finally(function() {
cleanup(null, value1, value2);
});
})
.finally(function() {
cleanup(null, value1, null);
});
})
.finally(function () {
cleanup(null, null, null);
});
};
Or another option, with atomic cleanup functions:
var doStuff = function () {
promise1()
.then(function (value1) {
return promise2()
.then(function (value2) {
return promise3(value1)
.then(successFunction)
.finally(function() {
cleanup3(value2);
});
})
.finally(function() {
cleanup2(value1);
});
})
.finally(function (err) {
cleanup1(err);
});
};
Really, I feel like there's not much you can do to clean this up. Event with vanilla try/catches, the best possible pattern is pretty similar to these.
Related
In the code below, I am trying to do the following:
Have Stats(), getOverallStats() and GetGroups() to run in parallel. Each returns a promise.
The forEach in GetGroups.then() should run sequentially to ensure the output is in the correct order.
Once ALL of the above is complete, then run some more code.
However, I am getting very confused with the promises! The logging gives me:
looping
here
looping
looping
But what I am looking for is here to be at the end.
Finally, at the moment I have hardcoded loopAgelist[1] for testing purposes. But, I actually want to be able to loop through loopAgelist[] with a timeout in between! I would appreciate if someone could explain some promise 'rules' to use in these complicated cases.
var loopAgeList;
var looppromises = [];
getAgeGroupList().then(function (loopAgeList) {
var statsPromise = Stats(loopAgeList[1]);
var oStatsPromise = getOverallStats();
var grpPromise = GetGroups(loopAgeList[1]).then(function (groups) {
var promise = Parse.Promise.as();
groups.forEach(function (grp) {
promise = promise.then(function () { // Need this so that the tables are drawn in the correct order (force to be in series)
console.log("looping")
if (grp != "KO"){
var standingsPromise = Standings(loopAgeList[1], grp);
looppromises.push(standingsPromise);
}
var fixPromise = GetFixtures(loopAgeList[1], grp);
looppromises.push(fixPromise);
return fixPromise;
});
});
return Parse.Promise.all(looppromises);
});
var promises = [statsPromise, oStatsPromise, grpPromise, looppromises];
Parse.Promise.all(promises).then(function(results) {
console.log("here");
});
});
The rewrite can be improved significantly by adopting a couple simple style rules: (1) there's no need to create a resolved promise and then chain to it (in fact, most would consider this an anti-pattern), (2) promises to be run together by iterating an array of operands is the perfect application of array .map (not reduce), (3) most importantly, smaller, testable, promise-returning functions always clears up the mystery.
Putting all that together, the main function can be as simple as this...
function loopOverOnce(agegroup) {
let statsPromise = Stats(agegroup);
let oStatsPromise = getOverallStats();
let grpPromise = GetGroups(agegroup).then(function(groups) {
return getStandingsAndFixturesForGroups(groups, agegroup);
});
return Parse.Promise.all([statsPromise, oStatsPromise, grpPromise]);
}
Let's write getStandingsAndFixturesForGroups. It's only job will be map the groups and aggregate promises to do work on each...
function getStandingsAndFixturesForGroups(groups, agegroup) {
let promises = groups.map(function(group) {
return getStandingsAndFixturesForGroup(group, agegroup);
});
return Parse.Promise.all(promises);
}
Now, getStandingsAndFixturesForGroup, a function to do the async work on a single group, conditionally for part of the work...
function getStandingsAndFixturesForGroup(group, agegroup) {
let promises = (group != "KO")? [ Standings(agegroup, grp) ] : [];
promises.push(GetFixtures(agegroup, group));
return Parse.Promise.all(promises); // this is your standings promise (conditionally) and fixtures promise
}
Done. I'd test this code in the reverse order that it's presented here.
EDIT The OP also asks how to perform several promises, serially, interspersed with timeouts. Here's my advice.
First, a slightly simpler version of your delay function, which is a good example when it is right to create a new promise (because there's nothing at bottom to call to get one)
function delay(interval) {
return new Promise(function(resolve, reject){
setTimeout(function() {resolve();}, interval);
});
};
And reducing is a good way to build a list of promises, including interspersed delays...
getAgeGroupList().then(function (loopAgeList) {
loopAgeList.reduce(function(promise, agegroup) {
return promise.then(function() {
let promises = [loopOverOnce(agegroup), delay(15000)];
return Promise.all(promises);
});
}, Promise.as());
});
A couple notes: this results in a sequence like loopOverOnce, timeout, loopOverOnce, timeout, ... etc.. If you'd like a timeout first, reverse the order of the little chain in the inner loop:
[ delay(15000), loopOverOnce(agegroup) ]
Final note is that all this could be made even shorter and prettier by adopting ES6 fat arrow syntax for anonymous functions, e.g.
loopAgeList.reduce((promise, agegroup) => {
promise.then(() => Promise.all([loopOverOnce(agegroup), delay(15000)]));
}, Promise.as());
I have re-written it using reduce and seem to have it working. Comments on this would be welcome (i.e. are there any issues with this code).
function loopOverOnce(agegroup) {
var statsPromise = Stats(agegroup);
var oStatsPromise = getOverallStats();
var grpPromise = GetGroups(agegroup).then(function (groups) {
function getStandingsAndFixtures(groups) {
var promise = Parse.Promise.as();
return groups.reduce(function (promise, grp) {
return promise.then(function (result) {
var standingsPromise = Parse.Promise.as();
if (grp != "KO") {
standingsPromise = Standings(agegroup, grp);
}
var fixPromise = GetFixtures(agegroup, grp);
console.log("fixPromise");
return Parse.Promise.all([standingsPromise, fixPromise]);
});
}, promise);
}
var sfPromise = getStandingsAndFixtures(groups).then(function () { console.log("Test1") });
return sfPromise;
});
return Parse.Promise.all([statsPromise, oStatsPromise, grpPromise]).then(function () { console.log("Test2") });
}
getAgeGroupList().then(function (loopAgeList) {
// https://stackoverflow.com/questions/39538473/using-settimeout-on-promise-chain
function delay(t, v) {
return new Promise(function (resolve) {
setTimeout(resolve.bind(null, v), t)
});
}
var promise = Parse.Promise.as();
loopAgeList.reduce(function (promise, agegroup) {
return promise.then(function () {
return delay(15000).then(function () {
return loopOverOnce(agegroup);
});
});
}, promise);
});
The problem is, that you pass a nested array to Promise.all:
var promises = [statsPromise, oStatsPromise, grpPromise, looppromises];
Just flatten that:
var promises = [statsPromise, oStatsPromise, grpPromise, ...looppromises];
// Or
var promises = [statsPromise, oStatsPromise, grpPromise].concat(looppromises);
However you still need to await promise somewhere to ensure that the chain finished its execution, otherwise looppromise will be empty.
All in all it might be better to use async / await to make everything much more readable:
(async function() {
const ageGroups = await getAgeGroupList();
const statsPromise = Stats(ageGroups[1]);
const overallStatsPromise = getOverallStats();
const groups = await GetGroups(ageGroups[1]);
for(const group of groups) {
const [standings, fixtures] = await Promise.all(
Standings(ageGroups[1], group),
GetFixtures(ageGroups[1], group)
);
// Do something with standings & fixtures
}
const [stats, overallStats] = await Promise.all(statsPromise, overallStatsPromise);
// Do whatever you want with stats etc.
})();
I´m using node/ epxress, mysql and bluebird.
I´m currently doing an async database operation after the client request it. Inside the callback of the first database operation I have to perform some calculations first and afterwards doing two more database queries which are required to provide the client the correct result.
My Code is separated into a Controller class, which handles the get/ post request. In the middle a service class for business logic, which talks to a database class which queries in the database.
I´m currently able to do perform the first and second database request.
getVacation(departmentID) {
return departmentDatabase.getVacation(departmentID)
.then(result => [ result, result.map(entry => this.getDateRange(new Date(entry.dateFrom), new Date(entry.dateTo))) ])
.spread(function(result, dateRange){
var mergedDateRange = [].concat.apply([], dateRange);
var counts = {};
mergedDateRange.forEach(function(x) { counts[x] = (counts[x] || 0)+1; });
return [{"vacationRequest": result, "dateRange": dateRange, "countedDateRange": counts}];
})
.then( result => [result, departmentDatabase.countUser(departmentID)])
.spread(function (result, userOfDepartmentCount){
console.log(userOfDepartmentCount);
console.log(result);
//console.log(blocked);
return departmentID; //return just for not running into timeout
})
.catch(err => {
// ...do something with it...
// If you want to propagate it:
return Promise.reject(err);
// Or you can do:
// throw err;
});
}
But when trying to perform the third I´m running into trouble.
For a solution of this problem I read the Bluebird Doc´s, which pointed me to .all() or (even better) .join(). But trying to use either of them didn´t worked for me.
If I try it with .join() is always results in join is not a function, which I find confusing because I can use all other function. I also tried to require
var Promise = require("bluebird");
var join = Promise.join;
But not even this helped.
Currently I just require Bluebird as Promise in my database class.
So here now my entire service class.
'use strict';
var departmentDatabase = require('../database/department');
var moment = require('moment');
class DepartmentService {
constructor() {
}
getVacation(departmentID) {
return departmentDatabase.getVacation(departmentID)
.then(result => [ result, result.map(entry => this.getDateRange(new Date(entry.dateFrom), new Date(entry.dateTo))) ])
.spread(function(result, dateRange){
var mergedDateRange = [].concat.apply([], dateRange);
var counts = {};
mergedDateRange.forEach(function(x) { counts[x] = (counts[x] || 0)+1; });
return [{"vacationRequest": result, "dateRange": dateRange, "countedDateRange": counts}];
})
//THIS DOES NOT WORK
.join(result => [result, departmentDatabase.countUser(departmentID), departmentDatabase.blockedDaysOfResponsible(departmentID)])
.spread(function (result, userOfDepartmentCount, blocked){
console.log(userOfDepartmentCount);
console.log(result);
console.log(blocked);
return departmentID;
})
.catch(err => {
// ...do something with it...
// If you want to propagate it:
return Promise.reject(err);
// Or you can do:
// throw err;
});
}
getDateRange(startDate, stopDate) {
var dateArray = [];
var currentDate = moment(startDate);
while (currentDate <= stopDate) {
dateArray.push(moment(currentDate).format('YYYY-MM-DD'))
currentDate = moment(currentDate).add(1, 'days');
}
return dateArray;
}
}
module.exports = new DepartmentService();
Is someone able to give me an example how to do it right?
EDIT:
Here an example code I´m using inside my databaseCall, to return the db result and the promise
return Promise.using(dbConnection.getConnection(), function (conn) {
return conn.queryAsync(sql, [departmentID])
.then(function (result) {
return result;
})
.catch(function (err) {
return err;
});
});
Promise.join is nice, but it might not suit your situation the best. Promise.all will combine several promises like you have there into a single resolution:
.then(result => Promise.all([result, departmentDatabase.countUser(departmentID), departmentDatabase.blockedDaysOfResponsible(departmentID)])]))
Then you could spread that result (an array) into a function call:
.spread(function(a, b, c) {
console.log("Results", a, b, c);
});
Promise.all takes an array of Promises, waits for all of them to resolve (or reject), and then continues with the results in an ordered array into the subsequent .then (or other promise) clause.
If your looking for a module that makes control flow with promises easier then you might like relign. Promise.all may do the trick here, but if you need the resolved results, then relign.parallel or relign.series might be better for you.
it is a common pattern that we cascade across a list of sources of data with the first success breaking the chain like this:
var data = getData1();
if (!data) data = getData2();
if (!data) data = getData3();
et cetera. if the getDataN() functions are asynchronous, however, it leads us to 'callback hell':
var data;
getData1(function() {
getData2(function () {
getData3(function () { alert('not found'); })
})
});
where the implementations may look something like:
function getData1(callback) {
$.ajax({
url: '/my/url/1/',
success: function(ret) { data = ret },
error: callback
});
}
...with promises I would expect to write something like this:
$.when(getData1())
.then(function (x) { data = x; })
.fail(function () { return getData2(); })
.then(function (x) { data = x; })
.fail(function () { return getData3(); })
.then(function (x) { data = x; });
where the second .then actually refers to the return value of the first .fail, which is itself a promise, and which I understood was chained in as the input to the succeeding chain step.
clearly I'm wrong but what is the correct way to write this?
In most promise libs, you could chain .fail() or .catch() as in #mido22's answer, but jQuery's .fail() doesn't "handle" an error as such. It is guaranteed always to pass on the input promise (with unaltered state), which would not allow the required "break" of the cascade if/when success happens.
The only jQuery Promise method that can return a promise with a different state (or different value/reason) is .then().
Therefore you could write a chain which continues on error by specifying the next step as a then's error handler at each stage.
function getDataUntilAsyncSuccess() {
return $.Deferred().reject()
.then(null, getData1)
.then(null, getData2)
.then(null, getData3);
}
//The nulls ensure that success at any stage will pass straight through to the first non-null success handler.
getDataUntilAsyncSuccess().then(function (x) {
//"success" data is available here as `x`
}, function (err) {
console.log('not found');
});
But in practice, you might more typically create an array of functions or data objects which are invoked in turn with the help of Array method .reduce().
For example :
var fns = [
getData1,
getData2,
getData3,
getData4,
getData5
];
function getDataUntilAsyncSuccess(data) {
return data.reduce(function(promise, fn) {
return promise.then(null, fn);
}, $.Deferred().reject());// a rejected promise to get the chain started
}
getDataUntilAsyncSuccess(fns).then(function (x) {
//"success" data is available here as `x`
}, function (err) {
console.log('not found');
});
Or, as is probably a better solution here :
var urls = [
'/path/1/',
'/path/2/',
'/path/3/',
'/path/4/',
'/path/5/'
];
function getDataUntilAsyncSuccess(data) {
return data.reduce(function(promise, url) {
return promise.then(null, function() {
return getData(url);// call a generalised `getData()` function that accepts a URL.
});
}, $.Deferred().reject());// a rejected promise to get the chain started
}
getDataUntilAsyncSuccess(urls).then(function (x) {
//"success" data is available here as `x`
}, function (err) {
console.log('not found');
});
As a beginner, stumbling across the same problem, I just realized how much simpler this has become with async and await:
The synchronous pattern
var data = getData1();
if (!data) data = getData2();
if (!data) data = getData3();
can now easily be applied to asynchronous code:
let data = await getData1();
if (!data) data = await getData2();
if (!data) data = await getData3();
Just remember to add an async to the function that this code is used in.
I've been using Bluebird a lot recently on a HAPI API development. I've just run into my first real problem, that perhaps my understanding or naivety has me stumped.
The following code is an example of what I am facing:-
var Promise = require('bluebird'),
stuff = require('../stuff');
module.exports = {
getSomething: function(request, reply) {
var p = Promise.resolve();
p = p.then(function() {
return db.find() //etc. etc.
});
p = p.then(function(resultFromPromise) {
//problems begin here
var data = stuff.doSomeReallyLongAndBoringFunction(resultFromPromise);
return data;
});
p.then(function(data) {
//no data here.
});
};
};
I've commented where the problems usually begin. the stuff.doSomeReallyLongAndBoringFunction() returns an object (using more promises concidently) and it's this object I want to access, but //no data here always fires before data returns. stuff.doSomeReallyLongAndBoringFunction() continues to run regardless and completes successfully, but after the code goes async, I don't know how to promise that function's return value back.
Can anyone offer any guidance? Please accept my apologies for any naivety in the question!
Help as always, is appreciated
NB just for clarity, stuff.doSomeReallyLongAndBoringFunction() does not return a Promise itself. Although, I did try return new Promise(reject, resolve) { }); manual wrap. It is simply a function that uses promises itself (successfully) to get data.
Update 1
stuff.doSomeReallyLongAndBoringFunction() is too big to post directly, but it does something like this:-
var Promise = require('bluebird'),
rp = require('request-promise');
module.exports = {
doSomeReallyLongAndBoringFunction: function() {
var p = Promise.resolve();
p = p.then(function() {
return db.find() //etc. etc.
});
p.then(function() {
rp(options).then(function(response){
//get some data from remote location
}).then(function(dataFromService) {
//do some jiggery pokery with said data
var marshalledData = dataFromService;
db.something.create({
Field: 'something'
}).exec(function(err, saved) {
return marshalledData;
});
});
}).catch(function(err) {
});
};
};
Update 2
Thank you Justin for your help. Here is the actual code, perhaps this may help?
Promise.resolve()
.then(function() {
if(typeof utils.intTryParse(place) !== 'number') {
return foursquare.createPlaceFromFoursquare(sso, place, request, reply);
} else {
return { Place: { PlaceId: place }};
}
}).then(function(placeObj) {
console.log('Place set as', placeObj); //always returns undefined, despite function actually completing after async op...
});
If your doSomeReallyLongAndBoringFunction is really running asynchronously, then it doesn't make sense to run it the way you have setup.
Edit - Here's a simple explanation of the way your code looks to be running vs a refactored version. It's been simplified , so you'll need to fill in the relevant sections with your actual implementation.
var Promise = require('bluebird');
function myAsync() {
setTimeout(function(){
return 'done sleeping';
}, 2000);
};
//The way your code is running
Promise.resolve()
.then(function(){
return 'hello';
})
.then(function(done){
console.log(done);
return myAsync(); //your error is here
})
.then(function(done){
console.log(done);
});
//refactored
Promise.resolve()
.then(function(){
return 'hello';
})
.then(function(done){
console.log(done);
return new Promise(function(resolve) {
setTimeout(function(){
resolve('done sleeping');
}, 2000);
});
})
.then(function(done){
console.log(done);
});
just for clarity, stuff.doSomeReallyLongAndBoringFunction() does not return a Promise itself.
And that's your problem. As it does something asynchronous and you want to get its result, it should return a promise. In fact, that's the case for every asynchronous function, especially then callbacks! It should be something like
module.exports = {
doSomeReallyLongAndBoringFunction: function() {
return db.find()
// ^^^^^^
.then(function() {
return rp(options).then(function(response){
// ^^^^^^
//get some data from remote location
}).then(function(dataFromService) {
//do some jiggery pokery with said data
var marshalledData = dataFromService;
return db.something.create({
// ^^^^^^
Field: 'something'
}).execAsyc();
});
}).catch(function(err) {
});
}
};
Your getSomething method has the same issues, and should look like this:
var createPlace = Promise.promisify(foursquare.createPlaceFromFoursquare);
module.exports = {
getSomething: function(request) {
var p;
if (typeof utils.intTryParse(place) !== 'number')
p = createPlace(sso, place, request); // this returns a promise!
else
p = Promise.resolve({Place: {PlaceId: place}});
return p.then(function(placeObj) {
// ^^^^^^
console.log('Place set as', placeObj);
});
}
};
See also these generic rules for promise development.
doSomeReallyLongAndBoringFunction needs to look like this:
doSomeReallyLongAndBoringFunction: function(param) {
var resolver = Promise.defer();
/*
* do some asynchronous task and when you are finished
* in the callback, do this:
*/
resolver.resolve(resultFromAsyncTask);
/*
*
*
*/
return resolver.promise;
}
I have the following code:
someService.fnReturnsPromise()
.then(function () {
return someService.fnReturnsAnotherPromise(someArg);
})
.then(function (resultsOfSecondFn) {
// do stuff with results
});
I feel as if this should work; however, resultsOfSecondFn isn't actually the results, it's the promise itself that I returned. To make it work the way I want, I have to do this:
someService.fnReturnsPromise()
.then(function () {
return someService.fnReturnsAnotherPromise(someArg);
})
.then(function (promiseReturn) {
promiseReturn.then(function (results) {
// do stuff with results
});
});
This is the pseudo-code for fnReturnsAnotherPromise:
someService.fnReturnsAnotherPromise = function (arg1) {
return anotherService.anotherFnThatReturnsPromise(arg1);
};
So really, it's just one extra layer, but a promise is getting returned either way. The code for anotherFnThatReturnsPromise is the simple paradigm of $q.defer(), return dfd.promise with some resolve()s.
Promises like Angular's are Promises/A+ compliant and are guaranteed to recursively assimilate promises. This is exactly to avoid nesting and simplify things like your case and is the point of promises.
So even if you have a promise that returns and a promise that returns a promise you can unwrap it in a single .then call. For example:
var p = $q.when(1); // Promise<Int>
var p2 = $q.when().then(function(){ return p; }); // Promise<Promise<Int>>
var p3 = $q.when().then(function(){ return p2; }); // Promise<Promise<Promise<Int>>>>
p3.then(function(result) {
console.log(result); // Logs 1, and Int and not p2's type
});
Or in your example:
someService.fnReturnsPromise()
.then(function() {
return someService.fnReturnsAnotherPromise(someArg);
})
.then(function(resultsOfSecondFn) {
// do work with results, it is already unwrapped
});
See this comparison with another language for perspective on promises not unwrapping.
you could try it this way:
someService.fnReturnsPromise().then(function () {
someService.fnReturnsAnotherPromise(someArg).then(function (results) {
console.log(results);
});
});
Hope it helps!