Probably asked before, but after the serious searching I'm still not able to find a proper solution. Please consider something like this:
function compute() {
asyncCall(args, function(err, result) {
});
/* 'join thread here' */
}
Even though asyncCall is asynchronous I'd like to use the result and return it from the function compute synchronously. asyncCall is a library call and I can't modify it in any way.
How to wait properly for the asynchronous result without setTimeout and watching a conditional variable? This is possible but suboptimal.
not sure how you can really use something that doesn't exist yet, but it's easy enough to return a slot where the result will be:
function compute() {
var rez=[];
asyncCall(args, function(err, result) {
rez[0]=result;
if(rez.onchange){ rez.onchange(result); }
});
/* 'join thread here' */
return rez;
}
now, you can refer to the [0] property of the return, and once the callback comes in, compute()[0] will have the result. It will also fire an event handler you can attach to the returned array that will fire when the data updates inside the callback.
i would use something more formal like a promise or secondary callback, but that's me...
EDIT: how to integrate a callback upstream:
// sync (old and busted):
function render(){
var myView=compute();
mainDiv.innerHTML=myView;
}
//async using my re-modified compute():
function render(){
var that=compute();
that.onchange=function(e){ mainDiv.innerHTML=e; }
}
see how making it wait only added a single wrapper in the render function?
There's no await syntax in browsers that is widely available. Your options are generally limited to Callback patterns or Promises.
NodeJS follows a callback pattern for most async methods.
function someAsyncMethod(options, callback) {
//callback = function(error, data)
// when there is an error, it is the first parameter, otherwise use null
doSomethingAsync(function(){
callback(null, response);
});
}
....
someAsyncMethod({...}, function(err, data) {
if (err) return alert("OMG! FAilZ!");
// use data
});
Another common implementation is promises, such as jQuery's .ajax() method...
var px = $.ajax({...});
px.data(function(data, xhr, status){
//runs when data returns.
});
px.fail(function(err,xhr, status){
//runs when an error occurs
});
Promises are similar to events...
Of the two methods above, the callback syntax tends to be easier to implement and follow, but can lead to deeply nested callback trees, though you can use utility patterns, methods like async to overcome this.
Related
I'm using node.js and the async package.
Here's the code I have:
async.waterfall(
[
function(callback) {
var data = getSomeData();
callback(null, data);
},
function(data, callback) {
someFunctionThatNeedsData(data);
callback(null, 'done');
}
],
function(err, result) {
}
);
getSomeData has an asynchronous HTTP request that grabs some data from a web service. I'd like to wait until I get a response, and then return that data and pass it to someFunctionThatNeedsData.
What I expected was that getSomeData -- including the callback inside of it -- would have to complete before moving on to invoke someFunctionThatNeedsData.
The problem is that, despite using the waterfall function here, data is undefined by the time it gets to someFunctionThatNeedsData.
Additionally, from console.log I can see that the end of getSomeData is reached before the callback inside of getSomeData even begins.
Am I using waterfall incorrectly, or is it just not the right tool here? If it's just not right, what can I use to achieve the desired effect?
Or do I have to resign to having deeply nested callbacks (which, with future work, I will) and have to just mitigate it by extracting inline code into named functions?
getSomeData() has an asynchronous http request that grabs some data from a web service.
This is the issue. The execution flow already continued to the callback and executed it. This is how asynchronous functions work!
You have to pass the callback to getSomeData, which calls it once the HTTP request finished. So yes: You may need to nest the callbacks.
If you have async operation. You don't necessary to use async.waterfall. You could just do that in a promise chain style.
getSomeData().then(function(data)
{
var changeData = changeYourData(data);
return changeData;
}).then(function(changedData)
{
// some more stuff with it. You can keep on forwarding to the next `then`
}).catch(function(err)
{
// if any error throw at any point will get catch here
}).finally(function()
{
// this one will guarantee get call no matter what,
// exactly the same like async.waterfall end of chain callback
});
This example will work with Q, When, and any promise lib that follow standard.
If you need to use async.waterfall (because you could drive it with an Array.map)
You just need to callback in your then
async.waterfall(
[
function(callback) {
// A
getSomeData().then(function(data)
{
callback(null, data);
});
// B - just throw the whole thing in
callback(null , getSomeData());
},
function(data, callback) {
// A
someFunctionThatNeedsData(data);
// B
data.then(function(resolvedData)
{
someFunctionThatNeedsData(resolvedData);
callback(null, 'done');
});
}
],
function(err, result) {
});
Hope this help.
I have this code:
var resources = myFunc();
myFunc2(resources);
The problem is that JavaScript calls myFunc() asynchronous, and then myFunc2(), but I don't have the results of myFunc() yet.
Is there a way to block the first call? Or a way to make this work?
The reason why this code doesn't work represents the beauty and pitfalls of async javascript. It doesn't work because it is not supposed to.
When the first line of code is executed, you have basically told node to go do something and let you know when it is done. It then moves on to execute the next line of code - which is why you don't have the response yet when you get here. For more on this, I would study the event-loop in greater detail. It's a bit abstract, but it might help you wrap your head around control flow in node.
This is where callbacks come in. A callback is basically a function you pass to another function that will execute when that second function is complete. The usual signature for a callback is (err, response). This enables you to check for errors and handle them accordingly.
//define first
var first = function ( callback ) {
//This function would do something, then
// when it is done, you callback
// if no error, hand in null
callback(err, res);
};
//Then this is how we call it
first ( function (err, res) {
if ( err ) { return handleError(err); }
//Otherwise do your thing
second(res)
});
As you might imagine, this can get complicated really quickly. It is not uncommon to end up with many nested callbacks which make your code hard to read and debug.
Extra:
If you find yourself in this situation, I would check out the async library. Here is a great tutorial on how to use it.
myFunc(), if asynchronous, needs to accept a callback or return a promise. Typically, you would see something like:
myFunc(function myFuncCallback (resources) {
myFunc2(resources);
});
Without knowing more about your environment and modules, I can't give you specific code. However, most asynchronous functions in Node.js allow you to specify a callback that will be called once the function is complete.
Assuming that myFunc calls some async function, you could do something like this:
function myFunc(callback) {
// do stuff
callSomeAsyncFunction(callback);
}
myFunc(myFunc2);
I have doubts on selecting async.js method for probing several alternatives and stop when the first of them is successful.
For example:
async.probeSeries([
function (callback) {
// try something and call callback
// without arguments - probing fails
callback();
},
function (callback) {
// try something and call callback
// with arguments - probing successful
callback(null, ok);
},
function (callback) {
// will be not executed, because
// the second one is successful
callback();
}
], function (err, result) {
// handle the result returned by the second probe
});
I think that using series and return the result as error way may be a workaround but is there a better way?
Maybe you are looking for detectSeries? It works a littlebit different than your example as it checks values from an array with the same function and then callbacks with one of these values, but maybe you can apply it to your problem.
Btw, this looks like the perfect use case for a promise library like Q, where you'd write
probe1().fail(probe2).fail(probe3).done(resulthandler, errhandler);
Callbacks are more and more a requirement in coding, especially when you think about Node.JS non-blocking style of working. But writing a lot of coroutine callbacks quickly becomes difficult to read back.
For example, imagine something like this Pyramid Of Doom:
// This asynchronous coding style is really annoying. Anyone invented a better way yet?
// Count, remove, re-count (verify) and log.
col.count(quertFilter, function(err, countFiltered) {
col.count(queryCached, function(err, countCached) {
col.remove(query, function(err) {
col.count(queryAll, function(err, countTotal) {
util.log(util.format('MongoDB cleanup: %d filtered and %d cached records removed. %d last-minute records left.', countFiltered, countCached, countTotal));
});
});
});
});
is something we see often and can easily become more complex.
When every function is at least a couple of lines longer, it starts to become feasible to separate the functions:
// Imagine something more complex
function mary(data, pictures) {
// Do something drastic
}
// I want to do mary(), but I need to write how before actually starting.
function nana(callback, cbFinal) {
// Get stuff from database or something
callback(nene, cbFinal, data);
}
function nene(callback, cbFinal, data) {
// Do stuff with data
callback(nini, cbFinal, data);
}
function nini(callback, data) {
// Look up pictures of Jeff Atwood
callback(data, pictures);
}
// I start here, so this story doesn't read like a book even if it's quite straightforward.
nana(nene, mary);
But there is a lot of passing vars around happening all the time. With other functions written in between, this becomes hard to read. The functions itself might be too insignificant on their own to justify giving them their own file.
Use an async flow control library like async. It provides a clean way to structure code that requires multiple async calls while maintaining whatever dependency is present between them (if any).
In your example, you'd do something like this:
async.series([
function(callback) { col.count(queryFilter, callback); },
function(callback) { col.count(queryCached, callback); },
function(callback) { col.remove(query, callback); },
function(callback) { col.count(queryAll, callback); }
], function (err, results) {
if (!err) {
util.log(util.format('MongoDB cleanup: %d filtered and %d cached records removed. %d last-minute records left.',
results[0], results[1], results[3]));
}
});
This would execute each of the functions in series; once the first one calls its callback the second one is invoked, and so on. But you can also use parallel or waterfall or whatever flow matches the flow you're looking for. I find it's much cleaner than using promises.
A different approach to callbacks are promises.
Example: jQuery Ajax. this one might look pretty familiar.
$.ajax({
url: '/foo',
success: function() {
alert('bar');
}
});
But $.ajax also returns a promise.
var request = $.ajax({
url: '/foo'
});
request.done(function() {
alert('bar');
});
A benefit is, that you simulate synchronous behavior, because you can use the returned promise instead of providing a callback to $.ajax.success and a callback to the callback and a callback.... Another advantage is, that you can chain / aggregate promises, and have error handlers for one promise-aggregate if you like.
I found this article to be pretty useful.
It describes the pro and cons of callbacks, promises and other techniques.
A popular implementation (used by e.g. AngularJS iirc) is Q.
Combined answers and articles. Please edit this answer and add libraries/examples/doc-urls in a straightforward fasion for everyone's benefit.
Documentation on Promises
Asynchronous Control Flow with Promises
jQuery deferreds
Asynchronous Libraries
async.js
async.waterfall([
function(){ // ... },
function(){ // ... }
], callback);
node fibers
step
Step(
function func1() {
// ...
return value
},
function func2(err, value) {
// ...
return value
},
function funcFinal(err, value) {
if (err) throw err;
// ...
}
);
Q
Q.fcall(func1)
.then(func2)
.then(func3)
.then(funcSucces, funcError)
API reference
Mode examples
More documentation
What's the best way/library for handling multiple asynchronous callbacks? Right now, I have something like this:
_.each(stuff, function(thing){
async(thing, callback);
});
I need to execute some code after the callback has been fired for each element in stuff.
What's the cleanest way to do this? I'm open to using libraries.
Since you're already using Underscore you might look at _.after. It does exactly what you're asking for. From the docs:
after _.after(count, function)
Creates a version of the function that will only be run after first being called count times. Useful for grouping asynchronous responses, where you want to be sure that all the async calls have finished, before proceeding.
There is a great library called Async.js that helps solve problems like this with many async & flow control helpers. It provides several forEach functions that can help you run callbacks for every item in a an array/object.
Check out:
https://github.com/caolan/async#forEach
// will print 1,2,3,4,5,6,7,all done
var arr = [1,2,3,4,5,6,7];
function doSomething(item, done) {
setTimeout(function() {
console.log(item);
done(); // call this when you're done with whatever you're doing
}, 50);
}
async.forEach(arr, doSomething, function(err) {
console.log("all done");
});
I recommend https://github.com/caolan/async for this. You can use async.parallel to do this.
function stuffDoer(thing) {
return function (callback) {
//Do stuff here with thing
callback(null, thing);
}
}
var work = _.map(stuff, stuffDoer)
async.parallel(work, function (error, results) {
//error will be defined if anything passed an error to the callback
//results will be an unordered array of whatever return value if any
//the worker functions passed to the callback
}
async.parallel() / async.series should suit your requirement. You can provide with a final callback that gets executed when all the REST calls succeed.
async.parallel([
function(){ ... },
function(){ ... }
], callback);
async.series([
function(){ ... },
function(){ ... }
], callback);
Have a counter, say async_count. Increase it by one every time you start a request (inside you loop) and have the callback reduce it by one and check if zero has been reached - if so, all the callbacks have returned.
EDIT: Although, if I were the one writing this, I would chain the requests rather than running them in parallel - in other words, I have a queue of requests and have the callback check the queue for the next request to make.
See my response to a similar question:
Coordinating parallel execution in node.js
My fork() function maintains the counter internally and automatically.