I tried to execute a series of events in my Node.js project with async as follows:
server.js:
var express = require('express');
var async = require('async');
...
app.get('/my_page', function (req, res) {
async.series([
function() { console.log("calling foo()"); },
function() { foo(); },
function() { console.log("foo() done"); },
]);
res.render('my_page', {});
}
but I only get the first console output and it's stuck.. the function()s were function(callback)s before. I thought it was waiting a value to be returned but removing them doesn't change the situation..
What am I doing wrong? (I'm newbie to Node.js)
Thanks for any help,
Please read the documentation. Each function should get callback as a parameter, and should call callback once it's done, thus telling async it's time to move on to the next function. So you should have something like:
function(callback) {
console.log("calling foo()");
callback();
},
If there was an error in one of the functions, call callback with the error as the 1st parameter. If you want res.render('my_page', {}); to be executed only after the last function is executed, wrap it in a function and put it as the 2nd parameter to async.series.
And of course, if non of the functions is asynchronous, you should consider not using async.
Firstly, I'm not sure why you're using async here, since none of the functions you've provided to the async.series function are asynchronous.
async.series is useful for calling multiple asynchronous functions in a row and avoiding callback hell.
var async = require("async");
function asyncA(callback){
// do something
callback(null, "value-a");
}
function asyncB(callback){
// do something else
callback(null, "value-b");
}
asyncA(function(err, valueA){
console.log(valueA); // "value-a"
asyncB(function(err, valueB){
console.log(valueB); // "value-b"
});
});
async.series([asyncA, asyncB], function(err, results){
console.log(results); // ["value-a", "value-b"]
}
I suggest doing some reading on asynchronous functions and callbacks and consider whether you really need the async library. I recommend the You Don't Know JS chapter on asynchronous javascript and the series as a whole :)
Related
I was unsure how node.js was able to realize what functions where async and which were not and how to create a custom async function.
Say I wanted to create a custom asynchronous function. I would be surprised if just because I called my last argument to the async function callback or cb that it would just know its an async function:
function f(arg1, callback){
//do stuff with arg1
arg1.doStuff()
//call callback
callback(null, arg1.result());
}
I tried something like that and it did not work async. How do you tell node.js that f is actually async?
NOTE: this answer was written in 2014, before the existence of async function, and before Promises gaining popularity. While the same principles apply as well, I would recommend reading on Promises before trying to get your head around how they relate to "traditional" callback-driven async functions.
To create a function that calls its callback asynchronously, you have to use some platform-provided async primitive (typically IO-related) on it - timers, reading from the filesystem, making a request etc.
For example, this function takes a callback argument, and calls it 100ms after:
function asyncFn(callback) {
setTimeout(() => {
callback();
}, 100);
}
A possible reason for making a function async when it doesn't need to be, is for API consistency. For example, suppose you have a function that makes a network request, and caches the result for later calls:
var cache = null;
function makeRequest(callback) {
if (!cache) {
makeAjax(result => {
cache = result;
callback(result);
});
} else {
callback(cache);
}
}
The problem is, this function is inconsistent: sometimes it is asynchronous, sometimes it isn't. Suppose you have a consumer like this:
makeRequest(result => doSomethingWithResult(result));
doSomethingElse();
The doSomethingElse function may run before or after the doSomethingWithResult function, depending on whether the result was cached or not. Now, if you use an async primitive on the makeRequest function, such as process.nextTick:
var cache = null;
function makeRequest(callback) {
if(!cache) {
makeAjax(result => {
cache = result;
callback(result);
});
} else {
process.nextTick(() => callback(cache));
}
}
The call is always async, and doSomethingElse always runs before doSomethingWithResult.
Only native functions (with access to the event loop) are asynchronous. You would need to call one of them to get asynchronity for your callback. See What is a simple example of an asynchronous javascript function?.
If you aren't using any, there's hardly a reason to make your function asynchronous.
I'm new to Node.js, so I'm still wrapping my head around asynchronous functions and callbacks. My struggle now is how to return a response after reading data from a file in an asynchronous operation.
My understanding is that sending a response works like this (and this works for me):
app.get('/search', function (req, res) {
res.send("request received");
});
However, now I want to read a file, perform some operations on the data, and then return the results in a response. If the operations I wanted to perform on the data were simple, I could do something like this -- perform them inline, and maintain access to the res object because it's still within scope.
app.get('/search', function (req, res) {
fs.readFile("data.txt", function(err, data) {
result = process(data.toString());
res.send(result);
});
});
However, the file operations I need to perform are long and complicated enough that I've separated them out into their own function in a separate file. As a result, my code looks more like this:
app.get('/search', function (req, res) {
searcher.do_search(res.query);
// ??? Now what ???
});
I need to call res.send in order to send the result. However, I can't call it directly in the function above, because do_search completes asynchronously. And I can't call it in the callback to do_search because the res object isn't in scope there.
Can somebody help me understand the proper way to handle this in Node.js?
To access a variable in a different function, when there isn't a shared scope, pass it as an argument.
You could just pass res and then access both query and send on the one variable within the function.
For the purposes of separation of concerns, you might be better off passing a callback instead.
Then do_search only needs to know about performing a query and then running a function. That makes it more generic (and thus reusable).
searcher.do_search(res.query, function (data) {
res.send(...);
});
function do_search(query, callback) {
callback(...);
}
The existing answers are perfectly valid, you can also use async/await keywords since ES2017. Using your own function:
app.get('/search', async(req, res, next) {
try {
const answer = await searcher.do_search(req.query);
res.send(answer);
}
catch(error) {
return next(error);
}
});
I'm using node.js and the async package.
Here's the code I have:
async.waterfall(
[
function(callback) {
var data = getSomeData();
callback(null, data);
},
function(data, callback) {
someFunctionThatNeedsData(data);
callback(null, 'done');
}
],
function(err, result) {
}
);
getSomeData has an asynchronous HTTP request that grabs some data from a web service. I'd like to wait until I get a response, and then return that data and pass it to someFunctionThatNeedsData.
What I expected was that getSomeData -- including the callback inside of it -- would have to complete before moving on to invoke someFunctionThatNeedsData.
The problem is that, despite using the waterfall function here, data is undefined by the time it gets to someFunctionThatNeedsData.
Additionally, from console.log I can see that the end of getSomeData is reached before the callback inside of getSomeData even begins.
Am I using waterfall incorrectly, or is it just not the right tool here? If it's just not right, what can I use to achieve the desired effect?
Or do I have to resign to having deeply nested callbacks (which, with future work, I will) and have to just mitigate it by extracting inline code into named functions?
getSomeData() has an asynchronous http request that grabs some data from a web service.
This is the issue. The execution flow already continued to the callback and executed it. This is how asynchronous functions work!
You have to pass the callback to getSomeData, which calls it once the HTTP request finished. So yes: You may need to nest the callbacks.
If you have async operation. You don't necessary to use async.waterfall. You could just do that in a promise chain style.
getSomeData().then(function(data)
{
var changeData = changeYourData(data);
return changeData;
}).then(function(changedData)
{
// some more stuff with it. You can keep on forwarding to the next `then`
}).catch(function(err)
{
// if any error throw at any point will get catch here
}).finally(function()
{
// this one will guarantee get call no matter what,
// exactly the same like async.waterfall end of chain callback
});
This example will work with Q, When, and any promise lib that follow standard.
If you need to use async.waterfall (because you could drive it with an Array.map)
You just need to callback in your then
async.waterfall(
[
function(callback) {
// A
getSomeData().then(function(data)
{
callback(null, data);
});
// B - just throw the whole thing in
callback(null , getSomeData());
},
function(data, callback) {
// A
someFunctionThatNeedsData(data);
// B
data.then(function(resolvedData)
{
someFunctionThatNeedsData(resolvedData);
callback(null, 'done');
});
}
],
function(err, result) {
});
Hope this help.
Probably asked before, but after the serious searching I'm still not able to find a proper solution. Please consider something like this:
function compute() {
asyncCall(args, function(err, result) {
});
/* 'join thread here' */
}
Even though asyncCall is asynchronous I'd like to use the result and return it from the function compute synchronously. asyncCall is a library call and I can't modify it in any way.
How to wait properly for the asynchronous result without setTimeout and watching a conditional variable? This is possible but suboptimal.
not sure how you can really use something that doesn't exist yet, but it's easy enough to return a slot where the result will be:
function compute() {
var rez=[];
asyncCall(args, function(err, result) {
rez[0]=result;
if(rez.onchange){ rez.onchange(result); }
});
/* 'join thread here' */
return rez;
}
now, you can refer to the [0] property of the return, and once the callback comes in, compute()[0] will have the result. It will also fire an event handler you can attach to the returned array that will fire when the data updates inside the callback.
i would use something more formal like a promise or secondary callback, but that's me...
EDIT: how to integrate a callback upstream:
// sync (old and busted):
function render(){
var myView=compute();
mainDiv.innerHTML=myView;
}
//async using my re-modified compute():
function render(){
var that=compute();
that.onchange=function(e){ mainDiv.innerHTML=e; }
}
see how making it wait only added a single wrapper in the render function?
There's no await syntax in browsers that is widely available. Your options are generally limited to Callback patterns or Promises.
NodeJS follows a callback pattern for most async methods.
function someAsyncMethod(options, callback) {
//callback = function(error, data)
// when there is an error, it is the first parameter, otherwise use null
doSomethingAsync(function(){
callback(null, response);
});
}
....
someAsyncMethod({...}, function(err, data) {
if (err) return alert("OMG! FAilZ!");
// use data
});
Another common implementation is promises, such as jQuery's .ajax() method...
var px = $.ajax({...});
px.data(function(data, xhr, status){
//runs when data returns.
});
px.fail(function(err,xhr, status){
//runs when an error occurs
});
Promises are similar to events...
Of the two methods above, the callback syntax tends to be easier to implement and follow, but can lead to deeply nested callback trees, though you can use utility patterns, methods like async to overcome this.
What's the best way/library for handling multiple asynchronous callbacks? Right now, I have something like this:
_.each(stuff, function(thing){
async(thing, callback);
});
I need to execute some code after the callback has been fired for each element in stuff.
What's the cleanest way to do this? I'm open to using libraries.
Since you're already using Underscore you might look at _.after. It does exactly what you're asking for. From the docs:
after _.after(count, function)
Creates a version of the function that will only be run after first being called count times. Useful for grouping asynchronous responses, where you want to be sure that all the async calls have finished, before proceeding.
There is a great library called Async.js that helps solve problems like this with many async & flow control helpers. It provides several forEach functions that can help you run callbacks for every item in a an array/object.
Check out:
https://github.com/caolan/async#forEach
// will print 1,2,3,4,5,6,7,all done
var arr = [1,2,3,4,5,6,7];
function doSomething(item, done) {
setTimeout(function() {
console.log(item);
done(); // call this when you're done with whatever you're doing
}, 50);
}
async.forEach(arr, doSomething, function(err) {
console.log("all done");
});
I recommend https://github.com/caolan/async for this. You can use async.parallel to do this.
function stuffDoer(thing) {
return function (callback) {
//Do stuff here with thing
callback(null, thing);
}
}
var work = _.map(stuff, stuffDoer)
async.parallel(work, function (error, results) {
//error will be defined if anything passed an error to the callback
//results will be an unordered array of whatever return value if any
//the worker functions passed to the callback
}
async.parallel() / async.series should suit your requirement. You can provide with a final callback that gets executed when all the REST calls succeed.
async.parallel([
function(){ ... },
function(){ ... }
], callback);
async.series([
function(){ ... },
function(){ ... }
], callback);
Have a counter, say async_count. Increase it by one every time you start a request (inside you loop) and have the callback reduce it by one and check if zero has been reached - if so, all the callbacks have returned.
EDIT: Although, if I were the one writing this, I would chain the requests rather than running them in parallel - in other words, I have a queue of requests and have the callback check the queue for the next request to make.
See my response to a similar question:
Coordinating parallel execution in node.js
My fork() function maintains the counter internally and automatically.