I would like to clarify - when is it correct to use setImmediate for a node callback.
The examples/articles I've studied argue it is best to use setImmediate to ensure a callback is asynchronous. The most common example is where a value might exist in a "cache" e.g.
const getData = function(id,callback) {
const cacheValue = cache[id];
if (cacheValue) {
// DON'T DO THIS ...
return callback(null,cacheValue);
// DO THIS ...
return setImmediate(callback,null,cacheValue);
}
return queryDB(id,function(err,result){
if (err){
return callback(err);
}
return callback(null,result);
});
};
Here's where it gets confusing. Most node examples of error handing in callbacks never seem to call setImmediate. From my example above:
if (err) {
return callback(err);
}
instead of:
if (err) {
return setImmediate(callback,err);
}
I read some arguments that say that setImmediate is not necessary in this case and indeed can impact performance, why is this? Is this example not the same as the example of accessing the cache.
Is it better to be consistent and always use setImmediate? In which case then, why not do the following:
const getData = function(id,callback) {
const cacheValue = cache[id];
if (cacheValue) {
return setImmediate(callback,null,cacheValue);
}
return queryDB(id,function(err,result){
if (err){
return setImmediate(callback,err);
}
return setImmediate(callback,null,result);
});
};
Quick answer
If you're calling the callback synchronously (before the host function has returned), you should use setImmediate(). If you are calling the callback asynchronously (after the function has returned), you do not need it and can call the callback directly.
Longer Answer
When you have an interface that accepts a callback and that callback is at least sometimes called asynchronously (meaning some indefinite time in the future after your function has returned), then it is a good practice to always call it asynchronously, even if the result is known immediately. This is so that the caller of your function and user of the callback will always see a consistent asynchronous interface.
As you seem aware of, a classic example of this would be with a cached result. If the result is in the cache, then the result is known immediately.
There is no law of Javascript-land that one MUST always call the callback asynchronously. Code may work OK if the callback is sometimes called synchronously and sometimes called asynchronously, but it is more subject to bugs caused by how the callback is used. If the callback is always called asynchronously, then the caller is less likely to accidentally create a bug by how they use the callback.
In the case of:
if (err) {
return callback(err);
}
My guess is that this is already in an asynchronous location. For example:
function someFunction(someUrl, callback) {
request(someURL, function(response, body, err) {
// this is already inside an async response
if (err) {
callback(err);
} else {
callback(body);
}
});
}
I read some arguments that say that setImmediate is not necessary in this case and indeed can impact performance, why is this? Is this example not the same as the example of accessing the cache.
In this case, the if (err) is already in an asynchronous callback part of code so there is no need for an additional setImmediate() there. The host function has already returned and thus calling the callback here without setImmediate() is already asynchronous timing. There is no need for an additional setImmediate().
The only time the setImmediate() is needed is when you are still in the synchronous body of the function and calling the callback there would call it before the host function returns, thus making the callback synchronous instead of asynchronous.
Summary
When is setImmediate required in node callback
So, to summarize. You should use setImmediate(callback) when you in code that is executing synchronously before the host function has returned. You do not need to use setImmediate(callback) when the code you are in is already in an asynchronous callback and the host function has already returned.
FYI, this is one reason to exclusively use promises in your asynchronous programming interfaces (instead of plain callbacks) because promises already handle this for you automatically. They guarantee that a .then() handler on a resolved promise will always be called asynchronously, even if the promise is resolved synchronously.
Related
I was unsure how node.js was able to realize what functions where async and which were not and how to create a custom async function.
Say I wanted to create a custom asynchronous function. I would be surprised if just because I called my last argument to the async function callback or cb that it would just know its an async function:
function f(arg1, callback){
//do stuff with arg1
arg1.doStuff()
//call callback
callback(null, arg1.result());
}
I tried something like that and it did not work async. How do you tell node.js that f is actually async?
NOTE: this answer was written in 2014, before the existence of async function, and before Promises gaining popularity. While the same principles apply as well, I would recommend reading on Promises before trying to get your head around how they relate to "traditional" callback-driven async functions.
To create a function that calls its callback asynchronously, you have to use some platform-provided async primitive (typically IO-related) on it - timers, reading from the filesystem, making a request etc.
For example, this function takes a callback argument, and calls it 100ms after:
function asyncFn(callback) {
setTimeout(() => {
callback();
}, 100);
}
A possible reason for making a function async when it doesn't need to be, is for API consistency. For example, suppose you have a function that makes a network request, and caches the result for later calls:
var cache = null;
function makeRequest(callback) {
if (!cache) {
makeAjax(result => {
cache = result;
callback(result);
});
} else {
callback(cache);
}
}
The problem is, this function is inconsistent: sometimes it is asynchronous, sometimes it isn't. Suppose you have a consumer like this:
makeRequest(result => doSomethingWithResult(result));
doSomethingElse();
The doSomethingElse function may run before or after the doSomethingWithResult function, depending on whether the result was cached or not. Now, if you use an async primitive on the makeRequest function, such as process.nextTick:
var cache = null;
function makeRequest(callback) {
if(!cache) {
makeAjax(result => {
cache = result;
callback(result);
});
} else {
process.nextTick(() => callback(cache));
}
}
The call is always async, and doSomethingElse always runs before doSomethingWithResult.
Only native functions (with access to the event loop) are asynchronous. You would need to call one of them to get asynchronity for your callback. See What is a simple example of an asynchronous javascript function?.
If you aren't using any, there's hardly a reason to make your function asynchronous.
Why does a function called after my promise execute before the promise's callback?
I read this in MDN, but didn't understand it
"Callbacks will never be called before the completion of the current
run of the JavaScript event loop."
I thought it meant that if I have any other statements after resolve() or reject() they will get executed before the callback is invoked. Though, that seems to be an incomplete understanding.
function myFunction() {
return new Promise( function(resolve, reject) {
const err = false;
if(err) {
reject("Something went wrong!!!");
}
else {
resolve("All good");
}
});
}
myFunction().then(doSuccess).catch(doError);
doOther();
function doError(err) {
console.log(err);
}
function doSuccess() {
console.log('Success');
}
function doOther() {
console.log("My Other Function");
}
Output:
My Other Function
Success
By specification, a promise .then() or .catch() callback is never called synchronously, but is called on a future tick of the event loop. That means that the rest of your synchronous code always runs before any .then() handler is called.
So, thus your doOther() function runs before either doSuccess() or doError() are called.
Promises are designed this way so that a promise .then() handler will be called with consistent timing whether the promise is resolved immediately or resolved some time in the future. If synchronous .then() handlers were allowed, then calling code would either have to know when it might get called synchronously or you'd be susceptible to weird timing bugs.
In the Promises/A+ specification which the promises in the ES6 specification were based on, it defines a `.then() handler like this:
promise.then(onFulfilled, onRejected)
and then has this to say about it:
2.2.4. onFulfilled or onRejected must not be called until the execution context stack contains only platform code. [3.1].
And, then it defines platform code like this:
Here “platform code” means engine, environment, and promise implementation code. In practice, this requirement ensures that onFulfilled and onRejected execute asynchronously, after the event loop turn in which then is called, and with a fresh stack. This can be implemented with either a “macro-task” mechanism such as setTimeout or setImmediate, or with a “micro-task” mechanism such as MutationObserver or process.nextTick. Since the promise implementation is considered platform code, it may itself contain a task-scheduling queue or “trampoline” in which the handlers are called.
Basically what this means is that .then() handlers are called by inserting a task in the event loop that will not execute until the currently running Javascript finishes and returns control back to the interpreter (where it can retrieve the next event). So, thus any synchronous Javascript code you have after the .then() handler is installed will always run before the .then() handler is called.
I had similar confusions about what it means by executing after the current loop. I was going through MDN docs on Promises, which says the following:
Callbacks added with then() will never be invoked before the completion of the current run of the JavaScript event loop.
This website http://latentflip.com/loupe/ video expains it pretty well, basically api's like setTimeout execute after the inbuilt js functions. But, the problem with callbacks is, if it doesn't use those apis then it might execute before the finish of current run of the event loop. Following examples show the difference:
Old school callbacks
var foo = function(then1) {
console.log("initial");
var i = 0;
while (i < 1000000000) {
i++;
}
then1();
}
function then11() {
console.log("res");
}
foo(then11);
console.log("end"); // unlike Promises, end is printed last
New Promise
var promise = new Promise(function(resolve, reject) {
console.log("Initial");
//while loop to 10000
var i = 0;
while(i<1000000000) { //long async task without ext apis
i++;
}
resolve("res");
});
promise.then(function(result) {
console.log(result); // "Stuff worked!"
});
console.log("end"); // end is printed before res!!!
I am confused on whether I should call done() after a function completes execution or return. I understand that calling done will mean that I have to pass it as a parameter to the function. What instances will one opt for calling return rather than done()?
i.e.
var foo = 2;
it('returns 2 on completion', function(done) {
expect(foo).toEqual(2);
done();
});
or
var foo = 2;
it('returns 2 on completion', function() {
expect(foo).toEqual(2);
return;
})
Whether you use a done() callback or simply return depends on the API you're using. A done() callback is necessary in a variety of asynchronous contexts. Off the top of my head:
Tests, like your example
Express middleware (they call it next() instead of done())
Methods in the async library (they call it callback() instead of done())
In all of those contexts, the done() callback is necessary because they need to do work that can't all finish before the return. For example, tests might include random timeouts, Express middleware might make network requests, etc.
Important note: every function returns. The done() callback is a convention you use on top of that when return isn't enough because it happens too soon.
I want to create a function, let's say a readConfig function.
readConfig(path [, callback(err, config)])
The callback here is optional. If there is no callback when called, the function will only use synchronous methods (readFile(path) and configParser(string) which return promises) to read the file and directly return the parsed config object. Exceptions are thrown directly. However when callback is provided, the function will use asynchronous methods and call the callback after finished. Exceptions are directly raised to callback but not thrown.
There are many similar codes to do sync and async work, so I want to use one method for both. How can a promise like function detect an async or sync call according to the caller's callback argument? And how can we ensure that the promise will act in synchronous way? Please show me a pattern for that.
I can't speak to the promise acting in a synchronous way, but in JavaScript you can do arguments.length to get the number of arguments passed to a function. If that equals 2 and typeof arguments[1] === 'function', then you have a second argument that is a function.
As for personal taste, I'd recommend just having 2 different functions, one of them ending in Sync, à la http://nodejs.org/api/fs.html. This module is replete with function pairs where one is async and the other isn't.
Also, remember that this is not async code:
function notAsync(cb) {
cb(null)
}
That callback executes in the same tick. You need to wrap it like so:
function async(cb) {
process.nextTick(function(){ cb(null)})
}
or something like that.
Whatever you decide, happy coding.
In this example, I'm using jQuery's deferred api. I wrap the callback in .when which allows you to attach a .then() to the callback, whether it is synchronous or asynchronous with a promise.
The solution should be to use your promise api's when. http://howtonode.org/promises
Live demo (click).
$('#sync').click(function() {
foo(function() {
});
});
$('#async').click(function() {
foo(function() {
var deferred = new $.Deferred();
setTimeout(function() {
deferred.resolve();
}, 500);
return deferred.promise();
});
});
function foo(callback) {
$.when(callback()).then(function() {
console.log('done!');
});
}
In node.js, is it possible to determine (using a function) whether a method is synchronous or asynchronous?
I'd like to write a function that does the following:
function isSynchonous(methodName) {
//if the method is synchronous, return true. Otherwise, return false.
}
From a language standpoint this is not possible, which I believe llambda's answer proves.
Functions can do things asynchronously but return something synchronously; say, return the number of async tasks that were fired off.
Functions can synchronously return promises... that represent asynchronous information. I would call such a method asynchronous but the language can't really tell that.
In some perverse sense every asynchronous function "returns" something... undefined if nothing else.
From an engineering standpoint:
read the documentation.
If the function accepts a callback, it is likely asynchronous. Look at the function signature.
Read the code.
Use "common sense." If the function does IO and returns a result from IO it must be, in some case, asynchronous. This includes file reading, reading from standard input, saving to a database, and HTTP/AJAX requests. Note streams are often used for this, which represents an asynchronous task, but the callbacks are special.
Furthermore there are functions that mix the two.
function(callback) {
if(ready) {
callback();
}
else {
setTimeout(callback, 5000);
}
}
Arguably this is very evil, and correct practice would be
if(ready) {
process.nextTick(callback);
}
so the function has uniform behavior.
However there is a hacky way to tell if anything asynchronous happened, at least in Node.js. See this discussion.
// untested!! please read the documentation on these functions before using yourself
var work = process._getActiveHandles().length + process._getActiveCallbacks().length;
foo;
var newWork = (process._getActiveHandles().length + process._getActiveCallbacks().length) - work;
if(newWork > 0) {
console.log("asynchronous work took place.");
}
This works because asynchronous work cannot resolve on the same tick, by definition, and because Node.js is single threaded.
You don't necessarily know. A function could even be randomly synchronous or asynchronous.
For example, a function that takes another function could execute that function immediately, or it could schedule to execute it at a later time using setImmediate or nextTick. The function could even randomly choose to call its passed function synchronously or asynchronous, such as:
console.log('Start')
function maybeSynchOrAsync(fun) {
var rand = Math.floor((Math.random() * 2))
if (rand == 0) {
console.log('Executing passed function synchronously')
fun()
console.log('Done.')
} else {
console.log('Executing passed function asynchronously via setImmediate')
setImmediate(fun)
console.log('Done.')
}
}
maybeSynchOrAsync(function () { console.log('The passed function has executed.') });
Further, technically speaking, every function call is synchronous. If you call a function F, and F queues a callback function to be invoked later (using setTimeout or whatever mechanism), the original function F still has a synchronous return value (whether it's undefined, a promise, a thunk, or whatever).
No, that's impossible. The methods aren't just marked synchronous or asynchronous, they either use callbacks or they don't.
Why exactly do you need to know, is the real question.
Having said that; that's possible with the new JS abstractions. Nowadays for the async functions those explicitly defined by a preceding async keyword you can perform a test like;
async function test(){}
var type = Object.getPrototypeOf(test).constructor.name;
console.log(type); // <- 'AsyncFunction'
cool..!
As for the normal functions those happen to return a promise, we must remember that it's just a sync function returning a promise object right away, synchronously. That means you have to check dynamically if a function returns a promise in advance, before invoking the function. I think that takes you to a type level programming adventure which exists in sophisticated languages like Haskell etc.