I read some questions about this topic here at stackoverflow but none of them seems to answer my doubts.
I know how to create async functions using process.nextTick:
var async_function = function(val, callback){
process.nextTick(function(){
callback(val);
});
};
I've also read about Promise, but how do you write async non-blocking functions without using libraries such as process, Promise, etc?
What's the native way? I thought first on this approach:
var async = function(val, cb) {
cb(val);
}
However, that function blocks:
async(3, function(val) {
console.log(val);
});
console.log(4);
// returns:
3
4
So what's the correct implementation? How could I write async functions without depending on those libraries?
You can use setTimeout - it's native function like a delay.
On Node.js, you need to be using process.nextTick. In the browser, you can use the postMessage hack which is much more efficient than setTimeout(function(){},0).
Here is a cross-browser implementation of the postMessage hack: https://github.com/timoxley/next-tick/blob/master/index.js
Break up your function's work into chunks and use process.nextTick to queue the next chunk once the current one has completed. That way you allow other queued callbacks to be executed between each chunk of work. When all chunks have completed, call the callback that was passed into your function as a parameter.
The above is assuming your function is not calling existing async APIs which would naturally make your function async without doing anything special.
Related
I'm looking to see if there is a way of taking a generic async function, such as AJAX, or web worker on message, and forcing it to behave synchronously.
First off - I really do need this behaviour, I am aware of all the arguments regarding async callbacks vs returns - but in this case, I truly need to wait for a callback.
If there is no generic solution available I'd settle for one that works purely on the Worker.prototype.onmessage as that is the primary use case here.
I'm fairly sure the answer will be "thats not possible" but I wanted to check before surrendering.
I've already looked into promises and generators, I'd prefer not to use a third party library (I'm aware of fiber/futures for Node).
A potential use case:
function main(){
worker = new Worker("someWorker.js")
var result = worker.onmessage(function(event){
return event.data;
})
//wait for worker.onmessage
return result;
}
If you want to preserve synchronous flow of function, the generator function is the only possible solution. You'll send out all async tasks like var result = yield new Worker('someWorker.js').
Outside the function* main you'll get the worker with mainFn.next().value and then you attach onmessage listener which will return resolved data into function* main and the function will be resumed.
The complete generator example looks like this
// write function as generator
function* main () {
var result = yield new Worker("someWorker.js");
return result;
}
// instantiate the generator and get the first yield value
var mainFn = main()
, worker = mainFn.next().value
// set onmessage listener, which is callback, that sends data back to *main()
worker.onmessage = function(event) {
mainFn.next(event.data);
}
This way your main() function will remain almost similar to original form, so you can reuse the original code, but you need to write surrounding logic, which wouldn't be simple and which would use callbacks anyway.
You seem to be aware what generators are and that it's not widely supported in the browsers wild, so if you want run this code on web page, you'll need ES6 transpiler Babel.
I've dealt with JavaScript years ago. So I'm familiar with the term "event" as an function which is automatically executed when a certain event happens. Please correct me in case my definition is wrong.
Now I have to deal with the language again. Try to get an idea from this promise-thing which is new to me. But can't find any good definition for it.
Can anyone give an easy to understand definition of what JavaScript promises are?
More over: They seem to me very similar to events. You define a callback-function which is automatically called when a certain event happens.
What's the difference between JavaScript-events and -promises?
For the first view, they are very similar. In events, you have a callback function, and in Promise you have a callback-function. Even more, technically, you can do almost similar stuff which Promises do only with Events.
Events and Promises both useful for Async code. I will write some abstract code to try explain. For example, you have some async code, and after that, it should alert something.
function someAsyncFunction() {
setTimeout(function() {
// some long async logic here
console.log('What the hell is Promises?');
}, 2000);
}
someAsyncFunction();
Timeout here is Async, because it will run your code not in main tread and run it after some time, you not sure when exactly it happens (ofcouse here it's around 2s. less-or-more).
So now imagine, that you need to do something with result of async action. For example you have function AsyncResultCalculator and you need to run that function. What you will do? You have few choices:
Pass a callback function to async code and run your function AsyncResultCalculator when async code complete it's work
Define some Event (for example 'DoSomethingAfterAsync' event) and trigger that event after async code is finished or failed.
Use promises
All this variants as result, will do only one thing - run your function AsyncResultCalculator. So, why we have 3 different ways to do the same result? Because it's cool! It's just different techniques to do the same stuff, but they change difficulty of your code. Some ways make your code more complicated, some ways make your code larger, some ways make your code more elegant.
So, i think you know how to run callback functions or how to trigger event, but what about Promises? How to use them? Very easy. Let's go back to Async Timeout and look:
function AsyncResultCalculator(res) {
// calculate result of async action!!!
console.log(res + 1);
}
function someAsyncFunction() {
return new Promise(function(resolve, reject) {
setTimeout(function() {
// some long async logic here
resolve(55); // or reject
}, 2000);
}
}
someAsyncFunction().then(AsyncResultCalculator);
So, what the point to use Promises here? Just because it's modern style for async actions. With promises your code will be more elegant and more easy to read for different developers. Also, promises is useful because you can build a CHAIN of function like
someAsyncFunction()
.then(function(){ ... })
.then(function(){ ... });
But, of course, in some cases, it's not bad to use callbacks or events. For example, events is more useful, if you need to catch async result in some different JS file and you don't want to uses promises or callbacks in original file with async action.
All JavaScript language is mix of events, callbacks and promises ;D Use it with wise.
Use events for DOM events or some complicated situations.
Use promises for async actions
Use callbacks for sync actions or where you don't want to use Promises
At their most basic, promises are a bit like event listeners except:
A promise can only succeed or fail once. It cannot succeed or fail twice, neither can it switch from success to failure or vice versa.
If a promise has succeeded or failed and you later add a success/failure callback, the correct callback will be called, even though the event took place earlier.
Check out this JavaScript Promises: an Introduction
Simple example without error handling:
var fs = require('fs');
function read(filename, cb) {
return cb(fs.readFileSync(filename));
}
Would this be the same, performance-wise with regards to the event loop, as:
fs.readFile(filename, cb);
The reason I ask is that some of the async functions in the crypto library still throw errors instead of passing them to the callback. I would like to just make a custom async function that does error handling the way I want it, but am worried about using the Sync versions of the native methods due to the bad reputation they have in the node community for bad performance. So, is it safe to go about this the way I proposed by just making a wrapper function?
No.
The whole point of async is that the actual work, which is done by readFileSync(), happens off the event loop.
It is completely impossible to call a synchronous function and make it async.
You cannot in any way make an asynchronous function suddenly behave synchronously in native node.js (e.g. without creating another process or using third party libraries that do some threading). It just cannot be done.
This does not help you at all:
function read(filename, cb) {
return cb(fs.readFileSync(filename));
}
It just implements a synchronous callback which doesn't really solve any problem. The code is still all synchronous so you get none of the benefits of async I/O. The important async I/O benefit you're after here is that you want the main JS event loop to not wait for the file I/O to happen. You want the node.js event loop to be able to do other things while the file I/O is happening in native code. As long as you use fs.readyFileSync(), this will never be the case, no matter what you put around it.
If you have some async crypto methods that are throwing exceptions synchronously rather than passing the error through to their callback, that is poorly designed code and I'd think twice about even using that code.
But, you can work around it by wrapping the crypto functions in your own try catch handlers that will call the callback if an error is thrown.
Let's suppose you have a crypto function like this:
function badCryptoFunc(arg1, arg2, callback)
You can wrap is like this:
function myCryptoFunc(arg1, arg2, callback) {
try {
badCryptoFunc(arg1, arg2, callback);
} catch(e) {
callback(e);
}
}
If the functions throws asynchronously, then even this won't catch the exception and the code is just bad and can't be safely used as there is no way to catch an async exception in most circumstances before it's called by the system, not by your code.
I'm using NodeJS to walk over a list of files and generate an MD5 hash for each one. Here's how I would normally do this synchronously:
// Assume files is already populated with an array of file objects
for(file in files) {
var currentFile = files[file];
currentFile.md5 = md5(file.path);
}
The problem here is that the MD5 function is asynchronous and actually has a callback function that is runs once the MD5 hash has been generated for the file. Thus, all of my currentFile.md5 variables are just going to be set to undefined.
Once I have gotten all of the MD5 hashes for all of the files I'll need to move onto another function to deal with that information.
How gnarly is the code going to get in order for me to do this asynchronously? What's the cleanest way to accomplish what I want to do? Are there common different approaches that I should be aware of?
To call an async function multiple times, you should make a function and call it in recursion like this.
I have assumed your md5 function has a callback with two params err and result.
var keys = Object.keys(files); // taking all keys in an array.
function fn() {
var currentFile = files[keys.shift()];
md5(currentFile, function (err, result) {
// Use result, store somewhere
// check if more files
if (keys.length) {
fn();
} else {
// done
}
});
}
One great approach is to use async. (Search on npm)
If you want to roll your own
Count the files, put that in a var
Everytime fs opens a file and calls your intermediate callback, compute and store the MD5
Also, decrement that counter.
When counter === 0, call a "final" callback, passing back all the MD5s.
To answer your questions (theoretically), in Javascript world, there are (at the moment) 2 different ways to deal with asynchronous code
Using callbacks. This is the most basic way that people start using Javascript know. However , there are plenty of libraries to help people deal with callback in a less painful way such as async, step. In your particular problem. Assuming that md5 is somehow weirdly asynchronous, you can use https://github.com/caolan/async#parallel to achieve it
Another way is to use promise, there are also plenty of promise-compliant libraries such as q, when. Basically, with a promise you have a nicer way to organize your code flow (IMO). With the problem above you can use when.all to gather the result of md5. However, you need to turn md5 into a promise-compliant function
To avoid "callback hell" you should introduce the world of promises to your Node toolset. I suggest q https://npmjs.org/package/q
Here is a post on SO that can help and give you an idea of the syntax how to use q.js promises to work with multiple asynchronous operations.
You essentially would run all your async functions with defered promises, the .then() chained method would fire when all promises are resolved and the function passed inside then() can process your MD5'd data.
I hope this helps.
What's the best way/library for handling multiple asynchronous callbacks? Right now, I have something like this:
_.each(stuff, function(thing){
async(thing, callback);
});
I need to execute some code after the callback has been fired for each element in stuff.
What's the cleanest way to do this? I'm open to using libraries.
Since you're already using Underscore you might look at _.after. It does exactly what you're asking for. From the docs:
after _.after(count, function)
Creates a version of the function that will only be run after first being called count times. Useful for grouping asynchronous responses, where you want to be sure that all the async calls have finished, before proceeding.
There is a great library called Async.js that helps solve problems like this with many async & flow control helpers. It provides several forEach functions that can help you run callbacks for every item in a an array/object.
Check out:
https://github.com/caolan/async#forEach
// will print 1,2,3,4,5,6,7,all done
var arr = [1,2,3,4,5,6,7];
function doSomething(item, done) {
setTimeout(function() {
console.log(item);
done(); // call this when you're done with whatever you're doing
}, 50);
}
async.forEach(arr, doSomething, function(err) {
console.log("all done");
});
I recommend https://github.com/caolan/async for this. You can use async.parallel to do this.
function stuffDoer(thing) {
return function (callback) {
//Do stuff here with thing
callback(null, thing);
}
}
var work = _.map(stuff, stuffDoer)
async.parallel(work, function (error, results) {
//error will be defined if anything passed an error to the callback
//results will be an unordered array of whatever return value if any
//the worker functions passed to the callback
}
async.parallel() / async.series should suit your requirement. You can provide with a final callback that gets executed when all the REST calls succeed.
async.parallel([
function(){ ... },
function(){ ... }
], callback);
async.series([
function(){ ... },
function(){ ... }
], callback);
Have a counter, say async_count. Increase it by one every time you start a request (inside you loop) and have the callback reduce it by one and check if zero has been reached - if so, all the callbacks have returned.
EDIT: Although, if I were the one writing this, I would chain the requests rather than running them in parallel - in other words, I have a queue of requests and have the callback check the queue for the next request to make.
See my response to a similar question:
Coordinating parallel execution in node.js
My fork() function maintains the counter internally and automatically.