I have this code:
var resources = myFunc();
myFunc2(resources);
The problem is that JavaScript calls myFunc() asynchronous, and then myFunc2(), but I don't have the results of myFunc() yet.
Is there a way to block the first call? Or a way to make this work?
The reason why this code doesn't work represents the beauty and pitfalls of async javascript. It doesn't work because it is not supposed to.
When the first line of code is executed, you have basically told node to go do something and let you know when it is done. It then moves on to execute the next line of code - which is why you don't have the response yet when you get here. For more on this, I would study the event-loop in greater detail. It's a bit abstract, but it might help you wrap your head around control flow in node.
This is where callbacks come in. A callback is basically a function you pass to another function that will execute when that second function is complete. The usual signature for a callback is (err, response). This enables you to check for errors and handle them accordingly.
//define first
var first = function ( callback ) {
//This function would do something, then
// when it is done, you callback
// if no error, hand in null
callback(err, res);
};
//Then this is how we call it
first ( function (err, res) {
if ( err ) { return handleError(err); }
//Otherwise do your thing
second(res)
});
As you might imagine, this can get complicated really quickly. It is not uncommon to end up with many nested callbacks which make your code hard to read and debug.
Extra:
If you find yourself in this situation, I would check out the async library. Here is a great tutorial on how to use it.
myFunc(), if asynchronous, needs to accept a callback or return a promise. Typically, you would see something like:
myFunc(function myFuncCallback (resources) {
myFunc2(resources);
});
Without knowing more about your environment and modules, I can't give you specific code. However, most asynchronous functions in Node.js allow you to specify a callback that will be called once the function is complete.
Assuming that myFunc calls some async function, you could do something like this:
function myFunc(callback) {
// do stuff
callSomeAsyncFunction(callback);
}
myFunc(myFunc2);
Related
I am at a point in my Node code where I need to retrieve 4 or 5 items from a database (MySQL), then do some math on the results and then return the result to the main part of the code which called the function. I can't really move on in the code until I have the data returned from these function or functions.
Everything I read says to NOT create synchronous functions because you take away all the joy and beauty of using Node. But, I literally can't move on with executing my code without the results from the functions.
So, don't I need a synchronous function here? If so, why does it feel so wrong? LOL.
I thought of doing one big outer function that is synchronous which contains the 4 or 5 functions that actually do the work. I can make the nested functions Async and make the outer function (the container) Synchronous.
Any thoughts on this? New to Node and just trying to do this right the first time.
Waiting for several i/o operations to finish before continuing is a common problem, especially with node+databases. I try to do as much
async as possible and only block when the logical flow of the program absolutely demands it. Your idea of "making the nested function async" seems fine.
You can leverage Promise.all to kick off all the async stuff. If you await the results of the promise, it won't go to the next line of code until the Promise.all is done
Here's a bit of code with silly but descriptive function names that I hope proves helpful.
async function processIAmTryingToDoAsAsynchAsPossible(allCommandsToExecute) {
try {
// go get all the data asynchronously and destructure the results. But only move to the next line
// once all the async calls have returned
const [firstResult, secondResult, thirdResult] = await Promise.all(allCommandsToExecute.map(eachCommandToExecute => executeSqlFunction(eachCommandToExecute)));
// assuming it is just math, this is not an i/o operation so it will execute "synchronously"
const finalMathProduct = doMathOnThisStuff(firstResult, secondResult, thirdResult);
// assuming this is an i/o operation and the final function returns something you need
return await functionWeCallAfterGettingTheData(finalMathProduct);
} catch (err) {
// You'll get here if the Promise.all has a reject, if the final function errors out,
// or if you throw an error in one of the other functions.
throw err;
}
}
I am newbie to Node js programming and hence want to understand the core concepts and practises very appropriately. AFAIK node js has non-blocking I/O allowing all disk and other I/O operations running async way while its JS runs in single thread managing the resource and execution paths using Event Loop. As suggested at many places developers are advised to write custom functions/methods using callback pattern e.g.
function processData(inputData, cb){
// do some computation and other stuff
if (err) {
cb(err, null);
}else{
cb(null, result);
}
}
callback = function(err, result){
// check error and handle
// if not error then grab result and do some stuff
}
processData(someData, callback)
// checking whether execution is async or not
console.log('control reached at the end of block');
If I run this code everything runs synchronously printing console message at last. What I believed was that the console message will print first of all followed by execution of processData function code which will in turn invoke the callback function. And if it happens this way, the event loop will be unblocked and would only execute the response to a request when the final response is made ready by 'callback' function.
My question is:- Is the execution sequence alright as per node's nature Or Am I doing something wrong Or missing an important concept?
Any help would be appreciate from you guys !!!
JavaScript (just as pretty much any other language) runs sequentially. If you write
var x = 1;
x *= 2;
x += 1;
you expect the result to be 3, not 4. That would be pretty bad. The same goes with functions. When you write
foo();
bar();
you expect them to run in the very same order. This doesn't change when you nest functions:
var foo = function() {};
var bar = function(clb) {
clb();
foo();
};
var callback = function() {
};
bar(callback);
The expected execution sequence is bar -> callback -> foo.
So the biggest damage people do on the internet is that they put equals sign between asynchronous programming and callback pattern. This is wrong. There are many synchronous functions using callback pattern, e.g. Array.prototype.forEach().
What really happens is that NodeJS has this event loop under the hood. You can tell it to schedule something and run later by calling multiple special functions: setTimeout, setInterval, process.nextTick to name few. All I/O also schedules some things to be done on the event loop. Now if you do
var foo = function() {};
var bar = function(clb) {
procress.nextTick(clb); // <-- async here
foo();
};
var callback = function() {
};
bar(callback);
The expected execution sequence is this bar -> (schedule callback) -> foo and then callback will fire at some point (i.e. when all other queued tasks are processed).
That's pretty much how it works.
You are writing synchronous code all the way down and expecting it to run asynchronously. Just because you are using callbacks does not mean it's an asynchronous operation. You should try using timers, api within your functions to feel node js non blocking asynchronous pattern. I hope The Node.js Event Loop will help you to get started.
Simple example without error handling:
var fs = require('fs');
function read(filename, cb) {
return cb(fs.readFileSync(filename));
}
Would this be the same, performance-wise with regards to the event loop, as:
fs.readFile(filename, cb);
The reason I ask is that some of the async functions in the crypto library still throw errors instead of passing them to the callback. I would like to just make a custom async function that does error handling the way I want it, but am worried about using the Sync versions of the native methods due to the bad reputation they have in the node community for bad performance. So, is it safe to go about this the way I proposed by just making a wrapper function?
No.
The whole point of async is that the actual work, which is done by readFileSync(), happens off the event loop.
It is completely impossible to call a synchronous function and make it async.
You cannot in any way make an asynchronous function suddenly behave synchronously in native node.js (e.g. without creating another process or using third party libraries that do some threading). It just cannot be done.
This does not help you at all:
function read(filename, cb) {
return cb(fs.readFileSync(filename));
}
It just implements a synchronous callback which doesn't really solve any problem. The code is still all synchronous so you get none of the benefits of async I/O. The important async I/O benefit you're after here is that you want the main JS event loop to not wait for the file I/O to happen. You want the node.js event loop to be able to do other things while the file I/O is happening in native code. As long as you use fs.readyFileSync(), this will never be the case, no matter what you put around it.
If you have some async crypto methods that are throwing exceptions synchronously rather than passing the error through to their callback, that is poorly designed code and I'd think twice about even using that code.
But, you can work around it by wrapping the crypto functions in your own try catch handlers that will call the callback if an error is thrown.
Let's suppose you have a crypto function like this:
function badCryptoFunc(arg1, arg2, callback)
You can wrap is like this:
function myCryptoFunc(arg1, arg2, callback) {
try {
badCryptoFunc(arg1, arg2, callback);
} catch(e) {
callback(e);
}
}
If the functions throws asynchronously, then even this won't catch the exception and the code is just bad and can't be safely used as there is no way to catch an async exception in most circumstances before it's called by the system, not by your code.
I'm writing a function that's supposed to grab some JSON from another server, either synchronously or asynchronously depending on how it's invoked. In pseudo-code I have:
function getData(async, callback) {
if (async) {
// Get the data...
return data;
}
else {
// Get the data...
callback(data);
}
}
It seems weird that the same function would either return a value or not... Am I violating any best practices here?
I don't see any point in having a single function that sometimes works async and sometimes not because the calling code HAS to know which it is doing and behave accordingly. May as well just have two functions then, one for sync and one for async that make the writing and understanding of the code much simpler.
In addition, the consequences for the user interface of doing any synchronous ajax call are generally bad so any good user experience design would pretty much always avoid synchronous ajax anyway.
If you were going to have only one function, then I'd suggest you give it an async design. Then the caller could write their code to the async style and it would work either way.
Also, you have your async logic backwards. It needs to call the callback when it's async.
function getData(async, callback) {
if (async) {
// get the data in async fashion
doAjaxCall(function(data) {
callback(data);
});
} else {
// get the data synchronously
callback(data);
}
}
As an interesting reference, the jQuery ajax function is modeled this way. It has an asynchronous design that can be passed an argument to work synchronous in some circumstances.
Probably asked before, but after the serious searching I'm still not able to find a proper solution. Please consider something like this:
function compute() {
asyncCall(args, function(err, result) {
});
/* 'join thread here' */
}
Even though asyncCall is asynchronous I'd like to use the result and return it from the function compute synchronously. asyncCall is a library call and I can't modify it in any way.
How to wait properly for the asynchronous result without setTimeout and watching a conditional variable? This is possible but suboptimal.
not sure how you can really use something that doesn't exist yet, but it's easy enough to return a slot where the result will be:
function compute() {
var rez=[];
asyncCall(args, function(err, result) {
rez[0]=result;
if(rez.onchange){ rez.onchange(result); }
});
/* 'join thread here' */
return rez;
}
now, you can refer to the [0] property of the return, and once the callback comes in, compute()[0] will have the result. It will also fire an event handler you can attach to the returned array that will fire when the data updates inside the callback.
i would use something more formal like a promise or secondary callback, but that's me...
EDIT: how to integrate a callback upstream:
// sync (old and busted):
function render(){
var myView=compute();
mainDiv.innerHTML=myView;
}
//async using my re-modified compute():
function render(){
var that=compute();
that.onchange=function(e){ mainDiv.innerHTML=e; }
}
see how making it wait only added a single wrapper in the render function?
There's no await syntax in browsers that is widely available. Your options are generally limited to Callback patterns or Promises.
NodeJS follows a callback pattern for most async methods.
function someAsyncMethod(options, callback) {
//callback = function(error, data)
// when there is an error, it is the first parameter, otherwise use null
doSomethingAsync(function(){
callback(null, response);
});
}
....
someAsyncMethod({...}, function(err, data) {
if (err) return alert("OMG! FAilZ!");
// use data
});
Another common implementation is promises, such as jQuery's .ajax() method...
var px = $.ajax({...});
px.data(function(data, xhr, status){
//runs when data returns.
});
px.fail(function(err,xhr, status){
//runs when an error occurs
});
Promises are similar to events...
Of the two methods above, the callback syntax tends to be easier to implement and follow, but can lead to deeply nested callback trees, though you can use utility patterns, methods like async to overcome this.