Lets say I have an synchronous function for example:
var sum = function(x,y){
return x+y
}
I want to call this function in asynchronously. How is that possible? Is the function below be considered an asynchronous function? If this is an asynchronous function then I second function's log should be logged before the first one? I know this is a very simple example which may not be an ideal case for async function. But I just wanted to get this fundamental clear.
function(a, b, c, d, e, f, function(){
console.log(result);
})
{
result = sum(a, b);
result = sum(result, c);
result = sum(result, d);
result = sum(result, e);
result = sum(result, f);
return
)};
function(a, b, function(){
console.log(result);
})
{
result = sum(a, b);
return
)};
Please help me. If this is not correct then please help me in as to How it should be written?
From your comment on Quentin's answer
How to create a function which has some manipulation inside it which is a normal synchronous function but the overall flow doesn't wait for this function to complete rather moves to the next function.
JavaScript, the language, doesn't have a feature to do that. So you look to the environment in which it's running to see if the environment has that feature.
NodeJS does. In fact, it has several of them: setTimeout (which is a bit like what browsers give you), setImmediate, and process.nextTick. For your use case, setImmediate is probably the best choice, but see this question and its answers for more on that. setImmediate looks like this:
setImmediate(functionToCall);
(It's a global, you don't have to require anything.)
That does exactly what your title asks for: An asynchronous call to a synchronous function.
From your comment below:
I just need to create an async function inside which I call a normal synchronous function. When I mean asynchronous function I mean that when I call this function it doesn't obstruct the flow and the flow goes to the next function and this function can do something(like logging) after completion
That's not quite the same thing your question asks. :-)
First off, there are no asynchronous functions (in JavaScript). When you call a function in JavaScript, that function is executed synchronously. So that means you have two choices:
The calling code can use setImmediate to tell Node to call the function asynchronously, or
The function can do that for itself, probably using a nested function.
Here's #1:
function foo() {
console.log("A");
setImmediate(bar);
console.log("B");
}
function bar() {
console.log("C");
}
foo();
which outputs:
A
B
C
Here's #2:
function foo() {
console.log("A");
bar("C");
console.log("B");
}
function bar(arg) {
setImmediate(run);
function run() {
console.log(arg);
}
}
foo();
...which also outputs
A
B
C
Note how "C" was passed as an argument to bar. bar does nothing but schedule the callback to its nested function run. run uses the argument.
How is that possible?
It isn't. Everything it does is a synchronous operation.
You use asynchronous code when dealing with things which are intrinsically asynchronous. This is limited, in JS, almost entirely to event handling.
Is the function below be considered an asynchronous function?
No. It has a callback, but everything it does is synchronous.
I'm not completely sure that I understand what you need, but I believe that what you're looking for is to create concurrent function calls, which means you probably need to run a potentially computationally intensive code, and you don't want it to block other code from executing while this computation runs.
Node.js is usually used in a single-threaded process, which means that only a single piece of code can run at a given time, but there are still some ways to do it.
One way to do it, as stated in #T.J. Crowder 's answer, is to queue your function call with process.nextTick so it'll run on the next cycle of the event-loop. This can help break down intensive computation into iterations and that each iteration will be queued to run on the next event-loop cycle after the last one. This can help in some cases, but not always you can break down computations into iterations that run async from each other. If you're looking for a more detailed explanation into how Javascript runs asynchronously, I suggest reading this book.
There are at least two other ways (that I know of) to achieve some sort of concurrency with Node.js- Workers/Threads, and child processes.
Child processes in short are just another Node.js process that is run by your main application. You can use the native *child_process* module of Node.js to run and manage your child processes if you want to go this way. Look at the official docs for more info.
The third way to do it is to use Workers in a very similar manner to how Web Workers are used in the browser. A Worker is a snippet of code that is run in a different OS thread than your main application. To me it seems that this is the easiest way to achieve concurrency, although some will argue it is not the most robust way of doing it, and that creating child processes is better.
There a very good explanation of how Workers function on MDN. To use Workers in Node.js you will need an external module such as webworker-threads.
Here's a code sample created from your sample with a snippet from their documentation:
var Worker = require('webworker-threads').Worker;
var worker = new Worker(function(){
onmessage = function(event) {
result = sum(event.data.a, event.data.b);
result = sum(result, event.data.c);
result = sum(result, event.data.d);
result = sum(result, event.data.e);
result = sum(result, event.data.f);
postMessage(result);
self.close();
};
});
worker.onmessage = function(event) {
console.log("The result is: " + event.data);
};
worker.postMessage({a: 1, b: 2, c: 3, d: 4, e: 5, f: 6});
// Do some other stuff that will happen while your Worker runs...
With all that said, most people will try to avoid running concurrent code in Node.js. Javascript's async nature makes it easy to write simple and concise back-end code, and usually you would want to keep it that way by not trying to introduce concurrency into your application, unless it is absolutely necessary.
In Javascript (leaving node aside for a moment) An asynchronous function should be called via setTimeout(). SetTimeout will not block the script execution while waiting:
function someThingThatTakesAWhile() {
for (var i = 0; i < someVeryLargeNumber; i++) {
//...
};
}
setTimeout(function () {
someThingThatTakesAWhile();
}, 0);
(haven't tested that, please forgive typos, &c)
But if you're wanting to do this in node, I don't think this is what you really want.
Related
Express documentation Production best practices: performance and reliability says:
Don’t use synchronous functions
Synchronous functions and methods tie up the executing process until
they return. A single call to a synchronous function might return in a
few microseconds or milliseconds, however in high-traffic websites,
these calls add up and reduce the performance of the app. Avoid their
use in production.
So my question is, in the context of node/express, if I have a function that accepts some static value and returns a calculated result (what I would normally consider a "synchronous function"), is it best practice to wrap that function inside a new Promise and resolve the result or does this create any significant unnecessary overhead? For example:
Current:
//inside my index.js
var myArgument = 'some long string';
var myResult = myFunction(myArgument);
function myFunction(myArgument){
var thisResult;
//some calculations
return thisResult;
}
New (and Improved?)
//inside my index.js
(async function() {
var myArgument = 'some long string';
var myResult = await myFunction(myArgument);
});
function myFunction(url) {
return new Promise((resolve, reject) => {
var thisResult;
//some calculations
if(thisResult){
resolve (thisResult);
} else {
reject (null)
}
});
}
Short answer: Nope.
The documentation is talking about not using synchronous versions of functions like readfileSync from nodeJS filesystem or bcrypt.compareSync for example. Synchronous calls block the event loop in nodeJS. So nothing happens while you are waiting for the synchronous call to finish. The whole program is on halt while this one method finishes. This is bad in a single threaded system like nodeJS.
There no reason to wrap functions that are just simple calculations or array manipulations with callbacks or promises.
Its just saying that if there's a library/method that offers synchronous version of the method, try to avoid that method.
Check out: https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
JavaScript execution in Node.js is single threaded, so concurrency
refers to the event loop's capacity to execute JavaScript callback
functions after completing other work. Any code that is expected to
run in a concurrent manner must allow the event loop to continue
running as non-JavaScript operations, like I/O, are occurring.
As an example, let's consider a case where each request to a web
server takes 50ms to complete and 45ms of that 50ms is database I/O
that can be done asynchronously. Choosing non-blocking asynchronous
operations frees up that 45ms per request to handle other requests.
This is a significant difference in capacity just by choosing to use
non-blocking methods instead of blocking methods.
The event loop is different than models in many other languages where
additional threads may be created to handle concurrent work.
Regarding additional overhead with wrapping everything in promises. The answer is still no.
You will experience no difference in
function sum(x,y) {
return x+y
}
const ans = sum(1,2)
console.log(ans) // 3
and
function sum(x,y) {
return Promise.resolve(x+y) // Shorthand for your new Promise
}
sum(1,2).then(ans => {
console.log(ans) //3
})
I am newbie to Node js programming and hence want to understand the core concepts and practises very appropriately. AFAIK node js has non-blocking I/O allowing all disk and other I/O operations running async way while its JS runs in single thread managing the resource and execution paths using Event Loop. As suggested at many places developers are advised to write custom functions/methods using callback pattern e.g.
function processData(inputData, cb){
// do some computation and other stuff
if (err) {
cb(err, null);
}else{
cb(null, result);
}
}
callback = function(err, result){
// check error and handle
// if not error then grab result and do some stuff
}
processData(someData, callback)
// checking whether execution is async or not
console.log('control reached at the end of block');
If I run this code everything runs synchronously printing console message at last. What I believed was that the console message will print first of all followed by execution of processData function code which will in turn invoke the callback function. And if it happens this way, the event loop will be unblocked and would only execute the response to a request when the final response is made ready by 'callback' function.
My question is:- Is the execution sequence alright as per node's nature Or Am I doing something wrong Or missing an important concept?
Any help would be appreciate from you guys !!!
JavaScript (just as pretty much any other language) runs sequentially. If you write
var x = 1;
x *= 2;
x += 1;
you expect the result to be 3, not 4. That would be pretty bad. The same goes with functions. When you write
foo();
bar();
you expect them to run in the very same order. This doesn't change when you nest functions:
var foo = function() {};
var bar = function(clb) {
clb();
foo();
};
var callback = function() {
};
bar(callback);
The expected execution sequence is bar -> callback -> foo.
So the biggest damage people do on the internet is that they put equals sign between asynchronous programming and callback pattern. This is wrong. There are many synchronous functions using callback pattern, e.g. Array.prototype.forEach().
What really happens is that NodeJS has this event loop under the hood. You can tell it to schedule something and run later by calling multiple special functions: setTimeout, setInterval, process.nextTick to name few. All I/O also schedules some things to be done on the event loop. Now if you do
var foo = function() {};
var bar = function(clb) {
procress.nextTick(clb); // <-- async here
foo();
};
var callback = function() {
};
bar(callback);
The expected execution sequence is this bar -> (schedule callback) -> foo and then callback will fire at some point (i.e. when all other queued tasks are processed).
That's pretty much how it works.
You are writing synchronous code all the way down and expecting it to run asynchronously. Just because you are using callbacks does not mean it's an asynchronous operation. You should try using timers, api within your functions to feel node js non blocking asynchronous pattern. I hope The Node.js Event Loop will help you to get started.
I have a question regarding synchronous run of included function. For example, I have the following code.
partitions(dataset, num_of_folds, function(train, test, fold) {
train_and_test(train,test, function(err, results)
{
})
})
where partitions runs num_of_folds times, for deffierent fold it returns different train and test sets. I want to run every iteration of partitions only when train_and_test is finished. How to do so?
Given your previous question: Optimum async flow for cross validation in node.js, I understand that partition is your own function, which splits the dataset into several parts, and then runs the callback (basically train_and_test here) for each of those parts.
Your issue now is that train_and_test is asynchronous, but you want to wait for each invocation to finish (which is signalled by the its own callback being called, I assume) before running the next one.
One trivial solution is do change your code to keep state, and run the next invocation from the callback. For instance:
exports.partitions = function(dataset, numOfPartitions, callback) {
var testSetCount = dataset.length / numOfPartitions;
var iPartition=0;
var iteration = function() {
if (iPartition<numOfPartitions)
{
var testSetStart = iPartition*testSetCount;
var partition = exports.partition(dataset, testSetStart, testSetCount);
callback(partition.train, partition.test, iPartition, iteration);
iPartition++;
}
};
iteration();
};
You'll then need to pass the additional callback down to your asynchronous function:
partitions(dataset, num_of_folds, function(train, test, fold, callback) {
train_and_test(train,test, function(err, results)
{
callback();
})
});
Note that I haven't tested any of the code above.
I want to run every iteration of partitions only when train_and_test is finished. How to do so?
That's entirely up to how partitions calls the callback you're giving it. If partitions calls the callback normally, then it will wait for the callback to finish before proceeding. But if it calls is asynchronously, for instance via nextTick or setTimeout or similar, then the only way you'd be able to tell it to wait would be if it provides a means of telling it that.
If I read your question another way: If train_and_test is asynchronous, and partitions isn't designed to deal with asynchronous callbacks, you can't make it wait.
I am working on this function on Node.js, with the following requirements:
As far as possible, maintain synchronous code flow (so that it is simple to read/follow)
Able to return value, so that it can be passed up to the higher-level function.
If asynchronous function has to be used, there has to be some way to "block" the flow between the asynchronous function and return statement, while at the same time not blocking the program and GUI.
Here is my code:
main.js
var some = require("./some.js");
main_fn = function () {
var result = some.some_fn();
if (result == "good") // do something
};
main_fn();
some.js
exports.some_fn = function () {
var result;
var someInterval = setInterval(function () {
// output to GUI
if (condition1) {
clearInterval(someInterval);
result = "good";
} else if (condition2) {
clearInterval(someInterval);
result = "bad";
}
// else continue with interval
}, 250);
// previous way: deasync
// it will not block event loop, however it will block the GUI ("freeze" the browser)
// require("deasync").loopWhile(function () { return result === undefined; });
// result will be undefined here if no code to "block" in between
// but I need result to be either "good" or "bad"
return result;
};
As you can see from the code, I had tried deasync (https://github.com/abbr/deasync). However, that will block the GUI. Are there any Node.js modules/workaround, something like deasync, that allow me to maintain this code structure (as far as possible) and meet my requirements on top?
It will be even better to find a solution without using native Node.js modules (modules which use C/C++ code, like deasync), as I may need to browserify the program in the future. However, I am fine with hearing any solutions from you. Thanks!
It is fundamentally impossible to "deasync" without blocking.
"Asynchronous" means that the result will be produced at some later time. "Synchronous" means that code execution will continue right now. These two simply do not go together. If a synchronous operation needs to wait for the completion of an asynchronous operation, it must suspend operation until the operation has completed; that means it'll block.
The friendliest syntax without devolving into callback hell is to use Promises. They sort of allow you to write synchronous-ish looking code while delegating the intricacies of asynchronous callbacks into the background.
In node.js, is it possible to determine (using a function) whether a method is synchronous or asynchronous?
I'd like to write a function that does the following:
function isSynchonous(methodName) {
//if the method is synchronous, return true. Otherwise, return false.
}
From a language standpoint this is not possible, which I believe llambda's answer proves.
Functions can do things asynchronously but return something synchronously; say, return the number of async tasks that were fired off.
Functions can synchronously return promises... that represent asynchronous information. I would call such a method asynchronous but the language can't really tell that.
In some perverse sense every asynchronous function "returns" something... undefined if nothing else.
From an engineering standpoint:
read the documentation.
If the function accepts a callback, it is likely asynchronous. Look at the function signature.
Read the code.
Use "common sense." If the function does IO and returns a result from IO it must be, in some case, asynchronous. This includes file reading, reading from standard input, saving to a database, and HTTP/AJAX requests. Note streams are often used for this, which represents an asynchronous task, but the callbacks are special.
Furthermore there are functions that mix the two.
function(callback) {
if(ready) {
callback();
}
else {
setTimeout(callback, 5000);
}
}
Arguably this is very evil, and correct practice would be
if(ready) {
process.nextTick(callback);
}
so the function has uniform behavior.
However there is a hacky way to tell if anything asynchronous happened, at least in Node.js. See this discussion.
// untested!! please read the documentation on these functions before using yourself
var work = process._getActiveHandles().length + process._getActiveCallbacks().length;
foo;
var newWork = (process._getActiveHandles().length + process._getActiveCallbacks().length) - work;
if(newWork > 0) {
console.log("asynchronous work took place.");
}
This works because asynchronous work cannot resolve on the same tick, by definition, and because Node.js is single threaded.
You don't necessarily know. A function could even be randomly synchronous or asynchronous.
For example, a function that takes another function could execute that function immediately, or it could schedule to execute it at a later time using setImmediate or nextTick. The function could even randomly choose to call its passed function synchronously or asynchronous, such as:
console.log('Start')
function maybeSynchOrAsync(fun) {
var rand = Math.floor((Math.random() * 2))
if (rand == 0) {
console.log('Executing passed function synchronously')
fun()
console.log('Done.')
} else {
console.log('Executing passed function asynchronously via setImmediate')
setImmediate(fun)
console.log('Done.')
}
}
maybeSynchOrAsync(function () { console.log('The passed function has executed.') });
Further, technically speaking, every function call is synchronous. If you call a function F, and F queues a callback function to be invoked later (using setTimeout or whatever mechanism), the original function F still has a synchronous return value (whether it's undefined, a promise, a thunk, or whatever).
No, that's impossible. The methods aren't just marked synchronous or asynchronous, they either use callbacks or they don't.
Why exactly do you need to know, is the real question.
Having said that; that's possible with the new JS abstractions. Nowadays for the async functions those explicitly defined by a preceding async keyword you can perform a test like;
async function test(){}
var type = Object.getPrototypeOf(test).constructor.name;
console.log(type); // <- 'AsyncFunction'
cool..!
As for the normal functions those happen to return a promise, we must remember that it's just a sync function returning a promise object right away, synchronously. That means you have to check dynamically if a function returns a promise in advance, before invoking the function. I think that takes you to a type level programming adventure which exists in sophisticated languages like Haskell etc.