I have a question regarding synchronous run of included function. For example, I have the following code.
partitions(dataset, num_of_folds, function(train, test, fold) {
train_and_test(train,test, function(err, results)
{
})
})
where partitions runs num_of_folds times, for deffierent fold it returns different train and test sets. I want to run every iteration of partitions only when train_and_test is finished. How to do so?
Given your previous question: Optimum async flow for cross validation in node.js, I understand that partition is your own function, which splits the dataset into several parts, and then runs the callback (basically train_and_test here) for each of those parts.
Your issue now is that train_and_test is asynchronous, but you want to wait for each invocation to finish (which is signalled by the its own callback being called, I assume) before running the next one.
One trivial solution is do change your code to keep state, and run the next invocation from the callback. For instance:
exports.partitions = function(dataset, numOfPartitions, callback) {
var testSetCount = dataset.length / numOfPartitions;
var iPartition=0;
var iteration = function() {
if (iPartition<numOfPartitions)
{
var testSetStart = iPartition*testSetCount;
var partition = exports.partition(dataset, testSetStart, testSetCount);
callback(partition.train, partition.test, iPartition, iteration);
iPartition++;
}
};
iteration();
};
You'll then need to pass the additional callback down to your asynchronous function:
partitions(dataset, num_of_folds, function(train, test, fold, callback) {
train_and_test(train,test, function(err, results)
{
callback();
})
});
Note that I haven't tested any of the code above.
I want to run every iteration of partitions only when train_and_test is finished. How to do so?
That's entirely up to how partitions calls the callback you're giving it. If partitions calls the callback normally, then it will wait for the callback to finish before proceeding. But if it calls is asynchronously, for instance via nextTick or setTimeout or similar, then the only way you'd be able to tell it to wait would be if it provides a means of telling it that.
If I read your question another way: If train_and_test is asynchronous, and partitions isn't designed to deal with asynchronous callbacks, you can't make it wait.
Related
This question already has answers here:
How to identify if a callback is going to be executed synchronously or asynchronously? [duplicate]
(3 answers)
Closed 3 years ago.
Hi! I am learning about callbacks and I understand that callbacks can be either synchronous or asynchronous.
I was reading about callbacks on https://www.w3schools.com/jquery/jquery_callback.asp and was quite confused.
There is this code:
$("button").click(function(){
$("p").hide("slow", function(){
alert("The paragraph is now hidden");
});
});
Can I check if there is a way to know if the above callback is Synchronous or Asynchronous?
I am guessing it is Synchronous above because it has to wait till the "slow" animation is over before the alert comes up. Is it by default in Javascript or Node.js all callbacks are synchronous unless you do something like setTimeOut or process.nextTick?
You have several questions here, so I'll try to answer them one by one:
Is it by default in Javascript or Node.js all callbacks are synchronous unless you do something like setTimeOut or process.nextTick?
In the browser you can kinda have this as the rule of thumb: Only setTimeout, setInterval, requests and events are asynchronous. Other built-in callbacks (like Array.prototype.map) are synchronous.
On Node.js it's more complicated: e.g. you can do file reading both synchronously and asynchronously. Then you just need to know that the callback is asynchronous by nature.
Can I check if there is a way to know if the above callback is Synchronous or Asynchronous?
Unfotunately without checking the source code of the method/function you're calling you cannot know if it's synchronous or asynchronous.
I am guessing it is Synchronous above because it has to wait till the "slow" animation is over before the alert comes up.
Exactly. You can find this from jQuery's documentation for .hide method:
complete
Type: Function()
A function to call once the animation is complete, called once per matched element.
A callback is a function that will get executed once an Asynchronous process is completed. Let's say we have an object person with the following data.
var person = {
id: 1034
name: 'Julio',
age: 23,
moneyInBank: 0
}
We want to get the the moneyInBank for this person but we don't have access to that information in our local environment. We need to get it from a database in another server, this could take a while depending internet connection, optimization, etc.
getMoneyInBank(person.id, callback) is going to go over that process and fetch the information we need. If we were to run the next script without the callback.
getMoneyInBank(person.id);
console.log(person.money);
We will get 0 in the output because getMoneyInBank has not finished by the time the log is executed. If we want to make sure we will print meaningful information we need to change our code to something like this.
getMoneyInBank(persion.id, function(){
console.log(person.money);
});
Using a callback the log only is going to be called when getMoneyInBank finishes.
You can easily check:
function isSync(func, ...args){
var isSync = false;
const res = func(...args, r => isSync = true);
if(res instanceof Promise) res.then(r => isSync = true);
return isSync;
}
console.log( isSync([].map.bind([])) );
Short answer:
You need to examine the caller and look for async operations.
Long answer:
A callback is a simple pattern by which a function is passed as a parameter to an higher order function and is called back it or further down the stack.
let mySynchronousFunction = () => 'Hello';
let myHigherOrderFunction = aFunc => {
return (aFunc() || 'Goodbye') + ' world!';
};
As soon as there is an I/O operation that would block the main thread, it instead will "branch out" and continue execution.
If we continue with our previous example, we would observe the following behavior:
let myAsyncFunction = () => {
return http.get('http://some.document')
.then(response => console.log(response)
);
};
myHigherOrderFunction(mySynchronousFunction); // 'Hello world!'
myHigherOrderFunction(myAsyncFunction); // 'Goodbye world!'
What happened here is that the main thread continued all the way until it had to wait for I/O and instead of blocking there, it went to the next instruction and took note of the point at which it needs to go when the I/O operation is no longer blocked.
So the next expression to evaluation in our code is:
return expression
But our expression branched out, as such it returns undefined. So we are left with:
return undefined
This means that our higher order function that was passed an async function got undefined when it called aFunc().
Once the I/O is done, the main thread returns to where it left off which is the function passed to Promise handler then. At this point the execution thread has branched out and is separated from main "thread".
Now for your question
The callback will be synchronous when the higher order function which calls it is calling it synchronously. Inversely if it is called within the context of the execution branch of an asynchronous operation it will be asynchronous.
This mean that you need to examine what the higher order function to which you pass your callback is doing and look for async operations.
In the context of your question let us examine the following code (HOF = Higher order function):
let mySynchronousHOC = aFunc => aFunc('Hello world!');
let myAsyncHOC = aFunc => {
http.get('http://hello.com')
.then(response => aFunc('Goodbye world!')
);
};
myAsyncHOC(msg => {
console.log(msg);
});
mySynchronousHOC(msg => {
console.log(msg);
});
The result of which will be:
'Hello world'
'Goodbye world'
Here we know that myAsyncHOC is asynchronous because it does an I/O operation.
I am not sure "synchronous" and "asynchronous" makes sense in this context. You can talk about sync or async when making ajax requests, which would mean that in a sync request everything else stops until the response is received, while in an asynchronous request something else can happen until the server sends the response.
In your particular case ( https://www.w3schools.com/jquery/jquery_callback.asp ) what happens is this:
in the "slow" case a timeout is set and the callback is called when the time is out
in the second case a timeout is set for hiding the paragraph but the function is called imediately on the next line, before the time is out
this is not sync/async, it is about event programming
I am newbie to Node js programming and hence want to understand the core concepts and practises very appropriately. AFAIK node js has non-blocking I/O allowing all disk and other I/O operations running async way while its JS runs in single thread managing the resource and execution paths using Event Loop. As suggested at many places developers are advised to write custom functions/methods using callback pattern e.g.
function processData(inputData, cb){
// do some computation and other stuff
if (err) {
cb(err, null);
}else{
cb(null, result);
}
}
callback = function(err, result){
// check error and handle
// if not error then grab result and do some stuff
}
processData(someData, callback)
// checking whether execution is async or not
console.log('control reached at the end of block');
If I run this code everything runs synchronously printing console message at last. What I believed was that the console message will print first of all followed by execution of processData function code which will in turn invoke the callback function. And if it happens this way, the event loop will be unblocked and would only execute the response to a request when the final response is made ready by 'callback' function.
My question is:- Is the execution sequence alright as per node's nature Or Am I doing something wrong Or missing an important concept?
Any help would be appreciate from you guys !!!
JavaScript (just as pretty much any other language) runs sequentially. If you write
var x = 1;
x *= 2;
x += 1;
you expect the result to be 3, not 4. That would be pretty bad. The same goes with functions. When you write
foo();
bar();
you expect them to run in the very same order. This doesn't change when you nest functions:
var foo = function() {};
var bar = function(clb) {
clb();
foo();
};
var callback = function() {
};
bar(callback);
The expected execution sequence is bar -> callback -> foo.
So the biggest damage people do on the internet is that they put equals sign between asynchronous programming and callback pattern. This is wrong. There are many synchronous functions using callback pattern, e.g. Array.prototype.forEach().
What really happens is that NodeJS has this event loop under the hood. You can tell it to schedule something and run later by calling multiple special functions: setTimeout, setInterval, process.nextTick to name few. All I/O also schedules some things to be done on the event loop. Now if you do
var foo = function() {};
var bar = function(clb) {
procress.nextTick(clb); // <-- async here
foo();
};
var callback = function() {
};
bar(callback);
The expected execution sequence is this bar -> (schedule callback) -> foo and then callback will fire at some point (i.e. when all other queued tasks are processed).
That's pretty much how it works.
You are writing synchronous code all the way down and expecting it to run asynchronously. Just because you are using callbacks does not mean it's an asynchronous operation. You should try using timers, api within your functions to feel node js non blocking asynchronous pattern. I hope The Node.js Event Loop will help you to get started.
Is there some way to make a number of synchronous ajax call in the same time? Lets say I have a number of ajax calls that have no dependency of each other. I want them to run in the same time but I can't use asynchronous calls because of some reason.
No, there is no way. Either you run them synchronously, one after another, or asynchronously, all at the same time :) What you want is absolute opposite and it can't be done, sorry :(
What I think you want - is to make the ajax calls in parallel and then make another action.
So you need to use the Async operations in parallel option - sync.js
Async operations in parallel -
http://alexeypetrushin.github.io/synchronize/docs/index.html
var sync = require('synchronize')
function read(a, cb) {
setTimeout(function(){
cb(null, a)
}, 1000)
}
// Runs in parallel
sync.fiber(function() {
sync.parallel(function() {
// You must call defer() manually within
// a parallel operation.
read(1, sync.defer())
read(2, sync.defer())
});
var results = sync.await()
// results now contains [1,2]
});
You should probably just use asynchronous calls. And do some kind of check to see if they've all been completed before doing what your script needs to do next, if that is the concern here. For example by using a setTimeout() function, or by using a global counter variable that is incremented with each ajax response.
I'm not sure why you can't call it asynchronously but you can try using Web Workers instead
http://www.w3schools.com/html/html5_webworkers.asp
I have this code:
var resources = myFunc();
myFunc2(resources);
The problem is that JavaScript calls myFunc() asynchronous, and then myFunc2(), but I don't have the results of myFunc() yet.
Is there a way to block the first call? Or a way to make this work?
The reason why this code doesn't work represents the beauty and pitfalls of async javascript. It doesn't work because it is not supposed to.
When the first line of code is executed, you have basically told node to go do something and let you know when it is done. It then moves on to execute the next line of code - which is why you don't have the response yet when you get here. For more on this, I would study the event-loop in greater detail. It's a bit abstract, but it might help you wrap your head around control flow in node.
This is where callbacks come in. A callback is basically a function you pass to another function that will execute when that second function is complete. The usual signature for a callback is (err, response). This enables you to check for errors and handle them accordingly.
//define first
var first = function ( callback ) {
//This function would do something, then
// when it is done, you callback
// if no error, hand in null
callback(err, res);
};
//Then this is how we call it
first ( function (err, res) {
if ( err ) { return handleError(err); }
//Otherwise do your thing
second(res)
});
As you might imagine, this can get complicated really quickly. It is not uncommon to end up with many nested callbacks which make your code hard to read and debug.
Extra:
If you find yourself in this situation, I would check out the async library. Here is a great tutorial on how to use it.
myFunc(), if asynchronous, needs to accept a callback or return a promise. Typically, you would see something like:
myFunc(function myFuncCallback (resources) {
myFunc2(resources);
});
Without knowing more about your environment and modules, I can't give you specific code. However, most asynchronous functions in Node.js allow you to specify a callback that will be called once the function is complete.
Assuming that myFunc calls some async function, you could do something like this:
function myFunc(callback) {
// do stuff
callSomeAsyncFunction(callback);
}
myFunc(myFunc2);
This question might be possible duplication. I am a noob to node.js and asynchronous programming palindrome. I have google searched and seen a lot of examples on this, but I still have bit confusion.
OK, from google search what I understand is that all the callbacks are handled asynchronous.
for example, let's take readfile function from node.js api
fs.readFile(filename, [options], callback) // callback here will be handled asynchronously
fs.readFileSync(filename, [options])
var fs = require('fs');
fs.readFile('async-try.js' ,'utf8' ,function(err,data){
console.log(data); })
console.log("hii");
The above code will first print hii then it will print the content of
the file.
So, my questions are:
Are all callbacks handled asynchronously?
The below code is not asynchronous, why and how do I make it?
function compute(callback){
for(var i =0; i < 1000 ; i++){}
callback(i);
}
function print(num){
console.log("value of i is:" + num);
}
compute(print);
console.log("hii");
Are all callbacks handled asynchronously?
Not necessarily. Generally, they are, because in NodeJS their very goal is to resume execution of a function (a continuation) after a long running task finishes (typically, IO operations). However, you wrote yourself a synchronous callback, so as you can see they're not always asynchronous.
The below code is not asynchronous, why and how do I make it?
If you want your callback to be called asynchronously, you have to tell Node to execute it "when it has time to do so". In other words, you defer execution of your callback for later, when Node will have finished the ongoing execution.
function compute(callback){
for (var i = 0; i < 1000; i++);
// Defer execution for later
process.nextTick(function () { callback(i); });
}
Output:
hii
value of i is:1000
For more information on how asynchronous callbacks work, please read this blog post that explains how process.nextTick works.
No, that is a regular function call.
A callback will not be asynchronous unless it is forced to be. A good way to do this is by calling it within a setTimeout of 0 milliseconds, e.g.
setTimeout(function() {
// Am now asynchronous
}, 0);
Generally callbacks are made asynchronous when the calling function involves making a new request on the server (e.g. Opening a new file) and it doesn't make sense to halt execution whilst waiting for it to complete.
The below code is not asynchronous, why and how do I make it?
function compute(callback){
for(var i =0; i < 1000 ; i++){}
callback(i);
}
I'm going to assume your code is trying to say, "I need to do something 1000 times then use my callback when everything is complete".
Even your for loop won't work here, because imagine this:
function compute(callback){
for(var i =0; i < 1000 ; i++){
DatabaseModel.save( function (err, result) {
// ^^^^^^ or whatever, Some async function here.
console.log("I am called when the record is saved!!");
});
}
callback(i);
}
In this case your for loop will execute the save calls, not wait around for them to be completed. So, in your example, you may get output like (depending on timing)
I am called when the record is saved
hii
I am called when the record is saved
...
For your compute method to only call the callback when everything is truely complete - all 1000 records have been saved in the database - I would look into the async Node package, which can do this easily for you, and provide patterns for many async problems you'll face in Node.
So, you could rewrite your compute function to be like:
function compute(callback){
var count = 0
async.whilst(
function() { return count < 1000 },
function(callback_for_async_module) {
DatabaseModel.save( function (err, result) {
console.log("I am called when the record is saved!!");
callback_for_async_module();
count++;
});
},
function(err) {
// this method is called when callback_for_async_module has
// been called 1000 times
callback(count);
);
console.log("Out of compute method!");
}
Note that your compute function's callback parameter will get called sometime after console.log("Out of compute method"). This function is now asynchronous: the rest of the application does not wait around for compute to complete.
You can put every callback call inside a timeout with one milisecond, that way they will be executed first when there are a thread free and all synchron tasks are done, then will the processor work through the stack of timeouts that want to be executet.