I've been trying to understand how Mike Bostock's queue.js works, but I can't see how it manages to work. The part I don't understand is how the code manages to continue executing callbacks. In particular, I am unsure about the pop() method (line 45). From my understanding, the method takes the next unprocessed, deferred function; appends a callback that (potentially) starts the next deferred function in the queue and executes when the immediately popped function finishes; then finally executes said function. My question is: what code executes this callback?
Each deferred function does not actually return anything -- they are expected to execute their final argument as a callback. For example, this will not work
var foo = function(i) {
console.log(i);
return i;
}
var finished = function(error, results) {
console.log(results);
}
queue(2)
.defer(foo, 1)
.defer(foo, 2)
.defer(foo, 3)
.defer(foo, 4)
.awaitAll(finished); // only prints "1" and "2", since foo() doesn't execute callbacks
However, if we modify foo to take a callback,
var foo = function(i, callback) {
console.log(i);
callback(null, i); // first argument is error reason, second is result
}
Then it will, as executing the callback causes queue to continue.
If I understand the code correctly, queue.await() and queue.awaitall() put the callback in the await instance variable, and then this is executed by notify().
Related
I've got some experience in PHP, but I'm starting out with javascript and jquery. I'm working on my first project. I thought that scripting is scripting, and there will be little difference between this and PHP. Well was I wrong. For the first time I saw that something which is first in the code executes last!
Please have a look at this function which is meant to get svg and store them in json object to use as inline svg later
var svgIcons = { "arrow_left": "", "arrow_right":"", } //json object with empty values
this.getIcons = function() {
for (var icon_name in svgIcons) {
if (svgIcons.hasOwnProperty(icon_name)) {
var url=PHP.plugin_url+'/includes/icons/'+icon_name+'.svg';
jQuery.get(url, function(data) {
svgIcons[icon_name]=data;
console.log('iterating');
console.log(svgIcons[icon_name]); //outputs svg
});
}
}
console.log('this should be after iteration');
console.log(svgIcons["arrow_left"]); //empty
}
this.getIcons(); //called at object initialization
But the output is:
this should be after iteration
iterating
#document (and svg inside it)
iterating
#document (and svg inside it)
What is the cause of this change of order? Is it the get() function? How do I avoid situations like this?
jQuery.get is asynchronous. You are iterating inside the callback for an AJAX call, so that gets executed whenever the AJAX call is completed.
AJAX callbacks, setTimeout and setInterval are some asynchronous Javascript functions. Some threads you might find useful:
How does Asynchronous Javascript Execution happen?
Are all javascript callbacks asynchronous? If not, how do I know which are?
Edit: Yes, the function call ends before any of the callback stuff happens. Basically the execution of your JS will be linear, placing functions on a call stack whenever they are called. On the call-stack they are executed one-by-one, line-by-line. However, when one of those lines calls an asynchronous function (like a setTimeout or AJAX), the current execution places the async function on the call-stack and immediately returns to complete itself. So something like:
function myFunc(){
console.log('a');
setTimeout(function(){
console.log('b');
},0)
console.log('c');
}
myFunc();
would always log:
a
c
b
...even though the setTimeout is 0.
So, in your case what must be happening is that you are assigning the AJAX-received data to svgIcons[icon_name] inside the async callback (obviously), while the rest of your code which uses the object svgIcons is in the sequential/normal execution. You either have to move the code that uses the object inside the async callback, or use promises (basically promises are functions that are executed after an async call is completed).
2nd Edit: So, the reason you are not able to set svgIcons[icon_name] inside the callback is related to the things I was mentioning in my comment. When synchronous functions are called, they are placed on top of the current stack and executed right away, before returning to the calling function. So if you called a sync function inside a loop:
function outer(){
function inner(){
console.log(i);
}
for(var i=0;i<3;i++)
inner();
}
outer();
the synchronous inner function would be executed right away inside each loop, and would have access to the current value of i, so it would output 0, 1, 2 (as expected).
If however, inner was asynchronous, e.g
function outer(){
for (var i=0;i<3;i++)
setTimeout(function(){console.log(i)},0);
}
Then you would get 3, 3, 3 as the output!
This is because the loop has already finished, including the final i++.
So now I think you can see the problem with your code. Upto calling jQuery.get you have access to the current value of icon_name, but once we are inside that asynchronous callback, the current value disappears and is replaced by the last value for it, because the loop already completed before any of the callbacks were executed.
Try something like this:
var svgIcons = {}
var props = ["arrow_left","arrow_right"];
this.getIcons = function() {
props.forEach(function(prop){
var url=PHP.plugin_url+'/includes/icons/'+prop+'.svg';
jQuery.get(url, function(data) {
svgIcons[prop]=data;
var fullyLoaded = false;
for(var i=0;i<props.length;i++) {
if(!svgIcons.hasOwnProperty(props[i])){
fullyLoaded = false;
break;
}
else fullyLoaded = true;
} // end for loop
if(fullyLoaded)
callMyFunctionWhereIUseSvgIconsData();
}); //end jQuery.get()
});//end forEach
}
this.getIcons()
This uses the forEach method, which is native to arrays (MDN reference). Inside the function passed to forEach, the first argument is always the current element of the array (which I named as prop). So there is no messy loop or i, and every executing function has access to its own prop property.
Then, inside the AJAX callback, I assign the current prop to the data received, and then loop through all the properties to check if the svgIcons object has received the properties. So fullyLoaded will only evaluate to true once all the callbacks have been executed and the global svgIcons has received all the properties and data. Hence, you can now call the function that uses the object.
Hope this helps, feel free to ask further or let me know if the console throws errors.
Any ajax calls are async therefore it can be run while the ajax call is taking place. If you want to call something after all calls are done then try this:
var svgIcons = { "arrow_left": "", "arrow_right":"", } //json object with empty values
var executing = 0;
this.getIcons = function() {
for (var icon_name in svgIcons) {
//store that this call has started
exectuing = executing + 1;
if (svgIcons.hasOwnProperty(icon_name)) {
var url=PHP.plugin_url+'/includes/icons/'+icon_name+'.svg';
console.log('this will run as you were expecting');
//this ajax call is then started and moves to next iteration
jQuery.get(url, function(data) {
//This is run after the ajax call has returned a response, not in the order of the code
svgIcons[icon_name]=data;
console.log('iterating');
console.log(svgIcons[icon_name]); //outputs svg
//if you want to call a function after evey call is comeplete then ignore the 'executing' part and just call the function here.
//decrement value as this call has finished
executing = executing - 1;
//if all have finished then call the function we want
if(executing === 0){
executeAfter();
}
});
}
}
console.log('this should be after iteration');
console.log(svgIcons["arrow_left"]); //empty
}
this.executeAfter(){
//This will be exectued after all of you ajax calls are complete.
}
this.getIcons(); //called at object initialization
I have a simple function to count the number of rows in a database and return the result. However I am getting the result returned as undefined, even though if I console.log the result it is coming up correctly from within the function. Here is the gist of what I am doing although I have stripped out a lot of code to simplify it.
$('#roomsList').append(getCount(currentRow.roomtype));
function getCount(roomtype) {
var query = "SELECT COUNT(*) FROM fixturesfittings WHERE roomtype = ?;"
localDatabase.transaction(function (trxn) {
trxn.executeSql(query, [propertyid,roomtype],function (transaction, results) {
return results.rows.item(0)["COUNT(*)"];
},errorHandler);
});
}
Can anyone help me?
The problem is localDataBase.transaction and exequteSql are asynchronous functions. They won't have their answer right away, which is why you pass the functions into them. Once they gets an answer, they calls your function, known as a callback. This will happen at a later point in the execution cycle, after getCount is long gone.
So getCount calls localDatabase.transaction, which gets to work but doesn't have anything immediately available, so getCount finishes before the data is available, and so getCount is returning undefined (the default return value in JavaScript functions).
You will probably need to rework your code to something like this:
getCount(function(count) {
$('#roomsList').append(count);
});
function getCount(callback) {
var query = '...';
localDatabase.transaction(function(trxn) {
trxn.exequteSql(query, ... function(transaction, results) {
callback(results);
});
}
}
This is a very common pattern JavaScript, and has lots of pitfalls and oddities to it. It takes some getting used to.
There are two equally serious problems with what you're doing:
Your outer function doesn't return anything. The only return statement in your code is returning from the inner function being passed to localDatabase.transaction.
Something like this:
function myFunction
inner_function = function () {
return 3; # Return from inner function, *not* myFunction
}
}
Even if you were attempting to communicate a value out of your inner function to the outer function and then returning from the outer function, it still wouldn't work; the inner function is being invoked asynchronously. That inner return results... is being invoked at some point in the future, long after getCount has finished executing and returned undefined. You need to pass in some form of callback into getCount; the callback will be invoked later and passed the count. Think of it this way:
function myFunc(callback) {
function inner_function = function(callback) {
// Invoke our callback sometime in the future
setTimeout(1000, callback);
}
// Inner function does some processing at some future time;
// give it a callback to invoke when its processing is done
inner_function(callback);
}
I'm trying to run a jQuery/Javascript function that searches an external JSON file, and returns true if the given variable is found as one of the usernames of the JSON.
This is what I have:
(The given variable player is equal to a)
function loggedIn(player) {
var result = false;
var liurl = "liplay.json";
$.getJSON(liurl, function (json) {
$.each(json, function (key, value) {
if (value.username == player) {
alert();
result = true;
} else {}
});
});
return result;
}
The External JSON file (liplay.json) is like this:
[{"username":"a"},{"username":"q"}]
For some reason, the code always returns false.
I tried running it one step at a time and I found out that for whatever reason, Chrome is running the first few lines of code until $.getJSON(liurl, function(json){, and then skipping until return result;. I can't tell why it's skipping the part where it runs the if/else statement. Any ideas on why this is happening and how to fix it?
The $.getJSON call executes your function in a callback, so you can't return the result since it spins off the call and jumps to the return statement. You will have to refactor things in order to use a callback once the JSON call has finished loading
$.getJSON is an asynchronous method - the function callback that you pass into it is what is executed once the asynchronous method has finished executing, but until then, the rest of the main function continues to execute, and inevitably returns false.
In my app I have the following:
client.on('test', function(req, fn) {
var returnArr = [];
redis.hkeys(req, function (err, replies) {
replies.forEach(function(reply, i) {
if (reply.indexOf('list.') > -1) {
redis.hgetall(reply.substring(5), function(err, r) {
returnArr.push({name:r['name'],index:i});
console.log(returnArr);
});
}
});
console.log(returnArr);
});
console.log(returnArr);
});
For some reason, the second and third logs contain a blank array even though the array is declared once at the beginnning of the event. Any ideas?
EDIT: Sorry, I changed the variable name when I posted it here without thinking. This happens when it's named anything.
Those redis calls are asynchronous. That's why you provide them with callbacks. The code won't work even if you fix the variable name for that reason.
To elaborate: the code in the callback to "hkeys" will be invoked when the data is available. The call will return immediately, however, so your array will have nothing in it at that point.
You cannot wrap asynchronous calls in a function and expect to return a value. It simply won't work.
Instead, the general pattern is to do exactly what the redis API (and virtually everything else in the node.js world; that's kind-of the whole point in fact): give your own function a callback argument to be invoked when appropriate. In your case, it'll be inside the "hgetall" callback that's the last one to be invoked. It should figure out that your results array has as many values in it as there are keys, and so it's time to call the callback passed in to your function.
(I should note that it's not clear what you're trying to do, given that the overall function appears to be a callback to something.)
Another approach would be to use some sort of "promise" pattern, though that's really just a restructuring of the same idea.
edit — the general pattern for an API with a callback would be something like this:
function yourAPI( param1, param2, callback ) {
// ...
some.asynchronousFunction( whatever, function( result ) {
callback( result );
}
}
Now in your case you're making multiple asynchronous service requests, and you'd need to figure out when it's time to invoke the callback. I think you'd probably want to iterate through the "replies" from the call to get the keys and extract the list of ones you want to fetch:
redis.hkeys(req, function (err, replies) {
var keys = [];
replies.forEach(function(reply, i) {
if (reply.indexOf('list.') > -1) {
keys.push( reply.substring(5) );
}
});
keys.forEach( function( key ) {
redis.hgetall(key, function(err, r) {
returnArr.push({name:r['name'],index:i});
if (returnArr.length === keys.length) {
// all values ready
callback( returnArr );
}
});
});
You cannot call your variable return
It is one of a few reserved words that you cannot use in your code as variables.
As Neal suggests don't use javascript reserved words for your variables, here is the list :
https://developer.mozilla.org/en/JavaScript/Reference/Reserved_Words
#Pointy answered this tersely already, but let me explain it a bit more clearly: Those nested functions are not being run in the order you think they are.
Node.js is non-blocking, and uses Javascript's implicit event loop to execute them when ready. Here's your code with line numbers:
/*01*/ client.on('test', function(req, fn) {
/*02*/ var returnArr = [];
/*03*/ redis.hkeys(req, function (err, replies) {
/*04*/ replies.forEach(function(reply, i) {
/*05*/ if (reply.indexOf('list.') > -1) {
/*06*/ redis.hgetall(reply.substring(5), function(err, r) {
/*07*/ returnArr.push({name:r['name'],index:i});
/*08*/ console.log(returnArr);
/*09*/ });
/*10*/ }
/*11*/ });
/*12*/ console.log(returnArr);
/*13*/ });
/*14*/ console.log(returnArr);
/*15*/ });
/*16*/ //Any other code you have after this.
So, what's the order of execution of this thing?
Line 1: Register the event handler for the 'test' event.
Line 16: Start running any other code to be run during this pass through the event loop
Line 2: A 'test' event has been received at some point by the event loop and is now being handled, so returnArr is initialized
Line 3: A non-blocking IO request is performed, and a callback function is registered to execute when the proper event is queued into the event loop.
Line 14-15: The last console.log is executed and this function is finished running, which should end the current event being processed.
Line 4: The request event returns and the callback is executed. The forEach method is one of the few blocking Node.js methods with a callback, so every callback is executed on every reply.
Line 5: The if statement is executed and either ends (goes to line 10) or enters the block (goes to line 6)
Line 6: A non-blocking IO request is performed, adding a new event to the event loop and a new callback to be run when the event comes back.
Line 9: Finishes the registration of the callback.
Line 10: Finishes the if statement
Line 11: Finishes the `forEach callbacks.
Line 12: Executes the second console.log request, which still has nothing in the returnArr
Line 7: One of the events returns and fires the event handler. The returnArr is given the new data.
Line 8: The first console.log is executed. Depending on which event this is, the length of the array will be different. Also the order of the array elements DOES NOT have to match the order of the replies listed in the replies array.
Essentially, you can look at the more deeply nested functions as executing after the entirety of the less-deeply nested functions (because that's what's happening, essentially), regardless of whether the method contains statements after nested non-blocking callback or not.
If this is confusing to you, you can write your callback code in a Continuation Passing Style so it's obvious that everything in the outer function is executed before the inner function, or you can use this nice async library to make your code look more imperative.
This, I think, answers your real question, rather than the one you've entered.
I'm trying to understand JSON, callbacks, etc within JS. From the altered example below, you'll see that I'm in a function callback from $.getJSON. Then, I jump into getSomething() and expect it to alter my result variable. It alters it within the scope of the function, but not when I jump out of that function.
You'll see from the 2 console.log()'s that the first one displays correct, and the second one doesn't. I'm sure the answer to my question has to do with returning variables via. callback, but could someone enlighten me :)
Thanks!
CODE:
$.getJSON('/cart.js', function (cart, textStatus) {
var result = '';
result += 'Sample Stuff';
StackOverflow.getSomething(param1, param2, function(a, b) {
for(j=0; j < b.length; j++) {
if (b.options[j] != 'Default Title') {
if (a.options[j].name.indexOf("Color") > -1) {
result += b.options[j].name;
console.log(result); // <-- It comes out correct (Sample Stuff + b.options...)
}
}
}
});
console.log(result); // <-- It comes out incorrect, just (Sample Stuff)
});
I guess StackOverflow.getSomething() runs an AJAX request? So what is defined inside of it's callback (looping through a and b) is not executed until the AJAX request is finished. What happens is that StackOverflow.getSomething is fired and then console.log(result) at the end of your code is executed immediately afterwards. By that time, the callback of StackOverflow.getSomething hasn't been run yet, and result hasn't been updated yet. Only "Sample stuff" is logged. But when the second console.log is executed in the callback after the AJAX request (of getSomething), result is updated and logged "correctly".
In other words, the execution order would be this
Set result to "Sample stuff"
StackOverflow.getSomething() fires an AJAX request with an attached callback function
console.log(result) logs "Sample stuff"
The ajax callback finishes and fires its callback function. result is updated accordingly by iterating over a and b
The callback function's console.log(result) logs the final value of result