We Keep Coding

JavaScript callback sequence of operations

I am reading a book called You don't know JS: Async and Performance. They give the following example of a problem with nested callbacks and I was wondering if someone could elaborate on the specifics for me.
doA (function(){
doC();
doD(function(){
doF();
})
doE();
});
doB();
According to the author, the code will execute in the order denoted alphabetically. Meaning doA, then doB ... . I may have been able to guess this based on experience, but I am trying to get a better grasp as to exactly why this happens.
Edit: I posted the question because the example in the book didn't make any sense to me and I was hoping to get some clarification. Maybe I should have just said that instead of trying to rescue the author with some explanation. I edited the question a little to try and make that clear.

Is it because the event loop runs for the entire outer "wrapper" first before it starts the inner wrapper?
No. If the order of execution really is A,B,C,D,E,F, then it is because that is how the functions are written to invoke their callbacks. If the functions were written differently, it could just as easily be A,C,D,E,F,B or A,C,D,F,E,B or, it could even just be A,B, if A does not accept a callback function.
All of this speculation...
Here is what I think is happening. The event loop is first created as doA and doB because JavaScript is not really "concerned" with the contents of those lines at first. When JavaScript runs the line doA(function... it then adds the callback function to the end of the event loop placing doC and doD behind doB.
... is more or less nonsense.
Here is a drastically simplified example:
function doA(callback) { callback(); }
doA(function () {
doB()
});
doC();
Here, the order is A,B,C because that is how A is written.
However, if we change doA so it invokes its callback asynchronously...
function doA(callback) { setTimeout(callback); }
... then the order changes completely to A,C,B.
This has nothing to do with JavaScript being "concerned" with any of the inner or outer code, or where JavaScript "chooses" to place the callbacks in the "event loop". These are not real things. JavaScript doesn't "choose" to do anything. It just executes your code. It's entirely about what each function does and whether the function accepts a callback and how that callback is invoked.

Do underscore.js functions, such as _.findWhere(), return results synchronously?

I was using _.findWhere() in underscore.js to find an object containing a specific property from a collection of such objects. For example:
var rules = _.findWhere(rules, {id: ruleId});
if (!rules) {
// do something
}
else {
// do something else
}
Then I started to worry about JavaScript being asynchronous. Do I know that _.findWhere() will finish populating the rules object by the time the second line is executed? Is this something that I need to consider for every method in underscore.js?
I found an answer that says _.each() is synchronous, but I'm not sure how that was determined, nor am I sure if that answer applies for _.findWhere() or other underscore.js functions.

I believe every single function of underscore.js is synchronous, so you don't need to worry about it: every line written after it will be executed after.
Anyway, if it was not synchronous (i.e. asynchronous) it would require a callback, such as
// DISCLAIMER: this is a fictionnal code, it is in fact synchronous
_.findWhere(rules, {id: ruleId}, function done() {
// code to be executed once finished
});
If you're still in doubt, you can just test in your browser's console _.findWhere([{a:true},{a:false}], {a:true}): if you see a result in the console then it was synchronous (note that I'm not familiar with underscore so not sure about my test-example)

They are synchronous.
See for yourself: _.findWhere, _.matches, _.find, _.findIndex, createIndexFinder. There is no instance of setTimeout in those functions.

The vast majority of JavaScript is synchronous unless stated otherwise.
Asynchronous functions will have callbacks or return promises. But again the majority of functions / methods in various libraries that have callbacks are not asynchronous.

Is Twisted's Deferred the same as a Promise in JavaScript?

I started using Twisted in a project that require asynchronous programming and the docs are pretty good.
So my question is, is a Deferred in Twisted the same as a Promise in Javascript? If not, what are the differences?

The answer to your question is both Yes and No depending on why you're asking.
Yes:
Both a Twisted Deferred and a Javascript Promise implement a mechanism for queuing synchronous blocks of code to be run in a given order while being decoupled from other synchronous blocks of code.
No:
So Javascript's Promise is actually more similar to Python's Future, and the airy-fairy way to explain this is to talk about the Promise and the Resolver being combined to make a Deferred, and to state that this affects what you can do with the callbacks.
This is all very well and good in that it's accurate, however it doesn't really make anything any clearer, and without typing thousands of words where I'm almost guaranteed to make a mistake, I'm probably better quoting someone who knows a little something about Python.
Guido van Rossum on Deferreds:
Here's my attempt to explain Deferred's big ideas (and there are a lot
of them) to advanced Python users with no previous Twisted experience.
I also assume you have thought about asynchronous calls before. Just
to annoy Glyph, I am using a 5-star system to indicate the importance
of ideas, where 1 star is "good idea but pretty obvious" and 5 stars
is "brilliant".
I am showing a lot of code snippets, because some ideas are just best
expressed that way -- but I intentionally leave out lots of details,
and sometimes I show code that has bugs, if fixing them would reduce
understanding the idea behind the code. (I will point out such bugs.)
I am using Python 3.
Notes specifically for Glyph: (a) Consider this a draft for a blog
post. I'd be more than happy to take corrections and suggestions for
improvements. (b) This does not mean I am going to change Tulip to a
more Deferred-like model; but that's for a different thread.
Idea 1: Return a special object instead of taking a callback argument
When designing APIs that produce results asynchronously, you find that
you need a system for callbacks. Usually the first design that comes
to mind is to pass in a callback function that will be called when the
async operation is complete. I've even seen designs where if you don't
pass in a callback the operation is synchronous -- that's bad enough
I'd give it zero stars. But even the one-star version pollutes all
APIs with extra arguments that have to be passed around tediously.
Twisted's first big idea then is that it's better to return a special
object to which the caller can add a callback after receiving it. I
give this three stars because from it sprout so many of the other good
ideas. It is of course similar to the idea underlying the Futures and
Promises found in many languages and libraries, e.g. Python's
concurrent.futures (PEP 3148, closely following Java Futures, both of
which are meant for a threaded world) and now Tulip (PEP 3156, using a
similar design adapted for thread-less async operation).
Idea 2: Pass results from callback to callback
I think it's best to show some code first:
class Deferred:
def __init__(self):
self.callbacks = []
def addCallback(self, callback):
self.callbacks.append(callback) # Bug here
def callback(self, result):
for cb in self.callbacks:
result = cb(result)
The most interesting bits are the last two lines: the result of each
callback is passed to the next. This is different from how things work
in concurrent.futures and Tulip, where the result (once set) is fixed
as an attribute of the Future. Here the result can be modified by each
callback.
This enables a new pattern when one function returning a Deferred
calls another one and transforms its result, and this is what earns
this idea three stars. For example, suppose we have an async function
that reads a set of bookmarks, and we want to write an async function
that calls this and then sorts the bookmarks. Instead of inventing a
mechanism whereby one async function can wait for another (which we
will do later anyway :-), the second async function can simply add a
new callback to the Deferred returned by the first one:
def read_bookmarks_sorted():
d = read_bookmarks()
d.addCallback(sorted)
return d
The Deferred returned by this function represents a sorted list of
bookmarks. If its caller wants to print those bookmarks, it must add
another callback:
d = read_bookmarks_sorted()
d.addCallback(print)
In a world where async results are represented by Futures, this same
example would require two separate Futures: one returned by
read_bookmarks() representing the unsorted list, and a separate Future
returned by read_bookmarks_sorted() representing the sorted list.
There is one non-obvious bug in this version of the class: if
addCallback() is called after the Deferred has already fired (i.e. its
callback() method was called) then the callback added by addCallback()
will never be called. It's easy enough to fix this, but tedious, and
you can look it up in the Twisted source code. I'll carry this bug
through successive examples -- just pretend that you live in a world
where the result is never ready too soon. There are other problems
with this design too, but I'd rather call the solutions improvements
than bugfixes.
Aside: Twisted's poor choices of terminology
I don't know why, but, starting with the project's own name, Twisted
often rubs me the wrong way with its choice of names for things. For
example, I really like the guideline that class names should be nouns.
But 'Deferred' is an adjective, and not just any adjective, it's a
verb's past participle (and an overly long one at that :-). And why is
it in a module named twisted.internet?
Then there is 'callback', which is used for two related but distinct
purposes: it is the preferred term used for a function that will be
called when a result is ready, but it is also the name of the method
you call to "fire" the Deferred, i.e. set the (initial) result.
Don't get me started on the neologism/portmanteau that is 'errback',
which leads us to...
Idea 3: Integrated error handling
This idea gets only two stars (which I'm sure will disappoint many
Twisted fans) because it confused me a lot. I've also noted that the
Twisted docs have some trouble explaining how it works -- In this case
particularly I found that reading the code was more helpful than the
docs.
The basic idea is simple enough: what if the promise of firing the
Deferred with a result can't be fulfilled? When we write
d = pod_bay_doors.open()
d.addCallback(lambda _: pod.launch())
how is HAL 9000 supposed to say "I'm sorry, Dave. I'm afraid I can't
do that" ?
And even if we don't care for that answer, what should we do if one of
the callbacks raises an exception?
Twisted's solution is to bifurcate each callback into a callback and
an 'errback'. But that's not all -- in order to deal with exceptions
raised by callbacks, it also introduces a new class, 'Failure'. I'd
actually like to introduce the latter first, without introducing
errbacks:
class Failure:
def __init__(self):
self.exception = sys.exc_info()[1]
(By the way, great class name. And I mean this, I'm not being
sarcastic.)
Now we can rewrite the callback() method as follows:
def callback(self, result):
for cb in self.callbacks:
try:
result = cb(result)
except:
result = Failure()
This in itself I'd give two stars; the callback can use
isinstance(result, Failure) to tell regular results apart from
failures.
By the way, in Python 3 it might be possible to do away with the
separate Failure class encapsulating exceptions, and just use the
built-in BaseException class. From reading the comments in the code,
Twisted's Failure class mostly exists so that it can hold all the
information returned by sys.exc_info(), i.e. exception class/type,
exception instance, and traceback but in Python 3, exception objects
already hold a reference to the traceback.There is some debug stuff
that Twisted's Failure class does which standard exceptions don't, but
still, I think most reasons for introducing a separate class have been
addressed.
But let's not forget about the errbacks. We change the list of
callbacks to a list of pairs of callback functions, and we rewrite the
callback() method again, as follows:
def callback(self, result):
for (cb, eb) in self.callbacks:
if isinstance(result, Failure):
cb = eb # Use errback
try:
result = cb(result)
except:
result = Failure()
For convenience we also add an errback() method:
def errback(self, fail=None):
if fail is None:
fail = Failure()
self.callback(fail)
(The real errback() function has a few more special cases, it can be
called with either an exception or a Failure as argument, and the
Failure class takes an optional exception argument to prevent it from
using sys.exc_info(). But none of that is essential and it makes the
code snippets more complicated.)
In order to ensure that self.callbacks is a list of pairs we must also
update addCallback() (it still doesn't work right when called after
the Deferred has fired):
def addCallback(self, callback, errback=None):
if errback is None:
errback = lambda r: r
self.callbacks.append((callback, errback))
If this is called with just a callback function, the errback will be a
dummy that passes the result (i.e. a Failure instance) through
unchanged. This preserves the error condition for a subsequent error
handler. To make it easy to add an error handler without also handling
a regular resullt, we add addErrback(), as follows:
def addErrback(self, errback):
self.addCallback(lambda r: r, errback)
Here, the callback half of the pair will pass the (non-Failure) result
through unchanged to the next callback.
If you want the full motivation, read Twisted's Introduction to
Deferreds; I'll just end by noting that an errback and substitute a
regular result for a Failure just by returning a non-Failure value
(including None).
Before I move on to the next idea, let me point out that there are
more niceties in the real Deferred class. For example, you can specify
additional arguments to be passed to the callback and errback. But in
a pinch you can do this with lambdas, so I'm leaving it out, because
the extra code for doing the administration doesn't elucidate the
basic ideas.
Idea 4: Chaining Deferreds
This is a five-star idea! Sometimes it really is necessary for a
callback to wait for an additional async event before it can produce
the desired result. For example, suppose we have two basic async
operations, read_bookmarks() and sync_bookmarks(), and we want a
combined operation. If this was synchronous code, we could write:
def sync_and_read_bookmarks():
sync_bookmarks()
return read_bookmarks()
But how do we write this if all operations return Deferreds? With the
idea of chaining, we can do it as follows:
def sync_and_read_bookmarks():
d = sync_bookmarks()
d.addCallback(lambda unused_result: read_bookmarks())
return d
The lambda is needed because all callbacks are called with a result
value, but read_bookmarks() takes no arguments.

Javascript callback functions differences

I would like to know the difference between 2 implementations of callback functions.
This:
$("#button").on('click', function () {
//do something
});
Versus having the function already defined.
$("#button").on('click', btnFunction);
function btnFunction() {
//do something
}
Are there any implications with one compared to another? Performance-wise is one faster?

The first uses an anonymous function and the second does not. There's no difference in both.
See:
Why do you need to invoke an anonymous function on the same line?

Some folks prefer the second form because it gives a function name when using the debugger and tracing, but there are ways to get the same functionality in the first form.
If you are attaching and removing the event handler based on changing conditions, the second form is much easier to maintain, however.

There's no difference at all, and there's no performance issue with neither one of them. The only difference is that in one of them you're defining the callback function as an anonymous function, this way you cannot reuse it.
The other way, where you define it else where and named it and then pass it as a callback, you're defining a function that you can later reuse in another part of your code.
For example: if you want to do something when the document is ready, and then do se exact same thing when some one press a button you can use something like this:
function getData() {
//do something
}
$(function() {
// Call the function once the DOM is ready
getData();
});
// Call the same function when the button is clicked
$("#refresh_button").on('click', getData);

In most cases the first one will be used, called Anonymous Functions
The second one will be used when the function is not only used inlined here, but also needs to be reused somewhere else.
But anyway it could be a personal preference.

The only real difference you could see is that stack trace (if an exception is thrown for example) will be better, i.e. easier to debug, when using the second one.

Just reuse-ability.
In the second case, you could call btnFunction() somewhere else if need be.

wait for async javascript function to return

I am using a function supplied by a third party library. This function takes a callback function as a parameter, but I would like to wait for this callback to be called before continuing. Is there a standard / accepted way to do this?

I am not sure if this is a possible solution for you but you can achieve the desired result by breaking your code into 2 functions.
Suppose this is what you intend to do:
Basically this is your original function:
function origFunc() {
codeBeforeThirdPartyFunc();
ThirdPartyFunc(oldCallBackFunc);
Wait();
codeAfterCallBackFunc();
}
You can modify the code flow with something like:
function newFunc() {
codeBeforeThirdPartyFunc();
ThirdPartyFunc(newCallBackFunc);
}
function newCallBackFunc() {
oldCallBackFunc();
codeAfterCallBackFunc();
}
This will eliminate the wait loop. And as far as I know, busy waiting doesn't work in IE (because ? God only knows)..

Here's another method of loading jQuery asynchronously, which doesn't depend on another script.

I don't know if Sharad's solution would work in all cases (for instance if you function calls are so far chained that you have to pass data as variables instead of parameters). Same problem with global variables. JavaScript just doesn't have a "wait" ability.
Although I had a similar problem and with jQuery, I ended up with a MacGyver type solution that gives you tons of control on when javascript functions execute. I just posted it here as an answer to my own question (but I it's an answer that's not checked - look for my username Emile): How to get a variable returned across multiple functions - Javascript/jQuery