We Keep Coding

Collect data to object with four async calls and handle the object onready

I have a handler (callback), an object to handle and four functions, which collect the data to object. In my case I wish to asynchronously call four data retrievers and when execution of all four is complete, handle the resulting object (something similar to the following):
var data = {};
function handle (jsObj) {}
// data retrieving
function getColorData () {}
function getSizeData () {}
function getWeightData () {}
function getExtraData () {}
data.color = getColorData();
data.size = getSizeData();
data.weight = getWeightData();
data.extra = getExtraData();
handle( data );
Of course, this code will not work properly. And if I chain data retrieving functions, they will be called one after another, right?
All four functions should be called asynchronously, cause they are being executed for too long to call them one by one.
Updated:
Thanks to everybody for your suggestions! I prefered $.Deferred(), but I found it slightly difficult to make it work the way I need. What I need is to asynchronously make a view, which requires four kinds of data (extraData, colorData, sizeData & weightData) and I have three objects: App, Utils & Tools.
Just a small description: view is created by calling App.getStuff passed App.handleStuff as a callback. Callback in the body of App.getStuff is called only $.when(App.getExtraData(), App.getColorData(), App.getSizeData(), App.getWeightData()). Before that Utils.asyncRequest passed Tools.parseResponse as a callback is called.
So, now the question is should I create four deferred objects inside each App.get*Data() and also return deferred.promise() from each of them?
And should I deferred.resolve() in the last function in my order (Tools.parseResponse for App.getExtraData in my example)?
var view,
App,
Utils = {},
Tools = {};
// Utils
Utils.asyncRequest = function (path, callback) {
var data,
parseResponse = callback;
// do something with 'data'
parseResponse( data );
};
// Tools
Tools.parseResponse = function (data) {
var output = {};
// do something to make 'output' from 'data'
/* So, should the deferred.resolve() be done here? */
deferred.resolve(output);
/// OR deferred.resolve();
/// OR return output;
};
// App
App = {
// Only one method really works in my example
getExtraData : function () {
var deferred = new jQuery.Deferred();
Utils.asyncRequest("/dir/data.txt", Tools.parseResponse);
return deferred.promise();
},
// Others do nothing
getColorData : function () { /* ... */ },
getSizeData : function () { /* ... */ },
getWeightData : function () { /* ... */ }
};
App.getStuff = function (callback) {
$.when(
App.getExtraData(),
App.getColorData(),
App.getSizeData(),
App.getWeightData()
)
.then(function (extraData, colorData, sizeData, weightData) {
var context,
handleStuff = callback;
// do something to make all kinds of data become a single object
handleStuff( context );
});
};
App.handleStuff = function (stuff) { /* ... */ };
/// RUN
view = App.getStuff( App.handleStuff );
I did not expect the code in my example above to work, it is for illustrative purposes.
I've been trying to solve this for quiet a long time and it still gives no result. The documentation for jQuery.Deferred() and discussions around this, unfortunately, did not help me. So, I would be very glad and greatful for any help or advise.

Conceptually, you would use a counter that gets incremented as each asynchronous call completes. The main caller should proceed after the counter has been incremented by all the asynchronous calls.

I think what you're looking for are Promises / Deferreds.
With promises you can write something like:
when(getColorData(), getSizeData(), getWeightData(), getExtraData()).then(
function (colorData, sizeData, weightData, extraData) {
handle(/*..*/);
}
)
The get*Data() functions will return a promise that they fulfill when their assynchronous call is complete.
Ex:
function getData() {
var promise = new Promise();
doAjax("getData", { "foo": "bar" }, function (result) {
promise.resolve(result);
});
return promise;
}
The when simply counts the number arguments, if all it's promises are resolved, it will call then with the results from the promises.
jQuery has an OK implementation: http://api.jquery.com/jQuery.when/

What I could suggest for this scenario would be something like that.
write a function like this
var completed = 0;
checkHandler = function() {
if(completed == 4) {
handle(data);
}
}
where completed is the number of positive callbacks you must receive.
As soon as every function receives a callback you can increment the "completed" counter and invoke the checkHandler function. and you're done!
in example
function getColorData() {
$.get('ajax/test.html', function(data) {
completed++;
checkHandler();
});
}

What does $.when.apply($, someArray) do?

I'm reading about Deferreds and Promises and keep coming across $.when.apply($, someArray). I'm a little unclear on what this does exactly, looking for an explanation that one line works exactly (not the entire code snippet). Here's some context:
var data = [1,2,3,4]; // the ids coming back from serviceA
var processItemsDeferred = [];
for(var i = 0; i < data.length; i++){
processItemsDeferred.push(processItem(data[i]));
}
$.when.apply($, processItemsDeferred).then(everythingDone);
function processItem(data) {
var dfd = $.Deferred();
console.log('called processItem');
//in the real world, this would probably make an AJAX call.
setTimeout(function() { dfd.resolve() }, 2000);
return dfd.promise();
}
function everythingDone(){
console.log('processed all items');
}

.apply is used to call a function with an array of arguments. It takes each element in the array, and uses each as a parameter to the function. .apply can also change the context (this) inside a function.
So, let's take $.when. It's used to say "when all these promises are resolved... do something". It takes an infinite (variable) number of parameters.
In your case, you have an array of promises; you don't know how many parameters you're passing to $.when. Passing the array itself to $.when wouldn't work, because it expects its parameters to be promises, not an array.
That's where .apply comes in. It takes the array, and calls $.when with each element as a parameter (and makes sure the this is set to jQuery/$), so then it all works :-)

$.when takes any number of parameters and resolves when all of these have resolved.
anyFunction.apply(thisValue, arrayParameters) calls the function anyFunction setting its context (thisValue will be the this within that function call) and passes all the objects in arrayParameters as individual parameters.
For example:
$.when.apply($, [def1, def2])
Is the same as:
$.when(def1, def2)
But the apply way of calling allows you to pass an array of unknown number of parameters. (In your code, you are saying that you data comes from a service, then that is the only way to call $.when)

Here, the code fully documented.
// 1. Declare an array of 4 elements
var data = [1,2,3,4]; // the ids coming back from serviceA
// 2. Declare an array of Deferred objects
var processItemsDeferred = [];
// 3. For each element of data, create a Deferred push push it to the array
for(var i = 0; i < data.length; i++){
processItemsDeferred.push(processItem(data[i]));
}
// 4. WHEN ALL Deferred objects in the array are resolved THEN call the function
// Note : same as $.when(processItemsDeferred[0], processItemsDeferred[1], ...).then(everythingDone);
$.when.apply($, processItemsDeferred).then(everythingDone);
// 3.1. Function called by the loop to create a Deferred object (data is numeric)
function processItem(data) {
// 3.1.1. Create the Deferred object and output some debug
var dfd = $.Deferred();
console.log('called processItem');
// 3.1.2. After some timeout, resolve the current Deferred
//in the real world, this would probably make an AJAX call.
setTimeout(function() { dfd.resolve() }, 2000);
// 3.1.3. Return that Deferred (to be inserted into the array)
return dfd.promise();
}
// 4.1. Function called when all deferred are resolved
function everythingDone(){
// 4.1.1. Do some debug trace
console.log('processed all items');
}

Unfortunately I can not agree with you guys.
$.when.apply($, processItemsDeferred).always(everythingDone);
Will call everythingDone as soon as one deferred gets rejected, even if there are other deferreds that are pending.
Heres the full script (I recommend http://jsfiddle.net/):
var data = [1,2,3,4]; // the ids coming back from serviceA
var processItemsDeferred = [];
for(var i = 0; i < data.length; i++){
processItemsDeferred.push(processItem(data[i]));
}
processItemsDeferred.push($.Deferred().reject());
//processItemsDeferred.push($.Deferred().resolve());
$.when.apply($, processItemsDeferred).always(everythingDone);
function processItem(data) {
var dfd = $.Deferred();
console.log('called processItem');
//in the real world, this would probably make an AJAX call.
setTimeout(function() { dfd.resolve(); }, 2000);
return dfd.promise();
}
function everythingDone(){
alert('processed all items');
}
It this a bug? I would like to use this like the gentleman above described it.

Maybe someone can find this useful:
$.when.apply($, processItemsDeferred).then(everythingDone).fail(noGood);
everythingDone isn't called in case of any reject

$.when alone makes it possible for a callback to be called when every promises passed to it are resolved/rejected. Normally, $.when takes a variable number of arguments, using .apply makes it possible to pass it an array of arguments, it's very powerful. For more info on .apply: https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Function/apply

Thanks for your elegant solution:
var promise;
for(var i = 0; i < data.length; i++){
promise = $.when(promise, processItem(data[i]));
}
promise.then(everythingDone);
Just one point: When using resolveWith to get some parameters, it breaks because of the initial promise set to undefined. What i did to make it work:
// Start with an empty resolved promise - undefined does the same thing!
var promise;
for(var i = 0; i < data.length; i++){
if(i==0) promise = processItem(data[i]);
else promise = $.when(promise, processItem(data[i]));
}
promise.then(everythingDone);

How to return a value from a deferred?

I am calling a method that has some logic in a deferred, and when that logic finishes, I want to return the value to the callee. See below:
//Callee.js
var myAssistant = new Assistant();
console.log(myAssistant.whatIsTheValue());
//Assistant.js
whatIsTheValue : function(someArg) {
var deferred = someService.getSomething();
deferred.then(lang.hitch(this, this._getTheValue));
//In theory, I want to return whatever this._getTheValue returns, how can I do that?!
}
_getTheValue() {
...
...
return xyz;
}

Deferreds are asynchronous operations. Therefore you can't return a variable in a normal way from them because they will not execute until after the current function context completes.
If you want to do more with that value you will need to do so in terms of another callback (IE chaining the then statements.)
The point of deferreds is to provide sequential operations for callbacks. So you can chain them to achieve the results you want. If you need the results to be available in your current execution context, you're going to have to find a synchronous (not deferreds) method of doing what you want.
So something like this
//Assistant.js
whatIsTheValue : function(someArg) {
var deferred = someService.getSomething();
var next = deferred.then(lang.hitch(this, this._getTheValue));
next.then(/*insert next function here*/);
}
You need to understand that using a deferred lang.hitch isn't going to execute until after whatistheValue is done operating. So instead of returning the value to whatever function called whatisthevalue, you're going to have to put the logic of processing that value into a new function and use that as an additional callback for your deferred. This will probably require some restructuring of your program.

I do not know what your lang.hitch does, but the solution should look like that:
Assistant.prototype.whatIsTheValue = function(someArg) {
var deferred = someService.getSomething();
return deferred.then(lang.hitch(this, this._getTheValue));
// ^^^^^^
};
var myAssistant = new Assistant();
myAssistant.whatIsTheValue().then(console.log); // use console.log.bind(console) in Chrome
// ^^^^ - it is a promise you return

use JQuery's $when instead.
Example
// assuming both getData and getLocation return their respective Promise
var combinedPromise = $.when(getData(), getLocation())
// function will be called when both getData and getLocation resolve
combinePromise.done(function(data,location){
alert("We got data: " + dataResult + " and location: " + location);
});
http://www.html5rocks.com/en/tutorials/async/deferred/

How to execute a Javascript function only after multiple other functions have completed?

My specific problem is that I need to execute a (potentially) large number of Javascript functions to prepare something like a batch file (each function call adds some information to the same batch file) and then, after all those calls are completed, execute a final function to send the batch file (say, send it as an HTML response). I'm looking for a general Javascript programming pattern for this.
Generalize problem:
Given the Javascript functions funcA(), funcB(), and funcC(), I would to figure out the best way to order execution so that funcC is only executed after after funcA and funcB have executed. I know that I could use nested callback functions like this:
funcA = function() {
//Does funcA stuff
funcB();
}
funcB = function() {
//Does funcB stuff
funcC();
}
funcA();
I could even make this pattern a little more general by passing in callback parameters, however, this solution becomes quite verbose.
I am also familiar with Javascript function chaining where a solution might look like:
myObj = {}
myObj.answer = ""
myObj.funcA = function() {
//Do some work on this.answer
return this;
}
myObj.funcB = function() {
//Do some more work on this.answer
return this;
}
myObj.funcC = function() {
//Use the value of this.answer now that funcA and funcB have made their modifications
return this;
}
myObj.funcA().funcB().funcC();
While this solution seems a little cleaner to me, as you add more steps to the computation, the chain of function executions grows longer and longer.
For my specific problem, the order in which funcA, funcB, etc. are executed DOES NOT matter. So in my solutions above, I am technically doing more work than is required because I am placing all the functions in a serial ordering. All that matters to me is that funcC (some function for sending the result or firing off a request) is only called after funcA and funcB have ALL completed execution. Ideally, funcC could somehow listen for all the intermediate function calls to complete and THEN would execute? I hoping to learn a general Javascript pattern to solve such a problem.
Thanks for your help.
Another Idea:
Maybe pass a shared object to funcA and funcB and when they complete execution mark the shared object like sharedThing.funcA = "complete" or sharedThing.funcB = "complete" and then somehow? have funcC execute when the shared object reaches a state where all fields are marked complete. I'm not sure how exactly you could make funcC wait for this.
Edit:
I should note that I'm using server-side Javascript (Node.js) and I would like to learn a pattern to solve it just using plain old Javascript (without the use of jQuery or other libraries). Surely this problem is general enough that there is a clean pure-Javascript solution?

If you want to keep it simple, you can use a counter-based callbacks system. Here's a draft of a system that allows when(A, B).then(C) syntax. (when/then is actually just sugar, but then again the whole system arguably is.)
var when = function() {
var args = arguments; // the functions to execute first
return {
then: function(done) {
var counter = 0;
for(var i = 0; i < args.length; i++) {
// call each function with a function to call on done
args[i](function() {
counter++;
if(counter === args.length) { // all functions have notified they're done
done();
}
});
}
}
};
};
Usage:
when(
function(done) {
// do things
done();
},
function(done) {
// do things
setTimeout(done, 1000);
},
...
).then(function() {
// all are done
});

If you don't use any asynchronous functions and your script doesn't break the order of execution, then the most simple solution is, as stated by Pointy and others:
funcA();
funcB();
funcC();
However, since you're using node.js, I believe you're going to use asynchronous functions and want to execute funcC after a async IO request has finished, so you have to use some kind of counting mechanisms, for example:
var call_after_completion = function(callback){
this._callback = callback;
this._args = [].slice.call(arguments,1);
this._queue = {};
this._count = 0;
this._run = false;
}
call_after_completion.prototype.add_condition = function(str){
if(this._queue[str] !== undefined)
throw new TypeError("Identifier '"+str+"' used twice");
else if(typeof str !== "String" && str.toString === undefined)
throw new TypeError("Identifier has to be a string or needs a toString method");
this._queue[str] = 1;
this._count++;
return str;
}
call_after_completion.prototype.remove_condition = function(str){
if(this._queue[str] === undefined){
console.log("Removal of condition '"+str+"' has no effect");
return;
}
else if(typeof str !== "String" && str.toString === undefined)
throw new TypeError("Identifier has to be a string or needs a toString method");
delete this._queue[str];
if(--this._count === 0 && this._run === false){
this._run = true;
this._callback.apply(null,this._args);
}
}
You can simplify this object by ignoring the identifier str and just increasing/decreasing this._count, however this system could be useful for debugging.
In order to use call_after_completion you simply create a new call_after_completion with your desired function func as argument and add_conditions. func will only be called if all conditions have been removed.
Example:
var foo = function(){console.log("foo");}
var bar = new call_after_completion(foo);
var i;
bar.add_condition("foo:3-Second-Timer");
bar.add_condition("foo:additional function");
bar.add_condition("foo:for-loop-finished");
function additional_stuff(cond){
console.log("additional things");
cond.remove_condition("foo:additional function");
}
for(i = 0; i < 1000; ++i){
}
console.log("for loop finished");
bar.remove_condition("foo:for-loop-finished");
additional_stuff(bar);
setTimeout(function(){
console.log("3 second timeout");
bar.remove_condition("foo:3-Second-Timer");
},3000);
JSFiddle Demo

If you don't want to use any helper libraries, than you need to write some helper yourself, there's no simple one line solution for this.
If you'd like to end with something that looks as readable as it would in synchronous case, try some deferred/promise concept implementation (it's still plain JavaScript), e.g. using deferred package you may end up with something as simple as:
// Invoke one after another:
funcA()(funcB)(funcC);
// Invoke funcA and funcB simultaneously and afterwards funcC:
funcA()(funcB())(funcC);
// If want result of both funcA and funcB to be passed to funcC:
deferred(funcA(), funcB())(funcC);

Have a look into jQuery's deferred objects. This provides a sophisticated means of controlling what happens when in an asynchronous environment.
The obvious use-case for this is AJAX, but it is not restricted to this.
Resources:
jQuery docs: deferred object
good introduction to deferred object patterns
Non-AJAX use for jQuery's deferred objects

I was looking for the same kind of pattern. I am using APIs that interrogate multiple remote data sources. The APIs each require that I pass a callback function to them. This means that I cannot just fire off a set of my own functions and wait for them to return. Instead I need a solution that works with a set of callbacks that might be called in any order depending on how responsive the different data sources are.
I came up with the following solution. JS is way down the list of languages that I am most familiar with, so this may not be a very JS idiom.
function getCallbackCreator( number_of_data_callbacks, final_callback ) {
var all_data = {}
return function ( data_key ) {
return function( data_value ) {
all_data[data_key] = data_value;
if ( Object.keys(all_data).length == number_of_data_callbacks ) {
final_callback( all_data );
}
}
}
}
var getCallback = getCallbackCreator( 2, inflatePage );
myGoogleDataFetcher( getCallback( 'google' ) );
myCartoDataFetcher( getCallback( 'cartodb' ) );
Edit: The question was tagged with node.js but the OP said, "I'm looking for a general Javascript programming pattern for this," so I have posted this even though I am not using node.

Nowadays, one can do something like this:
Let's say we have both funcA, funcB and funcC:
If one's want funcA and funcB results to be passed to funcC:
var promiseA = new Promise((resolve, reject) => {
resolve(await funcA());
});
var promiseB = new Promise((resolve, reject) => {
resolve(await funcB());
});
var promise = Promise.all([ promiseA, promiseB ]).then(results => {
// results = [result from funcA, result from funcB]
return funcC(results);
});
If one's want funcA, then funcB and then funcC:
var promise = (
new Promise(async resolve => resolve( await funcA() ))
).then(result_a => funcB(result_a)).then(result_b => funcC(result_b));
And finally:
promise.then(result_c => console.log('done.'));

how about:
funcC(funcB(funcA)));
I think the questions is because some of functions run longer and there might be a situation when we run funcC when funcA or funcB did not fininsh executing.

Is there a way to set a handler function for when a set of events has happened?

eg I have two concurrent AJAX requests, and I need the result from both to compute a third result. I'm using the Prototype library, so it might look something like this:
var r1 = new Ajax.Request(url1, ...);
var r2 = new Ajax.Request(url2, ...);
function on_both_requests_complete(resp1, resp2) {
...
}
One way would be to use polling, but I'm thinking there must be a better way.
Update: An acceptable solution must be free of race conditions.

On the callback function of each request, set a boolean such as
request1Complete and request2Complete
and call on_both_requests_complete(resp1,resp2).
In the handler function, check to see if both booleans are set. If not, just return and fall out of the function. The callback functions should be serialized, in that they cannot happen simultaneously, so this should work. If they could happen in parallel, you would break on a race condition.

This is how I would do it. The approach is a general one, which gives you more flexibility and reuse, and avoids coupling and the use of globals.
var makeEventHandler = function(eventMinimum, callback) {
var data = [];
var eventCount = 0;
var eventIndex = -1;
return function() {
// Create a local copy to avoid issues with closure in the inner-most function
var ei = ++eventIndex;
return function() {
// Convert arguments into an array
data[ei] = Array.prototype.slice.call(arguments);
// If the minimum event count has not be reached, return
if ( ++eventCount < eventMinimum ) {
return;
}
// The minimum event count has been reached, execute the original callback
callback(data);
};
};
};
General usage:
// Make a multiple event handler that will wait for 3 events
var multipleEventHandler = makeMultipleEventHandler(3, function(data) {
// This is the callback that gets called after the third event
console.log(data);
});
multipleEventHandler()(1,2,3);
var t = multipleEventHandler();
setTimeout(function() {t("some string");}, 1000);
multipleEventHandler()({a: 4, b: 5, c: 6});
Output from callback (condensed by Firebug):
[[1, 2, 3], ["some string"], [Object { a=4, more...}]]
Notice that the order of data in the final callback is in order of the calling events, even though the second "event" executes after the third.
To use this in context of your Ajax requests:
var onBothComplete = makeMultipleEventHandler(2, function(data) {
// Do something
...
});
new Ajax.Request(url1, {onComplete: onBothComplete()});
new Ajax.Request(url2, {onComplete: onBothComplete()});
Edit: I've updated the function to force data to always maintain the asynchronously received event data in the synchronously executed order (the previous caveat no longer exists).

Well, you have to remember that the JS implementation in browsers is not really concurrent, and use that to your advantage. So what you would want to do is in each handler check if the other has finished. Example in jQuery:
var other_done = false;
$.get('/one', function() {
if (other_done) both_completed();
other_done = true;
alert('One!');
});
$.get('/two', function() {
if (other_done) both_completed();
other_done = true;
alert('Two!');
});
function both_completed() {
alert('Both!');
}

Based on Justin Johnson's response to this question:
function sync(delays /* Array of Functions */, on_complete /* Function */) {
var complete_count = 0;
var results = new Array(delays.length);
delays.length.times(function (i) {
function on_progress(result) {
results[i] = result;
if (++complete_count == delays.length) {
on_complete(results);
}
}
delays[i](on_progress);
});
}
This assumes each delay accepts one argument: an "on progress" event handler, which takes one argument: the result that the delay is trying to compute. To complete the example in my original question, you'd use it like so:
var delays = [];
delays[0] = function (on_progress) {
new Ajax.Request(url1, {onSuccess: on_progress});
};
delays[1] = function (on_progress) {
new Ajax.Request(url2, {onSuccess: on_progress});
};
function on_complete(results) { alert(results.inspect()); }
sync(delays, on_complete);
The one thing I'm not sure of is whether this avoids a race condition. If the expression ++complete_count == delays.length always happens atomically, then this should work.

You can use the concept where you set temporary variables and wait for the "last" request to go. To do this, you can have the two handle functions set the tmp vars to the return val and then call your "on_both_requests_complete" function.
var processed = false;
var r1 = new Ajax.Request(...);
var r2 = new Ajax.Request(...);
(function() {
var data1;
var data2;
function handle_r1(data) {
data1 = data;
on_both_requests_complete();
};
function handle_r2(data) {
data2 = data;
on_both_requests_complete();
};
function on_both_requests_complete() {
if ( (!data1 || !data2) || processed) {
return;
}
processed = true;
/* do something */
};
}();