We Keep Coding

JavaScript initializing callback parameters down the callback chain

Note: I'm bootstrapping a reactjs app but this is a general JavaScript question.
I have a special module "locationUtils" that I am trying to keep in it's own package, but keeping that code separate is causing an eyesore with callbacks.
When I access one of it's methods I have to send a callback with it that only has one of its parameters initially defined, and in that method I get the other data parameter to initalize the other parameter.
Can I add in undefined parameters later like that in JavaScript, and is it good practice to initial parameters for a callback method as you go down the callback chain in general, or am I making a convoluted newbie mistake?
/********************Module 1******************************/
var bootStrapUI = function(callback) {
locationUtils.findData(findOtherData(callback));
}
//This gets called last to finalize bootstraping
var findOtherData = function(callback,originalFetchedData){
//use originalFetchedData to get more data
//bootStraping program with all rendering data
callback() //sends back a boolean confirming all fetched
}
/**********************Module2**********************************/
var findData = function(findOtherData){
var data = magicGetData();
findOtherData(findOtherData,data);//I initialized a param late here!
}

It's a good Javascript question, callbacks can become a serious hell for the uninitiated, particularly when they are nested and / or the order in which they return is important.
This is where promises come in: they are an essential tool for Javascript development and about to become part of the standard (in EcmaScript 6).
In essence: a promise is an object that is returned from a function with a method (callback) that is called when the asynchronous action (e.g. API call) has been completed. The difference between a promise and a callback is that promises allow you to structure how you handle the callbacks and, importantly, in what order.
I recently wrote a method that had to make 30 api calls with each call dependent on the results of the previous one (this was not a well designed api). Can you imagine trying to do that with callbacks? As it was, I created an array of promises and used jQuery.when() to handle things when all the api calls had completed.
For the moment we need to use a library for promises. jQuery: https://api.jquery.com/jquery.deferred/ is the obvious one but there are various other implementations that do much the same thing.
Update:
The question relates more specifically to the passing of arguments between callbacks and modifying the arguments as execution moves between them. This can be done easily by passing whatever info you need as an argument to your resolve method.
Typically it looks something like this (using jQuery):
var myAsyncMethod = function(info){
var deferred = $.Deferred();
$.getJSON(myUrl,
function(dataFromServer) {
// Do stuff with data
var newData = doSomething(dataFromServer);
deferred.resolve(newData);
});
});
return deferred.promise();
};
// Make initial method call
myAsyncMethod(myInitialData).then(
function(transformedData){
// transformed data from server is returned here.
}
);

How to structure Protractor promises

In this example I'm testing whether an element is located at the bottom of the page. I'm using the promises in Protractor/Webdriver, but I think I'm using it wrong because I don't think it's supposed to look this messy.
describe('Element', function(){
it('should be placed at bottom', function(){
element(by.id('page').getSize().then(function(dimP){
element(by.id('element').getLocation().then(function(locE){
element(by.id('element').getSize().then(function(dimE){
expect(locE.y + dimE.height).toEqual(dimP.height);
})
});
});
})
});
How can I do this in a cleaner way? I'm using Protractor version 2.1.0.
I have tried to do like this:
expect(element(by.id('page').getSize().height).toEqual(1000);
But it says Expected undefined to equal 1000. So it seems like I can't use return values as explained here: https://github.com/angular/protractor/blob/master/docs/control-flow.md
(I am using a page object in my actual test, so the variables are not as ugly as in the example.)

Since getSize() and getLocation() both return promises and you need actual values, you have to resolve all of the promises, either explicitly with then(), or let expect() implicitly do this for you.
I'm actually understanding your question as "How to flatten a promise chain in Protractor?". This is there promise.all() would help to resolve multiple promises at once which would save you from writing multiple nested then() callbacks:
var elm = element(by.id('element')),
page = element(by.id('page'));
var promises = [elm.getSize(), elm.getLocation(), page.getSize()];
protractor.promise.all(promises).then(function (values) {
expect(values[0].height + values[1].y).toEqual(values[2].height);
});
See also:
Flattening Promise Chains
How do I reference a promise returned from a chained function?
How do I access previous promise results in a .then() chain?

Node.JS - Q Module - Promises

I'm hoping someone who uses the Q module in Node.js can advise me.
So, my question is:
Is it typically necessary to create more than one promise if you are trying to perform multiple functions in sequence?
For example, I'm trying to create an application that:
1) Read data from a file (then)
2) Opens a database connection (then)
3) Executes a query via the database connection (then)
4) Store JSON dataset to a variable (then)
5) Close Database connection. (then)
6) Perform other code base on the JSON dataset I've stored to a variable.
In raw Node.js, each method of an object expects a callback, and in order to do these tasks in the proper order (and not simultaneously -which wouldn't work), I have to chain these callbacks together using an ugly amount of code nesting.
I discovered the Q module, which prevents nesting via the Promise concept.
However, in my first attempt at using Q, I'm trying to make a promise out of everything, and I think I might be over-complicating my code.
I'm think that maybe you only really have to create one promise object to perform the steps mentioned above, and that it may be unnecessary for me to convert every method to a promise via the Q.denodeify method.
For example, in my code I will be connecting to a db2 database using the ibm_db module. Probably due to misunderstanding, I've converted all the ibm_db methods to promises like this:
var ibmdb = require('ibm_db');
var q = require('q');
var ibmdbOpen = q.denodeify(ibmdb.open);
var ibmdbConn = q.denodeify(ibmdb.conn);
var ibmdbClose = q.denodeify(ibmdb.close);
var ibmdbQuery = q.denodeify(ibmdb.query);
Is this really necessary?
In order to do one thing after another in Q, is it necessary for me to denodeify each method I'll be using in my script?
Or, can I just create one promise at the beginning of the script, and use the q.then method to perform all the asynchronous functions in sequence (without blocking).

Is it typically necessary to create more than one promise if you are trying to perform multiple functions in sequence?
Yes, definitively. If you didn't have promises for all the intermediate steps, you'd have to use callbacks for them - which is just what you were trying to avoid.
I'm trying to make a promise out of everything
That should work fine. Indeed, you should try to promisify on the lowest possible level - the rule is to make a promise for everything that is asynchronous. However, there is no reason to make promises for synchronous functions.
Especially your steps 4 and 5 trouble me. Storing something in a variable is hardly needed when you have a promise for it - one might even consider this an antipattern. And the close action of a database should not go in a then handler - it should go in a finally handler instead.
I'd recommend not to use a linear chain but rather:
readFile(…).then(function(fileContents) {
return ibmdbOpen(…).then(function(conn) {
return imbmdbQuery(conn, …).finally(function() {
return imbdbClose(conn);
});
}).then(function(queriedDataset) {
…
});
});

Chaining or grouping multiple JavaScript function calls where order of operation is critical at times

I have created a web api method that carries out calculations using a json object I post to the method. My understanding is that a jquery post is asynchronous? Assuming this is so, I'd like to be able to chain together multiple calls to the js function that invokes this api method because certain calls are order-critical.
There are 80 calls to this asynchronous api method in a row. I don't want to do:
asyncMethodCall1(myParams).then(asyncMethodCall2(myParams).then...))
...
as there is no need to chain them like this as none of them depend on the other and can all run simultaneously. However at the end I need to do some other calculations that DO depend on the previous results having finished.
Is there a way of doing a sort of group of calls followed by a "then"? Is it simply a case of having something like:
mySynchronousFunction(params).then(function() {...do other calcs});
function mySynchronousFunction(params) {
asyncmethodcall1(myparams);
asyncmethodcall2(myparams);
asyncmethodcall3(myparams);
...
asyncmethodcall76(myparams);
}
or do I need to have a "setTimeOut" in there somewhere? JavaScript is something I'm tryign to learn and it's all a bit odd to me at the moment!
EDIT 1
Well I'm stumped.
I know it's a fundamental lack of understanding on my part that's causing the problem but it's very hard to find basic introductory stuff that someone coming from a synchronous language can follow and understand. Currently a resource I'm working through is this one and it seems to make some sense but it's still not sinking in: http://blog.mediumequalsmessage.com/promise-deferred-objects-in-javascript-pt1-theory-and-semantics
Currently I have this:
$.when(
individualImpactCalc('byWeightAndFactor', 'CO2e', articleResults().material_CO2e),
individualImpactCalc('byWeightAndFactor', 'Water', articleResults().material_Water),
individualImpactCalc('byWeightAndFactor', 'pH', articleResults().material_pH),
...lots of other calls in here...
).then(function () {
//do calculation totalling
alert("I'm done!");
}).fail(function() {
alert("Argh! I failed!");
});
...and it simply doesn't work as I want it to. I never get an alert showing. The impact calculations are done, the observables are updated, my page values change, but no alerts.
What am I fundamentally missing here?
Edit 2
What I was fundamentally missing was the difficulty in debugging chained promises! It actually was working once I cured a hidden reference error that wasn't bubbling up. Lots of painstaking stepping through javascript finally found it.
Just to add in another element to the mix of answers, I used the Q library as that was what is used in Durandal/Breeze anyway and it's just easy to keep the code consistent. I tried it with $.when and it worked just as well. I tried it with Promise and it failed with "Promise is not defined".
My working Q implementation:
var calcPromise = Q.all([
individualImpactCalc('byWeightAndFactor', 'CO2e', articleResults().material_CO2e),
individualImpactCalc('byWeightAndFactor', 'Water', articleResults().material_Water),
individualImpactCalc('byWeightAndFactor', 'pH', )]);
return calcPromise
.then(calcsDone)
.fail(calcsFailed);
function calcsDone() {
alert("all calcs done");
}
function calcsFailed() {
alert("Argh! Calc failure...");
}

What you are looking for is the jQuery when method:
http://api.jquery.com/jQuery.when/
In the case where multiple Deferred objects are passed to jQuery.when,
the method returns the Promise from a new "master" Deferred object
that tracks the aggregate state of all the Deferreds it has been
passed. The method will resolve its master Deferred as soon as all the
Deferreds resolve, or reject the master Deferred as soon as one of the
Deferreds is rejected.
Specifically you put the calls that don't depend on each other in when, and the ones that depend on them in the then.
$.when(/* independent calls */).then(/* dependent calls */);
So as an example if you want to run deffered 1 and 2 in paralel, then run 3, then run 4 you can do:
$.when(def1, def2).then(def3).then(def4);

You can use Promise.when() to wait for all of them.
function mySynchronousFunction(params) {
return Promise.when(asyncmethodcall1(myparams),
asyncmethodcall2(myparams),
...);
}
Promise.when returns a new promise, so you can chain this with .then().
See Asynchronous Programming in JavaScript with “Promises”

As others have pointed out, you can use $.when(...) to call all of the methods at once and only continue with .then() when all are done. But that doesn't solve your problem, because the methods are started all at once and there is no chaining. Calls to your web api can still come in and finish in random order.
Even though your example shows different async methods, your description sounds like you have only one method with different parameter values. So why not collecting those parameters in an array and then chaining the calls in a loop?
var params = [ 27, 38, 46, 83, 29, 22 ];
var promise = asyncmethodcall(params[0]);
for (var i=1; i<params.length; i++) {
promise = promise.then(buildAsyncmethodcall(params[i]));
}
function buildAsyncmethodcall(param) {
return function() {
return asyncmethodcall(param);
}
}
http://jsfiddle.net/qKS5e/ -- see here why I had to build the function using another function
If you really want to call different methods, you could write a jQuery plugin eg. $.whenSequential(...) to which you pass an array of functions that return Deferred objects like this:
$.whenSequential( // your cool new plugin
function() { return callasyncmethod1(123); },
function() { return callasyncmethod2(345); },
function() { return callasyncmethod3(678); }
);
The plugin method would work just like the for-loop above, but instead of params you iterate the function arguments and pass one by one to then().
Either way, you won't get around wrapping all calls in a function somehow to chain them, because otherwise they would get executed immediately (like with $.when).

jQuery deferred: "then" ... "when" ... where's the "join" or "blockUntilDone"?

I'm using the JayData.js library. It works quite well. However, I have a few situations where I've got a toArray() call deep in the function tree. Rather than trying to access my "busy" signal from there, I'd just as soon have the method block. Is that possible? I'm picturing something like "context.Groups.toArray(myObservableVar).block()".
Update 1: It appears that the JayData library returns a jQuery deferred object judging from the use of "then" and "when" operators on the return value. Is there a corresponding method to "join" -- meaning wait for the finish?

Indeed JayData toArray() (and all relevant data returning or saving/updating method) implements jQuery deferred. As from 1.0.5 you have to include the JayDataModules/deferred.js in order to this functionality to work.
For your use case $.when might be an answer:
var customers = context.Customers.toArray();
var products = context.Products.toArray();
var suppliers = context.Suppliers.toArray();
$.when(customers, products, suppliers).then(function(customers, products, suppliers) {
//we have everything here
//notice the the parameter names are shadowed as the var customers variable only
//holds a promise not the value
customers.forEach( ... );
products[12].ProductName = "X";
});

A blockUntilDone() method would go against the principles of deferred execution and continuations. JayData's toArray() is asynchronous because it is designed not to block the caller.
If you want this kind of code:
// Initialize context and groups...
var arrayOfGroups = context.Groups.toArray(); // Want synchronous behavior.
// Do something with 'arrayOfGroups'...
Trying to block until the deferred is resolved is not the solution. Move the last part of your code into a callback passed to toArray() instead:
// Initialize context and groups...
context.Groups.toArray(function(arrayOfGroups) {
// Do something with 'arrayOfGroups'...
});
Alternatively, bind to the returned promise with done() or then():
context.Groups.toArray().done(function(arrayOfGroups) {
// Do something with 'arrayOfGroups'...
});