I am using Async's Library Parallel method wrong? But how? - javascript

I am using the async library's parallel function, but I am running into a weird problem. I loop through an array of objects to create an array of functions called people. People gets passed into async's parallel method to execute in parallel. However, when I take a look at the parameter "people" that actually get passed into the "printFunction" it is wrong. The people parameter is always the last object in my people array.
So in this example, the console log in printFunction would print
{ 'name' : 'jar jar binks' }
{ 'name' : 'jar jar binks' }
Code:
var people = [{ 'name' : 'obi wan kenobi'}, { 'name' : 'jar jar binks' }];
// Create an array of tasks to be processed.
for(var i = 0; i < people.length; i++) {
tasks.push( function(callback) {
setTimeout( makePrintFunction(people[i], callback), 200);
});
}
// Process those tasks.
async.parallel(tasks, function(err, stuff) {
// ...do some stuff with the returned array here.
});
Make Print Function:
function makePrintFunction(people, next) {
return validateAndTrackFlag(people, next);
}
function printFunction(people, next) {
console.log(people); // Always prints: { 'name' : 'jar jar binks' }
next(null, true)
}
Obviously this is not the real code, I just changed the names and objects.
Thanks for any help in advance.

This is not an async problem, it's a closure problem.
Try this:
for(var i = 0; i < people.length; i++) {
(function (i){
tasks.push( function(callback) {
setTimeout( makePrintFunction(people[i], callback), 200);
});
})(i)
}
Or this:
people.forEach (function (p){
tasks.push( function(callback) {
setTimeout( makePrintFunction(p, callback), 200);
});
});

You are dealing with classical scoping issue. Try this:
for(var i = 0; i < people.length; i++) {
(function(i) {
tasks.push( function(callback) {
setTimeout( makePrintFunction(people[i], callback), 200);
});
})(i);
}
or even better
var create_scope = function(i) {
tasks.push( function(callback) {
setTimeout( makePrintFunction(people[i], callback), 200);
});
};
for (var i = 0; i < people.length; i++) {
create_scope(i);
}
or the best:
people.forEach(function(person) {
tasks.push( function(callback) {
setTimeout( makePrintFunction(person, callback), 200);
});
});
This happens because for loop does not create a scope, i.e. when functions are fired i is the last index. Now functions do create scope, so inside anonymous function i is the expected value.

Related

Loopback Callback inside For-Loop

In a for-loop I call a loopback function (model.find()) and use the callback method. In this method I need the i variable of the for-loop to specify a variable but it is not accessible from closure.
I already tried writing (i) or call(this,i) behind the callback function but it didn't work.
for (var i = 0; i < $scope.objects.length; i++) {
Priority.find({
filter: {
where: {priority: $scope.selectedPriority[i].priority}
}
}, function (prios) {
Priority.create({"priority": $scope.selectedPriority[i].priority //i is not accessible
}, function (priority) {
$scope.selectedPriority[i].priority = undefined; //i is not accessible
}, function (error) {
console.log(error);
});
}
});
}
Actually "i" should be defined but you'll always find it at its highest value (= $scope.objects.length - 1), the reason is that since Priority.find is asynchronous, once it returns an answer, the for loop has already done iterating.
To solve it, you could put body of the loop inside a function:
function find(i) {
Priority.find({
filter: {
where: {priority: $scope.selectedPriority[i].priority}
}
}, function (prios) {
Priority.create({"priority": $scope.selectedPriority[i].priority //i is not accessible
}, function (priority) {
$scope.selectedPriority[i].priority = undefined; //i is not accessible
}, function (error) {
console.log(error);
});
}
});
}
So then, the for loop becomes:
for (var i = 0; i < $scope.objects.length; i++) {
find(i);
}
essentially, you're "capturing" the value of "i" and send it as a function argument, as long as you're in the context of a specific function call, i stays fixed.

Passing variable into named callback

OK, let's keep it simple, I have the following :
function loopDaLoop(){
for (var i = 0; i < tempItemsLength; i++) {
var product = tempItems[i];
dust.render('product', product, addProductOrFinish);
}
}
and i'd like to get the current value of i inside my callback function
function addProductOrFinish(err, out) {
console.log(i); // undefined
}
I do know it's simple, really I do... Help?
Edit :
I know I'm supposed to use a closure so I tried and failed with this :
(function(i){
dust.render('product', product, addProductOrFinish);
};(i)
There are several different structures that could be used to solve this problem. The simplest is to use .bind() to add the desired parameter to the function call.
function loopDaLoop(){
for (var i = 0; i < tempItemsLength; i++) {
var product = tempItems[i];
dust.render('product', product, addProductOrFinish.bind(null, i));
}
}
function addProductOrFinish(i, err, out) {
console.log(i);
}
This will cause the value of this to change in addProductOrFinish if that was important. If so, you could work around that too, but it's not as simple.
Here's another approach using a closure that returns a function and preserves the value of this in case dust.render() is setting that:
function loopDaLoop(){
for (var i = 0; i < tempItemsLength; i++) {
var product = tempItems[i];
dust.render('product', product, getAddProductOrFinish(i));
}
}
function getAddProductOrFinish(loopArg) {
return function(err, out) {
return addProductOrFinish.call(this, loopArg, err, out);
}
}
function addProductOrFinish(i, err, out) {
console.log(i);
}
Or, if addProductOrFinish can be an inline function, then it can use the IIFE-type structure you tried like this:
function loopDaLoop(){
for (var i = 0; i < tempItemsLength; i++) {
var product = tempItems[i];
(function(i) {
dust.render('product', product, function(err, out) {
console.log(i);
// rest of your addProductOrFinish logic here
});
)(i);
}
}

Javascript end of For Loop chain a callback

How do you go about knowing when a For Loop is done iterating and attach a callback.
This is a sample loop of many loops within a function that inserts records into indexedDB.
if (Object.hasOwnProperty("Books")) {
for (var i = 0, j = Object["Books"].length; i < j; i++) {
server.Book.add({
title: Object["Books"][i].Cat,
content: Object["Books"][i]
});
}
}
I need to be able to know when each of the if statements loops are finished then attach a callback. All the loops are being fired asynchronously, and I need to run a function_final() just when all loops are finished not when they are fired.
EDIT
What I have tried so far :
InsertDB = {
addBook: function(Object) {
return $.Deferred(function() {
var self = this;
setTimeout(function() {
if (Object.hasOwnProperty("Book")) {
for (var i = 0, j = Object["Book"].length; i < j; i++) {
server.Book.add({
title: Object["Book"][i].id,
content: Object["Book"][i]
});
}
self.resolve();
}
}, 200);
});
},
addMagaz: function(Object) {
return $.Deferred(function() {
var self = this;
setTimeout(function() {
if (Object.hasOwnProperty("Magaz")) {
for (var i = 0, j = Object["Magaz"].length; i < j; i++) {
server.Magaz.add({
content: Object["Magaz"][i]
});
}
self.resolve();
}
}, 2000);
});
},
addHgh: function(Object) {
return $.Deferred(function() {
var self = this;
setTimeout(function() {
if (Object.hasOwnProperty("MYTVhighlights")) {
for (var i = 0, j = Object["MYTVhighlights"].length; i < j; i++) {
server.MYTVhighlights.add({
content: Object["MYTVhighlights"][i]
});
}
self.resolve();
}
}, 200);
});
}, ect...
then on a AJAX success callback :
success: function(data){
var Object = $.parseJSON(data);
$.when(InsertDB.addBook(Object),
InsertDB.addMagaz(Object),
InsertDB.addUser(Object),
InsertDB.addArticles(Object),
InsertDB.addHgh(Object),
InsertDB.addSomeC(Object),
InsertDB.addOtherC(Object)).done(final_func);
function final_func() {
window.location = 'page.html';
}
Here final_func is fired before looping ends..
Thanks
You can use JavaScript closures, just like this:
if (Object.hasOwnProperty("Books")) {
for (var i = 0, j = Object["Books"].length; i < j; i++) {
(function(currentBook)
server.Book.add({
title: currentBook.Cat,
content: currentBook
});
)(Object["Books"][i]);
}
function_final();
}
For more information about closures you can refer here.
Since you've said that server.Book.add() is asynchronous, you will need a method of knowing when that asynchronous operation is completed and you can then use that to build a system for knowing when all of them are done. So, the pivotal question (which I already asked as a comment earlier and you have not responded to) is how you can know when server.Book.add() is actually complete. If you're using an indexedDB, then somewhere inside that function, there is probably a request object that has an onsuccess and onerror methods that will tell you when that specific operation is done and that information needs to get surfaced out to server.Book.add() in some way, either as a completion callback or as a returned promise (those two options are how $.ajax() operates for it's asynchronous behavior.
Let's suppose that server.Book.add() returns a promise object that is resolved or rejected when the asychronous .add() operation is complete. If that was the case, then you could monitor the completion of all the operations in your loop like this:
if (obj.hasOwnProperty("Books")) {
var promises = [], p;
for (var i = 0, j = obj["Books"].length; i < j; i++) {
p = server.Book.add({
title: obj["Books"][i].Cat,
content: obj["Books"][i]
});
promises.push(p);
}
$.when.apply($, promises).done(function() {
// this is executed when all the promises returned by
// server.Book.add() have been resolved (e.g. completed)
}).error(function() {
// this is executed if any of the server.Book.add() calls
// had an error
});
}
Let's suppose that instead of server.Book.add() returning a promise, it has a couple callbacks for success and error conditions. Then, we could write the code like this:
if (obj.hasOwnProperty("Books")) {
var promises = [], p;
for (var i = 0, j = obj["Books"].length; i < j; i++) {
(function() {
var d = $.Deferred();
server.Book.add({
title: obj["Books"][i].Cat,
content: obj["Books"][i],
success: function() {
var args = Array.prototype.slice.call(arguments, 0);
d.resolve.apply(d, args);
},
error: function() {
var args = Array.prototype.slice.call(arguments, 0);
d.reject.apply(d, args);
}
});
promises.push(d.promise());
})();
}
$.when.apply($, promises).done(function() {
// this is executed when all the promises returned by
// server.Book.add() have been resolved (e.g. completed)
}).error(function() {
// this is executed if any of the server.Book.add() calls
// had an error
});
}
So, since you've not disclosed how server.Book.add() actually indicates its own completion, I can't say that either of these blocks of code work as is. This is meant to demonstrate how you solve this problem once you know how server.Book.add() communicates when it is complete.
Promises/Deferreds are not magic in any way. They don't know when an operation is completed unless that operation calls .resolve() or .reject() on a promise. So, in order to use promises, your async operations have to participate in using promises or you have to shim in a promise to an ordinary completion callback (as I've done in the second code block).
FYI, I also change your Object variable to obj because defining a variable named Object that conflicts with the built-in Object in the javascript language is a bad practice.
use jquery when functionality
$.when( function1 , function2 )
.then( myFunc, myFailure );
I'd write something like this in pure JS, consider it as pseudo code:
var totalForLoopsCount = 3; //Predict for loops count here
var forLoopsFinished = 0;
function finalFunction()
{
alert('All done!');
}
function forLoopFinished()
{
forLoopsFinished++;
if(forLoopsFinished == totalForLoopsCount)
{
finalFunction();
}
}
var length = 10; //The length of your array which you're iterating trough
for(var i=0;i<length;i++)
{
//Do anything
if(i == length-1)
{
forLoopFinished();
}
}
for(var i=0;i<length;i++)
{
//Do anything
if(i == length-1)
{
forLoopFinished();
}
}
for(var i=0;i<length;i++)
{
//Do anything
if(i == length-1)
{
forLoopFinished();
}
}

Async calling out generated functions in series

I am having problem with calling the generated functions in serial. I am using the async library and the code seems to work when there is no deep callback calling needed. When I add the real scenario it throws errors.
Here is the example which works, returns the array of 0 to 4:
Scrape.prototype.generatePageFunctions = function() {
var functionList = new Array(), self = this;
for (var i = 0; i < this.pageSet; i++) {
(function(i) {
functionList.push(function(cb) {
// Inner functions which will be called in seriers
var timeoutTime = parseInt(Math.random() * 5000 + 3000, 10);
setTimeout(function() {
self.setIndex(i);
//self.getSite(function)
cb(null, i);
}, timeoutTime);
});
})(i);
}
return functionList;
}
Scrape.prototype.run = function() {
var functionList = this.generatePageFunctions();
async.series(functionList, function(err, results) {
console.log('Job is completed ');
console.log(results);
});
}
Now adding the real scenario like downloading the site and then put in callback:
Scrape.prototype.generatePageFunctions = function() {
var functionList = new Array(), self = this;
for (var i = 0; i < this.pageSet; i++) {
(function(i) {
functionList.push(function(cb) {
// Inner functions which will be called in seriers
var timeoutTime = parseInt(Math.random() * 5000 + 3000, 10);
setTimeout(function() {
self.setIndex(i);
self.getSite(function(result) {
// Async callback to pass the data
cb(null, result);
});
}, timeoutTime);
});
})(i);
}
return functionList;
}
Error is like this, even if passing instead of result iterator variable i:
/home/risto/scrape/node_modules/async/lib/async.js:185
iterator(x.value, function (err, v) {
^
TypeError: Cannot read property 'value' of undefined
at /home/risto/scrape/node_modules/async/lib/async.js:185:23
at /home/risto/scrape/node_modules/async/lib/async.js:108:13
at /home/risto/scrape/node_modules/async/lib/async.js:119:25
at /home/risto/scrape/node_modules/async/lib/async.js:187:17
at /home/risto/scrape/node_modules/async/lib/async.js:491:34
at /home/risto/scrape/scraper/scrape.js:114:13
at /home/risto/scrape/scraper/scrape.js:64:16
at Object.<anonymous> (/home/risto/scrape/scraper/engines/google.js:58:12)
at Function.each (/home/risto/scrape/node_modules/cheerio/lib/api/utils.js:133:19)
at [object Object].each (/home/risto/scrape/node_modules/cheerio/lib/api/traversing.js:69:12)
// Edit
Only result which gets added into the complete callback is the first one, other functions are never called.
Also for information the functions return object literals if that does matter.
There is nothing wrong with your code. Creating a simple testcase shows that.
I created a mock:
Scrape = function() {
this.pageSet = 5;
}
Scrape.prototype.setIndex = function() {
}
Scrape.prototype.getSite = function(cb) {
cb('works');
}
and calling the run method it outputs the expected:
[ 'works', 'works', 'works', 'works', 'works' ]
So the problem is somewhere else. Have you tried to check the functionList variable in your run method?
Thank you #KARASZI István, all the code above is correct, the problem seemed in somewhere else. The deepest callback got called multiple times but the outer one got called only once.

Passing arrays of functions to async parallel

I have to pass an array of functions to the async.js module for node.js.
The normal way from the docs would be:
async.parallel([
function(callback){
setTimeout(function(){
callback(null, 'one');
}, 200);
},
function(callback){
setTimeout(function(){
callback(null, 'two');
}, 100);
},
],
// optional callback
function(err, results){
});
I tried like this:
for(var i = 0; i < jsonData.length; i++)
{
...
o.url = serviceurl;
o.title = jsonData[i];
var ff = function(callback){
obj.loadService(o.title,o.url,callback);
}
callItems.push(ff(function(){return true;}));
}
async.parallel(
callItems,
// optional callback
function(err, results){
console.log('all calls called without any errors');
}
);
That runs through but the main callback isn't called. And so I can't say if all parallel calls are done.
What am I missing here?
It looks like the closures are improperly formed in the for loop. Try an external function that returns the value you're currently assigning to ff. Example:
for(var i = 0; i < jsonData.length; i++)
{
...
o.url = serviceurl;
o.title = jsonData[i];
var ff = makeCallbackFunc(obj, o.title, o.url);
callItems.push(ff(function () {return true;}));
}
function makeCallbackFunc(obj, title, url) {
return function (callback) {
obj.loadService(title, url, callback);
};
}
I'm a bit confused by what you are adding to callitems - namely the result of calling ff with the function parameter - it won't be a callback, it will execute right away.

Categories