I have code that looks something like this in javascript:
forloop {
//async call, returns an array to its callback
}
After ALL of those async calls are done, I want to calculate the min over all of the arrays.
How can I wait for all of them?
My only idea right now is to have an array of booleans called done, and set done[i] to true in the ith callback function, then say while(not all are done) {}
edit: I suppose one possible, but ugly solution, would be to edit the done array in each callback, then call a method if all other done are set from each callback, thus the last callback to complete will call the continuing method.
You haven't been very specific with your code, so I'll make up a scenario. Let's say you have 10 ajax calls and you want to accumulate the results from those 10 ajax calls and then when they have all completed you want to do something. You can do it like this by accumulating the data in an array and keeping track of when the last one has finished:
Manual Counter
var ajaxCallsRemaining = 10;
var returnedData = [];
for (var i = 0; i < 10; i++) {
doAjax(whatever, function(response) {
// success handler from the ajax call
// save response
returnedData.push(response);
// see if we're done with the last ajax call
--ajaxCallsRemaining;
if (ajaxCallsRemaining <= 0) {
// all data is here now
// look through the returnedData and do whatever processing
// you want on it right here
}
});
}
Note: error handling is important here (not shown because it's specific to how you're making your ajax calls). You will want to think about how you're going to handle the case when one ajax call never completes, either with an error or gets stuck for a long time or times out after a long time.
jQuery Promises
Adding to my answer in 2014. These days, promises are often used to solve this type of problem since jQuery's $.ajax() already returns a promise and $.when() will let you know when a group of promises are all resolved and will collect the return results for you:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push($.ajax(...));
}
$.when.apply($, promises).then(function() {
// returned data is in arguments[0][0], arguments[1][0], ... arguments[9][0]
// you can process it here
}, function() {
// error occurred
});
ES6 Standard Promises
As specified in kba's answer: if you have an environment with native promises built-in (modern browser or node.js or using babeljs transpile or using a promise polyfill), then you can use ES6-specified promises. See this table for browser support. Promises are supported in pretty much all current browsers, except IE.
If doAjax() returns a promise, then you can do this:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push(doAjax(...));
}
Promise.all(promises).then(function() {
// returned data is in arguments[0], arguments[1], ... arguments[n]
// you can process it here
}, function(err) {
// error occurred
});
If you need to make a non-promise async operation into one that returns a promise, you can "promisify" it like this:
function doAjax(...) {
return new Promise(function(resolve, reject) {
someAsyncOperation(..., function(err, result) {
if (err) return reject(err);
resolve(result);
});
});
}
And, then use the pattern above:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push(doAjax(...));
}
Promise.all(promises).then(function() {
// returned data is in arguments[0], arguments[1], ... arguments[n]
// you can process it here
}, function(err) {
// error occurred
});
Bluebird Promises
If you use a more feature rich library such as the Bluebird promise library, then it has some additional functions built in to make this easier:
var doAjax = Promise.promisify(someAsync);
var someData = [...]
Promise.map(someData, doAjax).then(function(results) {
// all ajax results here
}, function(err) {
// some error here
});
Checking in from 2015: We now have native promises in most recent browser (Edge 12, Firefox 40, Chrome 43, Safari 8, Opera 32 and Android browser 4.4.4 and iOS Safari 8.4, but not Internet Explorer, Opera Mini and older versions of Android).
If we want to perform 10 async actions and get notified when they've all finished, we can use the native Promise.all, without any external libraries:
function asyncAction(i) {
return new Promise(function(resolve, reject) {
var result = calculateResult();
if (result.hasError()) {
return reject(result.error);
}
return resolve(result);
});
}
var promises = [];
for (var i=0; i < 10; i++) {
promises.push(asyncAction(i));
}
Promise.all(promises).then(function AcceptHandler(results) {
handleResults(results),
}, function ErrorHandler(error) {
handleError(error);
});
You can use jQuery's Deferred object along with the when method.
deferredArray = [];
forloop {
deferred = new $.Deferred();
ajaxCall(function() {
deferred.resolve();
}
deferredArray.push(deferred);
}
$.when(deferredArray, function() {
//this code is called after all the ajax calls are done
});
You can emulate it like this:
countDownLatch = {
count: 0,
check: function() {
this.count--;
if (this.count == 0) this.calculate();
},
calculate: function() {...}
};
then each async call does this:
countDownLatch.count++;
while in each asynch call back at the end of the method you add this line:
countDownLatch.check();
In other words, you emulate a count-down-latch functionality.
This is the most neat way in my opinion.
Promise.all
FetchAPI
(for some reason Array.map doesn't work inside .then functions for me. But you can use a .forEach and [].concat() or something similar)
Promise.all([
fetch('/user/4'),
fetch('/user/5'),
fetch('/user/6'),
fetch('/user/7'),
fetch('/user/8')
]).then(responses => {
return responses.map(response => {response.json()})
}).then((values) => {
console.log(values);
})
Use an control flow library like after
after.map(array, function (value, done) {
// do something async
setTimeout(function () {
// do something with the value
done(null, value * 2)
}, 10)
}, function (err, mappedArray) {
// all done, continue here
console.log(mappedArray)
})
In Node.js you can use async/await to control the async flow
async/await is supported in Node.js 7.6
util function to promisify callback is supported in Node.js v8
Sample Code:
const foo = async () => {
try {
const ids = [100, 101, 102];
const fetchFromExternalApi = util.promisify(fetchFromExternalApiCallback);
const promises = ids.map((id) => fetchFromExternalApi(id));
const dataList = await Promise.resolve(promises); // dataList is an array
return dataList;
} catch (err) {
// error handling
}
};
I see several response with Promise.all(), but this function stop if any promise generate an exception...
The best solution in 2022 is Promise.allSettled() (documentation here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled
A quick sample:
const allPromises = [];
for (r in records) {
const promise = update_async(r);
allPromises.push(promise);
};
await Promise.allSettled(allPromises);
At the end, you obtain with allPromises an array with the result of each promise:
when ok --> {status: “fulfilled”, value: xxx }
when error --> {status: "rejected", reason: Error: an error}
Related
I have code that looks something like this in javascript:
forloop {
//async call, returns an array to its callback
}
After ALL of those async calls are done, I want to calculate the min over all of the arrays.
How can I wait for all of them?
My only idea right now is to have an array of booleans called done, and set done[i] to true in the ith callback function, then say while(not all are done) {}
edit: I suppose one possible, but ugly solution, would be to edit the done array in each callback, then call a method if all other done are set from each callback, thus the last callback to complete will call the continuing method.
You haven't been very specific with your code, so I'll make up a scenario. Let's say you have 10 ajax calls and you want to accumulate the results from those 10 ajax calls and then when they have all completed you want to do something. You can do it like this by accumulating the data in an array and keeping track of when the last one has finished:
Manual Counter
var ajaxCallsRemaining = 10;
var returnedData = [];
for (var i = 0; i < 10; i++) {
doAjax(whatever, function(response) {
// success handler from the ajax call
// save response
returnedData.push(response);
// see if we're done with the last ajax call
--ajaxCallsRemaining;
if (ajaxCallsRemaining <= 0) {
// all data is here now
// look through the returnedData and do whatever processing
// you want on it right here
}
});
}
Note: error handling is important here (not shown because it's specific to how you're making your ajax calls). You will want to think about how you're going to handle the case when one ajax call never completes, either with an error or gets stuck for a long time or times out after a long time.
jQuery Promises
Adding to my answer in 2014. These days, promises are often used to solve this type of problem since jQuery's $.ajax() already returns a promise and $.when() will let you know when a group of promises are all resolved and will collect the return results for you:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push($.ajax(...));
}
$.when.apply($, promises).then(function() {
// returned data is in arguments[0][0], arguments[1][0], ... arguments[9][0]
// you can process it here
}, function() {
// error occurred
});
ES6 Standard Promises
As specified in kba's answer: if you have an environment with native promises built-in (modern browser or node.js or using babeljs transpile or using a promise polyfill), then you can use ES6-specified promises. See this table for browser support. Promises are supported in pretty much all current browsers, except IE.
If doAjax() returns a promise, then you can do this:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push(doAjax(...));
}
Promise.all(promises).then(function() {
// returned data is in arguments[0], arguments[1], ... arguments[n]
// you can process it here
}, function(err) {
// error occurred
});
If you need to make a non-promise async operation into one that returns a promise, you can "promisify" it like this:
function doAjax(...) {
return new Promise(function(resolve, reject) {
someAsyncOperation(..., function(err, result) {
if (err) return reject(err);
resolve(result);
});
});
}
And, then use the pattern above:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push(doAjax(...));
}
Promise.all(promises).then(function() {
// returned data is in arguments[0], arguments[1], ... arguments[n]
// you can process it here
}, function(err) {
// error occurred
});
Bluebird Promises
If you use a more feature rich library such as the Bluebird promise library, then it has some additional functions built in to make this easier:
var doAjax = Promise.promisify(someAsync);
var someData = [...]
Promise.map(someData, doAjax).then(function(results) {
// all ajax results here
}, function(err) {
// some error here
});
Checking in from 2015: We now have native promises in most recent browser (Edge 12, Firefox 40, Chrome 43, Safari 8, Opera 32 and Android browser 4.4.4 and iOS Safari 8.4, but not Internet Explorer, Opera Mini and older versions of Android).
If we want to perform 10 async actions and get notified when they've all finished, we can use the native Promise.all, without any external libraries:
function asyncAction(i) {
return new Promise(function(resolve, reject) {
var result = calculateResult();
if (result.hasError()) {
return reject(result.error);
}
return resolve(result);
});
}
var promises = [];
for (var i=0; i < 10; i++) {
promises.push(asyncAction(i));
}
Promise.all(promises).then(function AcceptHandler(results) {
handleResults(results),
}, function ErrorHandler(error) {
handleError(error);
});
You can use jQuery's Deferred object along with the when method.
deferredArray = [];
forloop {
deferred = new $.Deferred();
ajaxCall(function() {
deferred.resolve();
}
deferredArray.push(deferred);
}
$.when(deferredArray, function() {
//this code is called after all the ajax calls are done
});
You can emulate it like this:
countDownLatch = {
count: 0,
check: function() {
this.count--;
if (this.count == 0) this.calculate();
},
calculate: function() {...}
};
then each async call does this:
countDownLatch.count++;
while in each asynch call back at the end of the method you add this line:
countDownLatch.check();
In other words, you emulate a count-down-latch functionality.
This is the most neat way in my opinion.
Promise.all
FetchAPI
(for some reason Array.map doesn't work inside .then functions for me. But you can use a .forEach and [].concat() or something similar)
Promise.all([
fetch('/user/4'),
fetch('/user/5'),
fetch('/user/6'),
fetch('/user/7'),
fetch('/user/8')
]).then(responses => {
return responses.map(response => {response.json()})
}).then((values) => {
console.log(values);
})
Use an control flow library like after
after.map(array, function (value, done) {
// do something async
setTimeout(function () {
// do something with the value
done(null, value * 2)
}, 10)
}, function (err, mappedArray) {
// all done, continue here
console.log(mappedArray)
})
In Node.js you can use async/await to control the async flow
async/await is supported in Node.js 7.6
util function to promisify callback is supported in Node.js v8
Sample Code:
const foo = async () => {
try {
const ids = [100, 101, 102];
const fetchFromExternalApi = util.promisify(fetchFromExternalApiCallback);
const promises = ids.map((id) => fetchFromExternalApi(id));
const dataList = await Promise.resolve(promises); // dataList is an array
return dataList;
} catch (err) {
// error handling
}
};
I see several response with Promise.all(), but this function stop if any promise generate an exception...
The best solution in 2022 is Promise.allSettled() (documentation here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled
A quick sample:
const allPromises = [];
for (r in records) {
const promise = update_async(r);
allPromises.push(promise);
};
await Promise.allSettled(allPromises);
At the end, you obtain with allPromises an array with the result of each promise:
when ok --> {status: “fulfilled”, value: xxx }
when error --> {status: "rejected", reason: Error: an error}
I have code that looks something like this in javascript:
forloop {
//async call, returns an array to its callback
}
After ALL of those async calls are done, I want to calculate the min over all of the arrays.
How can I wait for all of them?
My only idea right now is to have an array of booleans called done, and set done[i] to true in the ith callback function, then say while(not all are done) {}
edit: I suppose one possible, but ugly solution, would be to edit the done array in each callback, then call a method if all other done are set from each callback, thus the last callback to complete will call the continuing method.
You haven't been very specific with your code, so I'll make up a scenario. Let's say you have 10 ajax calls and you want to accumulate the results from those 10 ajax calls and then when they have all completed you want to do something. You can do it like this by accumulating the data in an array and keeping track of when the last one has finished:
Manual Counter
var ajaxCallsRemaining = 10;
var returnedData = [];
for (var i = 0; i < 10; i++) {
doAjax(whatever, function(response) {
// success handler from the ajax call
// save response
returnedData.push(response);
// see if we're done with the last ajax call
--ajaxCallsRemaining;
if (ajaxCallsRemaining <= 0) {
// all data is here now
// look through the returnedData and do whatever processing
// you want on it right here
}
});
}
Note: error handling is important here (not shown because it's specific to how you're making your ajax calls). You will want to think about how you're going to handle the case when one ajax call never completes, either with an error or gets stuck for a long time or times out after a long time.
jQuery Promises
Adding to my answer in 2014. These days, promises are often used to solve this type of problem since jQuery's $.ajax() already returns a promise and $.when() will let you know when a group of promises are all resolved and will collect the return results for you:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push($.ajax(...));
}
$.when.apply($, promises).then(function() {
// returned data is in arguments[0][0], arguments[1][0], ... arguments[9][0]
// you can process it here
}, function() {
// error occurred
});
ES6 Standard Promises
As specified in kba's answer: if you have an environment with native promises built-in (modern browser or node.js or using babeljs transpile or using a promise polyfill), then you can use ES6-specified promises. See this table for browser support. Promises are supported in pretty much all current browsers, except IE.
If doAjax() returns a promise, then you can do this:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push(doAjax(...));
}
Promise.all(promises).then(function() {
// returned data is in arguments[0], arguments[1], ... arguments[n]
// you can process it here
}, function(err) {
// error occurred
});
If you need to make a non-promise async operation into one that returns a promise, you can "promisify" it like this:
function doAjax(...) {
return new Promise(function(resolve, reject) {
someAsyncOperation(..., function(err, result) {
if (err) return reject(err);
resolve(result);
});
});
}
And, then use the pattern above:
var promises = [];
for (var i = 0; i < 10; i++) {
promises.push(doAjax(...));
}
Promise.all(promises).then(function() {
// returned data is in arguments[0], arguments[1], ... arguments[n]
// you can process it here
}, function(err) {
// error occurred
});
Bluebird Promises
If you use a more feature rich library such as the Bluebird promise library, then it has some additional functions built in to make this easier:
var doAjax = Promise.promisify(someAsync);
var someData = [...]
Promise.map(someData, doAjax).then(function(results) {
// all ajax results here
}, function(err) {
// some error here
});
Checking in from 2015: We now have native promises in most recent browser (Edge 12, Firefox 40, Chrome 43, Safari 8, Opera 32 and Android browser 4.4.4 and iOS Safari 8.4, but not Internet Explorer, Opera Mini and older versions of Android).
If we want to perform 10 async actions and get notified when they've all finished, we can use the native Promise.all, without any external libraries:
function asyncAction(i) {
return new Promise(function(resolve, reject) {
var result = calculateResult();
if (result.hasError()) {
return reject(result.error);
}
return resolve(result);
});
}
var promises = [];
for (var i=0; i < 10; i++) {
promises.push(asyncAction(i));
}
Promise.all(promises).then(function AcceptHandler(results) {
handleResults(results),
}, function ErrorHandler(error) {
handleError(error);
});
You can use jQuery's Deferred object along with the when method.
deferredArray = [];
forloop {
deferred = new $.Deferred();
ajaxCall(function() {
deferred.resolve();
}
deferredArray.push(deferred);
}
$.when(deferredArray, function() {
//this code is called after all the ajax calls are done
});
You can emulate it like this:
countDownLatch = {
count: 0,
check: function() {
this.count--;
if (this.count == 0) this.calculate();
},
calculate: function() {...}
};
then each async call does this:
countDownLatch.count++;
while in each asynch call back at the end of the method you add this line:
countDownLatch.check();
In other words, you emulate a count-down-latch functionality.
This is the most neat way in my opinion.
Promise.all
FetchAPI
(for some reason Array.map doesn't work inside .then functions for me. But you can use a .forEach and [].concat() or something similar)
Promise.all([
fetch('/user/4'),
fetch('/user/5'),
fetch('/user/6'),
fetch('/user/7'),
fetch('/user/8')
]).then(responses => {
return responses.map(response => {response.json()})
}).then((values) => {
console.log(values);
})
Use an control flow library like after
after.map(array, function (value, done) {
// do something async
setTimeout(function () {
// do something with the value
done(null, value * 2)
}, 10)
}, function (err, mappedArray) {
// all done, continue here
console.log(mappedArray)
})
In Node.js you can use async/await to control the async flow
async/await is supported in Node.js 7.6
util function to promisify callback is supported in Node.js v8
Sample Code:
const foo = async () => {
try {
const ids = [100, 101, 102];
const fetchFromExternalApi = util.promisify(fetchFromExternalApiCallback);
const promises = ids.map((id) => fetchFromExternalApi(id));
const dataList = await Promise.resolve(promises); // dataList is an array
return dataList;
} catch (err) {
// error handling
}
};
I see several response with Promise.all(), but this function stop if any promise generate an exception...
The best solution in 2022 is Promise.allSettled() (documentation here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled
A quick sample:
const allPromises = [];
for (r in records) {
const promise = update_async(r);
allPromises.push(promise);
};
await Promise.allSettled(allPromises);
At the end, you obtain with allPromises an array with the result of each promise:
when ok --> {status: “fulfilled”, value: xxx }
when error --> {status: "rejected", reason: Error: an error}
I am looping through an array, and in each loop, I add to an array of promises that is then passed into $q.all. Each chain includes a confirm() dialogue and a modal. The sequence of events for the user should be confirm() - modal - confirm() - modal. Instead, I'm getting confirm() - confirm() - modal - modal. Also, I'd like a function refreshSelection() to execute after the last modal is closed, but it currently fires as soon as the last confirm() dialogue is closed.
var promiseArr = [];
var selection = []; // [obj1, obj1, obj3...]
for (var i = 0; i < selection.length; i++) {
promiseArr.push(getData(i));
}
$q.all(promiseArr)
.then(refreshSelection);
function getData(i) {
var opts = {}; // selection[i].id is param
return $http(opts)
.then(onGetSuccess(i));
}
function onGetSuccess(i) {
return function(result) {
var shouldOpenModal = confirm(selection[i].text);
if (shouldOpenModal == true) {
openModal();
}
}
}
function openModal() {
var copyPunchesModal = $uibModal.open({
// modal template options
}
});
copyPunchesModal.result.then(otherFunc, function () {
console.log("Modal closed");
});
}
function refreshSelection() {
selection = [];
}
I have also tried the following to no avail.
//...
function getData(i) {
var opts = {}; // selection[i].id is param
return $http(opts)
.then(onGetSuccess(i))
.then(openConfirm)
.then(openModal);
}
Help is much appreciated.
Q.all doesn't specify the order in which the promises will be resolved. If you want the promises to be completed in the order of the array you need to chain the promises so that the current promise is executed inside the .then() of the previous promise.
You can do this cleanly with reduce.
I've got an answer here for ES6 promises, here it is adapted to Q:
deferred = $q.defer();
deferred.resolve();
return selection.reduce(
(accumulator, current) => accumulator.then(()=>getData(current)),
deferred.promise
);
Note that you'll also need to modify your getData function to operate on the object itself (so currentSelection.text instead of selection[i].text, for example)
You want to use a $.when function, which takes as an argument a number of promises and executes a function when they all complete. Alternatively, you can pipe them all together if you are ok with them completing in sequential order.
https://api.jquery.com/jquery.when/
I wanted to do something similar to avoid overflowing a device my program was controlling - basically I needed to wait for the async transaction to complete before starting the next one. I used promises to enforce both the order of execution, and to block until each item was complete.
// the async thing we want to do, wrapped in a promise
//
function sendMessage = (message) {
var promise = new Promise (resolve, reject) {
// send message via an async function that provides success & fail callbacks, e.g. AJAX
// replace 'asyncFunction', it's just a placeholder in this example
asyncFunction (message,
function (response) { // success callback
if (/*response is to my liking*/) {
resolve ();
} else {
reject (new Error (/*why I didn't like the response*/));
}
},
function (error) { // failure callback
reject (error);
}
};
return promise;
}
// the array we want to process in order
var myMessages = ['hello', 'world', 'how', 'are', 'you', 'today?'];
// iterator, and recursive function to loop thru array
var idx = 0;
function next () {
if (idx < myMessages.length) {
sendMessage (myMessages[idx++]).then (
next, // maps to the 'resolve' call in sendMessage
function (err) { // maps to the 'reject' calls in sendMessage
throw err; // clearly, there's an option to be smarter than this
}
)
}
}
// start the recursion
next ();
I have this API call where i make sure the data return in the same order i send it. However, i realized thats not really what i want, i want to make sure the data is send and taken care of one at a time.
data[n] has returned before data[n+1] is send.
the reason for this is:
If i do it as seen below, the server still gets it in a random order, and therefor saves the data in my DB in a random order. (or well not random, heavier data gets processed slower)
var promiseArray = [];
for (var i = 0; i < data.length; i++) {
var dataPromise = $http.post('/api/bla/blabla', $httpParamSerializer(data[i]))
.then (function (response) {
//return data for chaining
return response.data;
});
promiseArray.push(dataPromise);
}
$q.all(promiseArray).then(function (dataArray) {
//succes
}).catch (function (errorResponse) {
//error
});
how can i make sure the data is send and processed and returned, one at a time in a smooth way ?
You could do something like this:
var i = -1;
processNextdata();
function processNextdata() {
i++;
if(angular.isUndefined(data[i]))
return;
$http.post('/api/bla/blabla', $httpParamSerializer(data[i]))
.then(processNextdata)
}
Update:
Callback after every result:
var i = -1;
processNextdata();
function processNextdata() {
i++;
if(angular.isUndefined(data[i]))
return;
$http.post('/api/bla/blabla', $httpParamSerializer(data[i]))
.then(function(result) {
// do something with a single result
return processNextdata();
}, errorCallback);
}
Callback after everything is done:
var i = -1, resultData = [];
processNextdata()
.then(function(result) {
console.log(result);
}, errorCallback);
function processNextdata() {
i++;
if(angular.isUndefined(data[i]))
return resultData;
$http.post('/api/bla/blabla', $httpParamSerializer(data[i]))
.then(function(result) {
resultData.push(result.data);
return processNextdata();
}, $q.reject);
}
When using the Promise.all([...]) method, the documentation shows the following:
The Promise.all(iterable) method returns a promise that resolves when all of the promises in the iterable argument have resolved, or rejects with the reason of the first passed promise that rejects.
What this tells us is that there is no expected order of synchronized operations, but in fact the promises run parallel to one another and can complete in any order.
In your case, there is an expected order that you want your promises to run in, so using Promise.all([...]) won't satisfy your requirements.
What you can do instead is execute individual promises, then if you have some that can run in parallel use the Promise.all([...]) method.
I would create a method that takes a request as an argument, then returns the generated promise:
function request (req) {
return new Promise(function (resolve, reject) {
request({
url: url
, port: <port>
, body: req
, json: <true/false>
, method: '<POST/GET>'
, headers: {
}
}, function (error, response, body) {
if (error) {
reject(error);
} else {
resolve(body);
}
});
});
You can then call this function and store the result:
var response = request(myRequest);
Alternatively, you could create an array of your requests and then call the function:
var requests = [request1, request2, ..., requestN];
var responses = [];
for (var i = 0; i < requests.length; i++) {
responses.push(request(requests[i]));
}
I have some code that I cant get my head around, I am trying to return an array of object using a callback, I have a function that is returning the values and then pushing them into an array but I cant access this outside of the function, I am doing something stupid here but can't tell what ( I am very new to Node.JS )
for (var index in res.response.result) {
var marketArray = [];
(function () {
var market = res.response.result[index];
createOrUpdateMarket(market, eventObj , function (err, marketObj) {
marketArray.push(marketObj)
console.log('The Array is %s',marketArray.length) //Returns The Array is 1.2.3..etc
});
console.log('The Array is %s',marketArray.length) // Returns The Array is 0
})();
}
You have multiple issues going on here. A core issue is to gain an understanding of how asynchronous responses work and which code executes when. But, in addition to that you also have to learn how to manage multiple async responses in a loop and how to know when all the responses are done and how to get the results in order and what tools can best be used in node.js to do that.
Your core issue is a matter of timing. The createOrUpdateMarket() function is probably asynchronous. That means that it starts its operation when the function is called, then calls its callback sometime in the future. Meanwhile the rest of your code continues to run. Thus, you are trying to access the array BEFORE the callback has been called.
Because you cannot know exactly when that callback will be called, the only place you can reliably use the callback data is inside the callback or in something that is called from within the callback.
You can read more about the details of the async/callback issue here: Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference
To know when a whole series of these createOrUpdateMarket() operations are all done, you will have to code especially to know when all of them are done and you cannot rely on a simple for loop. The modern way to do that is to use promises which offer tools for helping you manage the timing of one or more asynchronous operations.
In addition, if you want to accumulate results from your for loop in marketArray, you have to declare and initialize that before your for loop, not inside your for loop. Here are several solutions:
Manually Coded Solution
var len = res.response.result.length;
var marketArray = new Array(len), cntr = 0;
for (var index = 0, index < len; index++) {
(function(i) {
createOrUpdateMarket(res.response.result[i], eventObj , function (err, marketObj) {
++cntr;
if (err) {
// need error handling here
}
marketArray[i] = marketObj;
// if last response has just finished
if (cntr === len) {
// here the marketArray is fully populated and all responses are done
// put your code to process the marketArray here
}
});
})(index);
}
Standard Promises Built Into Node.js
// make a version of createOrUpdateMarket that returns a promise
function createOrUpdateMarketAsync(a, b) {
return new Promise(function(resolve, reject) {
createOrUpdateMarket(a, b, function(err, marketObj) {
if (err) {
reject(err);
return;
}
resolve(marketObj);
});
});
}
var promises = [];
for (var i = 0; i < res.response.result.length; i++) {
promises.push(createorUpdateMarketAsync(res.response.result[i], eventObj));
}
Promise.all(promises).then(function(marketArray) {
// all results done here, results in marketArray
}, function(err) {
// an error occurred
});
Enhanced Promises with the Bluebird Promise library
The bluebird promise library offers Promise.map() which will iterate over your array of data and produce an array of asynchronously obtained results.
// make a version of createOrUpdateMarket that returns a promise
var Promise = require('bluebird');
var createOrUpdateMarketAsync = Promise.promisify(createOrUpdateMarket);
// iterate the res.response.result array and run an operation on each item
Promise.map(res.response.result, function(item) {
return createOrUpdateMarketAsync(item, eventObj);
}).then(function(marketArray) {
// all results done here, results in marketArray
}, function(err) {
// an error occurred
});
Async Library
You can also use the async library to help manage multiple async operations. In this case, you can use async.map() which will create an array of results.
var async = require('async');
async.map(res.response.result, function(item, done) {
createOrUpdateMarker(item, eventObj, function(err, marketObj) {
if (err) {
done(err);
} else {
done(marketObj);
}
});
}, function(err, results) {
if (err) {
// an error occurred
} else {
// results array contains all the async results
}
});