We Keep Coding

How to handle response to several asynchronous request (AJAX calls) made in a loop

I need to collect logs for different devices from a backend, to export it to a csv-file. Problem is the number of devices can vary. So I ask the backend for the amount of devices and loop through the requests for the logs.
The problem is the for... loop runs too faster than I get the responses to $.post, so the loop finishes before I get the responses. After some research I could handle that behaviour, but now I was asked to add data to the request, for which I have no reference to the according device is stored. So I added the device names and spots I need to poll in an external js-file, so I have a defined list to loop through.
I tried to use the index of the for loop to fetch the device names, which didn't work, since the loop was too fast. Now I have created a workaround, by defining a counter and pushing the devices in another variable. This doesn't feel "clean" and there should be a better way to poll the data and keep track for which device it is.
The code so far:
function collectData() {
var outString = "";
var lots of stuff I can pre-fetch
var logs = function (outString, saveCSV) {
var postString;
var devices = [];
var count = 0;
for (i = 1; i <= maxDevice; i++) {
postString = build postString in loop
devices.push(spots[0][i - 1]);
$.post('/path/foobar.db',
postString,
function (data) {
outString += "Spotlist for: " + spots[0][count] + "\n";
count++;
outString += data.replace(/{/g, "").replace(/}/g, "").replace(/:/g, ";").replace(/,/g, "\n").replace(/\"/g, "");
outString += "\n\n";
});
postString = "/path/eventlog.csv?device=" + i;
$.get(postString,
function (data) {
outString += "Event Log: \n" + data + "\n";
});
postString = "/path/errorlog.csv?device=" + i;
$.get(postString,
function (data) {
outString += "Error Log: \n" + data + "\n";
});
}
$(document).ajaxStop(function () {
saveCSV(outString, filename);
$(this).unbind('ajaxStop');
});
};
var saveCSV = function (outString, filename) {
var tempString = "data:text/csv;charset=utf-8," + outString;
var encodedUri = encodeURI(tempString);
var a = document.getElementById("dlLink");
if (window.navigator.msSaveOrOpenBlob) {
blobObject = new Blob([outString], {type: 'text/csv;charset=utf-8'});
window.navigator.msSaveBlob(blobObject, filename);
} else
{
a.setAttribute("href", encodedUri);
a.setAttribute("download", filename);
a.click();
}
};
outString = lots of predefined and pre-fetched stuff
outString += "Device data: \n\n";
logs(outString, saveCSV);
}
The part which I am not satisfied with is:
for (i = 1; i <= maxDevice; i++) {
postString = "get = {" + i + ":en:[";
for (j = 0; j < spots[i].length; j++) {
postString += '"' + spots[i][j] + '",';
}
postString = postString.slice(0, -1) + "]}";
devices.push(spots[0][i - 1]);
$.post('/path/foobar.db',
postString,
function (data) {
outString += "Spotlist for: " + spots[0][count] + "\n";
count++;
outString += data.replace(/{/g, "").replace(/}/g, "").replace(/:/g, ";").replace(/,/g, "\n").replace(/\"/g, "");
outString += "\n\n";
});
To output the device that I collected the spots for I use the counter, to track device names. I have the hunch this is not the best and "cleanest" method, so I'd like to ask if there is any better way to deal with the asynchronity (sp?) in terms of collecting the right device for which the post is made and also to trigger the DL if everything is done.
Since my question doesn't seem to be clear, perhaps I need to narrow it down. The code works, but it seems just to be tinkered by me and there should be cleaner ways to
A) handle the posts/gets, since the outstring for CSV is just put together in the way the requests are answered, so not device 1 is the first in the csv, but the one which comes first. $(document).ajaxStop waits for everything to be finished, but not to be finished in the right order.
B) I need to relate the index of the for loop to the device I poll the data for. I used additional variables, that I count up to go through an additional array. Is there any better way?

The problem is that you need to run in order the methods that are invoked after you get the response to the AJAX calls.
To do so you must undertand that all jQuery AJAX calls return promises. Instead of passing the code to run as a parameter, you can do the following:
var functionToRunAfterResponse(params) = function(params) {
// do something with params
};
var promise = $.post(/*...*/); // some kind of ajax call
promise.then(functionToRunAfterResponse);
Note that the functionToRunAfterResponse will receive the response data as parameter. I.e. it's equivalent to:
promise.then(function(reponseData) {
functionToRunAfterResponse(responseData);
});
This is how it works for a simple call. See $.then and deferred and promise docs.
If you have to make a bunch of calls, you have to do the following:
store the promises of the AJAX calls, for example in an array
check that all the promises are fulfilled (i.e. that all responses have arrived)
run the code in the required order. I.e. run the promise.then in the right order to get the desired result.
To do so, you can use $.when.
The code structure should be like this (pseudocode):
var promises = [];
// Make the calls, and store the promises
for(/**/) {
promises.push( $.post or some other kind of ajax call );
}
// Ensure that all the responses have arrived
$.when.apply($, promises) // See below note
.then(function() {
for each promise in promises
promise[i].then(function(reponseData) {
// run the necessary code
});
NOTE: Here you've got a deeper explanation, but, basically, as $.when expects a series of promises, like this: $.when(promise1, promise2, ...) and you have an array promises[], you have to use apply to make the call, so that the items in the array are passed as individual parameters.
FINAL NOTES:
1) take into account that AJAX calls can fail. In that case, instead of a resolved promise, you get a failed promise. You should check it. If any of the promises fails, $.when will not work as desired:
The method [when] will resolve its master Deferred as soon as all the Deferreds resolve, or reject the master Deferred as soon as one of the Deferreds is rejected.
2) To handle errors, then has two parameters:
$.when.apply().then(function() { /* success */ }, function() { /* error */ });
3) When you make the calls as I've explained, they are all executed in parallell, and the responses will arrive in any order. I.e. there is no warranty that you'll receive all the responses in the order you made the calls. They can even fail, as I explained in 1) That's why you must use when and run then in order again
4) Working with asynchronous methods is great, but you have the responsibility to check that they didn't fail, and run in the right order the code that you need to run after you get the responses.

I can't understand your question, but the main problem is the asynchronity, right?
Try to call the functions asyncronously (summary version):
// Function that manage the data from ajax
var myResponseFunction = function (data) {
$('#response').html('data: ' + JSON.stringify(data));
};
// Ajax function, like your post with one parameter (add all you need)
function myAJAXFunction(callback) {
$.ajax({
type: 'POST',
url: '/echo/json/',
dataType: 'json',
data: {
json: JSON.stringify({
'foo': 'bar'
})
}
}).done(function(response) {
// Done!! Now is time to manage the answer
callback(response);
}).fail(function (jqXHR, textStatus, errorThrown) {
window.console.error('Error ' + textStatus + ': ' + errorThrown);
});
}
// Usually, this function it's inside "document.ready()".
// To avoid the ajax problem we call the function and "data manage function" as parameter.
for (i = 1; i <= maxDevice; i++) {
myAJAXFunction(myResponseFunction);
}
https://jsfiddle.net/erknrio/my1jLfLr/
This example is in spanish but in this answer you have the code commented in english :).
Sorry for my english :S.

Ajax call in loop

I know there are lot of question regarding this but still I am unable to find a proper answer which makes my code run properly.
I have one function defined to call ajax which I cannot change due to security issue. This is how I call that function
var JsonIQDetails = JSON.stringify(input);//Some input
//pram 1:MethodUrl, 2:JsonObject, 3:ReturnType, 4:SuccessCallBackFunction
InvokeAjaxCall(Url, JsonIQDetails, "json", Success);
I have array of objects (more than 500). Since JSON is getting very long so I am unable to make ajax call. Again due to security issue I can't change config file too. So JSON length cannot be increased.
I am dividing the array into small chunks of 100 and calling the method
for (i = 0, j = mainObject.length; i < j; i += chunk) {
var newSubObject = mainObject.slice(i, i + chunk);
InvokeAjaxCall(Url, newSubObject, "json", Success);
function Success(data) {
if (!data) {
alert("Failed");
break;
}
}
}
Its moving without completing the for loop and executing the next code. So I want first it to complete the for loop (Probably asynchronous)
Thanks in Advance..!!!

Ajax is by default Asynchronous, so you pretty much need to invoke the next part of your ajax call in your success function. Here is a recursive loop that takes care of that.
var ajaxRecursive = function(i, j, c){
if(i < j){
var newSubObject = mainObject.slice(i, i + chunk);
InvokeAjaxCall(Url, newSubObject , "json", function(data){
//do stuff with data
ajaxRecursive(i+=chunk, j,chunk);
});
}
}
ajaxRecursive(0, mainObject.length, chunk);
Supposing that the other variables within ajaxRecursive are defined globally.
Update description:
You can get rid of your "success" function and just create it annonymously.

How to make a callback for my loop function

I have a JSON file that keeps URLs. I made a $.each() loop to go through the JSON file and then do a SoundCloud function on each iteration of the loop. So in order to get same result of the loop I have to make a callback after each iteration for my SoundCloud function. Here is what I tried:
$.getJSON("http://www.json-generator.com/api/json/get/bLjOHIYsAy?indent=2", function(data){ //Link of the playlist
$.each(data.PlayListArray, function(key, val){ //navigate to array called PlayListArray
var songLink = val.URL; // the value of URL in the array
}, function(){// Callback Function
SC.get('/resolve', { url: songLink }, function(track) {
$("#demo").append("<p id= "+ track.id + ">" + track.title + "</p>");
});
});
});
Howevere this callback does not work and does not show the gathered data from SC.get() function of SoundCloud.
Any idea to make it working? Or How can I have deferred method to make a chain of functions ??
demo: http://jsfiddle.net/Fq2Rw/5/

What you're doing here is passing a 3rd parameter to each which is expecting only two. Unless I'm missing something about your exact goal, the code below should work as expected.
SC.initialize({
client_id: "b8f06bbb8e4e9e201f9e6e46001c3acb",
});
$.getJSON("http://www.json-generator.com/api/json/get/bLjOHIYsAy?indent=2", function(data){ //Link of the playlist
$.each(data.PlayListArray, function(key, val){ //navigate to array called PlayListArray
var songLink = val.URL; // the value of URL in the array
SC.get('/resolve', { url: songLink }, function(track) {
$("#demo").append("<p id= "+ track.id + ">" + track.title + "</p>");
});
});
});
EDIT: I now understand that you want to get the results in the order defined by the data.PlayListArray result set. I think you need to buffer the results and process them only when all answers have been received. The code below is probably not optimal but it may give you some new ideas.
There's is, of course, no way to control in which order the different 'SC.get()' will respond and, consequently, in which order your 'function(track)' callbacks will be called. Waiting for each answer before making the next call -- like your original post is suggesting -- is indeed another possibility, but it will be significantly slower than making all calls in parallel (like the code below does).
SC.initialize({
client_id: "b8f06bbb8e4e9e201f9e6e46001c3acb",
});
$.getJSON("http://www.json-generator.com/api/json/get/bLjOHIYsAy?indent=2", function(data){ //Link of the playlist
var answer = {};
$.each(data.PlayListArray, function(key, val){ //navigate to array called PlayListArray
var songLink = val.URL; // the value of URL in the array
SC.get('/resolve', { url: songLink }, function(track) {
answer[songLink] = track;
if(Object.keys(answer).length == data.PlayListArray.length) {
// we've got all results: let's process them by iterating on data.PlayListArray again
$.each(data.PlayListArray, function(key, val){
var track = answer[val.URL];
$("#demo").append("<p id= "+ track.id + ">" + track.title + "</p>");
});
}
});
});
});

If you want to do async calls in a loop, the rule of thumb is to use a recursive function instead of a for loop.
function loop(i){
if(i >= data.PlayListArray.length){
//DONE
}else{
SC.get('blabla', function(track){
//Loop body
loop(i+1);
});
}
}
loop(0);
As Arnaulid already pointed out, this sequential execution of callbacks is going to exibit high latency so first be sure that this is what you really want to do.

As already pointed out, $.getJSON takes arguments (url, callback), however, you can also chain .then(callback), which is the "promisy" way to do it.
Simplest way to ensure that the displayed list is in the same order as the array from which it is derived, is to append an empty p element, then populate it with data when it arrives. You can rely on closure formed by the $.each function to keep a reliable reference to the appended p.
$.getJSON("http://www.json-generator.com/api/json/get/bLjOHIYsAy?indent=2").then(function(data) { //Link of the playlist
var $demo = $("#demo");
$.each(data.PlayListArray, function(i, item) { //traverse the PlayListArray array
var $p = $("<p/>").appendTo($demo);//appending an empty <p></p> here ensures the observable playList will be in the same order as data.PlayListArray.
SC.get('/resolve', { url: item.URL }, function (track) {
$p.attr('id', track.id).text(track.title);//flesh out the <p></p> appended above. $p remains available due to closure formed by the outer function.
});
});
});
Note also that by assigning $demo ouside the loop avoids the inefficiency of discovering $("#demo") in the DOM on every iteration of the each loop.
Updated fiddle

jQuery .push into an Array in a .get call gives an empty result

Can anyone tell me why the below gives me an empty string? When I console.log(contentArray) in the $.get() callback function it shows the data but when I try to do it where it is in the code below, the result is empty.
sectionArray = [];
contentArray = [];
$(function () {
if (index == 1) {
$('menu:eq(' + (section - 1) + ') li a').each(function () {
sectionArray.push($(this).attr('href'));
});
var len = sectionArray.length;
for (var i = 0; i < len; i++) {
href2 = sectionArray[i];
$.get(href2, function (data) {
string = data.toString();
contentArray.push(string);
});
}
content = contentArray.toString();
console.log(content);
}

because ajax request ends after you call console.log() try this:
$.get(href2, function(data){
string = data.toString();
contentArray.push(string);
content = contentArray.toString();
console.log(content);
});
also do ajax request in loop is not best thing to do. that wont work as you want.
UPDATE:
also jQuery has async option set to false and your code should work but will work slow. Synchronous requests may temporarily lock the browser.
UPDATE 2
maybe try something like this(maybe not so good idea :D):
var countRequests = len;
$.get(href2, function(data){
string = data.toString();
contentArray.push(string);
countRequests = countRequests - 1;
if (countRequests == 0) {
content = contentArray.toString();
console.log(content);
// or create callback
}
});

The problem is that your $.get() ajax requests are executed asynchronously.
That is, the $.get() function returns immediately without waiting for the response, your entire for loop completes (queueing up multiple ajax requests), then your console.log() occurs at which point the array is still empty. Only after that do any of the ajax success handlers get called, regardless of how fast the ajax responses come back.
EDIT: Here is an answer from another question that shows how to do something after all the ajax calls have completed: https://stackoverflow.com/a/6250103/615754

Javascript/jQuery variables not giving expected values

Like others before me I'm struggling with scope in Javascript. (That and trying to read the darn stuff). I have checked some of the previous threads on this question but I cant seem to get them to apply correctly to my issuue.
In the example below, I want to manipulate the values in the tagsArr array, once the array has been fully populated. I declared the tagsArr variable outside the scope of the function in which it is populated in order to access it globally. But the variable doesn't seem to have the scope I expect - tagsArr.length is 0 at the point where I call output it to console on line 16.
$(function(){
var apiKey = [myapikey];
var tags = '';
var tagsArr = new Array();
$.getJSON('http://api.flickr.com/services/rest/?&method=flickr.people.getPublicPhotos&api_key=' + apiKey + '&user_id=46206266#N05&extras=date_taken,tags&format=json&jsoncallback=?', function(data){
$.each(data.photos.photo, function(i, item) {
var photoID = item.id;
$.getJSON('http://api.flickr.com/services/rest/?&method=flickr.photos.getInfo&api_key=' + apiKey + '&photo_id=' + photoID + '&format=json&jsoncallback=?', function(data){
if (data.photo.tags.tag != '') {
$.each(data.photo.tags.tag, function(j, item) {
tagsArr.push(item.raw);
});
}
});
tags = tagsArr.join('<br />');
console.debug(tagsArr.length);
});
$('#total-dragged').append(data.photos.total);
$('#types-dragged').append(tags);
});
});

Your calls to getJSON are asynchronous. Hence all the calls to the inner getJSON will still be outstanding by the time the console.debug line is reached. Hence the array length is still 0.
You need to run some extra code once the final getJSON call has completed.
$(function(){
var apiKey = [myapikey];
var tags = '';
var tagsArr = new Array();
$.getJSON('http://api.flickr.com/services/rest/?&method=flickr.people.getPublicPhotos&api_key=' + apiKey + '&user_id=46206266#N05&extras=date_taken,tags&format=json&jsoncallback=?', function(data){
var totalExpected = data.photos.total;
var totalFetched = 0;
$.each(data.photos.photo, function(i, item) {
var photoID = item.id;
$.getJSON('http://api.flickr.com/services/rest/?&method=flickr.photos.getInfo&api_key=' + apiKey + '&photo_id=' + photoID + '&format=json&jsoncallback=?', function(data){
if (data.photo.tags.tag != '') {
$.each(data.photo.tags.tag, function(j, item) {
tagsArr.push(item.raw);
totalFetched += 1;
if (totalFetched == totalExpected)
fetchComplete();
});
}
});
function fetchComplete()
{
tags = tagsArr.join('<br />');
console.debug(tagsArr.length);
}
});
$('#total-dragged').append(data.photos.total);
$('#types-dragged').append(tags);
});
});
This works assuming the total number of photos doesn't excede the default 100 per page, other wise you would need to tweak it.
That said I don't think using .each to fire off loads of getJSON requests makes a great deal of sense. I would refactor it so that only one call to getJSON is outstanding at any one time. Have the callback of one issue the next getJSON for the next photo until all have been pulled then do your completed code.

$.getJSON is asynchronous (the a in ajax). That means that by the time you get to console.debug(), getJSON is still getting. You'll need to do some extra work in the JSON callback.

The reason for this is that getJSON is an asynchronous request. after the call to $.getJSON, the javascript engine will move immediately on to the following two lines of code, and will output the length of your array, which is by then, zero-length. Not until after that does the getJSON request receive a response, and add items to the array.

The getJSON function is asynchronous, so when you call the debug function the array is still empty because the requests are not completed. Use the $.ajax function and set async:false and it will work.
$.ajax({
type: "GET",
url: 'http://api.flickr.com/services/rest/?&method=flickr.photos.getInfo&api_key=' + apiKey + '&photo_id=' + photoID + '&format=json&jsoncallback=?',
dataType: "json",
async:false,
success:function(data){
if (data.photo.tags.tag != '') {
$.each(data.photo.tags.tag, function(j, item) {
tagsArr.push(item.raw);
});
}
}
});

This isn't a scope issue - the problem is that getJSON is asynchronous, so it continues executing immediately after sending the request to flickr. By the time the browser executes console.debug the request hasn't returned and you haven't finished handling the response (and therefore haven't pushed any items into the array yet).
To solve this, find all the code that should only be executed when the array is full and move it into your getJSON callback method:
if (data.photo.tags.tag != '') {
$.each(data.photo.tags.tag, function(j, item) {
tagsArr.push(item.raw);
});
tags = tagsArr.join('<br />');
console.debug(tagsArr.length);
$('#total-dragged').append(data.photos.total);
$('#types-dragged').append(tags);
}

You may want to check the answer to this question I posted. There is some good information on scope issues in javascript.