We Keep Coding

Async Javascript: Waiting for data to be processed in a for loop before proceeding to a new function

I'm having issues understanding how to work around Javascript's asynchronous behavior in a forEach loop. This issue is quite complex (sorry), but the idea of the loop is as followed:
Loop through every item in an array
Make an HTTP request from a provider script
I then need to multiply every element of the array by a constant
Assign the new array to an item in an object
After the loop, take all the arrays and add them together into one array
The data will be assigned to the indvCoinPortfolioChartData array
I'm looking for any flaws in my event loop. I believe the battle is making this task synchronous, making sure my data is assigned before aggregating data.
The issue
When I'm adding all the arrays together, ONE dataset isn't summed up (I think because it's still being processed after the function is called). There is no error, but it doesn't have all the coin data in the final aggregated array.
This is the issue I see in the aggregatePortfolioChartData function. It begins the for loop with only 2 items in the array, and then later shows 3. The third item was not processed until after the for loop started.
image of console log (logged from aggregatePortfolioChartData function)
debug log when aggregation is successful
var indivCoinPortfolioChartData = {'data': []};
for(var i = 0; i < this.storedCoins.Coins.length; i++)
{
let entry = this.storedCoins.Coins[i];
localThis._data.getChart(entry.symbol, true).subscribe(res => {localThis.generateCoinWatchlistGraph(res, entry);});
localThis._data.getChart(entry.symbol).subscribe(res => {
if(entry.holdings > 0)
{
let data = res['Data'].map((a) => (a.close * entry.holdings));
indivCoinPortfolioChartData.data.push({'coinData': data});
localThis.loadedCoinData(loader, indivCoinPortfolioChartData);
}
else
{
localThis.loadedCoinData(loader, indivCoinPortfolioChartData);
}
});
}
Loaded Coin Data
loadedCoinData(loader, indivCoinPortfolioChartData)
{
this.coinsWithData++;
if(this.coinsWithData === this.storedCoins.Coins.length - 1)
{
loader.dismiss();
this.aggregatePortfolioChartData(indivCoinPortfolioChartData);
}
}
aggregatePortfolioChartData
aggregatePortfolioChartData(indivCoinPortfolioChartData)
{
console.log(indivCoinPortfolioChartData);
var aggregatedPortfolioData = [];
if(indivCoinPortfolioChartData.data[0].coinData)
{
let dataProcessed = 0;
for(var i = 0; i < indivCoinPortfolioChartData.data[0].coinData.length; i++)
{
for(var j = 0; j< indivCoinPortfolioChartData.data.length; j++)
{
let data = indivCoinPortfolioChartData.data[j].coinData[i];
if(data)
{
aggregatedPortfolioData[i] = (aggregatedPortfolioData[i] ? aggregatedPortfolioData[i] : 0) + data;
dataProcessed++;
}
else
{
dataProcessed++;
}
}
if(dataProcessed === (indivCoinPortfolioChartData.data[0].coinData.length) * (indivCoinPortfolioChartData.data.length))
{
console.log(dataProcessed + " data points for portfolio chart");
this.displayPortfolioChart(aggregatedPortfolioData);
}
}
}
}
Thank you for helping me get through this irksome issue.

Using response from one API call to do another API Call

I'm using AngularJS to build a site where one of the functions is to present Billboard(it's a music chart) listings for a specified date.
I want to present the songs in order, together with an image of the song.
First I'm calling this API:
http://billboard.modulo.site/
where I give a date and get a response of the top 10 songs for that date and data about each song.
The response from the Billboard API also includes a spotify id and I want to use that ID and call the Spotify Web API to get an image of that song, to complement the information I present about each song.
This is how it looks like in my controller:
var spotifyID = [];
$scope.spotifyImg = [];
musicService.getBillboard($scope.date).then(function(data){ //Response is top 10 songs for given date
$scope.status = "";
$scope.songlist = data;
for (var i = 0; i < data.length; i++) {
spotifyID[i] = data[i].spotify_id; //data[i].spotify_id returns the ID of the track, as given by the billboard API
}
$scope.getImages();
});
$scope.getImages = function() {
for (var i = 0; i < spotifyID.length; i++) {
if(spotifyID[i] !== null) {
musicService.getSpotify(spotifyID[i]).then(function(data){
$scope.spotifyImg[i] = data.album.images[0].url; //returns the appropriate image from the Spotify Web API
});
}
}
console.log($scope.spotifyImg);
}
And in my view it would look something like this:
<div ng-repeat = "song in songlist">
<div>{{ song.rank }}</div>
<div>
<img ng-src=" {{ spotifyImg[$index] }}"/>
</div>
</div>
However, it does not work.
When I'm checking the $scope.spotifyImg array in the console, it is of length 11 and only has one element in index 10 and that is the image of the last song(that is the 10th song).
I'm a bit confused as to why the $scope.spotifyImg array only contains one element in index 10. Also why is the array of length 11 when the spotifyID is of length 10?
Any ideas of how I could solve this?

The problem is that getSpotify is run asynchronous, when the responses to these calls come in, i is probably set to spotifyID.length - 1 which means that all callback functions set the $scope.spotifyImg[spotifyID.length - 1] element.
Try this:
$scope.spotifyImg = [];
musicService.getBillboard($scope.date).then(function(data){
$scope.status = "";
$scope.songlist = data;
for (var i = 0; i < data.length; i++) {
$scope.getImage(data[i].spotify_id, i);
}
});
$scope.getImage = function(id, index) {
musicService.getSpotify(id).then(function(data){
$scope.spotifyImg[index] = data.album.images[0].url;
});
}

create separate function and put the content of for loop inside that function and call that function inside the loop
$scope.getImages = function() {
for (var i = 0; i < spotifyID.length; i++) {
if (spotifyID[i] !== null) {
sampleFunc(i);
}
}
}
function sampleFunc(i) {
musicService.getSpotify(spotifyID[i]).then(function(data) {
$scope.spotifyImg[i] = data.album.images[0].url; //returns the appropriate image from the Spotify Web API
});
}
I think reason you get only last index of an array is when you are calling promise inside loop, loop does't wait until the promise returns. it just keep executing.At the time promise returns loop is executed and it;s getting last index of the array. That's why you need to separately call the promise from for loop

You can use IIFE
(function(i){
musicService.getSpotify(spotifyID[i]).then(function (data) {
$scope.spotifyImg[i] = data.album.images[0].url;
});
})(i)
So,your getImages function should be like this.
$scope.getImages = function () {
for (var i = 0; i < spotifyID.length; i++) {
if (spotifyID[i] !== null) {
(function(i){
musicService.getSpotify(spotifyID[i]).then(function (data) {
$scope.spotifyImg[i] = data.album.images[0].url;
});
})(i)
}
}
}

Try using this code
Js code
var spotifyID = [];
$scope.spotifyImg = [];
musicService.getBillboard($scope.date).then(function(data) { //Response is top 10 songs for given date
$scope.status = "";
$scope.songlist = data;
for (var i = 0; i < data.length; i++) {
spotifyID[i] = data[i].spotify_id; //data[i].spotify_id returns the ID of the track, as given by the billboard API
}
$scope.getImages(0);
});
$scope.getImages = function(index) {
if (index == spotifyID.length) {
return;
}
musicService.getSpotify(spotifyID[index]).then(function(data) {
$scope.spotifyImg[index] = data.album.images[0].url; //returns the appropriate image from the Spotify Web API
// call recursive here
$scope.getImages(index++);
});
}
}
call your getimages function recursively so that will add you images in array.

How to wait for multiple WebWorkers in a loop

I have the following issue with Web Workers in JS. I have a heavy duty application doing some simulation. The code runs in multiple Web Workers.
The main thread is running on a WebPage. But could also be a Web Worker, if it makes sense.
Example:
var myWebWorkers = [];
function openWorker(workerCount){
for(var i = 0; i < workerCount; i++){
myWebWorkers[i] = new Worker('worker.js');
myWebWorkers[i].onmessage = function(e){
this.result = e.data;
this.isReady = true;
}
}
}
function setWorkerData(somedata){
// somedata.length is always a multiple of myWebWorkers.length
var elementCntPerWorker = somedata.length / myWebWorkers.length;
myWebWorkers.forEach(function(worker, index){
worker.isReady = false;
worker.postMessage(
somedata.slice(index * elementCntPerWorker,
(index + 1) * elementCntPerWorker - 1));
});
}
var somedata = [...];
openWorker(8);
for(var i = 0; i < 10000; i++){
setWorkerData(somedata);
waitUntilWorkersAreDoneButAllowBrowserToReact(myWebWorkers);
if(x % 100) updateSVGonWebPage
}
function waitUntilWorkersAreDoneButAllowBrowserToReact(){
/* wait for all myWebWorkers-onchange event, but
allow browser to react and don't block a full Web Worker
Following example is my intension. But will not work, because
events are not executed until code excution stops.
*/
somedata = [];
for(var i = 0; i < myWebWorkers.length; i++){
while(!myWebWorkers[i].isReady);
somedata = somedata.concat(myWebWorkers.result);
}
}
What I need is really the waitUntilWorkersAreDoneButAllowBrowserToReact function or a concept to get this running. Every searching reagarding Mutex, sleep, etc ends in the following sentences: "JS is single threaded", "This will only work if you are not in a loop", "There is no reason to have a sleep function". etc.
Even when passing the main task to another Worker, I got the problem, that this thread is 100 % duty on checking, if the others are ready, which is waste of energy and processing power.
I would love to have a blocking function like myWebWorker.waitForReady(), which would allow events still to be handled. This would bring javascript to its next level. But may be I missed a simple concept that will do exactly this.
Thank you!

I would love to have a blocking function like myWebWorker.waitForReady()
No, that's not possible. All the statements you researched are correct, web workers stay asynchronous and will only communicate by messages. There is no waiting for events, not even on worker threads.
You will want to use promises for this:
function createWorkers(workerCount, src) {
var workers = new Array(workerCount);
for (var i = 0; i < workerCount; i++) {
workers[i] = new Worker(src);
}
return workers;
}
function doWork(worker, data) {
return new Promise(function(resolve, reject) {
worker.onmessage = resolve;
worker.postMessage(data);
});
}
function doDistributedWork(workers, data) {
// data size is always a multiple of the number of workers
var elementsPerWorker = data.length / workers.length;
return Promise.all(workers.map(function(worker, index) {
var start = index * elementsPerWorker;
return doWork(worker, data.slice(start, start+elementsPerWorker));
}));
}
var myWebWorkers = createWorkers(8, 'worker.js');
var somedata = [...];
function step(i) {
if (i <= 0)
return Promise.resolve("done!");
return doDistributedWork(myWebWorkers, somedata)
.then(function(results) {
if (i % 100)
updateSVGonWebPage();
return step(i-1)
});
}
step(1000).then(console.log);
Promise.all does the magic of waiting for concurrently running results, and the step function does the asynchronous looping using a recursive approach.

Retrieving a variable from within a function

I am attempting to pull information from the League of Legends API.
To simplify what I am doing, I am attempting to pull information about a user and their previous matches. The problem that I run into is that when I parse a JSON request, it returns a champion ID rather than their name (Ex: 412 rather than "Thresh").
The only solution I can see for this would be to make another JSON request and parse that data for the champion name. Currently what I have looks like this.
$.getJSON(championMasteryPHP, function (json) {
for (var i = 0; i < 20; i++) {
var champID = json[i].championId;
var championInfo = "http://example.com/champInfo.php?summonerid=" + champID;
$.getJSON(championInfo, function (json2) {
var champName = json2.name;
});
$('#champ').append("<li>"+champID+" - "+champName+"</li>")
}
});
I'm unable to access the champName variable due to it being nested within the second JSON function.
Is there a better way to do this?

$.getJSON(championMasteryPHP, function (json) {
for (var i = 0; i < 20; i++) {
var champID = json[i].championId;
var championInfo = "http://example.com/champInfo.php?summonerid=" + champID;
$.getJSON(championInfo, function (json2) {
var champName = json2.name;
$('#champ').append("<li>"+champID+" - "+champName+"</li>")
});
}
});
Just put it inside the second json request since you need to wait till that request is done anyway.

You should put the append statement in the callback because getJSON is an asynchronous method (does mean the Request is running in the background, and calls your function back when it got a response), so you should wait for the response first then you can append it to #champ :
$.getJSON(championMasteryPHP, function (json) {
for (var i = 0; i < 20; i++) {
var champID = json[i].championId;
var championInfo = "http://example.com/champInfo.php?summonerid=" + champID;
$.getJSON(championInfo, function (json2) {
var champName = json.name;
$('#champ').append("<li>"+champID+" - "+champName+"</li>")
});
}
});
Hope this helps.

Adding mutual friend count to each user in an array

I have an array of Facebook users (userList) and I want to store the number of mutual friends for each user in the array as a property (mfCount). I have checked that I am getting the correct number of mutual friends if I put in an individual user, but I'm not sure why I can't add this value to each user in the array?
function getfriends() {
FB.api('/me/friends', function(response) {
userList = userList.concat(response.data);
userCount = response.data.length;
for( i=0; i<response.data.length; i++) {
userId = response.data[i].id;
FB.api('/me/mutualfriends/'+userId+'/', function(response) {
userList[i].mfCount = response.data.length;
userCount--;
if(userCount === 0) { display_results();}
});
}
});
}

Have a look at the implementation below.
I've broken it out into multiple functions to separate each step.
When you're dealing with loops and callbacks, it becomes very important to keep track of what scope your anonymous functions are being defined in.
You can theoretically do it all in a one-liner like you were writing...
...but it gets very, very confusing as you go further and further into nested-callbacks.
One solution would be to make every variable inside each function 100% global, so that only i needs to have an enclosed reference. That's not really pretty, though.
Look through each function and take note of what parameters are going into the functions each step calls (or closures for callbacks). They're all needed (whether you separate them this way, or through closures in a one-liner or whatever).
The following worked just fine for me, inside of the Facebook developer sandbox (first time using the API).
The logs were for my benefit to see how the data was coming out, and to keep a basic stack-trace.
var userList = [],
userCount = 0;
function getfriends () {
//console.log("getFriends");
var url = "/me/friends";
FB.api(url, function (response) {
if (response.error && response.error.message) { return false; }
userList = userList.concat(response.data);
userCount = response.data.length;
compareAllFriends();
});
}
function compareAllFriends () {
//console.log("compareAllFriends");
var i = 0, l = userCount, userID;
for (; i < l; i += 1) {
userID = userList[i].id;
compareFriendsWith (i, userID);
}
}
function compareFriendsWith (i, id) {
//console.log("compareFriendsWith", i, id);
var path = "/me/mutualfriends/",
url = path + id + "/";
FB.api(url, (function (i) {
return function (response) {
//console.log(i, response);
var numFriends = (response.data) ? response.data.length : 0;
setMutualFriends(i, numFriends);
userCount -= 1;
//console.log(userCount);
if (userCount === 0) {
display_results();
//console.log("DISPLAYING");
}
};
}(i)));
}
function setMutualFriends (i, friendcount) { userList[i].mfCount = friendcount; }