The following example code confuses me...
"use strict";
var filesToLoad = [ 'fileA','fileB','fileC' ];
var promiseArray = [];
for( let i in filesToLoad ) {
promiseArray.push(
new Promise( function(resolve, reject ) {
setTimeout( function() {
resolve( filesToLoad[i] );
}, Math.random() * 1000 );
})
);
}
Promise.all( promiseArray ).then(function(value) {
console.log(value);
});
The reason I'm confused is that I was expecting a random ordered output on the console. But I always get the following...
[ 'fileA', 'fileB', 'fileC' ]
That confuses me little to say the least, but what really gets me scratching my head is when I change the let i to var i I get the following result....
[ 'fileC', 'fileC', 'fileC' ]
As someone who has only recently tried to fully understand Promises and not that long ago starting using let, I'm really stumped.
Further reading...
After getting lots of great answers I have refactored the example to get rid of the loop and i. Will seem obvious to most, but fun for me...
"use strict";
var filesToLoad = [ 'fileA','fileB','fileC' ];
function start( name ) {
return new Promise( function(resolve, reject ) {
setTimeout( function() {
resolve( name + '_done' );
}, Math.random() * 1000 );
});
}
Promise.all( filesToLoad.map(start) ).then(function(value) {
console.log(value);
});
It is because of closure. Read about it here and here.
Also let is block scoped whereas var is function scoped.
In case of using var i:
After timeout when the function is triggered, looping was completed and i was set to 2. so it got resolved with filesToLoad[2] for all the setTimeout functions.
In case of using let i:
Since it is block scoped, when function is resolved it remembers the state of i when setTimeOut was declared, so when it gets resolved it uses correct value of i.
Regarding the order of output in case of using let i.
Promise.all(Iterable<any>|Promise<Iterable<any>> input) -> Promise
Given an Iterable(arrays are Iterable), or a promise of an Iterable, which produces promises (or a mix of promises and values), iterate over all the values in the Iterable into an array and return a promise that is fulfilled when all the items in the array are fulfilled. The promise's fulfillment value is an array with fulfillment values at respective positions to the original array. If any promise in the array rejects, the returned promise is rejected with the rejection reason.
So irrespective of order in which your promises gets resolved, the result of promise.all will always have promise resolve value in correct order.
Why using let and var produces different results:
The reason that using let produces the desired result over var is that when using let you declare a block-scoped variable, such that when the loop moves to another iteration the value of i remains unaffected for the contents of the loop at that time.
Defining a var variable in the header of a for-loop does not mean that it exists only for the life of the for-loop, as you will notice if you do the following:
for (var i = 0; i < 10; i++) { /*...*/ }
console.log(i); //=> 10
// `i` is already declared and its value will be 10
for (; i < 20; i++) { /*...*/ }
console.log(i); //=> 20
You can avoid this problem altogether if you use Array#forEach, which does the job of filesToLoad[i] for you by giving you the next value within a callback function on each iteration:
filesToLoad.forEach((file) => {
promiseArray.push(
new Promise( function(resolve, reject ) {
setTimeout( function() {
resolve( file );
}, Math.random() * 1000 );
})
);
});
______
Does using either let or var affect the behaviour of Promise#all?
No. In your example, the position of the Promises in promiseArray defines in what order the values are added to the results array, not when each of those Promises is resolved. The fact that you resolve the Promises at random intervals does not move the position of the resolved value within promiseArray. What you have demonstrated is that Promise#all produces an array of values whose positions are mapped to the Promise that produced their value.
See this answer for more information about the behaviour of Promise#all:
All this means that the output is strictly ordered as the input as long as the input is strictly ordered (for example, an array).
It's because var's scope isn't a child of for, it's a sibling. So the loop runs once and sets i = 0. It then runs another 2 more times and sets i = 1 and then i = 2. After this has all happened, the timeouts then run and all run the resolve function and passes in resolve( filesToLoad[2] ). let works correctly because the value of let i is not overridden by the following loop iterations.
In short, the timeout only runs after the loop has already run 3 times and therefore passes in the same value. I've created a jsfiddle of a working version using var.
These are actually two different, unrelated problems.
Your first observation, that the output is in the original order, rather than a random order, is due to the behaviour of Promise.all. Promise.all returns a new promise that resolves to an array when all sub-promises have resolved, in the original order. You can think of it like Array.map, but for promises.
As stated in the Bluebird (a JS promise library) documentation:
The promise's fulfillment value is an array with fulfillment values at respective positions to the original array.
Your second problem, how changing let to var changes the output to have all of the same values, is due to array scoping. Put simply, a variable defined with var exists outside of the for loop scope. This means that on each iteration, you are actually modifying the value of the pre-existing variable, as opposed to creating a new variable. Since your code inside the Promise executes later, the variable will have the value from the last loop iteration.
let creates a variable that exists inside that for iteration. This means that, later on in the Promise, you are accessing a variable defined in that iterations scope, which isn't overridden by the next loop iteration.
That's the difference between let and var. Your issue is not directly related to Promises (besides the fact that they run async).
TL;DR
var declarations are moved to beginning of function by interpreter, so when timeouts executed the loop has already run to end of array on the same variable. let keeps the variable in scope, so each iteration of the loop used a different variable.
Background:
Javascript has a feature called hoisting, which means that a variable or function is always declared at the beginning of it's wrapping function. If the coder doesn't, the interpreter moves it. Hence:
var i;
for (i in array)
....
and
for (var i in array)
....
are equivalent, even though, having some reason, or other programming background, you would expect i to be scoped to the loop alone in the second case.
That's why
alert(i);
...
var i=5;
alerts undefined instead of throwing an error. As far as the interpreter is concerned - there was a declaration. It's like
var i;
alert(i);
...
i=5;
Now:
ES6/ES2015 has introduced let - which behaves as our first assumption in the second example above.
alert(i);
...
let i=5;
will actually throw an error. When reaching the alert there is no i to speak of.
In your case
With let you went through the loop with a new variable each time, since let was defined only to scope of the loop. Each iteration is a scope, each used a different variable (though all named i in their respective scopes)
When using var, you actually declared i before the loop
var i;
for( i in filesToLoad ) {
promiseArray.push(
new Promise( function(resolve, reject ) {
setTimeout( function() {
resolve( filesToLoad[i] );
}, Math.random() * 1000 );
})
);
}
so it's the same variable on each iteration (or, in each scope), meaning that the loop assigned the last item to i before any of the timeouts returned. That's why the result was 3 times the last item.
Related
Trouble finding the reason why JavaScript for loop is not executing. Wrote 2 simple functions below that I want to execute and run i.e.: Bought method should try to "simulate" synchronous code.
The problem is that for some reason the for loop in the addNodes() method is never executed. However, if I run this separately i.e. for example line by line
var result = [];
var addressBookNodes = await generateAddressBooksNodes();
addNodes(result, addressBookNodes);
that tells me that the code is running fine however most likely it has something to do with the asynchronous nature of the generateAddressBooksNodes method. If I simply run the command :
var addressBookNodes = await generateAddressBooksNodes();
in the browser, I get an array of objects exactly what I was expecting to get. Basically the generateAddressBooksNodes returns a promise and when that promise is resolved I can see the correct array returned however what I do not understand why the for loop is not executed if the nodes objects have at least one element as shown in the picture below.
function addNodes(result, nodes){
console.log("3");
console.log(nodes);
for (var num in nodes) {
console.log("4");
let singleNode = nodes[num];
console.log(singleNode);
console.log("5");
result.push(singleNode);
}
}
async function getAddressBookAndContactNodes() {
var result = [];
console.log("1");
var addressBookNodesPromise = generateAddressBooksNodes();
addressBookNodesPromise.then( (arrayOfAddressBookNodes) => {
console.log("2");
addNodes(result, arrayOfAddressBookNodes);
})
return result;
}
Update 26 August 2020 :
After poking around the "arrayOfAddressBookNodes" object i noticed something really strange. I added additional print statement and printed the length of the "arrayOfAddressBookNodes" array. The length of the array is 0 when runned in the function. I do not understand how the length can be 0 if the object is printed shortly before the for loop and as shown on the picture below the length there is :1. What the hell is going on here?
I found another article i.e. JavaScript Array.length returning 0 that is basically explaining this. And in one of the commends it has been mentioned to use Map instead of an Array. I decided to use Set, and still get the same error i.e. the size of the set is 0 although the Set contains an object. i.e. below is the code and the picture of that execution.
async function getAddressBookAndContactNodes() {
var result = new Set();
console.log("1");
var addressBookNodes = await generateAddressBooksNodes();
console.log("2");
console.log(addressBookNodes);
console.log("3");
console.log("addressBookNodes size : " + addressBookNodes.size);
addressBookNodes.forEach(function(value) {
result.add(value);
});
console.log("6");
console.log(result);
console.log("7");
return result;
}
example using set
all this is really confusing to someone having a c++ backgroud, it makes my head explode.
Update 2 : 26 August 2020.
Ok i solved my problem. The problem was that the the promises are not working withing the for loop everything is explained here.
i need to use the regular "for (index = 0; index < contactsArray.length; ++index) " instead of foreach. after that it all worked. Somehow this leaves the impression that the tools of the language are broken in so many ways.
If generateAddressBooksNodes is returning a promise, you can use async to wait for the results:
async function getAddressBookAndContactNodes() {
var result = [];
console.log("1");
var addressBookNodesPromise = await generateAddressBooksNodes();
// note await here. Also, unless you're using this at a later time in your code, you can save space by not assigning it to a variable and simply returning generateAddressBooksNodes
addNodes(result, arrayOfAddressBookNodes);
return result;
}
I realized when I do something like this:
for (const entity of someArr) {
console.log('start now!')
await doSomeAsycAction()
console.log('waited X secs!')
}
It prints out:
start now!
waited X secs!
start now!
waited X secs!
...
But when I use map:
arr.map(async entity => {
console.log('start now!')
await doSomeAsycAction()
console.log('waited X secs!')
})
It prints out:
start now!
start now!
start now!
...
waited X secs!
waited X secs!
waited X secs!
...
Can someone explain why this is the case?
The difference between the two flows is that the first one (using for, for..in or for..of) is running the loop iterations sequentially and the other one (using map, filter, reduce, forEach and so on) are running (in case async symbol is used somewhere in the mapping function) concurrently (kind of).
In for loops, the next iteration must wait for the previous one to finish. This allows you to do some async operations relevant for the next iteration.
In contrast, using the async methods runs each iteration independently, so you can't rely on other iterations in your current iteration. Those kind of functions receive a function as an argument and executes it immediately for every item in the array.
Think of every iteration as an independent promise execution. When running an async function, the await symbol tells that this operation might take a while (i.e I/O, DB calls, network operations...) and let's the code outside of the current executed function keep going (and resume later on, after the async call returns). The map function sees that the current iteration is busy and go on to the next one. Somewhen in the future it would resume and execute console.log('waited X secs!').
You can simulate the same behavior of async executions with for loop this way (would maybe help demonstrating the difference):
for (const entity of someArr) {
(async () => {
console.log('start now!')
await doSomeAsycAction()
console.log('waited X secs!')
})()
}
The async-await syntax is working per function scope, and map is defining a new function scope (the function passed as the param to the function) just like the (anonymous) function that gets executed each iteration in my example. Wish it helps to understand.
One important thing to notice is that each iteration of the map doesn't return the mapped value you were expecting for, but a promise that will be resolved with this value. So if you try to rely on one of the mapped array values - you must add an await right before it, otherwise the value type would still be a promise. Take a look at the following example:
let arr = [1];
arr = arr.map(async entity => incrementAsync(entity));
console.log(arr[0]) // would print an unresolved Promise object
console.log(await arr[0]) // would print 2
Basic array prototype loop functions like forEach, map, filter, find etc. don't wait for next iteration.Their basic behavior is to iterate not awaiting. If you want use like waiting function then try to use like below
for (const event of events) {
if (failure or conditional) {
continue;
}
}
This question already has answers here:
JavaScript closure inside loops – simple practical example
(44 answers)
Closed 4 years ago.
I am trying to call a promise inside a for loop. I need the call to have the relevant loop item. I read a bit about it but still can't seem to figure the correct way to do it.
var params;
var identifylayers =["parcels", "lots", "gasstation"];
for (i = 0; i < identifylayers.length; i++)
{
lname = identifylayers[i];
govmap.intersectFeatures()
.then(function (e)
{
alert( lname);
});
}
Running the function returns: gasstation,gasstation,gasstation
I want it to return: parcels, lots, gasstation
What am I doing wrong?
Its the async devil of JavaScript lurking in your code. :D
Observe it like this -
As soon as your program flow encounter a promise call which here is - intersectFeatures().then(fn (){}), it know that this call will eventually finish so the program flow will never attempt to execute the callback passed to .then() right at the moment instead it will keep it safe with itself and set a reminder that as soon as the intersectFeatures is finished it need to execute the callback passed to .then().
Next thing to notice is how the .then(callback) will have access to lname. Its by closures. So
Another important thing to notice is that this whole stuff is inside a loop. Your loop has no idea whether the code its iterating on is synchronous or asynchronous. Its only task is to iterate. And as we mentioned before in Point - 1 the .then(callback) will not be executed right at the moment. So by the time then .then(callback) will be executed your loop would already be finished and at the end of iteration lname holds gasstation. Hence your .then(callback) only print gasstation
Solution:
This is really old JS solution of iterating the async code and still ensuring the proper variable access using closures -
for (i = 0; i < identifylayers.length; i++)
{
lname = identifylayers[i];
(function (lname) {
govmap.intersectFeatures()
.then( function () {
alert(lname);
} )
})(lname);
}
This utilises the benefit of functional scope of a variable and idea of closures. Google it to find more information.
Its also referred to as befriending closures, I just made that up :D
Cheers!
Define lname in the loop. Currently it is the same instance for each iteration, so it gets overriden, and then the promise resolves and prints the last overwrite each time.
const lname = identifylayers[i];
It is getting overriden before the async code runs.
async function intersectFeatures() {
return "OK"
}
var params;
var identifylayers = ["parcels", "lots", "gasstation"];
for (let i = 0; i < identifylayers.length; i++) {
const lname = identifylayers[i];
intersectFeatures()
.then(function(e) {
alert(lname);
});
}
I'm having lots of trouble combining a for loop with promises. In essence, my workflow is:
Take an array of Artist objects in Node.js, and grab their
biographies and IDs using Mongoose.
Use their IDs to get all the artwork (more specifically, all the
artwork keywords) created by that artist.
Train them in a natural language processor classifier (this part
isn't super relevant- just know that I need to have access to
biography and keywords within the same outer scope.
However, I can't seem to use the bioText object inside the Mongoose promise, and I can't seem to return the keywords from the Mongoose promise.
if (!isBlank(bioText) && bioText){
bioText = artist.BIO;
artworkIdArray.forEach((id)=> {
console.log(bioText) // <- prints correct values
var getKey = function(){
var words = Artwork.findById(id).then((artwork)=>{
console.log(bioText) // <- empty string
if(artwork.artworkKeywords.length > 0){
keywords = artwork.artistKeywords;
return keywords;
}
}
)
return words;
}
t=getKey();
t.then((t)=>{
console.log(t); // <--- always undefined
})
}, this);
}
As you can see, I try to console.log() after exiting the promise, but the t object is always undefined.
Additionally,
I know that in my example bioText is available right up until the Mongoose find() promise. But I also don't know how to inject bioText into the promise (because then I could simply just train the classifier inside the then() statement.
So my post is basically two questions:
Can Promises use outside variables inside then()?
How do I return a variable declared inside then() so it is
available in the outer scope?
Can Promises use outside variables inside then()?
Yes, but you have to be careful with what happens to those variables later on.
That's why console.log(bioText) may log a value just before DB call, and then log null just after DB response.
I would suggest you try the immediate invokation pattern to ensure the value is the expected one:
(function(x){...; console.log(x);})(bioText)
How do I return a variable declared inside then() so it is available
in the outer scope?
You cannot, at least without complicated promise management, which you may not need here.
That's why t=getKey(); is always undefined.
I think you want to get all the values before going on with your process. So you should retrieve all from DB in a single query, then call all your code in the callback.
You can have a try with this:
Artwork
.find({_id: {$in: artworkIdArray}})
.then( (artworks) => {
var keywords = [];
artworks.forEach( (artwork) => {
if(artwork.artworkKeywords.length > 0)
keywords.append(artwork.artistKeywords);
});
// Now do your job with all the retrieved keywords
});
I constantly run into problems with this pattern with callbacks inside loops:
while(input.notEnd()) {
input.next();
checkInput(input, (success) => {
if (success) {
console.log(`Input ${input.combo} works!`);
}
});
}
The goal here is to check every possible value of input, and display the ones that pass an asynchronous test after confirmed. Assume the checkInput function performs this test, returning a boolean pass/fail, and is part of an external library and can't be modified.
Let's say input cycles through all combinations of a multi-code electronic jewelry safe, with .next incrementing the combination, .combo reading out the current combination, and checkInput asynchronously checking if the combination is correct. The correct combinations are 05-30-96, 18-22-09, 59-62-53, 68-82-01 are 85-55-85. What you'd expect to see as output is something like this:
Input 05-30-96 works!
Input 18-22-09 works!
Input 59-62-53 works!
Input 68-82-01 works!
Input 85-55-85 works!
Instead, because by the time the callback is called, input has already advanced an indeterminate amount of times, and the loop has likely already terminated, you're likely to see something like the following:
Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!
If the loop has terminated, at least it will be obvious something is wrong. If the checkInput function is particularly fast, or the loop particularly slow, you might get random outputs depending on where input happens to be at the moment the callback checks it.
This is a ridiculously difficult bug to track down if you find your output is completely random, and the hint for me tends to be that you always get the expected number of outputs, they're just wrong.
This is usually when I make up some convoluted solution to try to preserve or pass along the inputs, which works if there is a small number of them, but really doesn't when you have billions of inputs, of which a very small number are successful (hint, hint, combination locks are actually a great example here).
Is there a general purpose solution here, to pass the values into the callback as they were when the function with the callback first evaluated them?
If you want to iterate one async operation at a time, you cannot use a while loop. Asynchronous operations in Javascript are NOT blocking. So, what your while loop does is run through the entire loop calling checkInput() on every value and then, at some future time, each of the callbacks get called. They may not even get called in the desired order.
So, you have two options here depending upon how you want it to work.
First, you could use a different kind of loop that only advances to the next iteration of the loop when the async operation completes.
Or, second, you could run them all in a parallel like you were doing and capture the state of your object uniquely for each callback.
I'm assuming that what you probably want to do is to sequence your async operations (first option).
Sequencing async operations
Here's how you could do that (works in either ES5 or ES6):
function next() {
if (input.notEnd()) {
input.next();
checkInput(input, success => {
if (success) {
// because we are still on the current iteration of the loop
// the value of input is still valid
console.log(`Input ${input.combo} works!`);
}
// do next iteration
next();
});
}
}
next();
Run in parallel, save relevant properties in local scope in ES6
If you wanted to run them all in parallel like your original code was doing, but still be able to reference the right input.combo property in the callback, then you'd have to save that property in a closure (2nd option above) which let makes fairly easy because it is separately block scoped for each iteration of your while loop and thus retains its value for when the callback runs and is not overwritten by other iterations of the loop (requires ES6 support for let):
while(input.notEnd()) {
input.next();
// let makes a block scoped variable that will be separate for each
// iteration of the loop
let combo = input.combo;
checkInput(input, (success) => {
if (success) {
console.log(`Input ${combo} works!`);
}
});
}
Run in parallel, save relevant properties in local scope in ES5
In ES5, you could introduce a function scope to solve the same problem that let does in ES6 (make a new scope for each iteration of the loop):
while(input.notEnd()) {
input.next();
// create function scope to save value separately for each
// iteration of the loop
(function() {
var combo = input.combo;
checkInput(input, (success) => {
if (success) {
console.log(`Input ${combo} works!`);
}
});
})();
}
You could use the new feature async await for asynchronous calls, this would let you wait for the checkInput method to finish when inside the loop.
You can read more about async await here
I believe the snippet below achieves what you are after, I created a MockInput function that should mock the behaviour of your input. Note the Async and await keywords in the doAsyncThing method and keep an eye on the console when running it.
Hope this clarifies things.
function MockInput() {
this.currentIndex = 0;
this.list = ["05-30-96", "18-22-09", "59-62-53", "68-82-0", "85-55-85"];
this.notEnd = function(){
return this.currentIndex <= 4;
};
this.next = function(){
this.currentIndex++;
};
this.combo = function(){
return this.list[this.currentIndex];
}
}
function checkInput(input){
return new Promise(resolve => {
setTimeout(()=> {
var isValid = input.currentIndex % 2 > 0; // 'random' true or false
resolve( `Input ${input.currentIndex} - ${input.combo()} ${isValid ? 'works!' : 'did not work'}`);
}, 1000);
});
}
async function doAsyncThing(){
var input = new MockInput();
while(input.notEnd()) {
var result = await checkInput(input);
console.log(result);
input.next();
}
console.log('Done!');
}
doAsyncThing();