We can use await in a for loop; however, I am trying to figure out if this is ever a good practice.
I read on MDN: "When an await is encountered in code (either in an async function or in a module), the awaited expression is executed, while all code that depends on the expression's value is paused and pushed into the microtask queue."
I would interpret that as meaning perhaps that //2 and everything below it that depends on the result of //1 would be "pushed into the microtask queue" in each iteration - if my interpretation is correct.
Has an authority on the subject (e.g., MDN) written on if and when this is a good practice?
let zeros = new Array(10).fill(0);
(async () => {
for (let zero of zeros) {
var r = await new Promise((r)=>setTimeout(r.bind(null, 1), 10)); //1
console.log(zero);
}
console.log(r); //2
})();
It depends on what you want to do.
If for example you must send 3 HTTP requests, it is probably better to run them together like this :
await Promise.all([http_get(url_1), http_get(url_2), http_get(url_3)])
This way the whole duration of the operation is as long as the longest HTTP GET request, instead of being the sum of the duration of each request.
Related
I have an array of Ids, I need to iterate through all the Ids, and for each Ids of the array make an async call to retrieve a value from DB, then sums all the value gathered. I did something like this
let quantity = 0;
for (const id of [1,2,3,4]) {
const subQuantity = await getSubQuantityById(id);
quantity += subQuantity;
}
Is there a more elegant and coincise way to write this for in javascript?
It is totally fine because your case include an async operation. Using a forEach instead is not possible here at all.
Your for loop is perfectly clean. If you want to make it shorter you could even do:
let totalQuantity = 0;
for (const id of arrayOfIds) {
totalQuantity += await getSubQuantityById(id);
}
As-is, it may even be more clear than using += await as above.
Naming could be improved as suggested.
I find the following one liner suggested in comments more cryptic/dirty:
(await Promise.all([1,2,3,4].map(i => getSubQuantityById(id))).reduce((p, c) => p + c, 0)
Edit: Props to #vitaly-t, who indicates that using Promise.all the way this one liner does will result in uncontrollable concurrency and lead to troubles in the context of a database
I can't follow #vitaly-t's argument that concurrent database queries will cause "problems" - at least not when we are talking about simple queries and there is a "moderate" number of these queries.
Here is my version of doing the summation. Obviously, the console.log in the last .then() needs to be replaced by the actual action that needs to happen with the calculated result.
// a placeholder function for testing:
function getSubQuantityById(i){
return fetch("https://jsonplaceholder.typicode.com/users/"+i).then(r=>r.json()).then(u=>+u.address.geo.lat);
}
Promise.all([1,2,3,4].map(id => getSubQuantityById(id)))
.then(d=>d.reduce((p, c) => p + c,0))
.then(console.log)
Is there a more elegant and coincise way to write this for in javascript?
Certainly, by processing your input as an iterable. The solution below uses iter-ops library:
import {pipeAsync, map, wait, reduce} from 'iter-ops';
const i = pipeAsync(
[1, 2, 3, 4], // your list of id-s
map(getSubQuantityById), // remap ids into async requests
wait(), // resolve requests
reduce((a, c) => a + c) // calculate the sum
); //=> AsyncIterableExt<number>
Testing the iterable:
(async function () {
console.log(await i.first); //=> the sum
})();
It is elegant, because you can inject more processing logic right into the iteration pipeline, and the code will remain very easy to read. Also, it is lazy-executing, initiates only when iterated.
Perhaps even more importantly, such a solution lets you control concurrency, to avoid producing too many requests against the database. And you can fine-tune concurrency, by replacing wait with waitRace.
P.S. I'm the author of iter-ops.
Trouble finding the reason why JavaScript for loop is not executing. Wrote 2 simple functions below that I want to execute and run i.e.: Bought method should try to "simulate" synchronous code.
The problem is that for some reason the for loop in the addNodes() method is never executed. However, if I run this separately i.e. for example line by line
var result = [];
var addressBookNodes = await generateAddressBooksNodes();
addNodes(result, addressBookNodes);
that tells me that the code is running fine however most likely it has something to do with the asynchronous nature of the generateAddressBooksNodes method. If I simply run the command :
var addressBookNodes = await generateAddressBooksNodes();
in the browser, I get an array of objects exactly what I was expecting to get. Basically the generateAddressBooksNodes returns a promise and when that promise is resolved I can see the correct array returned however what I do not understand why the for loop is not executed if the nodes objects have at least one element as shown in the picture below.
function addNodes(result, nodes){
console.log("3");
console.log(nodes);
for (var num in nodes) {
console.log("4");
let singleNode = nodes[num];
console.log(singleNode);
console.log("5");
result.push(singleNode);
}
}
async function getAddressBookAndContactNodes() {
var result = [];
console.log("1");
var addressBookNodesPromise = generateAddressBooksNodes();
addressBookNodesPromise.then( (arrayOfAddressBookNodes) => {
console.log("2");
addNodes(result, arrayOfAddressBookNodes);
})
return result;
}
Update 26 August 2020 :
After poking around the "arrayOfAddressBookNodes" object i noticed something really strange. I added additional print statement and printed the length of the "arrayOfAddressBookNodes" array. The length of the array is 0 when runned in the function. I do not understand how the length can be 0 if the object is printed shortly before the for loop and as shown on the picture below the length there is :1. What the hell is going on here?
I found another article i.e. JavaScript Array.length returning 0 that is basically explaining this. And in one of the commends it has been mentioned to use Map instead of an Array. I decided to use Set, and still get the same error i.e. the size of the set is 0 although the Set contains an object. i.e. below is the code and the picture of that execution.
async function getAddressBookAndContactNodes() {
var result = new Set();
console.log("1");
var addressBookNodes = await generateAddressBooksNodes();
console.log("2");
console.log(addressBookNodes);
console.log("3");
console.log("addressBookNodes size : " + addressBookNodes.size);
addressBookNodes.forEach(function(value) {
result.add(value);
});
console.log("6");
console.log(result);
console.log("7");
return result;
}
example using set
all this is really confusing to someone having a c++ backgroud, it makes my head explode.
Update 2 : 26 August 2020.
Ok i solved my problem. The problem was that the the promises are not working withing the for loop everything is explained here.
i need to use the regular "for (index = 0; index < contactsArray.length; ++index) " instead of foreach. after that it all worked. Somehow this leaves the impression that the tools of the language are broken in so many ways.
If generateAddressBooksNodes is returning a promise, you can use async to wait for the results:
async function getAddressBookAndContactNodes() {
var result = [];
console.log("1");
var addressBookNodesPromise = await generateAddressBooksNodes();
// note await here. Also, unless you're using this at a later time in your code, you can save space by not assigning it to a variable and simply returning generateAddressBooksNodes
addNodes(result, arrayOfAddressBookNodes);
return result;
}
I realized when I do something like this:
for (const entity of someArr) {
console.log('start now!')
await doSomeAsycAction()
console.log('waited X secs!')
}
It prints out:
start now!
waited X secs!
start now!
waited X secs!
...
But when I use map:
arr.map(async entity => {
console.log('start now!')
await doSomeAsycAction()
console.log('waited X secs!')
})
It prints out:
start now!
start now!
start now!
...
waited X secs!
waited X secs!
waited X secs!
...
Can someone explain why this is the case?
The difference between the two flows is that the first one (using for, for..in or for..of) is running the loop iterations sequentially and the other one (using map, filter, reduce, forEach and so on) are running (in case async symbol is used somewhere in the mapping function) concurrently (kind of).
In for loops, the next iteration must wait for the previous one to finish. This allows you to do some async operations relevant for the next iteration.
In contrast, using the async methods runs each iteration independently, so you can't rely on other iterations in your current iteration. Those kind of functions receive a function as an argument and executes it immediately for every item in the array.
Think of every iteration as an independent promise execution. When running an async function, the await symbol tells that this operation might take a while (i.e I/O, DB calls, network operations...) and let's the code outside of the current executed function keep going (and resume later on, after the async call returns). The map function sees that the current iteration is busy and go on to the next one. Somewhen in the future it would resume and execute console.log('waited X secs!').
You can simulate the same behavior of async executions with for loop this way (would maybe help demonstrating the difference):
for (const entity of someArr) {
(async () => {
console.log('start now!')
await doSomeAsycAction()
console.log('waited X secs!')
})()
}
The async-await syntax is working per function scope, and map is defining a new function scope (the function passed as the param to the function) just like the (anonymous) function that gets executed each iteration in my example. Wish it helps to understand.
One important thing to notice is that each iteration of the map doesn't return the mapped value you were expecting for, but a promise that will be resolved with this value. So if you try to rely on one of the mapped array values - you must add an await right before it, otherwise the value type would still be a promise. Take a look at the following example:
let arr = [1];
arr = arr.map(async entity => incrementAsync(entity));
console.log(arr[0]) // would print an unresolved Promise object
console.log(await arr[0]) // would print 2
Basic array prototype loop functions like forEach, map, filter, find etc. don't wait for next iteration.Their basic behavior is to iterate not awaiting. If you want use like waiting function then try to use like below
for (const event of events) {
if (failure or conditional) {
continue;
}
}
I'm trying to make some checks before saving an array of objects (objects[]) to the DB (mongoDB using mongoose):
Those objects are already sorted by date, so objects[0].date is lower than objects[1].date.
Each object should check that last related saved object has a different value (to avoid saving the same info two times). This means that I've to query to the DB before each save, to make that check, AND each of these object MUST be stored in order to be able to make the check with the right object. If objects are not stored in order, the last related saved object might not be the correct one.
In-depth explanation:
HTTP request is send to an API. It returns an array of objects (sortered by date) that I want to process and save on my Mongo DB (using mongoose). I've to iterate through all these objects and, for each:
Look for the previous related object stored on DB (which COULD BE one of that array).
Check some values between the 'already stored' and the object to save to evaluate if new object must be saved or could be discarded.
Save it or discard it, and then jump to next iteration.
It's important to wait each iteration to finish because:
Items on array MUST be stored in DB in order: first those which lower date, because each could be modified by some object stored later with a higher date.
If next iteration starts before previous has finished, the query that searchs for the previous object could not find it if it hasn't been stored yet
Already tried:
Using promises or async/await on forEach/for loops only makes that iteration async, but it keeps launching all iterations at once.
I've tried using async/await functions inside forEach/for loops, even creating my own asyncForEach function as shown below, but none of this has worked:
Array.prototype.asyncForEach = function(fn) {
return this.reduce(
(promise, n) => promise.then(() => fn(n)),
Promise.resolve()
);
};
Test function:
let testArray = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
testArray.asyncForEach(function(element) {
setTimeout(() => {
console.log(element);
}, Math.random() * 500);
});
Provided example should show numbers on order in every case. It's not a problem if internal function (setTimeout in the example) should return a promise.
What I think I need is a loop that waits some function/promise between iterations, and only starts the next iteration when the first is already finished.
How could I do that? Thanks you in advance!
const myArray = ['a','b','c','d'];
async function wait(ms) { // comment 3
return new Promise(resolve => setTimeout(resolve, ms));
}
async function doSomething() {
await myArray.reduce(async (promise, item) => {
await promise; // comment 2
await wait(1000);
// here we could await something else that is async like DB call
document.getElementById('results').append(`${item} `);
}, Promise.resolve()); // comment 1
}
setTimeout(() => doSomething(), 1000);
<div id="results">Starting in 1 second <br/></div>
You can also use reduce and async await which you already said you've tried.
Basically, if you read how reduce works you can see that it accepts 2 parameters, first being callback to execute over each step and second optional initial value.
In the callback we have first argument being an accumulator which means that it accepts whatever the previous step returns or the optional initial value for first step.
1) You are giving initial value of promise resolve so that you start your first step.
2) Because of this await promise you will never go into next step until previous one has finished, since that is the accumulator value from previous step, which is promise since we said that callback is async. We are not resolving promise per say here, but as soon as the previous step is finish, we are going to implicitly resolve it and go to next step.
3) You can put for example await wait(30) to be sure that you are throttling the Ajax requests and not sending to many requests to 3rd party API's, since then there is no way that you will send more than 1000/30 requests per second, even if your code executes really fast on your machine.
Hm, ok i am not 100% sure if i understand your question in the right way. But if you try to perform an async array operation that awaits for your logic for each item, you can do it like follow:
async loadAllUsers() {
const test = [1,2,3,4];
const users = [];
for (const index in test) {
// make some magic or transform data or something else
await users.push(test[index])
}
return users;
}
Then you can simply invoke this function with "await". I hope that helps you.
In asyncForEach function you are resolving a Promise, setTimeout doesn't return a Promise.So if you convert your setTimeout to Promise. It will work as expected.
Here is the modified code:
testArray.asyncForEach(function(element) {
return new Promise((resolve, reject) => {
setTimeout(() => {
console.log(element);
return resolve(element)
}, Math.random() * 500);
})
});
I constantly run into problems with this pattern with callbacks inside loops:
while(input.notEnd()) {
input.next();
checkInput(input, (success) => {
if (success) {
console.log(`Input ${input.combo} works!`);
}
});
}
The goal here is to check every possible value of input, and display the ones that pass an asynchronous test after confirmed. Assume the checkInput function performs this test, returning a boolean pass/fail, and is part of an external library and can't be modified.
Let's say input cycles through all combinations of a multi-code electronic jewelry safe, with .next incrementing the combination, .combo reading out the current combination, and checkInput asynchronously checking if the combination is correct. The correct combinations are 05-30-96, 18-22-09, 59-62-53, 68-82-01 are 85-55-85. What you'd expect to see as output is something like this:
Input 05-30-96 works!
Input 18-22-09 works!
Input 59-62-53 works!
Input 68-82-01 works!
Input 85-55-85 works!
Instead, because by the time the callback is called, input has already advanced an indeterminate amount of times, and the loop has likely already terminated, you're likely to see something like the following:
Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!
Input 99-99-99 works!
If the loop has terminated, at least it will be obvious something is wrong. If the checkInput function is particularly fast, or the loop particularly slow, you might get random outputs depending on where input happens to be at the moment the callback checks it.
This is a ridiculously difficult bug to track down if you find your output is completely random, and the hint for me tends to be that you always get the expected number of outputs, they're just wrong.
This is usually when I make up some convoluted solution to try to preserve or pass along the inputs, which works if there is a small number of them, but really doesn't when you have billions of inputs, of which a very small number are successful (hint, hint, combination locks are actually a great example here).
Is there a general purpose solution here, to pass the values into the callback as they were when the function with the callback first evaluated them?
If you want to iterate one async operation at a time, you cannot use a while loop. Asynchronous operations in Javascript are NOT blocking. So, what your while loop does is run through the entire loop calling checkInput() on every value and then, at some future time, each of the callbacks get called. They may not even get called in the desired order.
So, you have two options here depending upon how you want it to work.
First, you could use a different kind of loop that only advances to the next iteration of the loop when the async operation completes.
Or, second, you could run them all in a parallel like you were doing and capture the state of your object uniquely for each callback.
I'm assuming that what you probably want to do is to sequence your async operations (first option).
Sequencing async operations
Here's how you could do that (works in either ES5 or ES6):
function next() {
if (input.notEnd()) {
input.next();
checkInput(input, success => {
if (success) {
// because we are still on the current iteration of the loop
// the value of input is still valid
console.log(`Input ${input.combo} works!`);
}
// do next iteration
next();
});
}
}
next();
Run in parallel, save relevant properties in local scope in ES6
If you wanted to run them all in parallel like your original code was doing, but still be able to reference the right input.combo property in the callback, then you'd have to save that property in a closure (2nd option above) which let makes fairly easy because it is separately block scoped for each iteration of your while loop and thus retains its value for when the callback runs and is not overwritten by other iterations of the loop (requires ES6 support for let):
while(input.notEnd()) {
input.next();
// let makes a block scoped variable that will be separate for each
// iteration of the loop
let combo = input.combo;
checkInput(input, (success) => {
if (success) {
console.log(`Input ${combo} works!`);
}
});
}
Run in parallel, save relevant properties in local scope in ES5
In ES5, you could introduce a function scope to solve the same problem that let does in ES6 (make a new scope for each iteration of the loop):
while(input.notEnd()) {
input.next();
// create function scope to save value separately for each
// iteration of the loop
(function() {
var combo = input.combo;
checkInput(input, (success) => {
if (success) {
console.log(`Input ${combo} works!`);
}
});
})();
}
You could use the new feature async await for asynchronous calls, this would let you wait for the checkInput method to finish when inside the loop.
You can read more about async await here
I believe the snippet below achieves what you are after, I created a MockInput function that should mock the behaviour of your input. Note the Async and await keywords in the doAsyncThing method and keep an eye on the console when running it.
Hope this clarifies things.
function MockInput() {
this.currentIndex = 0;
this.list = ["05-30-96", "18-22-09", "59-62-53", "68-82-0", "85-55-85"];
this.notEnd = function(){
return this.currentIndex <= 4;
};
this.next = function(){
this.currentIndex++;
};
this.combo = function(){
return this.list[this.currentIndex];
}
}
function checkInput(input){
return new Promise(resolve => {
setTimeout(()=> {
var isValid = input.currentIndex % 2 > 0; // 'random' true or false
resolve( `Input ${input.currentIndex} - ${input.combo()} ${isValid ? 'works!' : 'did not work'}`);
}, 1000);
});
}
async function doAsyncThing(){
var input = new MockInput();
while(input.notEnd()) {
var result = await checkInput(input);
console.log(result);
input.next();
}
console.log('Done!');
}
doAsyncThing();