We Keep Coding

Javascript for loop does not wait for fetch request to complete and moves onto next iteration

I have below code in javascript in which some asynchronous task is being performed:
async function fetchData(id){
for(let i=1;;++i){
res = await fetch(`https://some-api/v1/products/${id}/data?page=${i}`,{//headers here});
res = await res.json();
if(res.length==0) break;
else{ //do some work here and continue for next iteration}
}
}
async function callApi(){
var arr = [//list of id's here to pass to api one by one, almost 100 id's here];
await Promise.all(arr.map(async(e)=>{
await fetchData(e);
}));
}
callApi();
The above code looks fine to me, except that it doesn't work as expected. Ideally, what should happen is that unless one id's call is not completed( unless break condition not satisfies for one id), the for loop should not proceed to next iteration. Rather, I am getting totally different results. The api calls are happening in random order because the loop is not waiting the iteration to complete. My hard requirement is that unless one iteration is not complete, it should not move to next one.
await seems to have no effect here. Please guide me how can I achieve this. I am running out of ideas.
Thank You!

Your arr.map(...) is not awaiting the different fetchData calls before the next map call, so I'd turn this into a specific for loop to be sure it waits:
async function callApi(){
const arr = [...];
for(let i = 0; i < arr.length; i++){
await fetchData(arr[i]);
}
}
or alternatively use a for of
async function callApi(){
const arr = [...];
for(let a of arr){
await fetchData(a);
}
}
The fetchData function also looks like it could use some improvements with error handling, but since you shortened your code quite a bit, I'm assuming there is something like that going on there, too, and your issue is actually with the callApi() code instead, as the fetch and await looks good to me there.

You should decide either to use promises or async await. Don't mix them.
With promises you can always use funky abstractions but with a simple recursive approach you can do like
function fetchData(hds, id, page = 1, pages = []){
return fetch(`https://some-api/v1/products/${id}/data?page=${page}`,hds)
.then(r => r.ok ? r.json() : Promise.reject({status:r.status,pages})
.then(j => fetchData(hds, id, ++page, pages.push(doSomethingWith(j))))
.catch(e => (console.log(e.status), e.pages));
}
So we use recursion to fetch indefinitelly until the API says enough and r.ok is false.
At the callApi side you can use reduce since we have an ids array.
const ids = [/* ids array */],
hds = { /* headers object */ };
function callApi(ids){
return ids.reduce( (p,id) => p.then(_ => fetchData(hds,id))
.then(pages => process(pages))
, Promise.resolve(null)
)
.catch(e => console.log(e));
}
So now both accesses to the id and page data are working asynchronously but only fired once the previous one finishes. Such as
(id=1,page=1) then (id=1,page=2) then (id=1,page=3) then (process 3 pages of id=1) then
(id=2,page=1) then (id=2,page=2) then (process 2 pages of id=2) etc...
While I love the promises, you can also implement the same functionality with the asyc await abstraction. I believe the idea behind the invention of the async await is to mimic sync imperative code. But keep in mind that it's an abstraction over an abstraction and I urge you to learn promises by heart before even attemting to use async await. The general rule is to never mix both in the same code.
Accordingly the above code could have been written as follows by using async await.
async function fetchData(hds, id){
let page = 1,
pages = [],
res;
while(true){
res = await fetch(`https://some-api/v1/products/${id}/data?page=${page++}`,hds);
if (res.ok) pages.push(await res.json())
else return pages;
}
}
Then the callApi function can be implemented in a similar fashion
const ids = [/* ids array */],
hds = { /* headers object */ };
async function callApi(ids){
let pages;
for(let i = 0; i < ids.length; i++){
try {
pages = await fetchData(hds,ids[i]);
await process(pages); // no need for await if the process function is sync
}
catch(e){
console.log(e);
}
}
}

aysnc map is returning empty array

the following code block is returning an empty array but if I use for loop instead it works fine
let allMenus = [];
foodMenu = foodMenu.map(async menu => allMenus.push(await handleMenu(menu, DBName)))
console.log(foodMenu); // returns empty array
this return the data perfectly but I want to use map
let allMenus = [];
for (const menu in foodMenu) {
allMenus.push(await handleMenu(foodMenu[menu], DBName)); // this returns the data
}

A few things:
Firstly, Array#map expects a return value because it generates a new array. While your code works, it is not the intent of the method.
Using await in an array enumerator callback (Array#map, in yours case) will defer the execution, but it does not pause between callbacks. That means the code will be run, but it will not be resolved in sequence in the way you are expecting.
Do this:
let foodMenu = foodMenu.map(async menu => {
const newMenuItem = await handleMenu(menu, DBName)
console.log(newMenuItem)
allMenus.push(menu)
})
You will find that your return value, the empty array, will be printed first, and then your new menus printed afterwards. It is out of order
To resolve this, you either need to
Make it a loop, where await that will pause in the way you expect or
Map the promises into an array and wait for them all to finish, using Promise.all
let foodMenu = await Promise.all(foodMenu.map(menu => handleMenu(menu, DBName)))

Add key to array of objects in async function [duplicate]

This question already has answers here:
Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference
(7 answers)
Closed 11 months ago.
I have an api route that needs to take data from two sources, merge the data together into one object and then return. The issue I am having is I'm basically stuck in async/await hell and when pushing to a second array within the .then() block, the second array named
clone returns []. How can I make an api request, merge the data and return to the requester as needed?
Fetch code:
export default async function getProduct(product_id) {
const product = await fetch(
`${process.env.PRIVATE_APP_URL}/products/${product_id}.json`,
{
method: "GET",
headers: {
"Content-Type": "application/json",
},
}
).then((result) => {
return result.json();
});
return product.product;
}
API Handler:
const recharge_subscription_res = await rechargeAPI(
"GET",
`https://api.rechargeapps.com/subscriptions?customer_id=${recharge_customer.id}`
);
const closest_array = recharge_subscription_res.subscriptions.filter(
(e) => e.next_charge_scheduled_at == closest_date
);
let clone = [];
closest_array.forEach((element) => {
getProduct(element.shopify_product_id).then((product) => {
element.shopify_product_handle = product.handle;
element.shopify_product_image_url = product.image.src;
clone.push(element);
});
});
console.log(clone);
clone should log as an array of objects like closest_array, but instead logs as just an empty array. This isn't exactly like the other seemingly duplicate questions because typically their feature doesn't require sending the promise's data back to an external source. Most questions are related to the front end of things. My situation is with an Express.js API. Any help would be appreciated.

The original promise spec used .then(), and the new syntax hides then's with await. Style-wise, it makes sense to choose just one style and go with it.
In either style, there's a little challenge having to do with creating many promises in a loop. The js iteration functions (like map and forEach) take synchronous functions. The most common design is to create a collection of promises in a synchronous loop, then run them concurrently with Promise.all(). Taking both ideas into account...
You could (but don't have to) rewrite your network request like this...
// since we decorated "async" let's use await...
export default async function getProduct(product_id) {
const url = `${process.env.PRIVATE_APP_URL}/products/${product_id}.json`;
const options = { method: "GET", headers: { "Content-Type": "application/json" }};
const result = await fetch(url, options);
const product = await result.json();
return product.product;
}
await is not permitted at the top-level; it may be used only within an async function. Here, I'll make up a name and guess about the parameter
async function rechargeAndLookupProduct(recharge_customer) {
const base = 'https://api.rechargeapps.com/subscriptions';
const query = `customer_id=${recharge_customer.id}`;
const recharge_subscription_res = await rechargeAPI("GET",`${base}?${query}`);
const closest_array = recharge_subscription_res.subscriptions.filter(e =>
e.next_charge_scheduled_at == closest_date
);
// here's the important part: collect promises synchronously
// execute them together with Promise.all()
const promises = closest_array.map(element => {
return getProduct(element.shopify_product_id)
});
const allProducts = await Promise.all(promises);
// allProducts will be an array of objects that the promises resolved to
const clones = allProducts.map((product, i) => {
// use Object.assign so we'll really have a "clone"
let closest = Object.assign({}, closest_array[i]);
closest.shopify_product_handle = product.handle;
closest.shopify_product_image_url = product.image.src;
return closest;
});
// if I didn't make any typos (which I probably did), then
// clones ought to contain the result you expect
console.log(clones);
}

Your code has a flaw (in the section shown below). You have a dangling promise that you forgot to await or return.
When you log clone, none of the async getProduct operations have completed yet, and none of the elements have been pushed.
let clone = [];
closest_array.forEach((element) => {
getProduct(element.shopify_product_id).then((product) => {
element.shopify_product_handle = product.handle;
element.shopify_product_image_url = product.image.src;
clone.push(element);
}); // FLAW: dangling .then
});
console.log(clone); // FLAW: clone is not ready yet.
I would set it up more like this:
let clone = await Promise.all(closest_array.map((element) =>
getProduct(element.shopify_product_id).then((product) => {
element.shopify_product_handle = product.handle;
element.shopify_product_image_url = product.image.src;
return element;
})
));
console.log(clone);
It's a little sketchy to modify element the way you are (I wouldn't), but this way the getProduct calls are all in flight together for maximum efficiency. Promise.all handles awaiting all the promises and putting each's result into an array of results, which you can then await as a single promise since the calling function is async.

Reexecuting await/async/promise.all function with a new value

I am basically trying to send multiple API queries at once and wait for the result. The code works well for the first query, but I am actually struggling to reset the value of the prior async return. After the first execution is completed, it keeps returning the very first values. The goal is to assign random href(URL) with every function's execution and recall the API once again, so it can call the API once again with a newly randomized href and return new values. Here is the code, would be great is someone could help:
async function fetchData() {
const [test0_array_response,
test1_array_response,
test2_array_response,
] = await Promise.all([
fetch(random_href+'/test0/'),
fetch(random_href+'/test1/'),
fetch(random_href+'/test2/'),
]);
const test0_array = await test0_array_response.json();
const test1_array= await test1_array_response.json();
const test2_array = await test2_array_response.json();
return [test0_array, test1_array, test2_array];
}
function callApi() {
console.log(fetchData()) // returns a promise
};

From what I can see random_href is part of the closure of fetchData, so you need to pass it as argument inside callApi or wherever you use fetchData in order to use his correct value.
Take a look here for learn what I means with closure.
You should do something like this:
async function fetchData(random_href) {
const [test0_array_response,
test1_array_response,
test2_array_response,
] = await Promise.all([
fetch(random_href+'/test0/'),
fetch(random_href+'/test1/'),
fetch(random_href+'/test2/'),
]);
const test0_array = await test0_array_response.json();
const test1_array= await test1_array_response.json();
const test2_array = await test2_array_response.json();
return [test0_array, test1_array, test2_array];
}
function callApi(random_href) {
console.log(fetchData(random_href)) // returns a promise
};
Another approach could be to make an object random_container = { href: 'VALUE'} and use it's reference inside the fetchData function. So, for example, fetch(random_container.href+'/test0/'). In this way random_container will be part of the closure of fetchData but you'll be able to read the last value of random_container.href.

Firestore - getting data from a reference like 'collectionName/{id}' and iterating over the resulting list

I'm trying to get all data from 'anunturi_postate' reference list.
Variable 'post_data' pushed to the list looks ok in the loop, but outside the loop, the list it's empty.
I added an image from the console and the firestore emulator for a better view of what I'm trying to do.
// db = firebase.firestore()
async function postedByUser(){
let query = await db.collection("users").doc($current_user.uid).get()
let user_posts = []
// anunturi_postate it's a field name which holds an array
await query.data().anunturi_postate.forEach(async (doc) => {
let post = await db.doc(doc.path).get()
let post_data = await post.data()
console.log(post_data) // data looks ok
user_posts.push(post_data)
})
console.log("user_posts:", user_posts) // here it's empty
return user_posts
}
let user_posts
postedByUser().then(data => {
user_posts = data
})
// here it's empty
console.log("After func call: ", user_posts)

async/await does not work inside a forEach loop in the way you expect. The loop does not return a promise that you can await, and will not finish before moving on to the following code. Use a for loop instead. Also, you can use each reference as if it was a DocumentReference object.
let user_posts = []
const array = query.data().anunturi_postate
for (let ref of array) {
let post = await ref.get()
let post_data = await post.data()
console.log(post_data) // ok
user_posts.push(post_data)
})
console.log("user_posts:", user_posts)