I have an array of data and need to fetch data for each item and combine them.
// This is inside a Nextjs async API route
const data = ['item1','item2','item3']
let result = []
data.forEach(async (item)=>{
const res = await fetch(`https://websitename.com/${item}`)
result = [...result,...res]
}
console.log(result) //gives an empty array
Here, it returns an empty array even though for each item data is being fetched. How to make such requests?
}
Your code should be throwing errors I think? You're trying to declare result twice, but as the second time is within the forEach loop, it's actually a different variable, and is created and destroyed for each loop.
Assuming your fetch works, this should work:
const data = ['item1','item2','item3']
let result = []
data.forEach(async (item)=>{
const res = await fetch(`https://websitename.com/${item}`)
result = [...result,...res]
}
console.log(result)
May this could help you :)
async function doRequest(data) {
// process here
}
const requests = ['item1', 'item2', 'item3'];
const results = requests.map(async (val) => {
const response = await doRequest();
return response;
});
await Promise.all(requests);
Change:
const res = await fetch(`https://websitename.com/${item}`)
const result = [...result,...res]
To:
const response = await fetch(`https://websitename.com/${item}`);
const data = response.json(); // if your response is Json
result.push(data);
result should be const instead of let
Method .forEach() makes sync iteration. It means that an iteration does not wait resolving of your async callback, but just starts it. As a result forEach starts all requests and jumps to line with console.log(result). But at the moment none of the requests are done and array is still empty. If you wrap like that setTimeout(()=>console.log(result),3000) you will see that the array is filled with data.
If you want to make sequential calls:
(async function() {
const data = ['item1','item2','item3']
let result = []
for await (const item of data) {
const res = await fetch(`https://websitename.com/${item}`)
console.log(item);
result.push(item)
}
console.log(result)
})();
If you want to make parallel calls:
(async function() {
const data = ['item1','item2','item3']
let result = await Promise.all(data.map(async (item)=>{
return await await fetch(`https://websitename.com/${item}`)
}))
console.log(result)
})();
Related
I am new to js/react api calls and am trying to use 2 apis one to fetch a random movie
https://k2maan-moviehut.herokuapp.com/api/random
and the second to get movie
http://www.omdbapi.com/
where the t paramter in second is movie name, and here is my code
in my main component
const getMovies = async () => {
let movies = await fetchMovies();
console.log(movies);
return movies;
}
const movies = getMovies();
and i call these functions from another file
const getMovie = async () => {
const omdpURL = "http://www.omdbapi.com/?i=tt3896198&apikey=xxxxxx&t=";
let moviename = (await (await fetch("https://k2maan-moviehut.herokuapp.com/api/random")).json()).name
let movie = await (await fetch(omdpURL.concat(moviename))).json();
return movie;
}
export const fetchMovies = async() => {
let values = [];
for (let i = 0; i < 12; i++) {
let movie = await getMovie();
values.push(movie)
}
return values;
}
the problem is that when I try to see my movies in main component it returns a {Promise fulfilled Array(12)} while if I logged the movies while I call getMovies it gives me the result I want which is the 12 movie I called , how can I have the results I need
Here's a basic setup to help you get started:
const getMovie = async () => {
const randomMovieURL = "https://k2maan-moviehut.herokuapp.com/api/random";
const omdpURL = "https://www.omdbapi.com/?i=tt3896198&apikey=apiKey&t=";
// Fetch the first URL and get a random movie
const result = await fetch(randomMovieURL);
// Get the response and parse the JSON data coming back from this response
const movie = await result.json();
// Fetch the OMDB using the random movie name
const result2 = await fetch(omdpURL + movie.name);
// Once again, parse the JSON from the Response we got back
const movieData = await result2.json();
// Return the JSON data which contains the movie details
return movieData;
}
const fetchMovies = async () => {
const requests = [];
for (let i = 0; i < 12; i++) {
// Fill in the requests Array with the asynchronous operations
requests.push(getMovie())
}
// Run all 12 async operations in parallel to speed things up
const movies = await Promise.all(requests);
// Return the results:
return movies;
}
// fetchMovies() is async so we need to `await` for the result inside
// another async function:
async function main(){
const movies = await fetchMovies();
console.log(movies);
}
main();
// Alternative syntax using then():
fetchMovies().then( movies => console.log( movies ) );
Updated:
Keep in mind, that the correct way to loop over a list of async calls in parallel and get the results, is through Promise.all.
Using a normal for loop and await (as in your case) will result in running all 12 calls sequentially, which is much slower.
Regarding the question:
"the problem is that when I try to see my movies in main component it return a {Promise fulfilled Array(12)} while if I logged the movies while I call getMovies it gives me the result I want which is the 12 movie I called , how can I have the results I need"
The reason why you see a Promise fulfilled Array(12) in the main component is that the getMovies function is async and thus always returns a Promise not a normal value.
In order to see the value, you will need to either use await before the function call (const movies = await getMovies()) or use then (getMovies().then( movies => console.log(movies) )).
This is the reason why you see the actual value in the console.log inside getMovies. You are using await before the async function fetchMovies.
Lesson of the day
You have just stumbled upon one of the trickiest and hardest parts of JavaScript: asynchronous programming using Promises.
Make sure to go through the following MDN resources to get a comprehensive and solid foundation of Promises:
Using Promises
async function
await expression
I am not sure why you try to nest your await fetch clauses. If you do it like this it would work.
const getMovies = async () => {
const omdpURL = "http://www.omdbapi.com/?i=tt3896198&apikey=myApiKey&t=";
const res = await fetch(
"https://k2maan-moviehut.herokuapp.com/api/random"
);
var data = await res.json();
const movie = await fetch(omdpURL + data.name);
var moviedata = await movie.json();
console.log(moviedata);
};
getMovies();
I am calling an API and getting the total_tweet_count. If the meta tag contains a next_token, I fetch the next API (here I call the corresponding object) with the next_token and push the total_tweet_count to an array. However, if the response doesnt contain a next_token, I stop iterating and just push the total_tweet_count and return the array. For some reason, the code doesnt seem to run. Please help.
The below example should get the next_token from res1 and call res2 and push the tweet_count into the array. Then using the next_token of res2, fetch res3 data and push the count. Since res3 doesnt have a next_token, we end the loop and return the data. This is the expectation. Please advice.
const getData = async () => {
const totalCount = [];
const results = await Promise.resolve(data.res1);
totalCount.push(results.meta.total_tweet_count)
do {
const results1 = await Promise.resolve(data[`res${results.meta.next_token}`]);
totalCount.push(results1.meta.total_tweet_count)
}
while (results.meta.next_token)
return totalCount;
}
getData().then(res => console.log(res))
The final output I expect is [1,20, 8]
Please advice.
Fiddle Link: https://jsfiddle.net/czmwtj3e/
Your loop condition is constant:
const results = await Promise.resolve(data.res1);
//^^^^^^^^^^^^^
…
do {
…
}
while (results.meta.next_token)
What you are looking for is
const getData = async () => {
const totalCount = [];
let results = await Promise.resolve(data.res1);
totalCount.push(results.meta.total_tweet_count)
while (results.meta.next_token) {
results = await Promise.resolve(data[`res${results.meta.next_token}`]);
totalCount.push(results.meta.total_tweet_count)
}
return totalCount;
}
or
const getData = async () => {
const totalCount = [];
let results = {meta: {next_token: 1}};
do {
results = await Promise.resolve(data[`res${results.meta.next_token}`]);
totalCount.push(results.meta.total_tweet_count)
}
while (results.meta.next_token)
return totalCount;
}
Notice there is no const results1 in either of these, but an assignment to the mutable results variable.
As far as I can tell:
getData() is a wholly synchronous process acting on data therefore does not need to be an asyncFunction.
the required array of tweet counts can be created with a combination of Object.keys() and Array.prototype.reduce().
the .next_token property does not need to be used (unless it has some secret meaning beyond what has been explained in the question).
const getData = () => {
const keys = Object.keys(data); // array ['res1', 'res2', 'res3']
const totalCounts = keys.reduce((arr, key) => { // iterate the `keys` array with Array.prototype.reduce()
arr.push(data[key].meta.total_tweet_count); // push next tweet count onto the array
return arr; // return `arr` to be used in the next iteration
}, []); // start the reduction with empty array
return totalCounts; // [1, 20, 8]
};
In practice, you would:
pass data to the function
avoid intermediate variables
const getData = (data) => {
return Object.keys(data).reduce((arr, key) => {
arr.push(data[key].meta.total_tweet_count);
return arr;
}, []);
};
I want to push all the item-data into one array. However, when I try, it just makes an array for every item.
async function final() {
const response = await fetch('/api');
const data = await response.json();
console.log(data)
for (item of data) {
const num = item.atomicNumber;
const arr = []
arr.push(num)
console.log(arr)
}
}
Because you are creating a new array inside the loop. Analyzing your own code step by step and understanding what every line does is mandatory to be a good developer :-). Let's analyze it:
async function final() {
const response = await fetch('/api');
const data = await response.json();
console.log(data)
// Until here everything is fine
// You define a loop which will run from this point each time
for (item of data) {
const num = item.atomicNumber;
// You are creating a new array inside the loop. The loop, as its name says, will run one time per item.
const arr = []
arr.push(num)
console.log(arr)
}
}
To fix this, just move the array outside the loop, so it only runs one time:
async function final() {
const response = await fetch('/api');
const data = await response.json();
console.log(data)
// We create the array outside the loop
const arr = []
// Then define a loop which will run from this point each time
for (let item of data) { // Don't forget to define variables with the proper keyword (in this case, "let" is enough).
const num = item.atomicNumber;
arr.push(num)
}
// We log the array when the loop has ended, so it logs only one time
console.log(arr)
}
you need to put your empty array const arr = [] out of the loop, otherwise you're re-declaring the array at every iteration.
async function final() {
const response = await fetch('/api');
const data = await response.json();
console.log(data)
const arr = []
for (item of data) {
const num = item.atomicNumber;
arr.push(num)
}
console.log(arr)
}
You're re-declaring arr in every iteration of the loop. Move the declaration out of it, and just push into it every iteration:
async function final() {
const response = await fetch('/api');
const data = await response.json();
console.log(data)
const arr = [] // arr declared outside the loop
for (item of data) {
const num = item.atomicNumber;
arr.push(num) // but used inside it
console.log(arr)
}
}
try this:
async function final() {
const response = await fetch('/api');
const data = await response.json();
console.log(data)
const arr = []
for (item of data) {
const num = item.atomicNumber;
arr.push(num)
}
console.log(arr)
}
I'm fetching my user data and the map function is called several times for each user. I want to wait until all data was pushed to the array and then manipulate the data. I tried using Promise.all() but it didn't work.
How can I wait for this map function to finish before continuing?
Needless to say that the array user_list_temp is empty when printed inside the Promise.all().
const phone_list_promise_1 = await arrWithKeys.map(async (users,i) => {
return firebase.database().ref(`/users/${users}`)
.on('value', snapshot => {
user_list_temp.push(snapshot.val());
console.log(snapshot.val());
})
}
);
Promise.all(phone_list_promise_1).then( () => console.log(user_list_temp) )
I changed the code to this but I still get a wrong output
Promise.all(arrWithKeys.map(async (users,i) => {
const eventRef = db.ref(`users/${users}`);
await eventRef.on('value', snapshot => {
const value = snapshot.val();
console.log(value);
phone_user_list[0][users].name = value.name;
phone_user_list[0][users].photo = value.photo;
})
console.log(phone_user_list[0]);
user_list_temp.push(phone_user_list[0]);
}
));
console.log(user_list_temp); //empty array
}
It is possible to use async/await with firebase
This is how I usually make a Promise.all
const db = firebase.database();
let user_list_temp = await Promise.all(arrWithKeys.map(async (users,i) => {
const eventRef = db.ref(`users/${users}`);
const snapshot = await eventref.once('value');
const value = snapshot.value();
return value;
})
);
This article gives a fairly good explanation of using Promise.all with async/await https://www.taniarascia.com/promise-all-with-async-await/
Here is how I would refactor your new code snippet so that you are not mixing promises and async/await
let user_list_temp = await Promise.all(arrWithKeys.map(async (users,i) => {
const eventRef = db.ref(`users/${users}`);
const snapshot= await eventRef.once('value');
const value = snapshot.val();
console.log(value);
phone_user_list[0][users].name = value.name; // should this be hardcoded as 0?
phone_user_list[0][users].photo = value.photo; // should this be hardcoded as 0?
console.log(phone_user_list[0]);
return phone_user_list[0]; // should this be hardcoded as 0?
})
);
console.log(user_list_temp);
Here is a simple example that uses fetch, instead of firebase:
async componentDidMount () {
let urls = [
'https://jsonplaceholder.typicode.com/todos/1',
'https://jsonplaceholder.typicode.com/todos/2',
'https://jsonplaceholder.typicode.com/todos/3',
'https://jsonplaceholder.typicode.com/todos/4'
];
let results = await Promise.all(urls.map(async url => {
const response = await fetch(url);
const json = await response.json();
return json;
}));
alert(JSON.stringify(results))
}
If I understand your question correctly, you might consider revising your code to use a regular for..of loop, with a nested promise per user that resolves when the snapshot/value for that user is available as shown:
const user_list_temp = [];
/*
Use regular for..of loop to iterate values
*/
for(const user of arrWithKeys) {
/*
Use await on a new promise object for this user that
resolves with snapshot value when value recieved for
user
*/
const user_list_item = await (new Promise((resolve) => {
firebase.database()
.ref(`/users/${users}`)
.on('value', snapshot => {
/*
When value recieved, resolve the promise for
this user with val()
*/
resolve(snapshot.val());
});
}));
/*
Add value for this user to the resulting user_list_item
*/
user_list_temp.push(user_list_item);
}
console.log(user_list_temp);
This code assumes that the enclosing function is defined as an asynchronous method with the async keyword, seeing that the await keyword is used in the for..of loop. Hope that helps!
Consider this code
const response = await fetch('<my url>');
const responseJson = await response.json();
responseJson = _.sortBy(responseJson, "number");
responseJson[0] = await addEnabledProperty(responseJson[0]);
What addEnabledProperty does is to extend the object adding an enabled property, but this is not important. The function itself works well
async function addEnabledProperty (channel){
const channelId = channel.id;
const stored_status = await AsyncStorage.getItem(`ChannelIsEnabled:${channelId}`);
let boolean_status = false;
if (stored_status == null) {
boolean_status = true;
} else {
boolean_status = (stored_status == 'true');
}
return _.extend({}, channel, { enabled: boolean_status });
}
Is there a way to use _.map (or another system), to loop trough entire responseJson array to use addEnabledProperty against each element?
I tried:
responseJson = _.map(responseJson, function(channel) {
return addEnabledProperty(channell);
});
But it's not using async so it freeze the app.
I tried:
responseJson = _.map(responseJson, function(channel) {
return await addEnabledProperty(chanell);
});
But i got a js error (about the row return await addEnabledProperty(chanell);)
await is a reserved word
Then tried
responseJson = _.map(responseJson, async function(channel) {
return await addEnabledProperty(channell);
});
But I got an array of Promises... and I don't understand why...
What else!??
EDIT: I understand your complains about I didn't specify that addEnabledProperty() returns a Promise, but, really, I didn't know it. In fact, I wrote "I got an array of Promises... and I don't understand why "
To process your response jsons in parallel you may use Promise.all:
const responseJson = await response.json();
responseJson = _.sortBy(responseJson, "number");
let result = await Promise.all(_.map(responseJson, async (json) =>
await addEnabledProperty(json))
);
Since addEnabledProperty method is async, the following also should work (per #CRice):
let result = await Promise.all(_.map(responseJson, addEnabledProperty));
I found that I didn't have to put the async / await inside of the Promise.all wrapper.
Using that knowledge, in conjunction with lodash chain (_.chain) could result in the following simplified version of the accepted answer:
const responseJson = await Promise.all( _
.chain( response.json() )
.sortBy( 'number' )
.map( json => addEnabledProperty( json ) )
.value()
)
How about using partial.js(https://github.com/marpple/partial.js)
It cover both promise and normal pattern by same code.
_p.map([1, 2, 3], async (v) => await promiseFunction());
You can use Promise.all() to run all the promises in your array.
responseJson = await Promise.all(_.map(responseJson, (channel) => {
return addEnabledProperty(channel);
}));
If you want to iterate over some object, you can use keys first to get an array of keys and then just loop over your keys while awaiting for necessary actions.
This works when you want to wait until every previous iteration step is finished before getting into the next one.
Here is a consolidated asyncMap function that you can use for your object:
async function asyncMap(obj, cb) {
const result = {};
const keysArr = keys(obj);
let keysArrLength = keysArr.length;
while (keysArrLength-- > 0) {
const key = keysArr[keysArrLength];
const item = obj[key];
// eslint-disable-next-line no-await-in-loop
result[key] = await cb(item, key);
}
return result;
}
And then, for your example case:
responseJson = await asyncMap(responseJson, addEnabledProperty);
Otherwise use Promise.all like was proposed above to run all the iteration steps in parallel