I am new to js/react api calls and am trying to use 2 apis one to fetch a random movie
https://k2maan-moviehut.herokuapp.com/api/random
and the second to get movie
http://www.omdbapi.com/
where the t paramter in second is movie name, and here is my code
in my main component
const getMovies = async () => {
let movies = await fetchMovies();
console.log(movies);
return movies;
}
const movies = getMovies();
and i call these functions from another file
const getMovie = async () => {
const omdpURL = "http://www.omdbapi.com/?i=tt3896198&apikey=xxxxxx&t=";
let moviename = (await (await fetch("https://k2maan-moviehut.herokuapp.com/api/random")).json()).name
let movie = await (await fetch(omdpURL.concat(moviename))).json();
return movie;
}
export const fetchMovies = async() => {
let values = [];
for (let i = 0; i < 12; i++) {
let movie = await getMovie();
values.push(movie)
}
return values;
}
the problem is that when I try to see my movies in main component it returns a {Promise fulfilled Array(12)} while if I logged the movies while I call getMovies it gives me the result I want which is the 12 movie I called , how can I have the results I need
Here's a basic setup to help you get started:
const getMovie = async () => {
const randomMovieURL = "https://k2maan-moviehut.herokuapp.com/api/random";
const omdpURL = "https://www.omdbapi.com/?i=tt3896198&apikey=apiKey&t=";
// Fetch the first URL and get a random movie
const result = await fetch(randomMovieURL);
// Get the response and parse the JSON data coming back from this response
const movie = await result.json();
// Fetch the OMDB using the random movie name
const result2 = await fetch(omdpURL + movie.name);
// Once again, parse the JSON from the Response we got back
const movieData = await result2.json();
// Return the JSON data which contains the movie details
return movieData;
}
const fetchMovies = async () => {
const requests = [];
for (let i = 0; i < 12; i++) {
// Fill in the requests Array with the asynchronous operations
requests.push(getMovie())
}
// Run all 12 async operations in parallel to speed things up
const movies = await Promise.all(requests);
// Return the results:
return movies;
}
// fetchMovies() is async so we need to `await` for the result inside
// another async function:
async function main(){
const movies = await fetchMovies();
console.log(movies);
}
main();
// Alternative syntax using then():
fetchMovies().then( movies => console.log( movies ) );
Updated:
Keep in mind, that the correct way to loop over a list of async calls in parallel and get the results, is through Promise.all.
Using a normal for loop and await (as in your case) will result in running all 12 calls sequentially, which is much slower.
Regarding the question:
"the problem is that when I try to see my movies in main component it return a {Promise fulfilled Array(12)} while if I logged the movies while I call getMovies it gives me the result I want which is the 12 movie I called , how can I have the results I need"
The reason why you see a Promise fulfilled Array(12) in the main component is that the getMovies function is async and thus always returns a Promise not a normal value.
In order to see the value, you will need to either use await before the function call (const movies = await getMovies()) or use then (getMovies().then( movies => console.log(movies) )).
This is the reason why you see the actual value in the console.log inside getMovies. You are using await before the async function fetchMovies.
Lesson of the day
You have just stumbled upon one of the trickiest and hardest parts of JavaScript: asynchronous programming using Promises.
Make sure to go through the following MDN resources to get a comprehensive and solid foundation of Promises:
Using Promises
async function
await expression
I am not sure why you try to nest your await fetch clauses. If you do it like this it would work.
const getMovies = async () => {
const omdpURL = "http://www.omdbapi.com/?i=tt3896198&apikey=myApiKey&t=";
const res = await fetch(
"https://k2maan-moviehut.herokuapp.com/api/random"
);
var data = await res.json();
const movie = await fetch(omdpURL + data.name);
var moviedata = await movie.json();
console.log(moviedata);
};
getMovies();
Related
I have an array of data and need to fetch data for each item and combine them.
// This is inside a Nextjs async API route
const data = ['item1','item2','item3']
let result = []
data.forEach(async (item)=>{
const res = await fetch(`https://websitename.com/${item}`)
result = [...result,...res]
}
console.log(result) //gives an empty array
Here, it returns an empty array even though for each item data is being fetched. How to make such requests?
}
Your code should be throwing errors I think? You're trying to declare result twice, but as the second time is within the forEach loop, it's actually a different variable, and is created and destroyed for each loop.
Assuming your fetch works, this should work:
const data = ['item1','item2','item3']
let result = []
data.forEach(async (item)=>{
const res = await fetch(`https://websitename.com/${item}`)
result = [...result,...res]
}
console.log(result)
May this could help you :)
async function doRequest(data) {
// process here
}
const requests = ['item1', 'item2', 'item3'];
const results = requests.map(async (val) => {
const response = await doRequest();
return response;
});
await Promise.all(requests);
Change:
const res = await fetch(`https://websitename.com/${item}`)
const result = [...result,...res]
To:
const response = await fetch(`https://websitename.com/${item}`);
const data = response.json(); // if your response is Json
result.push(data);
result should be const instead of let
Method .forEach() makes sync iteration. It means that an iteration does not wait resolving of your async callback, but just starts it. As a result forEach starts all requests and jumps to line with console.log(result). But at the moment none of the requests are done and array is still empty. If you wrap like that setTimeout(()=>console.log(result),3000) you will see that the array is filled with data.
If you want to make sequential calls:
(async function() {
const data = ['item1','item2','item3']
let result = []
for await (const item of data) {
const res = await fetch(`https://websitename.com/${item}`)
console.log(item);
result.push(item)
}
console.log(result)
})();
If you want to make parallel calls:
(async function() {
const data = ['item1','item2','item3']
let result = await Promise.all(data.map(async (item)=>{
return await await fetch(`https://websitename.com/${item}`)
}))
console.log(result)
})();
If not then how can you work with 2 .json files at the same time? It does't seem to work so if there exists another method I should try and use that. If you could provide the correct syntax to achieve the goal it would be appreciated. Does the second .json file ever get processed?
sync function populate() {
const requestURL = 'nascar.json';
const request = new Request(requestURL);
const response = await fetch(request);
const nascarDrivers = await response.json();
findDriver(nascarDrivers);
}
async function texas() {
const requestURL = 'texasMS.json';
const request = new Request(requestURL);
const response = await fetch(request);
const texasLaps = await response.json();
findLaps(texasLaps);
}
If you run them one by one, without awaiting them, they will run in parallel (sort-of).
populate()
texas()
If you want to wait for the results as they come back, you can use Promise.all:
const promises = [
populate(),
texas(),
]
Promise.all(promises).then((results) => {
const [populateRes, texasRes] = results
// ...
})
// or
const results = await Promise.all(promises)
blogCategorie and blog are both CRUD-objects. blogCatergorie.getAll gives me all the category objects. I loop through each category and push the name of the category in to an array (categorieTitles). So far so good. In the map function i want to retrieve all the blogs from that category with blog.getBlogsFromCategory, but it looks like the compiler ignores the await in front of this promise. It pushes the blogs at the end of the code when i already did the `console.log('Blog containers'). Did i do something wrong?
Code
let categorieTitles = [];
let blogContainers = [];
await blogCategorie.getAll().then((allCategories) => {
console.log('after 1st await');
allCategories.map(async (categorie) => {
categorieTitles.push(categorie.name);
await blog.getBlogsFromCategory(categorie.id).then((blogs) => {
console.log("after 2nd await");
let blogGroup = [];
blogs.map((blog) => {
blogGroup.push(blog);
});
blogContainers.push(blogGroup);
});
});
});
console.log('Categorie titels', categorieTitles);
console.log('Blog containers', blogContainers);
output
after 1st wait
Categorie titels [ 'Kinderen en psychologie', 'Efficiënt lesgeven' ]
Blog container []
after 2nd await
after 2nd await
async-await inside a map do not work.
Also, if you are using await, then ideally you should not use the .then syntax, since you can store the response of your asynchronous operation in a variable.
You can change the map to a for-in loop as below:
const allCategories = await blogCategorie.getAll();
console.log('after 1st await');
for (const categories of allCategories) {
categorieTitles.push(categorie.name);
const blogs = await blog.getBlogsFromCategory(categorie.id);
console.log("after 2nd await");
let blogGroup = [];
blogs.map((blog) => {
blogGroup.push(blog);
});
blogContainers.push(blogGroup);
};
I'm fetching my user data and the map function is called several times for each user. I want to wait until all data was pushed to the array and then manipulate the data. I tried using Promise.all() but it didn't work.
How can I wait for this map function to finish before continuing?
Needless to say that the array user_list_temp is empty when printed inside the Promise.all().
const phone_list_promise_1 = await arrWithKeys.map(async (users,i) => {
return firebase.database().ref(`/users/${users}`)
.on('value', snapshot => {
user_list_temp.push(snapshot.val());
console.log(snapshot.val());
})
}
);
Promise.all(phone_list_promise_1).then( () => console.log(user_list_temp) )
I changed the code to this but I still get a wrong output
Promise.all(arrWithKeys.map(async (users,i) => {
const eventRef = db.ref(`users/${users}`);
await eventRef.on('value', snapshot => {
const value = snapshot.val();
console.log(value);
phone_user_list[0][users].name = value.name;
phone_user_list[0][users].photo = value.photo;
})
console.log(phone_user_list[0]);
user_list_temp.push(phone_user_list[0]);
}
));
console.log(user_list_temp); //empty array
}
It is possible to use async/await with firebase
This is how I usually make a Promise.all
const db = firebase.database();
let user_list_temp = await Promise.all(arrWithKeys.map(async (users,i) => {
const eventRef = db.ref(`users/${users}`);
const snapshot = await eventref.once('value');
const value = snapshot.value();
return value;
})
);
This article gives a fairly good explanation of using Promise.all with async/await https://www.taniarascia.com/promise-all-with-async-await/
Here is how I would refactor your new code snippet so that you are not mixing promises and async/await
let user_list_temp = await Promise.all(arrWithKeys.map(async (users,i) => {
const eventRef = db.ref(`users/${users}`);
const snapshot= await eventRef.once('value');
const value = snapshot.val();
console.log(value);
phone_user_list[0][users].name = value.name; // should this be hardcoded as 0?
phone_user_list[0][users].photo = value.photo; // should this be hardcoded as 0?
console.log(phone_user_list[0]);
return phone_user_list[0]; // should this be hardcoded as 0?
})
);
console.log(user_list_temp);
Here is a simple example that uses fetch, instead of firebase:
async componentDidMount () {
let urls = [
'https://jsonplaceholder.typicode.com/todos/1',
'https://jsonplaceholder.typicode.com/todos/2',
'https://jsonplaceholder.typicode.com/todos/3',
'https://jsonplaceholder.typicode.com/todos/4'
];
let results = await Promise.all(urls.map(async url => {
const response = await fetch(url);
const json = await response.json();
return json;
}));
alert(JSON.stringify(results))
}
If I understand your question correctly, you might consider revising your code to use a regular for..of loop, with a nested promise per user that resolves when the snapshot/value for that user is available as shown:
const user_list_temp = [];
/*
Use regular for..of loop to iterate values
*/
for(const user of arrWithKeys) {
/*
Use await on a new promise object for this user that
resolves with snapshot value when value recieved for
user
*/
const user_list_item = await (new Promise((resolve) => {
firebase.database()
.ref(`/users/${users}`)
.on('value', snapshot => {
/*
When value recieved, resolve the promise for
this user with val()
*/
resolve(snapshot.val());
});
}));
/*
Add value for this user to the resulting user_list_item
*/
user_list_temp.push(user_list_item);
}
console.log(user_list_temp);
This code assumes that the enclosing function is defined as an asynchronous method with the async keyword, seeing that the await keyword is used in the for..of loop. Hope that helps!
I'm trying to make a JSON file eventually that will contain all of the results from Google Maps Reviews but I can only get one/latest review to output...
Can anyone help me as to how to make this into an array to get all the reviews?
const puppeteer = require('puppeteer');
let scrape = async () => {
const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']});
const page = await browser.newPage();
await page.goto('https://www.google.com/maps/place/Microsoft/#36.1275216,-115.1728651,17z/data=!3m2!4b1!5s0x80c8c416a26be787:0x4392ab27a0ae83e0!4m7!3m6!1s0x80c8c4141f4642c5:0x764c3f951cfc6355!8m2!3d36.1275216!4d-115.1706764!9m1!1b1');
await page.waitFor(1000);
const result = await page.evaluate(async () => {
let fullName = document.querySelector('.section-review-title').innerText;
let postedDate = document.querySelector('.section-review-publish-date').innerText;
let starRating = document.querySelector('.section-review-stars').getAttribute("aria-label");
let review = document.querySelector('.section-review-text').innerText;
return {
fullName,
postedDate,
starRating,
review
}
});
browser.close();
return result;
};
scrape().then((value) => {
console.log(value); // Success!
});
Thank you!
In general document.querySelectorAll gives you all results and not just the first.
In specific to your use case, what you want to do is getting a handle on ALL reviews first (before processing them).
I checked the url you provided and would start this way (Puppeteer style):
await page.$$('.section-review-content') will return a promise that resolves to an array with all reviews as ElementHandles.
Then you loop through the array and operate on every ElementHandle like this:
await ElementHandle.$eval('.section-review-title', el => el.innerText)
So for example, inside your scrape function you would have (I shortened your scenario a little):
...
await page.goto('https://www.google.com/maps/place/Microsoft/#36.1275216,-115.1728651,17z/data=!3m2!4b1!5s0x80c8c416a26be787:0x4392ab27a0ae83e0!4m7!3m6!1s0x80c8c4141f4642c5:0x764c3f951cfc6355!8m2!3d36.1275216!4d-115.1706764!9m1!1b1');
await page.waitFor(1000);
const reviews = await page.$$(".section-review-content");
for (const review of reviews) {
const reviewTitle = await review.$eval(
".section-review-title",
div => div.innerText
);
console.log('\n' + reviewTitle);
}
...
Check out the Puppeteer API how page.$$ works.