I have this code that returns the speed score of a website through google API. As sometimes the value is not correct, i read on a post, that is a good practice to make the request few times and then make the median of the score . How i can make multiple Api request simultaneously ?
I tried something like this
function medianSpeed() {
let values = [];
function call() {
return fetch(url)
.then((response) => response.json())
.then((json) => {
const speed = +json.lighthouseResult.categories.performance.score;
values.push(speed);
});
}
const promise1 = call(),
promise2 = call(),
promise3 = call();
const promises = [promise1, promise2, promise3];
Promise.allSettled(promises).then(() => {
return values;
});
}
now the thing is if i call medianSpeed() the code doesn't wait the end of Promise.allSettled it returns straight undefined
Inside medianSpeed() add return before the call to Promise.allSettled():
return Promise.allSettled(promises).then(() => {
return values;
});
and when calling medianSpeed(), calc the median in .then() callback:
medianSpeed().then(values=>calcMedian(values))
Your issue here is that medianSpeed isn't returning a promise, so there's nothing to wait for.
If you did return Promise.allSettled... and then did a then block after calling the function: medianSpeed().then(values => {}) then you would get the values back.
I assume that the url variable is coming from somewhere else?
Related
I'm currently fetching data from an API and I need to do multiple GET requests (using axios). After all those GET requests are completed, I return a resolved promise.
However, I need to do these GET requests automatically based on an array list:
function do_api_get_requests() {
return promise = new Promise(function(resolve, reject) {
API_IDs = [0, 1, 2];
axios.get('https://my.api.com/' + API_IDs[0])
.then(data => {
// Do something with data
axios.get('https://my.api.com/' + API_IDs[1])
.then(data => {
// Do something with data
axios.get('https://my.api.com/' + API_IDs[2])
.then(data => {
// Do something with data
// Finished, resolve
resolve("success");
}
}
}
}
}
This works but the problem is API_IDs isn't always going to be the same array, it will change. So I'm not sure how to chain these requests automatically.
Since you said it may be a variable length array and you show sequencing the requests, you can just loop through the array using async/await:
async function do_api_get_requests(API_IDS) {
for (let id of API_IDS) {
const data = await axios.get(`https://my.api.com/${id}`);
// do something with data here
}
return "success";
}
And, since you said the list of API ids would be variable, I made it a parameter that you can pass into the function.
If you wanted to run all the API requests in parallel (which might be OK for a small array, but might be trouble for a large array) and you don't need to run them in a specific order, you can do this:
function do_api_get_requests(API_IDS) {
return Promise.all(API_IDS.map(async (id) => {
const data = await axios.get(`https://my.api.com/${id}`);
// do something with data here for this request
})).then(() => {
// make resolved value be "success"
return "success";
});
}
Depending upon your circumstances, you could also use Promise.allSettled(). Since you don't show getting results back, it's not clear whether that would be useful or not.
You can use Promise.all() method to do all API requests at the same time, and resolve when all of them resolves.
function do_api_get_requests() {
const API_IDs = [0, 1, 2];
let promises = [];
for (const id of API_IDS) {
promises.push(axios.get(`https://my.api.com/${id}`));
}
return Promise.all(promises);
}
If you use Bluebird.js (a better promise library, and faster than the in-built Promise), you can use Promise.each(), Promise.mapSeries(), or Promisme.reduce() to do what you want.
http://bluebirdjs.com
I have this script that get random videos in the web.
The function getRandomVideo() returns a Promise with a list of urls.
What I want to do is print the data by calling main().then(data => console.log(data)). The problem is that data is being printed before the function is done running. So when I start the program I get undefined and then after the function is done I get the actual data.
I thought that what is inside .then() would run just after the promise is returned.
Does anyone know what is happening?
const main = async () => {
let allData = [];
getRandomVideo().then((videoLinks) => {
allData = videoLinks;
return new Promise((resolve) => {
resolve(allData);
});
});
};
main().then((data) => console.log(data));
As others have said, your code is full of anti-patterns in an attempt to avoid the basics of asynchronous programming with promises. All you need is this:
const main = function() => {
return getRandomVideo();
}
main().then(data => console.log(data)).catch(err => console.log(err));
Or, of course, you don't even need main() at all. You can just do:
getRandomVideo().then(data => console.log(data)).catch(err => console.log(err));
Some anti-patterns in your original code:
Not returning anything from your main() function.
Assigning an asynchronous value to a higher scoped variable and hoping that it somehow fixes asynchronous programming issues. It is nearly always a warning that you're doing something wrong when you assign an asynchronously retrieved value to a higher scoped variable because the code in the higher scope generally has no idea when that value will actually be available. Only within the .then() handler so you know when the value is present.
Manually creating a promise for no particular reason.
Marking a function async for no reason.
No error handling (e.g. .catch() handler) when calling a function that returns a promise.
If you want to manipulate the results of getRandomVideo before you send it back to the caller, then you could have a reason to use async/await
const main = async () => {
let videos = await getRandomVideo();
// define some filter function for the videos array and
// return the filtered result as the resolved value
return videos.filter(...);
}
main().then(data => console.log(data)).catch(err => console.log(err));
I think we can understand main function step by step:
let allData = []; //Defined allData variable
call getRandomVideo function
The thread wait getRandomVideo completed and return videoLinks array when you call .then function
allData = videoLinks; //Asign videoLinks array to allData variable
return new Promise((resolve) => { resolve(allData) }) //Return a Promise, that will tell the thread that you will do that later, on the other hand, after few second it's will comback and return all data by call resolve function
End of main function
==>> So as you can see, main function don't return a Promise imediately, you can't use .then to wait until it completed
Solution: Let wrap main function detail by a Promise like that
const main= () => {
return new Promise((resolve) => {
getRandomVideo().then((videoLinks) => {
resolve(videoLinks);
});
});
}
And clean your code by drop allData variable.
I am trying to combine .fetch(), .then(). and .all(). But I can't get it right. My case looks like this:
fetch() a record from an API. (My Item)
then() Search in this record for more API links. (Here I find the links to the media that belong to the item. Each item can have any number of media).
all() Execute an API call for each of these links with .fetch()
then() if all previous promises are fulfilled, execute the render
function that depends on this data.
// Prepare Request Header
this.url_params = new URLSearchParams(window.location.search);
let httpHeader = {};
let request = new Request(api_itemURL + this.url_params.toString(), {
method: "GET",
headers: new Headers(httpHeader),
});
// The first fetch I have wrapped a seperate class. I still need it from other places.
omekaAPI
.getItemOnly(request)
.then(
function (result) {
this.itemObject = result;
let media_array = Array.from(result["o:media"], (x) => x["#id"]);
// Up to here it works wonderfully. I have list with all links. But after that it doesn't even jump into the all.
console.log(media_array);
return media_array;
}.bind(this)
)
.all(
media_array.map((media_array) =>
fetch(url).then(function () {
console.log("Its works!?!");
})
)
);
I don't know what I'm doing wrong and could use a push in the right direction.
Follow-Up Question
Today I have now tried to process the input. Unfortunately, I cheered too early. I only got back the underfilled Promise. With which my loop of course does not work. I then tried to append .then(). Both inside Promise.all() and after the .then() in which I assumed it. Unfortunately this has the same result. I thought then should go back a step and read about Promises in general. Is it possible that I have to do this with await/async? But then I have to write the complete function differently or? Also, my babel/uglify completely freaks out when I have one of these commands inside. It would be great if someone could give me a hint if I can get anywhere with my code here?
getItemWithAllMedia(request) {
return fetch(request)
.then((response) => {
return response.json();
})
.then((data) => {
console.log(data);
return data;
})
.then((result) => {
let media_links = Array.from(result["o:media"], (x) => x["#id"]);
return media_links;
})
.then((media_links) => {
return Promise.all(media_links.map((url) => fetch(url)));
})
.then((media_response) => {
/// debugging results from promise.all
console.log(media_response); // (2) [Response, Response]
console.log(media_reponse[0]); // ReferenceError: media_reponse is not defined
console.log(media_reponse[0].json()); // ReferenceError: media_reponse is not defined
/// desired function: all responses to json and push to an array
media_response.forEach((element) => {
media_data.push(element.json());
});
/// show created array for debugging
console.log(media_data);
return media_data;
})
In my code below I get an empty array on my console.log(response) but the console.log(filterdIds) inside the getIds function is showing my desired data. I think my resolve is not right.
Note that I run do..while once for testing. The API is paged. If the records are from yesterday it will keep going, if not then the do..while is stopped.
Can somebody point me to the right direction?
const axios = require("axios");
function getToken() {
// Get the token
}
function getIds(jwt) {
return new Promise((resolve) => {
let pageNumber = 1;
const filterdIds = [];
const config = {
//Config stuff
};
do {
axios(config)
.then((response) => {
response.forEach(element => {
//Some logic, if true then:
filterdIds.push(element.id);
console.log(filterdIds);
});
})
.catch(error => {
console.log(error);
});
} while (pageNumber != 1)
resolve(filterdIds);
});
}
getToken()
.then(token => {
return token;
})
.then(jwt => {
return getIds(jwt);
})
.then(response => {
console.log(response);
})
.catch(error => {
console.log(error);
});
I'm also not sure where to put the reject inside the getIds function because of the do..while.
The fundamental problem is that resolve(filterdIds); runs synchronously before the requests fire, so it's guaranteed to be empty.
Promise.all or Promise.allSettled can help if you know how many pages you want up front (or if you're using a chunk size to make multiple requests--more on that later). These methods run in parallel. Here's a runnable proof-of-concept example:
const pages = 10; // some page value you're using to run your loop
axios
.get("https://httpbin.org") // some initial request like getToken
.then(response => // response has the token, ignored for simplicity
Promise.all(
Array(pages).fill().map((_, i) => // make an array of request promisess
axios.get(`https://jsonplaceholder.typicode.com/comments?postId=${i + 1}`)
)
)
)
.then(responses => {
// perform your filter/reduce on the response data
const results = responses.flatMap(response =>
response.data
.filter(e => e.id % 2 === 0) // some silly filter
.map(({id, name}) => ({id, name}))
);
// use the results
console.log(results);
})
.catch(err => console.error(err))
;
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>
The network tab shows the requests happening in parallel:
If the number of pages is unknown and you intend to fire requests one at a time until your API informs you of the end of the pages, a sequential loop is slow but can be used. Async/await is cleaner for this strategy:
(async () => {
// like getToken; should handle err
const tokenStub = await axios.get("https://httpbin.org");
const results = [];
// page += 10 to make the snippet run faster; you'd probably use page++
for (let page = 1;; page += 10) {
try {
const url = `https://jsonplaceholder.typicode.com/comments?postId=${page}`;
const response = await axios.get(url);
// check whatever condition your API sends to tell you no more pages
if (response.data.length === 0) {
break;
}
for (const comment of response.data) {
if (comment.id % 2 === 0) { // some silly filter
const {name, id} = comment;
results.push({name, id});
}
}
}
catch (err) { // hit the end of the pages or some other error
break;
}
}
// use the results
console.log(results);
})();
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>
Here's the sequential request waterfall:
A task queue or chunked loop can be used if you want to increase parallelization. A chunked loop would combine the two techniques to request n records at a time and check each result in the chunk for the termination condition. Here's a simple example that strips out the filtering operation, which is sort of incidental to the asynchronous request issue and can be done synchronously after the responses arrive:
(async () => {
const results = [];
const chunk = 5;
for (let page = 1;; page += chunk) {
try {
const responses = await Promise.all(
Array(chunk).fill().map((_, i) =>
axios.get(`https://jsonplaceholder.typicode.com/comments?postId=${page + i}`)
)
);
for (const response of responses) {
for (const comment of response.data) {
const {name, id} = comment;
results.push({name, id});
}
}
// check end condition
if (responses.some(e => e.data.length === 0)) {
break;
}
}
catch (err) {
break;
}
}
// use the results
console.log(results);
})();
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>
(above image is an except of the 100 requests, but the chunk size of 5 at once is visible)
Note that these snippets are proofs-of-concept and could stand to be less indiscriminate with catching errors, ensure all throws are caught, etc. When breaking it into sub-functions, make sure to .then and await all promises in the caller--don't try to turn it into synchronous code.
See also
How do I return the response from an asynchronous call? and Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference which explain why the array is empty.
What is the explicit promise construction antipattern and how do I avoid it?, which warns against adding a new Promise to help resolve code that already returns promises.
To take a step back and think about why you ran into this issue, we have to think about how synchronous and asynchronous javascript code works together. Your
synchronous getIds function is going to run to completion, stepping through each line until it gets to the end.
The axios function invocation is returning a Promise, which is an object that represents some future fulfillment or rejection value. That Promise isn't going to resolve until the next cycle of the event loop (at the earliest), and your code is telling it to do some stuff when that pending value is returned (which is the callback in the .then() method).
But your main getIds function isn't going to wait around... it invokes the axios function, gives the Promise that is returned something to do in the future, and keeps going, moving past the do/while loop and onto the resolve method which returns a value from the Promise you created at the beginning of the function... but the axios Promise hasn't resolved by that point and therefore filterIds hasn't been populated.
When you moved the resolve method for the promise you're creating into the callback that the axios resolved Promise will invoke, it started working because now your Promise waits for axios to resolve before resolving itself.
Hopefully that sheds some light on what you can do to get your multi-page goal to work.
I couldn't help thinking there was a cleaner way to allow you to fetch multiple pages at once, and then recursively keep fetching if the last page indicated there were additional pages to fetch. You may still need to add some additional logic to filter out any pages that you batch fetch that don't meet whatever criteria you're looking for, but this should get you most of the way:
async function getIds(startingPage, pages) {
const pagePromises = Array(pages).fill(null).map((_, index) => {
const page = startingPage + index;
// set the page however you do it with axios query params
config.page = page;
return axios(config);
});
// get the last page you attempted, and if it doesn't meet whatever
// criteria you have to finish the query, submit another batch query
const lastPage = await pagePromises[pagePromises.length - 1];
// the result from getIds is an array of ids, so we recursively get the rest of the pages here
// and have a single level array of ids (or an empty array if there were no more pages to fetch)
const additionalIds = !lastPage.done ? [] : await getIds(startingPage + pages, pages);
// now we wait for all page queries to resolve and extract the ids
const resolvedPages = await Promise.all(pagePromises);
const resolvedIds = [].concat(...resolvedPages).map(elem => elem.id);
// and finally merge the ids fetched in this methods invocation, with any fetched recursively
return [...resolvedIds, ...additionalIds];
}
First of all, there are some issues with console.log in Google Chrome not functioning as expected. This is not the case as I am working in VSCode.
We begin with two async calls to the server.
promise_a = fetch(url)
promise_b = fetch(url)
Since fetch results are also promises, .json() will needed to be called on each item. The helper function process will be used, as suggested by a Stackoverflow user -- sorry lost the link.
let promiseResults = []
let process = prom => {
prom.then(data => {
promiseResults.push(data);
});
};
Promise.all is called. The resulting array is passed to .then where forEach calls process on item.json() each iteration and fulfilled promises are pushed to promiseResults.
Promise.all([promise_a, promise_b])
.then(responseArr => {
responseArr.forEach(item => {
process(item.json());
});
})
No argument is given to the final .then block because promiseResults are in the outer scope. console.log show confusing results.
.then(() => {
console.log(promiseResults); // correct results
console.log(promiseResults[0]); // undefined ?!?
})
Any help will be greatly appreciated.
If you are familiar with async/await syntax, I would suggest you not to use an external variable promiseResults, but return the results on the fly with this function:
async function getJsonResults(promisesArr) {
// Get fetch promises response
const results = await Promise.all(promisesArr);
// Get JSON from each response promise
const jsonResults = await Promise.all(results.map(r => r.json()));
return jsonResults
}
This is usage example:
promise_a = fetch(url1)
promise_b = fetch(url2)
getJsonResults([promise_a, promise_b])
.then(theResults => console.log('All results:', theResults))
Use theResults variable to extract necessary results.
You can try this, it looks the array loop is not going properly in the promise env.
Specifically: the promiseResults is filled after you are logging.
var resultAll = Promise.all([promise_a, promise_b])
.then(responseArr => {
return Promise.all(responseArr.map(item => return item.json()));
});
resultAll.then(promiseResults => {
console.log(promiseResults);
});