Send many parallel fetch request (more than max browser support) - javascript

I want to know: if I want to send multiple fetch request, that exceeds max browser parallel request support, can I send all request in parallel
and browser automatically will handle them? or I should first shrink them to sufficient size bunches and chain them.
Promise.all([allpromisses])
or any other solutions like:
function fetchAll(urls) {
const requestPromises = urls.map(url => {
return fetch(url).then(response => response.json());
});
requestPromises.reduce((chain, requestPromise) => {
return chain.then(() => requestPromise)
.then(data => data);
}, Promise.resolve());
}
or
getBunch([promises1]).then(getBunch([promises2]). ...

If you don't want to continue until all of the fetches return, it's worth just using Promise.all() and letting the browser handle queuing the requests. One nice thing is that Promise.all() will "fail fast". Meaning that unless you're handing errors individually, one failed promise will reject the Promise.all().
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all#Promise.all_fail-fast_behaviour

Related

How to wait for multiple nested promises to resolve?

I am attempting to make several http requests (using axios), one to each url in a list. As soon as an individual request resolves, I want to make another request to a url constructed from the previous request's response. Finally, when all secondary requests are resolved, I want to do something with all of their responses.
I have attempted to accomplish this as follows:
let promises = [];
urls.forEach(url => {
const promise = axios.get(url);
promise.then(response => {
promises.push(axios.get(response.data.url))
});
});
Promise.all(promises).then(responses => {
// do something with responses
})
I expect the responses of all secondary url requests to be resolved when .then(responses => {}) triggers, yet when this code triggers the responses array is empty.
How can I achieve something like this?
Return the nested axios.get to a mapping callback, so that Promise.all will wait for all nested requests to complete first.
Promise.all(
urls.map(
url => axios.get(url).then(
response => axios.get(response.data.url)
)
)
)
.then((results) => {
// ...
})
This is logical considering that very first promise.then() statement is asynchronous. The urls.forEach althoughy seemingly asynchronous, it is synchronous. Thus the Promise.all() is called before any promises are pushed to your array.
You could set it up like so:
let promises = [];
urls.forEach(url => {
promises.push(axios.get(url).then((response) => {
return axios.get(response.data.url);
});
});
Promise.all(promises).then(responses => {
// do something with responses
});
This chains the .then() directly onto the first axios request, which is then pushed onto the promises array.

Multiple fetch request in a raw

For a project, I have to generate a PDF that lists the vehicles in a DB. To do this, I use the JSPDF library.
1st step: I get data from a DB (via asynchronous requests on an API) and images on a server that I store in an Array.
2nd step: I call a function that generates the PDF with JSPDF.
The problem is that I need to have all my data retrieved before calling my generatePDF function otherwise the fields and images are empty because they have not yet been retrieved from the DB or the server.
A solution I found is to use the setTimeout function to put a delay between each call. However, this makes the code very slow and inflexible because you have to change the timeout manually depending on the number of data and images to retrieve. Moreover, it is impossible to determine exactly how long it will take to retrieve the data, especially since this can vary depending on the state of the network, so you have to allow for a margin which is often unnecessary.
Another solution is to use callbacks or to interweave fetch / ajax calls with .then / .done calls, but this becomes very complicated when it comes to retrieving the images since they are retrieved one by one and there are more than a hundred of them.
What would be the easiest way to do this in a clean and flexible way?
Thanks for your help and sorry for the long text, I tried to be as clear as possible :)
To do a series of asynchronous things in order, you start the next operation in the fulfillment handler of the previous operation.
An async function is the easiest way:
async function buildPDF() {
const response = await fetch("/path/to/the/data");
if (!response.ok) {
throw new Error(`HTTP error ${response.status}`);
}
const data = await response.json(); // Or `.text()` or whatever
const pdf = await createPDF(data); // Assuming it returns a promise
// ...
}
If you can't use async functions in your environment and don't want to transpile, you can write your fulfullment handlers as callbacks:
function buildPDF() {
return fetch("/path/to/the/data")
.then(response => {
if (!response.ok) {
throw new Error(`HTTP error ${response.status}`);
}
return response.json(); // Or `.text()` or whatever
})
.then(data => createPDF(data))
.then(pdf => {
// ...
});
}
Note that I'm returning the result of the chain, so that the caller can handle errors.

Promise: skip all fulfill and reject reactions but execute .finally

I have function called request:
function request (endpoint) {
return axios.request(endpoint).then(api.onSuccess).catch(api.onError)
}
api.onSuccess:
onSuccess (response) {
let breakChain = false
... some logic goes here ...
return breakChain ? (new Promise(() => {})) : response
}
api.onError:
onError (error) {
let breakChain = false
... some logic goes here ...
if (breakChain) {
return new Promise(() => {})
} else {
throw error
}
}
api holds a lot of functions that represent different API calls based on provided endpoints data and return request(endpoint).
Currenly I have code, as you can see, that return Promise with empty executor that is always in pending state to stop the chain of subsequent .then(...) and .catch(...) handlers from execution as they just infinitely wait for that Promise to settle. This is done to handle certain API responses that have common response handling (like responses with code >= 500).
The problem is that now I need a call to .finally() (like in Vue cookbook - https://v2.vuejs.org/v2/cookbook/using-axios-to-consume-apis.html#Dealing-with-Errors) to restore some component's state nevertheless of whether there is an error or not, but this approach of pending Promise is an obstacle.
The question is: is it possible to skip all subsequent .then(...) and .catch(...) calls within one of such handlers to go directly to .finally()?
Update. I haven't mentioned the important bit - api.onSuccess and api.onError are basic handlers. In another application components there are additional handlers appended to the end of that basic chain presented in request function. Usual chain of some API call has a following resulting form:
return axios.request(endpoint).then(api.onSuccess).catch(api.onError).then((response) => {...}).catch(() => {...}).finally(() => {...})
(sometimes there is no .finally() or .catch(...) block)
Is it possible to skip all subsequent .then(...) and .catch(...) calls within one of such handlers to go directly to .finally()?
No.
Currenly I stop the chain by just infinitely waiting - yet this approach of pending Promise is an obstacle.
Indeed, don't do that. You can skip then handlers by using rejections (exceptions) for flow control, but the more appropriate way is to handle this by nesting the part of the chain to be skipped inside an if statement.
This is done to handle certain API responses that have common response handling (like responses with code >= 500)
For that, you should use something like
return axios.request(endpoint).then(response => {
…
}).catch(error => {
if (api.handleCommonError(error)) return; // returns false if it couldn't handle the error
…
}).finally(() => {
…
});
Yes, you cannot hide this kind of error handling inside an api.request function.
You can use async and await. All modern browsers support them, and your bundler can make them compatible with older browsers.
For example:
async function request (endpoint) {
try {
const response = await axios.request(endpoint);
return api.onSuccess(response);
} catch (err) {
api.onError(err);
} finally {
// Always executed, even if no error was thrown
}
}
You can also do it more traditionally:
function request (endpoint) {
return axios.request(endpoint).then(api.onSuccess, api.onError).then(() => {
// Code is always executed after error / success
}
}

how can I get the result for the [ Promise { <pending> } ]? [duplicate]

This is more of a conceptual question. I understand the Promise design pattern, but couldn't find a reliable source to answer my question about promise.all():
What is(are) the correct scenario(s) to use promise.all()
OR
Are there any best practices to use promise.all()? Should it be ideally used only if all of the promise objects are of the same or similar types?
The only one I could think of is:
Use promise.all() if you want to resolve the promise only if all of the promise objects resolve and reject if even one rejects.
I'm not sure anyone has really given the most general purpose explanation for when to use Promise.all() (and when not to use it):
What is(are) the correct scenario(s) to use promise.all()
Promise.all() is useful anytime you have more than one promise and your code wants to know when all the operations that those promises represent have finished successfully. It does not matter what the individual async operations are. If they are async, are represented by promises and your code wants to know when they have all completed successfully, then Promise.all() is built to do exactly that.
For example, suppose you need to gather information from three separate remote API calls and when you have the results from all three API calls, you then need to run some further code using all three results. That situation would be perfect for Promise.all(). You could so something like this:
Promise.all([apiRequest(...), apiRequest(...), apiRequest(...)]).then(function(results) {
// API results in the results array here
// processing can continue using the results of all three API requests
}, function(err) {
// an error occurred, process the error here
});
Promise.all() is probably most commonly used with similar types of requests (as in the above example), but there is no reason that it needs to be. If you had a different case where you needed to make a remote API request, read a local file and read a local temperature probe and then when you had data from all three async operations, you wanted to then do some processing with the data from all three, you would again use Promise.all():
Promise.all([apiRequest(...), fs.promises.readFile(...), readTemperature(...)]).then(function(results) {
// all results in the results array here
// processing can continue using the results of all three async operations
}, function(err) {
// an error occurred, process the error here
});
On the flip side, if you don't need to coordinate among them and can just handle each async operation individually, then you don't need Promise.all(). You can just fire each of your separate async operations with their own .then() handlers and no coordination between them is needed.
In addition Promise.all() has what is called a "fast fail" implementation. It returns a master promise that will reject as soon as the first promise you passed it rejects or it will resolve when all the promises have resolved. So, to use Promise.all() that type of implementation needs to work for your situation. There are other situations where you want to run multiple async operations and you need all the results, even if some of them failed. Promise.all() will not do that for you directly. Instead, you would likely use something like Promise.settle() for that situation. You can see an implementation of .settle() here which gives you access to all the results, even if some failed. This is particularly useful when you expect that some operations might fail and you have a useful task to pursue with the results from whatever operations succeeded or you want to examine the failure reasons for all the operations that failed to make decisions based on that.
Are there any best practices to use promise.all()? Should it be
ideally used only if all of the promise objects are of the same or
similar types?
As explained above, it does not matter what the individual async operations are or if they are the same type. It only matters whether your code needs to coordinate them and know when they all succeed.
It's also useful to list some situations when you would not use Promise.all():
When you only have one async operation. With only one operation, you can just use a .then() handler on the one promise and there is no reason for Promise.all().
When you don't need to coordinate among multiple async operations.
When a fast fail implementation is not appropriate. If you need all results, even if some fail, then Promise.all() will not do that by itself. You will probably want something like Promise.allSettled() instead.
If your async operations do not all return promises, Promise.all() cannot track an async operation that is not managed through a promise.
Promise.all is for waiting for several Promises to resolve in parallel (at the same time). It returns a Promise that resolves when all of the input Promises have resolved:
// p1, p2, p3 are Promises
Promise.all([p1, p2, p3])
.then(([p1Result, p2Result, p3Result]) => {
// This function is called when p1, p2 and p3 have all resolved.
// The arguments are the resolved values.
})
If any of the input Promises is rejected, the Promise returned by Promise.all is also rejected.
A common scenario is waiting for several API requests to finish so you can combine their results:
const contentPromise = requestUser();
const commentsPromise = requestComments();
const combinedContent = Promise.all([contentPromise, commentsPromise])
.then(([content, comments]) => {
// content and comments have both finished loading.
})
You can use Promise.all with Promise instance.
It's hard to answer these questions as they are the type that tend to answer themselves as one uses the available APIs of a language feature. Basically, it's fine to use Promises any way that suits your use case, so long as you avoid their anti-patterns.
What is(are) the correct scenario(s) to use promise.all()
Any situation in which an operation depends on the successful resolution of multiple promises.
Are there any best practices to use promise.all()? Should it be ideally used only if all of the promise objects are of the same or similar types?
Generally, no and no.
I use promise.all() when I have to do some requests to my API and I don't want to display something before the application loads all the data requested, so I delay the execution flow until I have all the data I need.
Example:
What I want to do I want to load the users of my app and their products (imagine that you have to do multiple requests) before displaying a table in my app with the user emails and the product names of each user.
What I do next I send the requests to my API creating the promises and using promise.all()
What I do when all the data has been loaded Once the data arrives to my app, I can execute the callback of promises.all() and then make visible the table with the users.
I hope it helps you to see in which scenario makes sense to use promises.all()
As #joews mentioned, probably one of the important features of Promise.all that should be explicitly indicated is that it makes your async code much faster.
This makes it ideal in any code that contains independent calls (that we want to return/finish before the rest of the code continues), but especially when we make frontend calls and want the user's experience to be as smooth as possible.
async function waitSecond() {
return new Promise((res, rej) => {
setTimeout(res, 1000);
});
}
function runSeries() {
console.time('series');
waitSecond().then(() => {
waitSecond().then(() => {
waitSecond().then(() => {
console.timeEnd('series');
});
});
});
}
function runParallel() {
console.time('parallel');
Promise.all([
waitSecond(),
waitSecond(),
waitSecond(),
]).then(() => {
console.timeEnd('parallel');
});
}
runSeries();
runParallel();
I tend to use promise all for something like this:
myService.getUsers()
.then(users => {
this.users = users;
var profileRequests = users.map(user => {
return myService.getProfile(user.Id); // returns a promise
});
return Promise.all(profileRequests);
})
.then(userProfilesRequest => {
// do something here with all the user profiles, like assign them back to the users.
this.users.forEach((user, index) => {
user.profile = userProfilesRequest[index];
});
});
Here, for each user we're going off and getting their profile. I don't want my promise chain to get out of hand now that i have x amount of promises to resolve.
So Promise.all() will basically aggregate all my promises back into one, and I can manage that through the next then. I can keep doing this for as long as a like, say for each profile I want to get related settings etc. etc. Each time I create tonnes more promises, I can aggregate them all back into one.
Promise.all-This method is useful for when you want to wait for more than one promise to complete or The Promise.all(iterable) method returns a promise that resolves when all of the promises in the iterable argument have resolved, or rejects with the reason of the first passed promise that rejects.
2.Just use Promise.all(files).catch(err => { })
This throws an error if ANY of the promises are rejected.
3.Use .reflect on the promises before .all if you want to wait for all
promises to reject or fulfill
Syntax -Promise.all(iterable);
Promise.all passes an array of values from all the promises in the iterable object that it was passed.
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
var isCallFailed = false;
function myEndpoint1() {
return isCallFailed ? Promise.reject("Bohoo!") :Promise.resolve({"a":"a"});
}
function myEndpoint2() {
return Promise.resolve({"b":"b"});
}
Promise.all([myEndpoint1(), myEndpoint2()])
.then(values => {
var data1 = values[0];
var data2 = values[1];
alert("SUCCESS... data1: " + JSON.stringify(data1) + "; data2: " + JSON.stringify(data2));
})
.catch(error => {
alert("ERROR... " + error);
});
you can try another case by making isCallFailed = true.
Use Promise.all only when you need to run a code according to the result of more than one asynchronous operations using promises.
For example:
You have a scenario like, You need to download 2000 mb file from server, and at the same time you are going to free the user storage to make sure it can save the downloaded file.
And you need to save only in case if the file is downloaded successfully and the storage space is created successfully.
you will do like this.
your first asynchronous operation
var p1 = new Promise(function(resolve, reject) {
// you need to download 2000mb file and return resolve if
// you successfully downloaded the file
})
and your second asynchronous operation
var p2 = new Promise(function(resolve, reject) {
// you need to clear the user storage for 2000 mb
// which can take some time
})
Now you want to save only when both of the promises resolved successfully, otherwise not.
You will use promise.all like this.
Promise.all([p1,p2]).then((result)=>{
// you will be here only if your both p1 and p2 are resolved successfully.
// you code to save the downloaded file here
})
.catch((error)=>{
// you will be here if at-least one promise in p1,p2 is rejected.
// show error to user
// take some other action
})
Promise.all can be used in a scenario when there is a routine which is validating multiplerules based on particular criteria and you have to execute them all in parallel and need to see the results of those rules at one point. Promise.all returns the results as an array which were resolved in your rule vaidator routine.
E.g.
const results = await Promise.all([validateRule1, validateRule2, validateRule3, ...]);
then results array may look like (depending upon the conditions) as for example: [true, false, false]
Now you can reject/accept the results you have based on return values. Using this way you won't have to apply multiple conditions with if-then-else.
If you are interested only Promise.all then read below Promise.all
Promise (usually they are called "Promise") - provide a convenient way to organize asynchronous code.
Promise - is a special object that contains your state. Initially, pending ( «waiting"), and then - one of: fulfilled ( «was successful") or rejected ( «done with error").
On the promise to hang callbacks can be of two types:
unFulfilled - triggered when the promise in a state of "completed
successfully."
Rejected - triggered when the promise in the "made in error."
The syntax for creating the Promise:
var promise = new Promise(function(resolve, reject) {
// This function will be called automatically
// It is possible to make any asynchronous operations,
// And when they will end - you need to call one of:
// resolve(result) on success
// reject(error) on error
})
Universal method for hanging handlers:
promise.then(onFulfilled, onRejected)
onFulfilled - a function that will be called with the result with
resolve.
onRejected - a function that will be called when an error reject.
With its help you can assign both the handler once, and only one:
// onFulfilled It works on success
promise.then(onFulfilled)
// onRejected It works on error
promise.then(null, onRejected)
Synchronous throw - the same that reject
'use strict';
let p = new Promise((resolve, reject) => {
// то же что reject(new Error("o_O"))
throw new Error("o_O");
});
p.catch(alert); // Error: o_O
Promisification
Promisification - When taking asynchronous functionality and make it a wrapper for returning PROMIS.
After Promisification functional use often becomes much more convenient.
As an example, make a wrapper for using XMLHttpRequest requests
httpGet function (url) will return PROMIS, which upon successful data loading with the url will go into fulfilled with these data, and in case of error - in rejected with an error information:
function httpGet(url) {
return new Promise(function(resolve, reject) {
var xhr = new XMLHttpRequest();
xhr.open('GET', url, true);
xhr.onload = function() {
if (this.status == 200) {
resolve(this.response);
} else {
var error = new Error(this.statusText);
error.code = this.status;
reject(error);
}
};
xhr.onerror = function() {
reject(new Error("Network Error"));
};
xhr.send();
});
}
As you can see, inside the function XMLHttpRequest object is created and sent as usual, when onload / onerror are called, respectively, resolve (at the status 200) or reject.
Using:
httpGet("/article/promise/user.json")
.then(
response => alert(`Fulfilled: ${response}`),
error => alert(`Rejected: ${error}`)
);
Parallel execution
What if we want to implement multiple asynchronous processes simultaneously and to process their results?
The Promise class has the following static methods.
Promise.all(iterable)
Call Promise.all (iterable) receives an array (or other iterable object) and returns PROMIS PROMIS, which waits until all transferred PROMIS completed, and changes to the state "done" with an array of results.
For example:
Promise.all([
httpGet('/article/promise/user.json'),
httpGet('/article/promise/guest.json')
]).then(results => {
alert(results);
});
Let's say we have an array of URL.
let urls = [
'/article/promise/user.json',
'/article/promise/guest.json'
];
To download them in parallel, you need to:
Create for each URL corresponding to PROMIS.
Wrap an array of PROMIS in Promise.all.
We obtain this:
'use strict';
let urls = [
'/article/promise/user.json',
'/article/promise/guest.json'
];
Promise.all( urls.map(httpGet) )
.then(results => {
alert(results);
});
Note that if any of Promise ended with an error, the result will
Promise.all this error.
At the same time the rest of PROMIS ignored.
For example:
Promise.all([
httpGet('/article/promise/user.json'),
httpGet('/article/promise/guest.json'),
httpGet('/article/promise/no-such-page.json') // (нет такой страницы)
]).then(
result => alert("не сработает"),
error => alert("Ошибка: " + error.message) // Ошибка: Not Found
)
In total:
Promise - is a special object that stores its state, the current
result (if any), and callbacks.
When you create a new Promise ((resolve, reject) => ...) function
argument starts automatically, which should call resolve (result) on
success, and reject (error) - error.
Argument resolve / reject (only the first, and the rest are ignored)
is passed to handlers on this Promise.
Handlers are appointed by calling .then / catch.
To transfer the results from one processor to another using Channing.
https://www.promisejs.org/patterns/

Why Use JS Promises Instead of IF/ELSE / Ternary?

I'm doing some reading up on JS Promises to up-skill.
Here's my quandry:
Say you want to console.log('we done, bruh!') AFTER your data's come back.
so with a Promise, you might say:
let iWantToLogOut = function() {
let data = fetch('https://jsonplaceholder.typicode.com/users')
return new Promise((resolve) => {
resolve(data)
})
}
And then resolve that promise like:
iWantToLogOut().then((dataBack)
=> databack.json())
.then((json) => {
console.log('We done, bruh! Look: ', json)
})
So that's great. You get your API data back and then we log our msg out.
But isn't it just way easier to go:
let data = fetch('https://jsonplaceholder.typicode.com/users');
data ? console.log('we done, bruh!') : null;
I'm probably over-simplifying/missing something (because... well... i'm retarded) but I just want to make sure i'm really understanding Promises first before i move onto Async/Await.
But isn't it just way easier to go:
let data = fetch('https://jsonplaceholder.typicode.com/users');
data ? console.log('we done, bruh!') : null;
It would be, but it doesn't work. What fetch returns is a promise, not the result of the operation. You can't return the result of an asynchronous process. More: How do I return the response from an asynchronous call?
In the upcoming ES2017 spec, though, we have syntactic sugar around promise consumption which will let you write this:
let data = await fetch('https://jsonplaceholder.typicode.com/users');
// --------^^^^^
console.log('we done, bruh!');
Note we don't even need the conditional, because await converts a promise rejection into an exception.
That code would need to be in an async function, e.g.:
(async function() {
let data = await fetch(/*...*/);
// use data here
})();
The JavaScript engines in some browsers already support async/await, but to use it in the wild, you'll want to transpile with Babel or similar.
Note: You've shown
so with a Promise, you might say:
let iWantToLogOut = function() {
let data = fetch('https://jsonplaceholder.typicode.com/users')
return new Promise((resolve) => {
resolve(data)
})
}
There are a couple of problems with that code:
It never settles the promise you created if the fetch fails.
It calls something data which is not data, it's a promise of data (that's mostly style, but it's misleading).
It exhibits the promise creation anti-pattern. You already have a promise (from fetch), no need to create another.
iWantToLogOut should be simply:
let iWantToLogOut = function() {
return fetch('https://jsonplaceholder.typicode.com/users');
};
That returns a promise that will be resolved with the data, or of course rejected. Which you'd then consume with promise methods or await (within an async function).
It is not a matter of easy.
Usually network calls should be handle asynchronously(I don't want to the anti-pattern of synchronous AJAX calls). At that point you have few options to handle it:
Callbacks
Promises
Observables
In you code above, when it's synchronous, the fetch should return immediately with a promise that will be resolve to the data only when the server has responded. Only then you can check the data for it's content. Further. Because every promise can be fulfilled or failed, in your then you can have a handler for each instead of using the ternary.
From the latest spec:
Synchronous XMLHttpRequest outside of workers is in the process of being removed from the web platform as it has detrimental effects to the end user’s experience. (This is a long process that takes many years.) Developers must not pass false for the async argument when current global object is a Window object. User agents are strongly encouraged to warn about such usage in developer tools and may experiment with throwing an InvalidAccessError exception when it occurs.

Categories