I'm writing a then() statement for extracting the json data from an array of responses from fetch(). In the code below queries is an array of promises returned by a series of calls to fetch(). I'm using async/await for the response because otherwise the promises would be returned without resolving (I found a solution in this question).
My first attempt worked properly, when I push into jsonified I obtain an array with the promises as elements:
return Promise.all(queries)
.then(async(responses)=> {
let jsonified = [];
for (let res of responses){
jsonified.push(await(res.json()));
}
return jsonified;
}.then(data=> ...
But when I went for refactoring and tried to use Array.reduce() I realised that when I push into the accumulator instead of obtaining an array with a promise as element, acc is assigned to be a promise instead.
.then(responses=> {
return responses.reduce(async(acc, next) => {
acc.push(await(next.json()));
return acc;
}, [])
})
I can use the first version without any issue and the program works properly, but whats happening inside Array.reduce()? Why pushing a promise into the accumulator returns a promise intead of an array? How could I refactor the code with Array.reduce()?
Although it's not what you've asked, you could avoid the pain of having to use reduce, and just utilise the Promise.all() that you are already using:
return Promise.all(queries.map(q => q.then(res => res.json()))
.then(data => {...})
It's a much shorter way and less of a headache to read when you come back to it.
Have the accumulator's initial value be a Promise that resolves to an empty array, then await the accumulator on each iteration (so that all prior iterations resolve before the current iteration runs)
.then(responses=> {
return responses.reduce(async (accPromiseFromLastIter, next) => {
const arr = await accPromiseFromLastIter;
arr.push(await next.json());
return arr;
}, Promise.resolve([]))
})
(That said, your original code is a lot clearer, I'd prefer it over the .reduce version)
Live demo:
const makeProm = num => Promise.resolve(num * 2);
const result = [1, 2, 3].reduce(async(accPromiseFromLastIter, next) => {
const arr = await accPromiseFromLastIter;
arr.push(await makeProm(next));
return arr;
}, Promise.resolve([]));
result.then(console.log);
Unless you have to retrieve all data in serial, consider using Promise.all to call the .json() of each Promise in parallel instead, so that the result is produced more quickly:
return Promise.all(queries)
.then(responses => Promise.all(responses.map(response => response.json())));
If the queries are an array of Responses that were just generated from fetch, it would be even better to chain the .json() call onto the original fetch call instead, eg:
const urls = [ ... ];
const results = await Promise.all(
urls.map(url => fetch(url).then(res => res.json()))
);
This way, you can consume the responses immediately when they come back, rather than having to wait for all responses to come back before starting to process the first one.
Related
I'm currently fetching data from an API and I need to do multiple GET requests (using axios). After all those GET requests are completed, I return a resolved promise.
However, I need to do these GET requests automatically based on an array list:
function do_api_get_requests() {
return promise = new Promise(function(resolve, reject) {
API_IDs = [0, 1, 2];
axios.get('https://my.api.com/' + API_IDs[0])
.then(data => {
// Do something with data
axios.get('https://my.api.com/' + API_IDs[1])
.then(data => {
// Do something with data
axios.get('https://my.api.com/' + API_IDs[2])
.then(data => {
// Do something with data
// Finished, resolve
resolve("success");
}
}
}
}
}
This works but the problem is API_IDs isn't always going to be the same array, it will change. So I'm not sure how to chain these requests automatically.
Since you said it may be a variable length array and you show sequencing the requests, you can just loop through the array using async/await:
async function do_api_get_requests(API_IDS) {
for (let id of API_IDS) {
const data = await axios.get(`https://my.api.com/${id}`);
// do something with data here
}
return "success";
}
And, since you said the list of API ids would be variable, I made it a parameter that you can pass into the function.
If you wanted to run all the API requests in parallel (which might be OK for a small array, but might be trouble for a large array) and you don't need to run them in a specific order, you can do this:
function do_api_get_requests(API_IDS) {
return Promise.all(API_IDS.map(async (id) => {
const data = await axios.get(`https://my.api.com/${id}`);
// do something with data here for this request
})).then(() => {
// make resolved value be "success"
return "success";
});
}
Depending upon your circumstances, you could also use Promise.allSettled(). Since you don't show getting results back, it's not clear whether that would be useful or not.
You can use Promise.all() method to do all API requests at the same time, and resolve when all of them resolves.
function do_api_get_requests() {
const API_IDs = [0, 1, 2];
let promises = [];
for (const id of API_IDS) {
promises.push(axios.get(`https://my.api.com/${id}`));
}
return Promise.all(promises);
}
If you use Bluebird.js (a better promise library, and faster than the in-built Promise), you can use Promise.each(), Promise.mapSeries(), or Promisme.reduce() to do what you want.
http://bluebirdjs.com
I need to recursively go through JSON and in some cases call remote API. I need to return the whole JSON modified at the end but I cannot figure out how to wait until all promises are fulfilled
const getObjectsOfRelated = (xmlAsJson, token) => {
if (testIfIwantCallApi()) {
const jsonToReturn = JSON.parse(JSON.stringify(xmlAsJson))
jsonToReturn.elements = callApi(xmlAsJson.text).then(result => {
return result.data
})
return jsonToReturn
}
if (xmlAsJson.elements) {
const jsonToReturn = JSON.parse(JSON.stringify(xmlAsJson))
jsonToReturn.elements = xmlAsJson.elements.map(res => getObjectsOfRelated(res, token))
return jsonToReturn
}
return xmlAsJson
}
Even if I try to hack it using setTimeout the result does not include parts that were created using external API.
This way the code returns correct structure with promises instead of values I want it either return completed promises or be able to wait until the promises are fulfilled.
Wrap plain return values in Promise's using Promise.resolve:
const getObjectsOfRelated = (xmlAsJson, token) => {
if (testIfIwantCallApi()) {
const jsonToReturn = JSON.parse(JSON.stringify(xmlAsJson))
return callApi(xmlAsJson.text).then(result => {
jsonToReturn.elements = result.data;
return jsonToReturn;
})
}
if (xmlAsJson.elements) {
const jsonToReturn = JSON.parse(JSON.stringify(xmlAsJson))
Promise.all(xmlAsJson.elements.map(res => getObjectsOfRelated(res,
token)).then((results) => {
jsonToReturn.elements = results.map(result => result.data);
return jsonToReturn;
});
}
return Promise.resolve(xmlAsJson);
}
This way you will consistently return promises and you can use your function like this:
getObjectsOfRelated(xmlAsJson, token).then(result => console.log(result))
You can use "Promise.all"...
For a simple array, you map a function over the array:
The function returns a promise for the "new value" of each element.
If you use Bluebird promises, you can even return a mixture of Promises and plain values.
( without having to wrap plain values in "Promise.resolve" )
Then you pass the array of promises to "Promise.all()", which waits for all of them to complete.
To transform a tree-shaped data structure (like JSON), you do the same sort of thing, but recursively. Each node in the tree would use "Promise.all" to wait for all its child-nodes=, and the root node would only "resolve" when every node in the tree has resolved.
Note that "Promise.all" is going to run all of your ASYNC functions at the same time. If you don't want that, you can instead use "Promise.mapSeries", which does the same thing, but it waits for each async function before starting the next. This can be better if you have large data and don't want to start too many simultaneous async functions at the same time.
First of all, there are some issues with console.log in Google Chrome not functioning as expected. This is not the case as I am working in VSCode.
We begin with two async calls to the server.
promise_a = fetch(url)
promise_b = fetch(url)
Since fetch results are also promises, .json() will needed to be called on each item. The helper function process will be used, as suggested by a Stackoverflow user -- sorry lost the link.
let promiseResults = []
let process = prom => {
prom.then(data => {
promiseResults.push(data);
});
};
Promise.all is called. The resulting array is passed to .then where forEach calls process on item.json() each iteration and fulfilled promises are pushed to promiseResults.
Promise.all([promise_a, promise_b])
.then(responseArr => {
responseArr.forEach(item => {
process(item.json());
});
})
No argument is given to the final .then block because promiseResults are in the outer scope. console.log show confusing results.
.then(() => {
console.log(promiseResults); // correct results
console.log(promiseResults[0]); // undefined ?!?
})
Any help will be greatly appreciated.
If you are familiar with async/await syntax, I would suggest you not to use an external variable promiseResults, but return the results on the fly with this function:
async function getJsonResults(promisesArr) {
// Get fetch promises response
const results = await Promise.all(promisesArr);
// Get JSON from each response promise
const jsonResults = await Promise.all(results.map(r => r.json()));
return jsonResults
}
This is usage example:
promise_a = fetch(url1)
promise_b = fetch(url2)
getJsonResults([promise_a, promise_b])
.then(theResults => console.log('All results:', theResults))
Use theResults variable to extract necessary results.
You can try this, it looks the array loop is not going properly in the promise env.
Specifically: the promiseResults is filled after you are logging.
var resultAll = Promise.all([promise_a, promise_b])
.then(responseArr => {
return Promise.all(responseArr.map(item => return item.json()));
});
resultAll.then(promiseResults => {
console.log(promiseResults);
});
sup guys, i'm working on this firebase project and i need to iterate trought a subcollection of all sales of all stores in the root collection and sum their values... the problem that i'm getting is that i'm getting the sum printed before the iteration. I'm new to TS and Firebase... this is what i got so far:
export const newBilling = functions.firestore.document('billings/{billId}').onCreate(event =>
{
const valueArray = []
const feeArray = []
const storesCollection = afs.collection('stores').where('active', '==', true).get().then(stores => {
stores.forEach(store => {
const salesCollection = afs.collection('stores').doc(store.id).collection('sales').get().then(sales => {
sales.forEach(sale => {
return valueArray.push(sale.data().value) + feeArray.push(sale.data().fee)
// other aproach
// valueArray.push(sale.data().value)
// feeArray.push(sale.data().fee)
})
})
})
}).catch(error => {console.log(error)})
let cashbackSum, feeSum : number
cashbackArray.forEach(value => {
cashbackSum += value
})
feeArray.forEach(value => {
feeSum += value
})
console.log(cashbackSum, feeSum)
return 0
})
TKS =)
You're not using promises correctly. You've got a lot of get() method call, each of which are asynchronous and return a promise, but you're never using them to case the entire function to wait for all the work to complete. Calling then() doesn't actually make your code wait - it just runs the next bit of code and returns another promise. Your final console.log is executing first because none of the work you kicked off ahead of it is complete yet.
Your code actually needs to be substantially different in order to work correctly, and you need to return a promise from the entire function that resolves only after all the work is complete.
You can learn better how to use promises in Cloud Functions by going through the video tutorials.
I am creating a script in node.js (V8.1.3) which looks at similar JSON data from multiple API's and compares the values. To be more exact I am looking at different market prices of different stocks (actually cryptocurrencies).
Currently, I am using promise.all to wait for all responses from the respective APIs.
let fetchedJSON =
await Promise.all([getJSON(settings1), getJSON(settings2), getJSON(settings3) ... ]);
However, Promise.all throws an error if even just one promise rejects with an error. In the bluebird docos there is a function called Promise.some which is almost what I want. As I understand it takes an array of promises and resolves the two fastest promises to resolve, or otherwise (if less than 2 promises resolve) throws an error.
The problem with this is that firstly, I don't want the fastest two promises resolved to be what it returns, I want any successful promises to be returned, as long as there is more than 2. This seems to be what Promise.any does except with a min count of 1. (I require a minimum count of 2)
Secondly, I don't know how many Promises I will be awaiting on (In other words, I don't know how many API's I will be requesting data from). It may only be 2 or it may be 30. This depends on user input.
Currently writing this it seems to me there is probably just a way to have a promise.any with a count of 2 and that would be the easiest solution. Is this possible?
Btw, not sure if the title really summarizes the question. Please suggest an edit for the title :)
EDIT: Another way I may be writing the script is that the first two APIs to get loaded in start getting computed and pushed to the browser and then every next JSON that gets loaded and computed after it. This way I am not waiting for all Promises to be fulfilled before I start computing the data and passing results to the front end. Would this be possible with a function which also works for the other circumstances?
What I mean kind of looks like this:
Requesting JSON in parallel...
|-----JSON1------|
|---JSON-FAILS---| > catch error > do something with error. Doesn't effect next results.
|-------JSON2-------| > Meets minimum of 2 results > computes JSON > to browser.
|-------JSON3---------| > computes JSON > to browser.
How about thening all the promises so none fail, pass that to Promise.all, and filter the successful results in a final .then.
Something like this:
function some( promises, count = 1 ){
const wrapped = promises.map( promise => promise.then(value => ({ success: true, value }), () => ({ success: false })) );
return Promise.all( wrapped ).then(function(results){
const successful = results.filter(result => result.success);
if( successful.length < count )
throw new Error("Only " + successful.length + " resolved.")
return successful.map(result => result.value);
});
}
This might be somewhat clunky, considering you're asking to implement an anti-pattern, but you can force each promise to resolve:
async function fetchAllJSON(settingsArray) {
let fetchedJSON = await Promise.all(
settingsArray.map((settings) => {
// force rejected ajax to always resolve
return getJSON(settings).then((data) => {
// initial processing
return { success: true, data }
}).catch((error) => {
// error handling
return { success, false, error }
})
})
).then((unfilteredArray) => {
// only keep successful promises
return dataArray.filter(({ success }) => success)
})
// do the rest of your processing here
// with fetchedJSON containing array of data
}
You can use Promise.allSettled([]). the difference is that allSettled will return an array of objects after all the promises are settled regardless if successful or failed. then just find the successful o whatever you need.
let resArr = await Promise.allSettled(userNamesArr.map(user=>this.authenticateUserPassword(user,password)));
return resArr.find(e=>e.status!="rejected");
OR return resArr.find(e=>e.status=="fulfilled").
The other answers have the downside of having to wait for all the promises to resolve, whereas ideally .some would return as soon as any (N) promise(s) passes the predicate.
let anyNPromises = (promises, predicate = a => a, n = 1) => new Promise(async resolve => {
promises.forEach(async p => predicate(await p) && !--n && resolve(true));
await Promise.all(promises);
resolve(false);
});
let atLeast2NumbersGreaterThan5 = promises => anyNPromises(promises, a => a > 5, 2);
atLeast2NumbersGreaterThan5([
Promise.resolve(5),
Promise.resolve(3),
Promise.resolve(10),
Promise.resolve(11)]
).then(a => console.log('5, 3, 10, 11', a)); // true
atLeast2NumbersGreaterThan5([
Promise.resolve(5),
Promise.resolve(3),
Promise.resolve(10),
Promise.resolve(-43)]
).then(a => console.log('5, 3, 10, -43', a)); // false
atLeast2NumbersGreaterThan5([
Promise.resolve(5),
Promise.resolve(3),
new Promise(() => 'never resolved'),
Promise.resolve(10),
Promise.resolve(11)]
).then(a => console.log('5, 3, unresolved, 10, 11', a)); // true