I'm working on a function that uses Array.reduce, and I need to add an asynchronous API call inside the reduce function. This requires me to use an async function for the callback I pass into reduce, since I'm using await inside the function to wait for the asynchronous API call.
I'm having some trouble writing the reduce correctly. Here's how it currently is (working):
const result = records.reduce((array, currValue) => {
//do stuff
return array
}, [])
Here's what I tried to change it to:
const result = records.reduce(async(array, currentValue) => {
// do stuff
someValue = await asyncCall(currentValue)
array.add(someValue)
return array
}, [])
The error I'm getting is 'No overload matches this call'.
This seems to make sense to me, since reduce takes in a callback that returns an array, and async functions return a callback, not an array. But when I read other examples of how to pass async functions into .reduce, they all seem to just pass an async function into reduce with no problem.
Here are a few links I looked at:
https://advancedweb.hu/how-to-use-async-functions-with-array-reduce-in-javascript/
JavaScript array .reduce with async/await
https://gyandeeps.com/array-reduce-async-await/
The moment I declare the reduction function into async, I get the no matching overloads error, which makes sense to me. I'm not sure how this seems to work for other people.
First: reduce probably isn't the best tool to use for this. It looks like you're just adding entries to an array. reduce is overcomplicated for that task, particularly if you're doing something asynchronous. Instead, a looping construct that you can use in an async function is much, much simpler.
I'll start with reduce, then go to the looping construct.
reduce works synchronously. If you pass an async function in as its callback, the promise that function returns will be the accumulator value seen by the next callback. So if one of the steps in the reduce operation needs to be asynchronous and return a promise, every step after it has to be asynchronous returning a promise (for simplicity, it's best to just make every step return a promise); and the result of the reduce will be a promise for the eventual final value, not the final value itself. You can't make an asynchronous call synchronous, and you can't make a synchronous operation (reduce) wait for an asynchronous result.
So, all of your callbacks will be dealing with promises. It'll look a bit like this:
const result = await records.reduce(async(arrayPromise, currentValue) => {
// −−−−−−−−−−−−^^^^^−−−−−−−−−−−−−−−−−−−−−−^^^^^^^^^^^^
const array = await arrayPromise // <=====
// do stuff
someValue = await asyncCall(currentValue)
array.push(someValue) // <==== `push` rather than `add`, presumably
return array
}, Promise.resolve([]))
// ^^^^^^^^^^^^^^^^−−^
Of course, since that uses await, it has to be in an async function. Otherwise:
records.reduce(async(arrayPromise, currentValue) => {
const array = await arrayPromise // <=====
// do stuff
someValue = await asyncCall(currentValue)
array.push(someValue)
return array
}, Promise.resolve([]))
.then(result => {
// ...use `result` here
})
.catch(error => {
// ...handle/report error here...
})
You're better off with a looping construct that natively supports being part of an async function:
const result = []
for (const currentValue of records) {
someValue = await asyncCall(currentValue)
result.push(someValue)
}
// ...use `result` here...
or even
const result = []
for (const currentValue of records) {
result.push(await asyncCall(currentValue))
}
// ...use `result` here...
If you need to do this in a function that isn't an async function, you'll be dealing explicitly with a promise, which would look like:
(async () => {
const result = []
for (const currentValue of records) {
result.push(await asyncCall(currentValue))
}
return result
})()
.then(result => {
// ...use `result` here
})
.catch(error => {
// ...handle/report error here...
})
I think the simplest thing would be as following
const promises = records.reduce((array, currentValue) => {
// do stuff
someValue = asyncCall(currentValue)
array.add(someValue)
return array
}, [])
const results= Promise.all(promises);
If the use-case for your reduce function is more complicated, please post more code or create a sandbox
Related
How do I check that every element pass the test if a test is async and can throw an error
const exists = async () => {//can throw an error}
const allExists = [12, 5, 8, 130, 44].every(exists);
You can't use synchronous methods like every with functions that do asynchronous work, because the synchronous method won't wait for the asynchronous result. It's possible to write an async every, though:
async function every(array, callback) {
for (const element of array) {
const result = await callback(element);
if (!result) {
return false;
}
}
return true;
}
Note that because it relies on asynchronous information, it too delivers its result asynchronously. You have to await it (or use .then/.catch, etc.).
That version works in series, only calling the callback for the second entry when the callback for the first has finished its work. That lets you short-circuit (not call the callback for later entries when you already know the answer), but may make it take longer in overall time. Another approach is to do all of the calls in parallel, then check their results:
Promise.all(array.map(callback))
.then(flags => flags.every(Boolean))
.then(result => {
// ...`result` will be `true` or `false`...
});
I have a function that calls an API, and the API accepts a callback:
const callApi = async (param1, param2) => {
api.doSomething(param1, param2, (result) => {
// I want to return the result as the result of callApi
}
}
And I have a list/array of objects on which I want to call the Api function, treat the results one by one, and then sort them and pick the one that fits my criteria.
I do NOT want to shove all the logic in the callback.
I know I could, but it would result in ugly looking code that will be hard to read 1 year from now.
Something like:
let results = myArrayOfObjects.map(function(myObject){
callApi(myObject.field1, myObject.field2)
.then(
// here I'd like to get the result of api.doSomething
// do more stuff here with that result
)
// return an object here, or a promise
})
// do something with the results after all objects in myArrayOfObjects have been processed
// for example, sort()
If the results are promises, I am thinking of using Promise.all to wait for all of them to complete.
Otherwise, it would be even easier to work with the results.
Thank you.
You could first promisify your API so that it returns a Promise that resolves to your result provided by your callback. It might be worth checking your API implementation and seeing if it already returns a Promise (if that's the case, then you can just return that Promise and ditch the callback)
const callApi = (param1, param2) => {
return new Promise(resolve => {
api.doSomething(param1, param2, (result) => {
resolve(result);
});
});
}
Once you have callApi returning a Promise, you can map over your array and call callApi() to fire your API request, as well as to return the Promise. At this point you can extend your promise chain by attached a .then() to it and return a transformed version of your Promise:
const mapped = myArrayOfObjects.map(myObject => {
return callApi(myObject.field1, myObject.field2)
.then(result => // result of api.doSomething
// do more stuff here with that result
return result2; // return the transformed result (ie: `result2`)
);
});
The above piece of code can be re-written a little more nicely using async/await isntead:
const mapped = myArrayOfObjects.map(async myObject => {
const result = await callApi(myObject.field1, myObject.field2);
// do more stuff here with that result
return result2; // result2 represents the "mapped"/transformed version of `result`
});
Once you have an array of promises mapping to your desired value, you can use Promise.all() to wait for all Promises to resolve and retrieve your result:
Promise.all(mapped).then(results => {
// results is an array based on the mapped values
});
Simple Example (using setTimeout to simulate an asynchronous API request):
const callApi = (n) => {
return new Promise(resolve => {
setTimeout(() => {
const result = Math.random() + n;
resolve(result);
}, 1000);
});
}
const mapped = [1, 2, 3].map(async n => {
const result = await callApi(n);
const result2 = `Random number is: ${result}`;
return result2; // result2 represents the "mapped"/transformed version of `result`
});
Promise.all(mapped).then(results => {
console.log(results);
});
Instead of using .then(), create a function that does all of the stuff you want it to do and then send that function into callApi as a parameter.
const doStuff = (results) => {
// Do stuff here
}
callApi(myObject.field1, myObject.field2, doStuff)
And then in the callApi function:
const callApi = async (param1, param2, stuffDoerFunc) => {
api.doSomething(param1, param2, stuffDoerFunc);
}
Now that I think about it, you could probably even simplify it further:
const doStuff = (results) => {
// Do stuff here
}
api.doSomething(myObject.field1, myObject.field2, doStuff);
// Wrap your object/array processor in a standalone function
function processObjects() {
let results = myArrayOfObjects.map(function(myObject){
callApi(myObject.field1, myObject.field2)
.then(
// here I'd like to get the result of api.doSomething
// do more stuff here with that result
)
// return an object here, or a promise
})
}
// Wrap your post-processing in a second function
function processResults() {
// do something with the results after all objects in myArrayOfObjects have been processed
// for example, sort()
}
// Create a master function to call, which calls both functions in order,
// and does not let the second one start until the first one completes
async function processObjectsThenResults() {
await processObjects();
await processResults();
}
All this wizardry is unnecessary in a sequential functional language, but in the synchronous swamp of JS you need to force it to wait until your first group of commands finishes before starting the post-processing, otherwise it will try to just do it all at once and overlap and mayhem will ensue!
To modify this approach to pass results, you could just push them from one function to the other. A hacky way to do this would be passing the result of the first function into the second, like this:
async function processObjectsThenResults() {
someArrayVariable = await processObjects();
await processResults(someArrayVariable);
}
You would need to adjust the second function to be able to receive and interpret the params in the format that the first function outputs them to.
My sample code:
let name;
Login.findOne().then(() => {
name = 'sameer';
}); // consider this is async code
console.log(name);
So the above code works async, so now my console.log became undefined.
I used callbacks to make the code work like synchronous.
My callback code:
let name;
const callback = () => {
console.log(name);
};
Login.findOne().then(() => {
name = 'sameer';
callback();
});
Now its working perfectly,
My question is how you replace this small code with promises and async await instead of callbacks?
await lets you write asynchronous code in a somewhat synchronous fashion:
async function doIt() {
let name = await Login.findOne();
console.log(name);
// You can use the result here
// Or, if you return it, then it becomes the resolved value
// of the promise that this async tagged function returns
return name;
}
// so you can use `.then()` to get that resolved value here
doIt().then(name => {
// result here
}).catch(err => {
console.log(err);
});
The plain promises version would be this:
function doIt() {
// make the query, return the promise
return Login.findOne();
}
// so you can use `.then()` to get that resolved value here
doIt().then(name => {
// result here
}).catch(err => {
console.log(err);
});
Keep in mind that await can only be used inside an async function so sooner or later, you often still have to use .then() to see when everything is done. But, many times, using await can simplify sequential asynchronous operations.
It makes a lot more difference if you have multiple, sequential asynchronous operations:
async function doIt() {
let result1 = await someFunc1();
let result2 = await someFunc2(result1 + 10);
return someFunc3(result2 * 100);
}
Without await, this would be:
function doIt() {
return someFunc1().then(result1 => {
return someFunc2(result1 + 10);
}).then(result2 => {
return someFunc3(result2 * 100);
});
}
Add in more logic for processing the intermediate results or branching of the logic flow and it gets more and more complicated without await.
For more examples, see How to chain and share prior results with Promises and how much simpler the await version is.
Though you can use anonymous functions, I am just going to declare a function called printName like so, for clarity.
function printName(name) {
// if name provided in param, then print the name, otherwise print sameer.
console.log(name || 'sameer')
}
With promise, you can do:
Login.findOne().then(printName).catch(console.error)
With async/await. It has to be in a function declared async.
async function doLogin() {
try {
const name = await Login.findOne()
printName(name)
} catch(e) {
console.error(e)
}
}
So your current approach is already promise based, you can add the console.log directly under name = 'sameer'; and get the result you're looking for.
See here for an overview of promises prototype - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/then
Login.findOne().then(() => {
name = 'sameer';
console.log(name);
});
If you wanted to use async/await, you'd need to wrap this logic in a async function, but you could then use it like this:
async function someFunction() {
const resultFromFindOne = await Login.findOne();
name = 'sameer';
console.log(name)
}
how you replace this small code with promises and async await instead of callbacks
Here you go, 2 ways to do it:
/* Let's define a stand-in for your Login object so we can use Stack Snippets */
const Login = {
findOne: (name) => Promise.resolve(name) // dummy function
}
/* using promises */
Login.findOne('sameer')
.then(name => name.toUpperCase()) // the thing you return will be passed to the next 'then'. Let's uppercase just for fun
.then(name => console.log('**FROM PROMISE**', name))
/* using async/await */
async function main() { // (A) only inside async can you use await
const name = await Login.findOne('sameer') // as mentioned in (A)
console.log('**FROM AWAIT**', name)
}
main() // trigger our async/await test
Cheers,
I have a function with multiple forEach loops:
async insertKpbDocument(jsonFile) {
jsonFile.doc.annotations.forEach((annotation) => {
annotation.entities.forEach(async (entity) => {
await this.addVertex(entity);
});
annotation.relations.forEach(async (relation) => {
await this.addRelation(relation);
});
});
return jsonFile;
}
I need to make sure that the async code in the forEach loop calling the this.addVertex function is really done before executing the next one.
But when I log variables, It seems that the this.addRelation function is called before the first loop is really over.
So I tried adding await terms before every loops like so :
await jsonFile.doc.annotations.forEach(async (annotation) => {
await annotation.entities.forEach(async (entity) => {
await this.addVertex(entity);
});
await annotation.relations.forEach(async (relation) => {
await this.addRelation(relation);
});
});
But same behavior.
Maybe it is the log function that have a latency? Any ideas?
As we've discussed, await does not pause a .forEach() loop and does not make the 2nd item of the iteration wait for the first item to be processed. So, if you're really trying to do asynchronous sequencing of items, you can't really accomplish it with a .forEach() loop.
For this type of problem, async/await works really well with a plain for loop because they do pause the execution of the actual for statement to give you sequencing of asynchronous operations which it appears is what you want. Plus, it even works with nested for loops because they are all in the same function scope:
To show you how much simpler this can be using for/of and await, it could be done like this:
async insertKpbDocument(jsonFile) {
for (let annotation of jsonFile.doc.annotations) {
for (let entity of annotation.entities) {
await this.addVertex(entity);
}
for (let relation of annotation.relations) {
await this.addRelation(relation);
}
}
return jsonFile;
}
You get to write synchronous-like code that is actually sequencing asynchronous operations.
If you are really avoiding any for loop, and your real requirement is only that all calls to addVertex() come before any calls to addRelation(), then you can do this where you use .map() instead of .forEach() and you collect an array of promises that you then use Promise.all() to wait on the whole array of promises:
insertKpbDocument(jsonFile) {
return Promise.all(jsonFile.doc.annotations.map(async annotation => {
await Promise.all(annotation.entities.map(entity => this.addVertex(entity)));
await Promise.all(annotation.relations.map(relation => this.addRelation(relation)));
})).then(() => jsonFile);
}
To fully understand how this works, this runs all addVertex() calls in parallel for one annotation, waits for them all to finish, then runs all the addRelation() calls in parallel for one annotation, then waits for them all to finish. It runs all the annotations themselves in parallel. So, this isn't very much actual sequencing except within an annotation, but you accepted an answer that has this same sequencing and said it works so I show a little simpler version of this for completeness.
If you really need to sequence each individual addVertex() call so you don't call the next one until the previous one is done and you're still not going to use a for loop, then you can use the .reduce() promise pattern put into a helper function to manually sequence asynchronous access to an array:
// helper function to sequence asynchronous iteration of an array
// fn returns a promise and is passed an array item as an argument
function sequence(array, fn) {
return array.reduce((p, item) => {
return p.then(() => {
return fn(item);
});
}, Promise.resolve());
}
insertKpbDocument(jsonFile) {
return sequence(jsonFile.doc.annotations, async (annotation) => {
await sequence(annotation.entities, entity => this.addVertex(entity));
await sequence(annotation.relations, relation => this.addRelation(relation));
}).then(() => jsonFile);
}
This will completely sequence everything. It will do this type of order:
addVertex(annotation1)
addRelation(relation1);
addVertex(annotation2)
addRelation(relation2);
....
addVertex(annotationN);
addRelation(relationN);
where it waits for each operation to finish before going onto the next one.
foreach will return void so awaiting it will not do much. You can use map to return all the promises you create now in the forEach, and use Promise.all to await all:
async insertKpbDocument(jsonFile: { doc: { annotations: Array<{ entities: Array<{}>, relations: Array<{}> }> } }) {
await Promise.all(jsonFile.doc.annotations.map(async(annotation) => {
await Promise.all(annotation.entities.map(async (entity) => {
await this.addVertex(entity);
}));
await Promise.all(annotation.relations.map(async (relation) => {
await this.addRelation(relation);
}));
}));
return jsonFile;
}
I understand you can run all the addVertex concurrently. Combining reduce with map splitted into two different set of promises you can do it. My idea:
const first = jsonFile.doc.annotations.reduce((acc, annotation) => {
acc = acc.concat(annotation.entities.map(this.addVertex));
return acc;
}, []);
await Promise.all(first);
const second = jsonFile.doc.annotations.reduce((acc, annotation) => {
acc = acc.concat(annotation.relations.map(this.addRelation));
return acc;
}, []);
await Promise.all(second);
You have more loops, but it does what you need I think
forEach executes the callback against each element in the array and does not wait for anything. Using await is basically sugar for writing promise.then() and nesting everything that follows in the then() callback. But forEach doesn't return a promise, so await arr.forEach() is meaningless. The only reason it isn't a compile error is because the async/await spec says you can await anything, and if it isn't a promise you just get its value... forEach just gives you void.
If you want something to happen in sequence you can await in a for loop:
for (let i = 0; i < jsonFile.doc.annotations.length; i++) {
const annotation = jsonFile.doc.annotations[i];
for (let j = 0; j < annotation.entities.length; j++) {
const entity = annotation.entities[j];
await this.addVertex(entity);
});
// code here executes after all vertix have been added in order
Edit: While typing this a couple other answers and comments happened... you don't want to use a for loop, you can use Promise.all but there's still maybe some confusion, so I'll leave the above explanation in case it helps.
async/await does not within forEach.
A simple solution: Replace .forEach() with for(.. of ..) instead.
Details in this similar question.
If no-iterator linting rule is enabled, you will get a linting warning/error for using for(.. of ..). There are lots of discussion/opinions on this topic.
IMHO, this is a scenario where we can suppress the warning with eslint-disable-next-line or for the method/class.
Example:
const insertKpbDocument = async (jsonFile) => {
// eslint-disable-next-line no-iterator
for (let entity of annotation.entities) {
await this.addVertex(entity)
}
// eslint-disable-next-line no-iterator
for (let relation of annotation.relations) {
await this.addRelation(relation)
}
return jsonFile
}
The code is very readable and works as expected. To get similar functionality with .forEach(), we need some promises/observables acrobatics that i think is a waste of effort.
I'm getting compile time error in this code:
const someFunction = async (myArray) => {
return myArray.map(myValue => {
return {
id: "my_id",
myValue: await service.getByValue(myValue);
}
});
};
Error message is:
await is a reserved word
Why can't I use it like this?
You can't do this as you imagine, because you can't use await if it is not directly inside an async function.
The sensible thing to do here would be to make the function passed to map asynchronous. This means that map would return an array of promises. We can then use Promise.all to get the result when all the promises return. As Promise.all itself returns a promise, the outer function does not need to be async.
const someFunction = (myArray) => {
const promises = myArray.map(async (myValue) => {
return {
id: "my_id",
myValue: await service.getByValue(myValue)
}
});
return Promise.all(promises);
}
If you want to run map with an asynchronous mapping function you can use the following code:
const resultArray = await Promise.all(inputArray.map(async (i) => someAsyncFunction(i)));
How it works:
inputArray.map(async ...) returns an array of promises - one for each value in inputArray.
Putting Promise.all() around the array of promises converts it into a single promise.
The single promise from Promise.all() returns an array of values - the individual promises each resolve to one value.
We put await in front of Promise.all() so that we wait for the combined promise to resolve and store the array of resolved sub-promises into the variable resultArray.
In the end we get one output value in resultArray for each item in inputArray, mapped through the function someAsyncFunction. We have to wait for all async functions to resolve before the result is available.
That's because the function in map isn't async, so you can't have await in it's return statement. It compiles with this modification:
const someFunction = async (myArray) => {
return myArray.map(async (myValue) => { // <-- note the `async` on this line
return {
id: "my_id",
myValue: await service.getByValue(myValue)
}
});
};
Try it out in Babel REPL
So… it's not possible to give recommendation without seeing the rest of your app, but depending on what are you trying to do, either make the inner function asynchronous or try to come up with some different architecture for this block.
Update: we might get top-level await one day: https://github.com/MylesBorins/proposal-top-level-await
it will be 2 instructions, but just shift "await" with the extra instruction
let results = array.map((e) => fetch('....'))
results = await Promise.all(results)
I tried all these answers but no one works for my case because all answers return a promise object not the result of the promise like this:
{
[[Prototype]]: Promise
[[PromiseState]]: "fulfilled"
[[PromiseResult]]: Array(3)
0: ...an object data here...
1: ...an object data here...
2: ...an object data here...
length: 3
[[Prototype]]: Array(0)
}
Then I found this answer https://stackoverflow.com/a/64978715/8339172 that states if map function is not async or promise aware.
So instead of using await inside map function, I use for loop and await the individual item because he said that for loop is async aware and will pause the loop.
When you want each remapped value resolved before moving on to the next, you can process the array as an asynchronous iterable.
Below, we use library iter-ops, to remap each value into promise, and then produce an object with resolved value, because map itself shouldn't be handling any promises internally.
import {pipe, map, wait, toAsync} from 'iter-ops';
const i = pipe(
toAsync(myArray), // make asynchronous
map(myValue => {
return service.getByValue(myValue).then(a => ({id: 'my_id', myValue: a}))
}),
wait() // wait for each promise
);
(async function() {
for await (const a of i) {
console.log(a); // print resulting objects
}
})
After each value, we use wait to resolve each remapped value as it is generated, to keep resolution requirement consistent with the original question.