This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed 11 months ago.
I have a patch endpoint that takes a body of fields to update, iterates over each field, and updates the correct value. I want to do something send a response once all those things are done, but I do not know how to await a loop of operations. If it was a singular operation I could just add .then(), but that would not work in this case. What is an elegant solution to this?
Code:
const updateUser = (req, res) => {
const db = mongoConnection.getDb();
for (key in req.body) {
db.collection('users').updateOne(
{username: req.query.username},
{$set: {[key]: req.body[key]}}
)
}
// send response once for loop is done
}
You could mark the outer function async and await each DB update inside the loop. Then you'll know that after the loop completes, all DB updates are done.
An even better way is to run the updates in parallel since they do not depend on each other. You can use Promise.allSettled() which takes an array of promises and resolves when the last one is finished.
const updateUser = async (req, res) => {
const db = mongoConnection.getDb();
const dbUpdates = Object.entries(req.body).map((key, value) => {
return db.collection('users').updateOne(
{ username: req.query.username },
{ $set: { [key]: value }}
);
});
await Promise.allSettled(dbUpdates);
// Send response, at this point all DB updates are done
};
I think an easy answer would be hiding the for loop in an async function, then use it in the endpoint.
const updateUserField = async () =>
db.collection("users").updateOne(
{
username: req.query.username,
},
{
$set: { [key]: req.body[key] },
}
);
const updateUserEndpoint = (req, res) => {
for (let field of req.body) {
await updateUserField(field);
}
res.status(200).send('updated');
}
Then, for testing, you could wrap the endpoint with a function that injects the function itself:
const updateUserEndpoint = (userUpdateFunc) => userUpdateFunc(req, res);
This pattern is called dependency injection - when writing unit tests for the endpoint, you'd just replace userUpdateFunc with whatever mocked function you want to use. This removes the need to monkeypatch the function in the test.
Related
I followed the youtube Node & Express project tutorial and Im confused facing these code:
This is in async js file:
const asyncHWrapper = (fn) => {
return async (req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
};
module.exports = asyncHWrapper;
And this is the usage:
const Task = require("../models/taskModel");
const asyncWrapper = require("../middleware/async");
const { createCustomError } = require("../errors/customErrors");
const getAllTasks = asyncWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
});
Im just confused about these questions:
Does is necessary to return an arrow function in the asyncWrapper? Why doesn't just call the function?
Where do the params (req,res) in the asyncWrapper function come from? In the "fn" function declaration?
Why should I write two pairs of async and await in the wrapper and when I call it?
Thanks a lot!
The youtube tutorial link :Node.js / Express Course - Build 4 Projects
Let me analyze the code first, and then I'll answer your question.
So here is your wrapper
const asyncHWrapper = (fn) => {
return async (req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
};
and here is how it is used
const getAllTasks = asyncWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
});
the asyncWrapper accept an fn param, in this case, it is this function:
async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
}
after asyncHWrapper is called with the above function, it will return another function, in this case, the return function is assigned as the name getAllTasks.
Now for your question:
Does is necessary to return an arrow function in the asyncWrapper? Why doesn't just call the function?
Well basically you can write it like this
const asyncHWrapper = async (fn, req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
And call it like this
await asyncHWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
}, req, res, next)
But in this case, it's just a normal function with callback, it's not a wrapper, and its name shouldn't be asyncHWrapper.
Where do the params (req,res) in the asyncWrapper function come from? In the "fn" function declaration?
No, it comes from getAllTasks, your fn is just consuming two values (req, res), and the next param will be used for error handling. So when you call getAllTask, you must pass in three params like this getAllTasks(req, res, next)
Why should I write two pairs of async and await in the wrapper and when I call it?
I'm not sure what you meant by "two pairs of async and await". I assume you're referring to await when calling getAllTasks and await when calling fn?
That's just because both of them are async functions.
I hope that this answer can help you think about the concept of a "wrapper". Let's say we have a function divide:
const divide = (a,b) => a/b;
You can use this in normal code quite easily:
x = divide(10,5); // sets x to 2.
But you may decide that you care about the possibility of errors. In this case, division by zero is a possibility. You could certainly include some error handling code where you define the divide function. But you could also choose to "wrap" divide with an error handler. This way, we can keep the error handling aspects away from the main division logic. We would like to be able to define a safeDivide function like this:
const safeDivide = catchErrors(divide);
In the same way that divide is a function that takes two arguments, safeDivide also has to be a function that takes two arguments. So the catchErrors wrapper will have to return a function. We will start with something like this:
const catchErrors = (fn) => {
return (p,q) => fn(p,q);
}
If you pass a function fn to catchErrors, it will return a function. That returned function takes two arguments p and q, and returns fn(p,q). So far, this doesn't really achieve anything (except limiting the number of arguments). But now, we can add a try/catch block. I'll do it in a couple of steps, because the notation can be confusing.
The first step is to put an explicit return inside the inner arrow function.
const catchErrors = (fn) => {
return (p,q) => {
return fn(p,q);
}
}
This is technically the same code - it just looks slightly different. Now we add the try/catch.
const catchErrors = (fn) => {
return (p,q) => {
try {
return fn(p,q);
} catch (e) {
console.log("Error occurred. Continuing with null result.");
return null;
}
}
}
So now, the function returned from catchErrors will do the same as the original function, except when an exception is thrown, in which case it will return null. (I'm not saying that this is the best general way to handle exceptions, but it's useful as an example, and it's related to the original question.)
So now look again at where we use this catchErrors wrapper function.
const safeDivide = catchErrors(divide);
When you call the wrapper catchErrors with function divide, it doesn't actually do any dividing - it doesn't yet have any numbers to divide. Instead, it builds up a new function that, whenever it is called, would do the dividing, and catch any exception that arises. I hope that answers your first question.
Your second question is where req and res come from. They are names given to arguments that will be passed to the function. They can be passed to the wrapped function (along with a 3rd argument next), and they will also be passed to the inner (nameless) function which includes calls to Task.find and res.status(200). Those arguments will be provided by the Express (or other) web framework.
I will leave your 3rd question, on the async/await aspects of the wrapper, for another answer.
Is there any way I can make the below code run synchronously in a way where I can get all of the productLine ids and then loop through and delete all of them, then once all of this is complete, get all of the productIds and then loop through and delete all of them?
I really want to be able to delete each set of items in batch, but the next section can't run until the first section is complete or there will be referential integrity issues.
// Delete Product Lines
axios.get('https://myapi.com/ProductLine?select=id')
.then(function (response) {
const ids = response.data.value
ids.forEach(id => {
axios.delete('https://myapi.com/ProductLine/' + id)
})
})
.catch(function (error) {
})
// Delete Products (I want to ensure this runs after the above code)
axios.get('https://myapi.com/Product?select=id')
.then(function (response) {
const ids = response.data.value
ids.forEach(id => {
axios.delete('https://myapi.com/Product/' + id)
})
})
.catch(function (error) {
})
There's a lot of duplication in your code. To reduce code duplication, you can create a helper function that can be called with appropriate arguments and this helper function will contain code to delete product lines and products.
async function deleteHelper(getURL, deleteURL) {
const response = await axios.get(getURL);
const ids = response.data.value;
return Promise.all(ids.map(id => (
axios.delete(deleteURL + id)
)));
}
With this helper function, now your code will be simplified and will be without code duplication.
Now to achieve the desired result, you could use one of the following ways:
Instead of two separate promise chains, use only one promise chain that deletes product lines and then deletes products.
const prodLineGetURL = 'https://myapi.com/ProductLine?select=id';
const prodLineDeleteURL = 'https://myapi.com/ProductLine/';
deleteHelper(prodLineGetURL, prodLineDeleteURL)
.then(function() {
const prodGetURL = 'https://myapi.com/Product?select=id';
const prodDeleteURL = 'https://myapi.com/Product/';
deleteHelper(prodGetURL, prodDeleteURL);
})
.catch(function (error) {
// handle error
});
Use async-await syntax.
async function delete() {
try {
const urls = [
[ prodLineGetURL, prodLineDeleteURL ],
[ prodGetURL, prodDeleteURL ]
];
for (const [getURL, deleteURL] of urls) {
await deleteHelper(getURL, deleteURL);
}
} catch (error) {
// handle error
}
}
One other thing that you could improve in your code is to use Promise.all instead of forEach() method to make delete requests, above code uses Promise.all inside deleteHelper function.
Your code (and all other answers) are executing delete requests sequentially, which is huge waste of time. You should use Promise.all() and execute in parallel...
// Delete Product Lines
axios.get('https://myapi.com/ProductLine?select=id')
.then(function (response) {
const ids = response.data.value
// execute all delete requests in parallel
Promise.all(
ids.map(id => axios.delete('https://myapi.com/ProductLine/' + id))
).then(
// all delete request are finished
);
})
.catch(function (error) {
})
All HTTP request are asynchronous but you can make it sync-like. How? Using async-await
Suppose you have a function called retrieveProducts, you need to make that function async and then await for the response to keep processing.
Leaving it to:
const retrieveProducts = async () => {
// Delete Product Lines
const response = await axios.get('https://myapi.com/ProductLine?select=id')
const ids = response.data.value
ids.forEach(id => {
axios.delete('https://myapi.com/ProductLine/' + id)
})
// Delete Products (I want to ensure this runs after the above code)
const otherResponse = await axios.get('https://myapi.com/Product?select=id') // use proper var name
const otherIds = response.data.value //same here
otherIds.forEach(id => {
axios.delete('https://myapi.com/Product/' + id)
})
}
But just keep in mind that it's not synchronous, it keeps being async
i was reading about how to solve this problem, and everyone says that i should use "async" and "await", but i dont know how to put it propely on my code. (I'm making a new JSON, and i should send it to the front-end, but i need to wait to the JSON get ready inside the forEach loop first, then render on the screen).
router.get('/', (req, res) => {
Controle.find().lean().then( (controles) => {
var controles_cadastrados = []
controles.forEach((controle, index, array)=>{
Sensor.find({id_mac: controle.sensor}).lean().then((sensor)=>{
controle.sensor_nome = sensor[0].nome
Atuador.find({id_mac: controle.atuador}).lean().then((atuador)=>{
controle.atuador_nome = atuador[0].nome
controles_cadastrados.push(controle)
console.log(controles_cadastrados)
})
})
})
//wait to send the response
res.render('controle/controles', {controles: controles_cadastrados })
}).catch((erro) => {
console.log('Erro ao carregar controles')
})
})
I have try it in so many ways, but none seens to working good.
Sorry if i made some mistake.
You aren't properly waiting for all your asynchronous operations to be done before calling res.render() thus your array is empty or partially populated when you try to use it. So, you need to use the promises to be able to track when everything is done.
You have a couple choices. You can run all your request in parallel or in sequence. I'll show an example of both:
Here's processing all the database requests in parallel. This chains the various promises and use Promise.all() to know when they are all done. Results in the controles_cadastrados are not in any particular order, but this will probably run quicker than processing everything sequentially.
router.get('/', (req, res) => {
Controle.find()
.lean()
.then(controles => {
const controles_cadastrados = [];
Promise.all(controles.map(controle => {
return Sensor.find({ id_mac: controle.sensor })
.lean()
.then(sensor => {
controle.sensor_nome = sensor[0].nome;
return Atuador.find({ id_mac: controle.atuador })
.lean()
.then(atuador => {
controle.atuador_nome = atuador[0].nome;
controles_cadastrados.push(controle);
console.log(controles_cadastrados);
});
});
})).then(() => {
//wait to send the response
res.render('controle/controles', {
controles: controles_cadastrados,
});
});
}).catch(erro => {
console.log('Erro ao carregar controles');
res.sendStatus(500);
});
});
And, here's how you would sequence the operations using async/await:
router.get('/', async (req, res) => {
try {
let controles = await Controle.find().lean();
const controles_cadastrados = [];
for (let controle of controles) {
let sensor = await Sensor.find({ id_mac: controle.sensor }).lean();
controle.sensor_nome = sensor[0].nome;
let atuador = await Atuador.find({ id_mac: controle.atuador }).lean()
controle.atuador_nome = atuador[0].nome;
controles_cadastrados.push(controle);
console.log(controles_cadastrados);
}
//wait to send the response
res.render('controle/controles', {
controles: controles_cadastrados,
});
} catch(e) {
console.log(e, 'Erro ao carregar controles');
res.sendStatus(500);
}
});
Also, note that all possible rejected promises or other errors are captured here and a response is always sent to the incoming http request, even when there's an error.
I've got a https onRequest cloud function that writes a doc. That doc triggers another cloud function, which adds a field to the doc based on a calculation.
Any suggestions how to make the res of the https onRequest function to wait until that field is added and to respond to the https with the value of that field?
exports = module.exports = functions.https.onRequest(async (req, res) => {
const user = await admin.firestore().collection("users").doc();
const enquiryId = await admin.firestore().collection("users").doc(user.id).collection("enquiries").doc();
await enquiryId.set({
value: req.query.value;
});
let getQ = functions.firestore.document('users/{user.id}/enquiries/{enquiryId.id}').onUpdate((change, context) => {
const newValue = change.after.data();
q = newValue.quote;
return q
});
getQ;
return res.status(200).send(getQ);
});
Update following your comments below:
Since the "cloud function gets triggered when we write to the firestore from mobile or web" you could set up a listener to the users/{user.id}/enquiries/{enquiryId.id} document.
You just have to send back the values of user.id and enquiryId.id as response to the HTTP Cloud Function, as follows:
await enquiryDocRef.set(data);
return res.status(200).send({userDocID : user.id, enquiryDocId: enquiryId.id});
and you use these two values to set the listener. The doc is here: https://firebase.google.com/docs/firestore/query-data/listen
This way, when reaching step 4 described in your comment, i.e. when "based on the calculation, a field in the (users/{user.id}/enquiries/{enquiryId.id}) doc gets updated", your listener will be triggered and you will get this updated field value in your front-end.
Initial answer:
If you want the HTTP Cloud Function to "wait until that field is added and to respond to the https with the value of that field", you just have to wait that the asynchronous set() operation is completed before sending back the response.
You can detect that the operation is completed when the Promise returned by the set() method resolves.
Since you do await enquiryId.set({...}) you are therefore very close to the solution: await will make the Cloud Function wait until that Promise settles (is not pending anymore), so you just have to return the response right after this line, as follows:
Therefore just do as follows:
exports = module.exports = functions.https.onRequest(async (req, res) => {
try {
const userDocRef = await admin.firestore().collection("users").doc();
const enquiryDocRef = await admin.firestore().collection("users").doc(userDocRef.id).collection("enquiries").doc();
const data = {
value: req.query.value
};
await enquiryDocRef.set(data);
return res.status(200).send(data);
} catch (error) {
return res.status(400).send(error);
}
});
I am new to await/async in Jquery. When I am trying to use it in my js file, It is getting executed, but not as expected.
async updateProduct (product) {
product.travelers.forEach((traveler, travelerOrder) => this.updateProductwithPassengerName(traveler) )
return product
}
async updateProductwithPassengerName (traveler) {
const travelerId = traveler.profile_id;
var body = await this.userServiceClient.request('/v2/users/' + travelerId + '/info','GET')
traveler.first_name = body.first_name
traveler.last_name = body.last_name
return traveler
}
async request (path, method, body) {
const options = {
method: method,
body: body,
headers: this.headers(),
credentials: 'same-origin'
}
const response = await fetch(this.relativeUrl(path), options)
if (!response.ok) {
throw new HttpError('Technical error occured', response.status)
}
return response.json().catch(function () {
throw new HttpError('No results found', response.status)
})
}
Those are the 3 functions. What is happening now is that
traveler.first_name = body.first_name
traveler.last_name = body.last_name
these are not setting in synchronous manner( after
var body = await this.userServiceClient.request('/v2/users/' +
travelerId + '/info','GET')
. These are executing after a quiet a long time.
What I am doing here is that, for each traveler I am setting first_name and last_name. And updating the product object. Since this setting of values is happening after a long time, product object is getting updated later, by the time UI page renders. I want this setting values to happening before doing anything else in jquery.
Looking for help
The problem is that forEach will not await the asynchronous result and so your first function returns a promise that immediately resolves, without waiting for the requests to finish.
Here is how to correct:
async updateProduct (product) {
await Promise.all(product.travelers.map((traveler, travelerOrder) => this.updateProductwithPassengerName(traveler) ));
return product
}
Or, if the backbone cannot deal with all these concurrent requests, then wait for each of them to complete before issuing the next:
async updateProduct (product) {
for (let traveler of product.travelers) {
await this.updateProductwithPassengerName(traveler);
}
return product
}
when using asyn/await in JQuery or angularjs or nodejs, use promise. Promise will complete the process and then return either success or fail. Then only execute next steps.Go through below links and you will get idea.
Modern Asynchronous JavaScript with Async and Await
async & await in Javascript