I want to get the entry at index gameNumber in the history collection. From that entry I want to take team1Player1 and find that player in the rating collection and then set his games to an arbitary number. This code works sometimes but idk what makes it not work
app.post("/delete", function (req, res) {
var gameNumber = req.body.test;
History.find({}, async function (err, historyRecords) {
if (err) {
console.log(err);
return;
}
await Rating.updateOne({name: historyRecords.reverse()[gameNumber-1].team1Player1}, { games: 9999});
})
// res.redirect("/delete");
});
Before answering your question, I want to note a few issues with your code.
You are mixing callback style functions with async await (promise based). You are using await on Rating, why not do it on History as well?
Your are using an expressive db which allows you to do complex queries, use it to your advantage.
Due to your passing a callback, but trying to render /delete at the end, the /delete will execute before your callback is invoked due to the callbacks async nature. In order for your code to complete, you either go full callback style on everything, or promise based, or of course, async await, which is syntactic sugar on top of promises.
Lets query the db for the lists at index: index, take note that I used limit. In your example you take all of them into memory, both wasting resources + network bandwidth, always limit your queries!
Here is an example with async await:
app.post("/delete", async function (req, res, next) {
var gameNumber = req.body.test
const [ gameRetrieved ] = await History.find({ gameNumber }).limit(1)
await Rating.updateOne({ name: gameRetrieved.team1Player1 }, { games: 9999 })
});
Here is an example with callbacks:
app.post("/delete", async function (req, res, next) {
var gameNumber = req.body.test
return History.find({ gameNumber }).limit(1).exec((err, [ gameRetrieved ]) => {
return Rating.updateOne({ name: gameRetrieved.team1Player1 }, { games: 9999 }).exec((err, rating)=>{
res.render('/delete')
})
})
});
I did not account for error handling as that is not the scope here.
To sum things up: a callback/promise is async, doing anything outside of its scope will execute before/after it depending on the situation, which in your case sometimes finished before rendering delete and sometimes did not.
Related
I followed the youtube Node & Express project tutorial and Im confused facing these code:
This is in async js file:
const asyncHWrapper = (fn) => {
return async (req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
};
module.exports = asyncHWrapper;
And this is the usage:
const Task = require("../models/taskModel");
const asyncWrapper = require("../middleware/async");
const { createCustomError } = require("../errors/customErrors");
const getAllTasks = asyncWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
});
Im just confused about these questions:
Does is necessary to return an arrow function in the asyncWrapper? Why doesn't just call the function?
Where do the params (req,res) in the asyncWrapper function come from? In the "fn" function declaration?
Why should I write two pairs of async and await in the wrapper and when I call it?
Thanks a lot!
The youtube tutorial link :Node.js / Express Course - Build 4 Projects
Let me analyze the code first, and then I'll answer your question.
So here is your wrapper
const asyncHWrapper = (fn) => {
return async (req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
};
and here is how it is used
const getAllTasks = asyncWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
});
the asyncWrapper accept an fn param, in this case, it is this function:
async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
}
after asyncHWrapper is called with the above function, it will return another function, in this case, the return function is assigned as the name getAllTasks.
Now for your question:
Does is necessary to return an arrow function in the asyncWrapper? Why doesn't just call the function?
Well basically you can write it like this
const asyncHWrapper = async (fn, req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
next(error);
}
};
And call it like this
await asyncHWrapper(async (req, res) => {
const tasks = await Task.find({});
res.status(200).json({ tasks });
}, req, res, next)
But in this case, it's just a normal function with callback, it's not a wrapper, and its name shouldn't be asyncHWrapper.
Where do the params (req,res) in the asyncWrapper function come from? In the "fn" function declaration?
No, it comes from getAllTasks, your fn is just consuming two values (req, res), and the next param will be used for error handling. So when you call getAllTask, you must pass in three params like this getAllTasks(req, res, next)
Why should I write two pairs of async and await in the wrapper and when I call it?
I'm not sure what you meant by "two pairs of async and await". I assume you're referring to await when calling getAllTasks and await when calling fn?
That's just because both of them are async functions.
I hope that this answer can help you think about the concept of a "wrapper". Let's say we have a function divide:
const divide = (a,b) => a/b;
You can use this in normal code quite easily:
x = divide(10,5); // sets x to 2.
But you may decide that you care about the possibility of errors. In this case, division by zero is a possibility. You could certainly include some error handling code where you define the divide function. But you could also choose to "wrap" divide with an error handler. This way, we can keep the error handling aspects away from the main division logic. We would like to be able to define a safeDivide function like this:
const safeDivide = catchErrors(divide);
In the same way that divide is a function that takes two arguments, safeDivide also has to be a function that takes two arguments. So the catchErrors wrapper will have to return a function. We will start with something like this:
const catchErrors = (fn) => {
return (p,q) => fn(p,q);
}
If you pass a function fn to catchErrors, it will return a function. That returned function takes two arguments p and q, and returns fn(p,q). So far, this doesn't really achieve anything (except limiting the number of arguments). But now, we can add a try/catch block. I'll do it in a couple of steps, because the notation can be confusing.
The first step is to put an explicit return inside the inner arrow function.
const catchErrors = (fn) => {
return (p,q) => {
return fn(p,q);
}
}
This is technically the same code - it just looks slightly different. Now we add the try/catch.
const catchErrors = (fn) => {
return (p,q) => {
try {
return fn(p,q);
} catch (e) {
console.log("Error occurred. Continuing with null result.");
return null;
}
}
}
So now, the function returned from catchErrors will do the same as the original function, except when an exception is thrown, in which case it will return null. (I'm not saying that this is the best general way to handle exceptions, but it's useful as an example, and it's related to the original question.)
So now look again at where we use this catchErrors wrapper function.
const safeDivide = catchErrors(divide);
When you call the wrapper catchErrors with function divide, it doesn't actually do any dividing - it doesn't yet have any numbers to divide. Instead, it builds up a new function that, whenever it is called, would do the dividing, and catch any exception that arises. I hope that answers your first question.
Your second question is where req and res come from. They are names given to arguments that will be passed to the function. They can be passed to the wrapped function (along with a 3rd argument next), and they will also be passed to the inner (nameless) function which includes calls to Task.find and res.status(200). Those arguments will be provided by the Express (or other) web framework.
I will leave your 3rd question, on the async/await aspects of the wrapper, for another answer.
This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed 11 months ago.
I have a patch endpoint that takes a body of fields to update, iterates over each field, and updates the correct value. I want to do something send a response once all those things are done, but I do not know how to await a loop of operations. If it was a singular operation I could just add .then(), but that would not work in this case. What is an elegant solution to this?
Code:
const updateUser = (req, res) => {
const db = mongoConnection.getDb();
for (key in req.body) {
db.collection('users').updateOne(
{username: req.query.username},
{$set: {[key]: req.body[key]}}
)
}
// send response once for loop is done
}
You could mark the outer function async and await each DB update inside the loop. Then you'll know that after the loop completes, all DB updates are done.
An even better way is to run the updates in parallel since they do not depend on each other. You can use Promise.allSettled() which takes an array of promises and resolves when the last one is finished.
const updateUser = async (req, res) => {
const db = mongoConnection.getDb();
const dbUpdates = Object.entries(req.body).map((key, value) => {
return db.collection('users').updateOne(
{ username: req.query.username },
{ $set: { [key]: value }}
);
});
await Promise.allSettled(dbUpdates);
// Send response, at this point all DB updates are done
};
I think an easy answer would be hiding the for loop in an async function, then use it in the endpoint.
const updateUserField = async () =>
db.collection("users").updateOne(
{
username: req.query.username,
},
{
$set: { [key]: req.body[key] },
}
);
const updateUserEndpoint = (req, res) => {
for (let field of req.body) {
await updateUserField(field);
}
res.status(200).send('updated');
}
Then, for testing, you could wrap the endpoint with a function that injects the function itself:
const updateUserEndpoint = (userUpdateFunc) => userUpdateFunc(req, res);
This pattern is called dependency injection - when writing unit tests for the endpoint, you'd just replace userUpdateFunc with whatever mocked function you want to use. This removes the need to monkeypatch the function in the test.
I have a async function like this:
exports.myFunction = async (req, res, next) => {
if (some condition) {
next()
}
try {
const results = await axios.get(`https://a-domain.com/url/path`);
const info = results.data;
const docRef = await addDoc(collection(db, "name_of_collec"), info);
console.log("Document written with ID: ", docRef.id);
} catch (e) {
console.error("Error adding document: ", e);
}
next();
};
Here, response from the first request is required in the second request. But according to my knowledge since I am making asynchronous requests, the two requests should be made at the same time.
What I cannot understand:
So, how it is possible that there is no error occurring?
How is it that, sometimes the no document is added to Firestore? Like it is skipped somehow...
What modification can be done so that await addDoc(collection(db, "name_of_collec") executes compulsorily?
Context info:
I am using Express.js and Node.js to make a backend
I am using Axios to make the first request and the second one is adding a document to Firestore.
Firestore is a NoSQL database inside Google's Firebase.
But according to my knowledge since I am making asynchronous requests,
the two requests should be made at the same time.
asynchronous doesn't mean that the requests are made at the same time. It means the code won't wait for the responses from these requests. Requests are made in the manner you add them in the code and responses will be asynchronous i.e the response will be received at some point in future time.
And there is no guarantee in what manner they will be received.
However, when you sue async/await, the code execution waits at await keyword for the response based on its success or failure it calls the next flow. It is nothing but syntactic sugar on promises. Under the hood, this is how your code will be executed:
exports.myFunction = async (req, res, next) => {
if (some condition) {
next()l
}
axios.get(`https://a-domain.com/url/path`).then(result => result).then(data => {
const info = data;
addDoc(collection(db, "name_of_collec"), info).then(response => {
console.log("Document written with ID: ", docRef.id);
}).catch(e => {
console.error(e);
})
}).catch(e => {
console.error(e);
})
next();
};
So, how it is possible that there is no error occurring?
It is not necessary that the error occurs when you request. If that is the case catch handler should be activated.
How is it that, sometimes the no document is added to Firestore? Like
it is skipped somehow...
The reason might be info is an empty object and firestore automatically removes the empty document.
There is no issue, but there is not data info that is empty sometimes that skips the addition(it does request firestore, but firestore removes the empty docs)
exports.myFunction = async (req, res, next) => {
if (some condition) {
next()
}
try {
const results = await axios.get(`https://a-domain.com/url/path`);
const info = results.data;
const is_Info_Data = Array.isArray(info) ? info?.length >= 1: Object.keys(info) !== 0
if (is_Info_Data) {
const docRef = await addDoc(collection(db, "name_of_collec"), info);
console.log("Document written with ID: ", docRef.id);
} else {
console.log(`Info is empty ${info} no addition to DB`);
}
next();
} catch (e) {
console.error("Error adding document: ", e);
next(e);
}
};
I am currently stuck in asynchronous hell.
In my React, I have a page /menu, that would load data from my mongo instance via expressjs api.
In my database, called menu, i have collections which represent a meal-type eg "breakfast", "lunch" etc. In those collections, the documents for every item looks like this bread collection example:
{
_id: 2398jcs9dn2f9f,
name: "Ciabatta",
desc: "Italian bread",
imageURI: "image01.jpg",
reviews: []
}
This is my api that would be called when the page loads
exports.getAllFoods = (req, res, next) => {
const db = mongoose.connection
const allCollections = {}
try {
db.db.listCollections().toArray((err, collections) => {
collections.forEach((k) => {
allCollections[k.name] = []
})
Object.keys(allCollections).map(k => {
let Meal = mongoose.model(k, MealSchema)
meal = Meal.find((err, docs) => {
allCollections[k] = docs
console.log(allCollections)
})
})
res.send(allCollections)
})
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
The last output of the console.log(allCollections) produces this:
{ snacks:
[ { review: [],
tags: [],
_id: 5fcec3fc4bc5d81917c9c1fe,
name: 'Simosa',
description: 'Indian food',
imageURI: 'image02.jpg',
__v: 0 } ],
breads:
[ { review: [],
tags: [],
_id: 5fcec41a4bc5d81917c9c1ff,
name: 'Ciabatta',
description: 'Italian bread',
imageURI: 'image02.jpg',
__v: 0 } ],
}
This is exactly what I need, but I am stuck in figuring out how to send to React. What am I to do to send the above json? The res.send(allCollections) gives me this:
{
"snacks": [],
"breads": [],
"drinks": []
}
I understand why the above is being sent, but I dont know what I need to do to address it.
This is my React on page load
useEffect(() => {
axios
.get('http://localhost:8888/api/allFoods')
.then((res) => {
setMealTypes(res.data)
})
.catch((err) => [
console.log(err)
])
}, [])
Ultimately, I need the json outputted in console as I wanted to loop through that data and use the key as a title, and then list the values from the value array eg
<div>
<h2>Breads</h2>
<img src=image01.jpg/>
<h3>Ciabatta</h3>
<p>Italian bread</p>
...
</div>
...
I'd appreciate any help, and any docs I should read to help and improve my javascript understandings
I'd prefer to solve this using async/await and Promise.all, replacing most callbacks.
Because you're calling the DB when you're iterating through an array, you have the most annoying callback situation: how do you issue a bunch of async things and then get the results after? You'll need something else to ensure all callbacks are called before sending the results.
Async/await means we can declare a function is async, and await the results of an async operation. async/await is annoying in JS because it abstracts away callbacks and is actually creating a Promise underneath. Complicating things further, async/await doesn't solve issuing multiple async functions, so again we have to rely on this fancy Promise.all() function combined with map-ing the desired input array to async functions.
Original:
Object.keys(allCollections).map(k => {
let Meal = mongoose.model(k, MealSchema)
meal = Meal.find((err, docs) => {
allCollections[k] = docs
console.log(allCollections)
})
});
Suggested async/await:
await Promise.all(Object.keys(allCollections).map(async k => {
let Meal = mongoose.model(k, MealSchema)
let docs = await Meal.find();
allCollections[k] = docs;
console.log(allCollections);
}));
Another advantage is error handling. If any errors happen in the callback of the original example, they won't be caught in this try/catch block.
async/await handles errors like you'd expect, and errors will end up in the catch block.
...
// Now that we have awaited all async calls above, this should be executed _after_ the async calls instead of before them.
res.send(allCollections);
})
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
Technically Promise.all() returns an array of results, but we can ignore that since you're formatting an Object anyway.
There is plenty of room to optimize this further. I might write the whole function as something like:
exports.getAllFoods = async (req, res, next) => {
const db = mongoose.connection.db;
try {
let collections = await db.listCollections().toArray();
let allCollections = {};
collections.forEach((k) => {
allCollections[k.name] = [];
})
// For each collection key name, find docs from the database
// await completion of this block before proceeding to the next block
await Promise.all(Object.keys(allCollections).map(async k => {
let Meal = mongoose.model(k, MealSchema)
let docs = await Meal.find();
allCollections[k] = docs;
}));
// allCollections should be populated if no errors occurred
console.log(allCollections);
res.send(allCollections);
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
Completely untested.
You might find these links more helpful than my explanation:
https://javascript.info/async-await
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
https://medium.com/dailyjs/the-pitfalls-of-async-await-in-array-loops-cf9cf713bfeb
I hope this will help you : You need to first use the stringify method before sending the collections from the express api and then use JSON.parse on the React front end to restore the object.
PS: can you do a console.log(allCollections) one line above res.send(allCollections)?
You need to send it to the front-end in a JSON format.
replace res.send(allCollections) with res.json(allCollections)
This is how I am currently getting data from mongodb:
users.get(base_URL, (req, res) => {
UserModel.find({}, (err, docs) => {
res.render("Users/index", {
title: "All Users here",
user_list: docs
});
});
});
Now, as you can see this is an express application. What I would like, is to simple call a function so that I can get the value from the docs variable inside the mongodb model callback. How do I do this, ideally, I want to see something like this:
users.get(base_URL, (req, res) => {
res.render('<some_jade_file_here>', {
title: "Yes, got it right",
user_list: getAllUsers();
});
});
Ideally, I just want to call a function. How can I do this, since having to put render inside of a mongodb call is a problem, since you may want to query a bunch of things from the database, and it might not even be just one database. I'm struggling a little since I'm not all that used to callbacks.
Any help would be deeply appreciated. If you're wondering about this syntax () => {}, thats just an anonymous function in typescript.
You can't do it without callbacks, but you can use an async flow control library like async to help manage the nest of callbacks. In this case you probably want to use async.parallel.
Using that you can do something like:
users.get(base_URL, (req, res) => {
var data = {
title: "Yes, got it right"
};
async.parallel([
(callback) => {
UserModel.find({}, (err, docs) {
data.user_list = docs;
callback(err);
});
},
(callback) => {
// Other query that populates another field in data
}
], (err, results) => {
// Called after all parallel functions have called their callback
res.render('<some_jade_file_here>', data);
});
});