I am currently stuck in asynchronous hell.
In my React, I have a page /menu, that would load data from my mongo instance via expressjs api.
In my database, called menu, i have collections which represent a meal-type eg "breakfast", "lunch" etc. In those collections, the documents for every item looks like this bread collection example:
{
_id: 2398jcs9dn2f9f,
name: "Ciabatta",
desc: "Italian bread",
imageURI: "image01.jpg",
reviews: []
}
This is my api that would be called when the page loads
exports.getAllFoods = (req, res, next) => {
const db = mongoose.connection
const allCollections = {}
try {
db.db.listCollections().toArray((err, collections) => {
collections.forEach((k) => {
allCollections[k.name] = []
})
Object.keys(allCollections).map(k => {
let Meal = mongoose.model(k, MealSchema)
meal = Meal.find((err, docs) => {
allCollections[k] = docs
console.log(allCollections)
})
})
res.send(allCollections)
})
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
The last output of the console.log(allCollections) produces this:
{ snacks:
[ { review: [],
tags: [],
_id: 5fcec3fc4bc5d81917c9c1fe,
name: 'Simosa',
description: 'Indian food',
imageURI: 'image02.jpg',
__v: 0 } ],
breads:
[ { review: [],
tags: [],
_id: 5fcec41a4bc5d81917c9c1ff,
name: 'Ciabatta',
description: 'Italian bread',
imageURI: 'image02.jpg',
__v: 0 } ],
}
This is exactly what I need, but I am stuck in figuring out how to send to React. What am I to do to send the above json? The res.send(allCollections) gives me this:
{
"snacks": [],
"breads": [],
"drinks": []
}
I understand why the above is being sent, but I dont know what I need to do to address it.
This is my React on page load
useEffect(() => {
axios
.get('http://localhost:8888/api/allFoods')
.then((res) => {
setMealTypes(res.data)
})
.catch((err) => [
console.log(err)
])
}, [])
Ultimately, I need the json outputted in console as I wanted to loop through that data and use the key as a title, and then list the values from the value array eg
<div>
<h2>Breads</h2>
<img src=image01.jpg/>
<h3>Ciabatta</h3>
<p>Italian bread</p>
...
</div>
...
I'd appreciate any help, and any docs I should read to help and improve my javascript understandings
I'd prefer to solve this using async/await and Promise.all, replacing most callbacks.
Because you're calling the DB when you're iterating through an array, you have the most annoying callback situation: how do you issue a bunch of async things and then get the results after? You'll need something else to ensure all callbacks are called before sending the results.
Async/await means we can declare a function is async, and await the results of an async operation. async/await is annoying in JS because it abstracts away callbacks and is actually creating a Promise underneath. Complicating things further, async/await doesn't solve issuing multiple async functions, so again we have to rely on this fancy Promise.all() function combined with map-ing the desired input array to async functions.
Original:
Object.keys(allCollections).map(k => {
let Meal = mongoose.model(k, MealSchema)
meal = Meal.find((err, docs) => {
allCollections[k] = docs
console.log(allCollections)
})
});
Suggested async/await:
await Promise.all(Object.keys(allCollections).map(async k => {
let Meal = mongoose.model(k, MealSchema)
let docs = await Meal.find();
allCollections[k] = docs;
console.log(allCollections);
}));
Another advantage is error handling. If any errors happen in the callback of the original example, they won't be caught in this try/catch block.
async/await handles errors like you'd expect, and errors will end up in the catch block.
...
// Now that we have awaited all async calls above, this should be executed _after_ the async calls instead of before them.
res.send(allCollections);
})
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
Technically Promise.all() returns an array of results, but we can ignore that since you're formatting an Object anyway.
There is plenty of room to optimize this further. I might write the whole function as something like:
exports.getAllFoods = async (req, res, next) => {
const db = mongoose.connection.db;
try {
let collections = await db.listCollections().toArray();
let allCollections = {};
collections.forEach((k) => {
allCollections[k.name] = [];
})
// For each collection key name, find docs from the database
// await completion of this block before proceeding to the next block
await Promise.all(Object.keys(allCollections).map(async k => {
let Meal = mongoose.model(k, MealSchema)
let docs = await Meal.find();
allCollections[k] = docs;
}));
// allCollections should be populated if no errors occurred
console.log(allCollections);
res.send(allCollections);
} catch (error) {
console.log(error)
res.send('unable to get all collections')
}
}
Completely untested.
You might find these links more helpful than my explanation:
https://javascript.info/async-await
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
https://medium.com/dailyjs/the-pitfalls-of-async-await-in-array-loops-cf9cf713bfeb
I hope this will help you : You need to first use the stringify method before sending the collections from the express api and then use JSON.parse on the React front end to restore the object.
PS: can you do a console.log(allCollections) one line above res.send(allCollections)?
You need to send it to the front-end in a JSON format.
replace res.send(allCollections) with res.json(allCollections)
Related
I have a project collection. Under the project schema have a property called tasks which is an array with the ids of the tasks. Now I am trying to write an api for singleProject, which should include the project details with a property taskDetails which will contain the detail of the tasks under the project. And for getting the task details I am using the ids which I stored in the task property under project. Below is the code I am using :
exports.getSingleProject = async (req, res) => {
req.project.logo = undefined;
await req.project.tasks.map((taskItem, index) => {
taskSchema.find({ _id: taskItem.toString() }).exec((err, task) => {
req.project.tasksDetails.push(task);
});
});
responseMessages(
res,
200,
true,
"Single project fetched successfully.",
req.project
);
};
but in the response getting taskDetails as an blank array. I guess I am using async await in wrong place. Need some help on this. Thanks in advance
The await keyword only works with a promise: an array is not a promise, but you can map to a promise and wait for all promises to resolve using Promise.all - see example below:
const tasks = await Promise.all(req.project.tasks.map(item => new Promise((accept, reject) => {
taskSchema.find({ _id: taskItem.toString() }).exec((err, task) => {
if(err) {
reject(err);
return;
}
accept(task);
});
});
The tasks variable would be an array of your tasks, which you can return in the response.
I have two schemas, User, and Product. The User schema is where I store all the product ID and the number of items that the user added to the cart.
When the user makes a request to '/checkout' it should update the quantity and then remove it from the cart. I am having an issue when I checkout the quantity is not updating the quantity.
router.post('/checkout', auth, catchAsync(async (req, res) => {
const user = await User.findById(req.session.userId);
const err = [];
if (user && user.products.length > 0) {
user.products.map(async (product) => {
let total = 0;
const p = await Product.findById(product.productID);
if (p.quantity > 0 && p.quantity > product.quantity) {
console.log('IN');
total = p.quantity - product.quantity;
console.log(total);
await Product.findOneAndUpdate({ _id: product.productID }, { $set: { quantity: total } });
} else {
err.push(`Item, ${p.name} is sold out`);
}
});
await User.findOneAndUpdate({ _id: req.session.userId }, { $set: { products: [] } });
if (err.length) {
return res.status(500).json({ message: err });
}
return res.status(200).json({ message: 'OK' });
}
return res.status(200).json({ message: 'Empty cart' });
}));
User Schema:
Product Schema:
I believe the problem in your code is at the user.products.map(...) function, because you never wait for all the promises you create in the map to resolve.
In other words, the map function returns an array of pending promises, but it will not wait for them to be done, and therefore the execution continues through the rest of the code reaching the res.status(...) before any of the code in map had been executed.
You have different options to solve it, but mainly you need to take care of the array of promises returned by the map function and wait for their completion, before you end your code. There is a very good explanation of how to handle this situation with async/await at Google Developers Web fundamentals guide.
I usually leverage Promise.all() function, which returns a single promise from the array of promises, and therefore you can wait until the code in map is executed in parallel for each item in the array (i.e. product in your case). You can read more about it at MDN documentation.
// ...
let promisesArray = user.products.map(async product => {...});
// promisesArray should look like: [Promise { <pending> }, Promise { <pending> }, … ]
// Using Promise.all we wait for each of them to be done in parallel
await Promise.all(promisesArray);
// Now you are certain the code in the map has been executed for each product
// ...
A good practice as well is to use try {} catch(err) {} block around the Promise.all() to handle cases of some promise being rejected.
Im creating a live search box on express and it shows 2 errors.
(1) TypeError (2) Unhandled Promise rejection
CODE:
router. post('/search-phrasing', async (req, res) => {
const {
phrasing
} = req.body;
const
phrasingArray = phrasing.trim().split(' ');
phrasingArray.map(async (phrasing) => {
let suggestions = [];
await Response.find({
entities: {
$regex: new RegExp(phrasing)
}
}).sort({
phrasing: 'asc'
}).then((data) => {
if (data[0]) {
suggestions.push({
id: data[0]._id,
phrasing: data[0].phrasing
});
res.send(suggestions);
}
}).catch((err) => console.log(err));
});
});
Don't attempt to loop async functions this way as it is not required, and certainly don't send responses in a loop. Instead you should .map() the list of regular expressions to $in:
router.post('/search-phrasing', (req, res) => {
const { phrasing } = req.body;
if (phrasing == undefined || ( typeof(phrasing) != 'string' ) ) {
console.error("phrasing is required as a string");
return res.end(); // really should have better error handling
}
const phrasingArray = phrasing.trim().split(' ');
Response.find({ entities: { $in: phrasingArray.map(e => RegExp(e)) })
.sort('phrasing')
.select('phrasing')
.then(suggestions => res.send(suggestions))
.catch(err => console.error(err));
})
The $in operator accepts an array of arguments to match. It also happens to accept regular expressions as those arguments. It's basically shorthand for the $or operator but always applying to the one field.
Attempting to do this otherwise is executing multiple statements with the database, awaiting various promises and attempting to construct a single response from all of that. It's simply not necessary when there are query expressions which already handle this.
Also check your input types. Don't blindly presume you supplied the required data to the POST body. Check for it being present as is shown here, otherwise you get exceptions
Created an Rest Post-type API in nodeJS,in which :
I am executing two queries here.
1. Firstly Executing the query on answers table to fetch user-Id and answer detail as well in that table. // i have check in the my console they show me two user-Id
2.Second ,executing the query on users table to fetch users detail on the basis of user-Id that i pass in find function. // check my console they show me two answer object because i have two user-Id.
------------------ above process work fine ---------------------
Now, i got stuck because , i have two merge both result into one object.
I'm doing this but isn't work perfect.Help me out here..!!
My code : -
app.post('/getTopAns', function(req, res) {
console.log("inside getTopAns ");
var questionId=req.body.questionId;
mongoose.model('answers').find({
questionId:questionId,
compliance:"Y"
}, function(err, ansResult){
for (var i = 0;i<ansResult.length;i++) {
mongoose.model('users').findOne({
userId:ansResult[i].userId,
}, function(err,usrResult){
var obj = {
followerLength : usrResult.follower.length,
upvote : ansResult[i].upvote
}
})
console.log(obj);
}
});
})
Maybe you should first log ansResult to see if db is returning this. If it is, then check usrResult. If both are returned, I usually use lodash's assign or merge to merge two objects into one
I recommend using promises instead of callback to handle async easier with node.js. The flow is to get the first call result, map the array of result into a list of promises which we can wait for the result by using Promise.all. Finally map the second results which the first list.
app.post("/getTopAns", function(req, res) {
console.log("inside getTopAns ");
var questionId = req.body.questionId;
mongoose.model("answers")
.find({
questionId: questionId,
compliance: "Y"
})
.then(ansResult => {
const promises = ansResult.map(ans => {
return mongoose.model("users")
.findOne({ userId: ansResult[i].userId })
})
return Promise.all(promises)
.then(usrResults => {
return usrResults.map((usrResult, i) => {
followerLength: usrResult.follower.length,
upvote: ansResult[i].upvote
})
})
})
.then(results => {
console.log('got all results here', results)
})
});
It can be even better using async/await.
app.post("/getTopAns", async (req, res) => {
console.log("inside getTopAns ");
var questionId = req.body.questionId;
const ansResult = mongoose.model("answers")
.find({
questionId: questionId,
compliance: "Y"
})
const promises = ansResult.map(ans => {
return mongoose.model("users")
.findOne({ userId: ansResult[i].userId })
})
const usrResults = await Promise.all(promises)
const results = usrResults.map((usrResult, i) => {
followerLength: usrResult.follower.length,
upvote: ansResult[i].upvote
})
console.log('got all results here', results)
});
More information can be found here: https://medium.com/#ThatGuyTinus/callbacks-vs-promises-vs-async-await-f65ed7c2b9b4
In Node.js, I have a Promise.all(array) resolution with a resulting value that I need to combine with the results of another asynchronous function call. I am having problems getting the results of this second function, since it resolves later than the promise.all. I could add it to the Promise.all, but it would ruin my algorithm. Is there a way to get these values outside of their resolutions so I can modify them statically? Can I create a container that waits for their results?
To be more specific, I am reading from a Firebase realtime database that has been polling API data. I need to run an algorithm on this data and store it in a MongoDB archive. But the archive opens aynchronously and I can't get it to open before my results resolve (which need to be written).
Example:
module.exports = {
news: async function getNews() {
try {
const response = await axios.get('https://cryptopanic.com/api/posts/?auth_token=518dacbc2f54788fcbd9e182521851725a09b4fa&public=true');
//console.log(response.data.results);
var news = [];
response.data.results.forEach((results) => {
news.push(results.title);
news.push(results.published_at);
news.push(results.url);
});
console.log(news);
return news;
} catch (error) {
console.error(error);
}
},
coins: async function resolution() {
await Promise.all(firebasePromise).then((values) => {
//code
return value
}
}
I have tried the first solution, and it works for the first entry, but I may be writing my async function wrong on my second export, because it returns undefined.
You can return a Promise from getNews
module.exports = {
news: function getNews() {
return axios.get('https://cryptopanic.com/api/posts/?auth_token=518dacbc2f54788fcbd9e182521851725a09b4fa&public=true')
.then(res => {
// Do your stuff
return res;
})
}
}
and then
let promiseSlowest = news();
let promiseOther1 = willResolveSoon1();
let promiseOther2 = willResolveSoon2();
let promiseOther3 = willResolveSoon3();
Promise.all([ promiseOther1, promiseOther2, promiseOther3 ]).then([data1,
data2, data3] => {
promiseSlowest.then(lastData => {
// data1, data2, data3, lastData all will be defined here
})
})
The benefit here is all promises will start and run concurrently so your total waiting time will be equal to the time taken by the promiseSlowest.
Check the link for more:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function#Examples