i'm building a crud app for a project,
people can add view and delete entries, i'm using nodejs, js and fireabse for this.
i have such a firestore database structure:
entries:
--entry id
--entry data
users:
--user id
--user email (query)
-- my entries (collection)
-- entries id
--entry id
now i want to display all users entries,
so i created this module.exports function:
module.exports = {
//get all the users entries from database with passed in uid ('/dashboard')
getUserEntries: async uid => {
//get all his entries docs id from "myEntries" collection in an, array
const usersEntriesId = await db
.collection("users")
.doc(uid)
.collection("myEntries")
.get()
.then(entries => {
return entries.docs.map(entry => entry.data().entry);
})
.catch(err => console.log(err));
console.log(usersEntriesId); //works fine, logs array with ids
const userEntriesDocs = usersEntriesId.map(async id => {
const entry = await db
.collection("entries")
.doc(id)
.get()
.then(entry => {
return entry.data();
});
console.log("hello:", entry); //works fine returns logs entry data
return entry;
});
console.log("hello1: ", userEntriesDocs); //doesnt work i get "hello1: [ Promise { <pending> },
// Promise { <pending> },
//Promise { <pending> } ]"
//i got three entries, that's why i get 3 times "Promise { <pending> }"
}
};
so how do i resolve that?
Thanks
well, async function returns Promise, it how they works under the hood. If there was no .map you could just await on that function or use it as arbitrary Promise with .then() and .catch. But since there is array of Promise you need Promise.all to wait until all are resolved.
const userEntriesDocs = await Promise.all(usersEntriesId.map(async id => {
....
);
Beware: unlike .allSettled, .all() will fail immediately if any of subsequent Promise fails. So if for any reason you want to have data from those requests that succeeded, you need more complex logic.
As an alternative you may go through loop manually:
const userEntriesDocs = [];
for(const docPromise of userEntriesId.map(.....)) {
userEntriesDocs.push(await docPromise);
}
But to me await Promise.all[...] is more readable.
Also I highlight there is array of Promises(requests have already been sent). If you try sending requests inside the loop like
const userEntriesDocs = [];
for(const id of userEntriesId) {
userEntriesDocs.push(await db
.collection("entries")
.doc(id)
.get()
.then(entry => {
return entry.data();
})
);
}
you will find that requests are going strictly one-by-one, not in parallel. That will require much more time to process the list.
Related
I am trying to make a chat app and using firestore and cloud functions.
My firestore structure is
Users -> name,username,email,etc...
Conversations -> members:[memberId1,memberId2]
and when there is a new conversation created I am adding a conversation collection to Users/{memberId}/Conversation collection but either its not getting created or it takes long time to gets created
this is my cloud function to add Conversation info to Users collection
functions.firestore
.document('Conversations/{conversationId}')
.onCreate((snapshot:any, context:any) => {
const data = snapshot.data();
const conversationId = context.params.conversationId;
if (data) {
const members = data.members;
if(!Array.isArray(members)){
console.log("Members is not an array")
return null;
}
for ( const member of members) {
const currentUserId = member;
const remainingUserIDs = members.filter((u:string) => u !== currentUserId);
remainingUserIDs.forEach(async (m:string) => {
return admin
.firestore()
.collection('Users')
.doc(m)
.get()
.then((_doc) => {
const userData = _doc.data();
if (userData) {
return admin
.firestore()
.collection('Users')
.doc(currentUserId)
.collection('Conversations')
.doc(m)
.create({
conversationId: conversationId,
image: userData.profilePic,
username: userData.username,
unseenCount: 0,
userId: m,
});
}
return null;
})
.catch((err) => {
console.log(err);
return null;
});
});
}
}
return null;
});
and i see in the logs its gets called without error but no data getting added or it takes long to gets created.
what am I doing wrong ?
and another interesting thing is I have tested the function on emulator and it works fine but on production there is a problem
Your code is not waiting for each of the promises in the foreach loop to resolve before returning from the overall function. The function is just returning null immediately. Calling then and catch on a promise doesn't make the code pause.
Your function must return a promise that resolves only after all other async work is finished. If you don't, the function will terminate early and clean up before the work is complete. This means you have to track all the promises that you kick off in the foreach loop. I suggest reading these to better understand the problem and get some ideas on how to manage this:
Node JS Promise.all and forEach
How to use promise in forEach loop of array to populate an object
How to call promise inside forEach?
This method is supposed to return a list of users but returning an empty array. When I console log inside the forEach loop, it prints the data but not outside of it.
get getAllUsers() {
const usersList = [];
firebase
.firestore()
.collection('users')
.get()
.then(snapshot => {
snapshot.forEach(user => {
usersList.push(user.data());
console.log(user.data());
});
});
return usersList; // Array [] or undefined when indexed
}
On Home.js, I call it here.
componentDidMount() {
const list = fireStoreDB.getAllUsers;
console.log(list);
}
Array [] is the console log of the method and the objects are the console log from the forEach loop.
You got an empty array, because your function is synchronous and firebase returns a promise which is asynchronous. So that's why in most cases you will get an empty array first, and after that response from firebase.
Try this one:
async getAllUsers() {
const usersList = await firebase
.firestore()
.collection('users')
.get()
.then(snapshot => {
return snapshot.map(user => user.data(););
});
return usersList;
}
And then in Home.js
componentDidMount() {
fireStoreDB.getAllUsers().then(list => console.log(list));
}
I'm creating a cloud function that receives a call from the client including the current user id. What I'm trying to achieve is get a list of friend ids from the "users" collection under the current user id and then for each friend id get a list of objects from a collection named "locations". The code below has the correct queries but there is some confusion around the promise handling and the return statements. How can I correctly return the result from inside the second 'then' ? Currently returning HttpsCallableResult with data as an empty map even thought it should include 2 objects.
exports.friendsData = functions.https.onCall((data, context) => {
return admin
.firestore()
.collection('users')
.doc(data.id)
.get()
.then(snapshot => {
const friendsIds = snapshot.data().friends;
console.log('friendsIds', friendsIds);
return Promise.all(
friendsIds.map(id => {
console.log('id', id);
return admin
.firestore()
.collection('locations')
.where('userid', '==', id)
.get();
})
);
})
.then(locationSnapshots => {
const results = [];
for (const locationSnap of locationSnapshots) {
const data = locationSnap.data();
console.log('data', data);
results.push(data);
}
console.log('resuts', resuts);
return resuts;
})
.catch(reason => {
return reason;
});
});
EDIT
locations collection has documents with the autogenerated ids and each document has field named "userid" which is used for the where query
Image for location collection structure
Updated code based on comments. locationSnapshots should be an array of query snapshots for each promise added before. Still getting undefined.
I'll document the things wrong we've discovered so far in an answer:
First, you need a .get() after the .where(...) to get the promise.
Second, if friendsIds is an array and you're trying to iterate the contents of the array, change this:
for (const id in friendsIds)
to this:
for (const id of friendsIds)
When you use in, it iterates properties of the object which in the case of an array would be the array indexes, not the array values.
Even better would be switch to use .map() since you're trying to create an array from an array. Change this:
const promises = [];
for (const id in friendsIds) {
const p = admin
.firestore()
.collection('locations')
.where('userid', '==', id)
.get();
promises.push(p);
}
return Promise.all(promises);
to this:
return Promise.all(friendsIds.map(id => {
return admin
.firestore()
.collection('locations')
.where('userid', '==', id)
.get();
}));
I'm quite desperate, because of this annoying error. I can't find any information about that on the internet.
Actually my application works fine, but if I start multiple queries and wait for them with Promise.all(), the server crashes. But step by step:
In my index.js, I'm initializing the connection pool with 60 max. connections, and exporting the variable.
const pool = mariadb.createPool(config.mariaDB);
module.exports.pool = pool
Later I'm importing the index file as "app" and using app.pool.query(...) to query my database.
Next, I have a post request /getUsersGroups. A User can be a member of multiple groups, so I expect the response to be an array of objects. Each object contains information about the group. Moreover there is a members field inside this object. This field also is an array containing information about the members, so s.th. like that:
[
{
"groupId": 125758,
"title": "test",
"members": [
{userId: 5, name:Max, }, ...]
}, ...
]
Therefore I have a post request, which looks like that. I'm getting all group ids the user is a member. Then for each groupId, I call the method Group.groupInfo.
This returns a promise, therefore I'm collecting all promises in an array and wait for all of them, before sending the response.
router.post("/getUsersGroups", (req, res) => {
let userId = req.body.userId;
let result = []
let promises = []
Group.getGroupsByUser(userId) // this gets all group ids the user is a member
.then((item) => {
let groups = item.map(x => objectParser.parseUnderscoreToCamel(x))
for (let i = 0; i < item.length; i++) {
let groupId = item[i].groupId
promises.push( //adding promises
Group.groupInfo(groupId, userId)
.then((item) => {
result.push(item)
})
.catch((err)=>{console.log(err)}))
}
})
.then(() => {
// waiting for all promises to finish
return Promise.all(promises)
.then(() => {
res.status(200).json({groups: result})
})
})
.catch((err) => {
console.log(err)
}
})
}
So the Promise Group.groupInfo looks like this. It first selects the group information and then for each member inside the group it calls another Promise User.userInfo. The pattern is the same like in the method above. So I push the Promise to an array and wait for them to finish:
module.exports.groupInfo = (groupId, userId)=>{
let groupInfo = {}
let promises = []
return new Promise(((resolve, reject) => {
app.pool.query("SELECT* FROM Groups WHERE group_id=?", [groupId])
.then((item) =>{
groupInfo = item[0]
groupInfo["members"] = [];
return app.pool.query("SELECT user_id FROM Users_groups WHERE group_id=?", [groupId])
.then((item) =>{
let members = [] //contains all user ids
for(let i=0; i<item.length;i++){
members.push(item[i].userId)
}
members.forEach((memberId)=>{
// for each memberID call User.userInfo
promises.push(User.userInfo(userId,memberId)
.then((item)=>{
groupInfo.members.push(item)}))
}
})
})
})
.then(()=>{
// wait for all Promises
return Promise.all(promises)
})
.then(() =>{
return resolve(groupInfo)
})
.catch((err)=>{
return reject(err)
})
}))
}
User.userInfo itself, makes a query to the Database to collect the information about the user.
If I'm calling this, the server crashes and giving me this error:
activeConnections[conn.threadId] = conn;
TypeError: Cannot read property 'threadId' of undefined
at Pool.handleTaskQueue (****\node_modules\mariadb\lib\pool.js:431:30)
at _combinedTickCallback (internal/process/next_tick.js:132:7)
at process._tickCallback (internal/process/next_tick.js:181:9)
I guess the problem is somehow with the Promise.all() right?
But I cannot find any information about this error. I'm thankful for every hint!
My solution was not to use Promise.all but to call the Promises sequentially, but seems not right to me.
return groupsId.reduce((promise, groupId) => {
return promise.then(() => {
return Group.groupInfo(groupId, userId)
.then((item)=>{
result.push(item)
console.log(result)
})
.catch((err)=>{
throw err
})
})
}, Promise.resolve())
I cannot rely on this, as long as I don't understand the error.
Thanks!
In Node.js, I have a Promise.all(array) resolution with a resulting value that I need to combine with the results of another asynchronous function call. I am having problems getting the results of this second function, since it resolves later than the promise.all. I could add it to the Promise.all, but it would ruin my algorithm. Is there a way to get these values outside of their resolutions so I can modify them statically? Can I create a container that waits for their results?
To be more specific, I am reading from a Firebase realtime database that has been polling API data. I need to run an algorithm on this data and store it in a MongoDB archive. But the archive opens aynchronously and I can't get it to open before my results resolve (which need to be written).
Example:
module.exports = {
news: async function getNews() {
try {
const response = await axios.get('https://cryptopanic.com/api/posts/?auth_token=518dacbc2f54788fcbd9e182521851725a09b4fa&public=true');
//console.log(response.data.results);
var news = [];
response.data.results.forEach((results) => {
news.push(results.title);
news.push(results.published_at);
news.push(results.url);
});
console.log(news);
return news;
} catch (error) {
console.error(error);
}
},
coins: async function resolution() {
await Promise.all(firebasePromise).then((values) => {
//code
return value
}
}
I have tried the first solution, and it works for the first entry, but I may be writing my async function wrong on my second export, because it returns undefined.
You can return a Promise from getNews
module.exports = {
news: function getNews() {
return axios.get('https://cryptopanic.com/api/posts/?auth_token=518dacbc2f54788fcbd9e182521851725a09b4fa&public=true')
.then(res => {
// Do your stuff
return res;
})
}
}
and then
let promiseSlowest = news();
let promiseOther1 = willResolveSoon1();
let promiseOther2 = willResolveSoon2();
let promiseOther3 = willResolveSoon3();
Promise.all([ promiseOther1, promiseOther2, promiseOther3 ]).then([data1,
data2, data3] => {
promiseSlowest.then(lastData => {
// data1, data2, data3, lastData all will be defined here
})
})
The benefit here is all promises will start and run concurrently so your total waiting time will be equal to the time taken by the promiseSlowest.
Check the link for more:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function#Examples