We Keep Coding

Why does startAfter in this firebase query isn't working?

I have the next function using firebase in an JavaScript app. The query works fine the first time because lastDoc isn't defined so it gets the first 7 documents in the database, but the next time when there's a lastDoc, the query keeps returning the same first 7 docs. The lastDoc variable is updated and indeed gets the value of the next doc.
const getPosts = async ({loadMore=false, lastDoc}: { loadMore: boolean, lastDoc?: any}) => {
let col = collection(db, "posts");
let q = query(col,
orderBy("updateDate", "desc"),
limit(7),
startAfter(lastDoc));
const querySnapshot = await getDocs(q);
let newLastDoc = querySnapshot.docs[querySnapshot.size-1];
let posts = querySnapshot.docs.map(doc => {
if(doc.data().inactive == false || !doc.data().inactive) return {...doc.data(), id: doc.id}
else return null
}).filter((post: any) => post !== null);
return {posts, lastDoc: newLastDoc};
}
The first time the lastDoc is undefined, and returns docs 1, 2, 3, 4, 5, 6 and 7.
The second time the lastDoc is:
{
_firestore: {...},
_userDataWriter: {...},
_key: {...},
_document: {..., data: {DATA_FROM_THE_7th_DOC}},
_converter: null
}
, and keeps returning docs 1, 2, 3, 4, 5, 6 and 7
Why isn't it working?

In the legacy version of the JavaScript SDK, startAfter was defined as:
Query<T>#startAfter(...fieldValues: any[]): Query<T>;
Query<T>#startAfter(snapshot: DocumentSnapshot<any>): Query<T>;
In the modern JavaScript SDK, startAfter is defined as:
function startAfter(...fieldValues: unknown[]): QueryStartAtConstraint;
function startAfter(snapshot: DocumentSnapshot<any>): QueryStartAtConstraint;
This means that your code should be functioning as expected. HOWEVER, based on the provided project code from the comments, this is for an Express API, not client-side JavaScript.
Your query isn't working as expected because lastDoc is not a true DocumentSnapshot object at all and is instead a JavaScript version of it churned out by res.json(). Instead of returning a mangled DocumentSnapshot object to the client, we should instead just send back the data we need to pass as the startAfter arguments. In our case, we'll send back just the updateDate parameter inside of lastDocData:
return {
posts,
lastDocData: {
updateDate: newLastDoc.get("updateDate")
}
};
Then when the data comes back to the API, we'll pass it to startAfter:
startAfter(lastDocData.updateDate)) // Assumes lastDocData is { updateDate: number }
While reviewing methods/getPosts.ts, I noticed lines similar to the following:
let q = !loadMore
? category && category !== "TODOS"
? query(
col,
where("school", "==", school?.code),
where("category", "==", category),
where("inactive", "==", false),
orderBy("updateDate", "desc"),
limit(7)
)
: query(
col,
where("school", "==", school.code),
where("inactive", "==", false),
orderBy("updateDate", "desc"),
limit(7)
)
: category && category === "TODOS" // note: condition does not match above
? query(
col,
where("school", "==", school?.code),
where("inactive", "==", false),
orderBy("updateDate", "desc"),
limit(7),
startAfter(lastDocData.updateDate))
)
: query(
col,
where("school", "==", school.code),
where("category", "==", category),
where("inactive", "==", false),
orderBy("updateDate", "desc"),
limit(7),
startAfter(lastDocData.updateDate)
);
Because query takes a variable number of arguments, you can use the spread operator (...) to optionally include some arguments. As a simplified example:
let includeC = false,
result = [];
result.push(
'a',
'b',
...(includeC ? ['c'] : []),
'd'
);
console.log(result); // => ['a', 'b', 'd']
includeC = true;
result = [];
result.push(
'a',
'b',
...(includeC ? ['c'] : []),
'd'
);
console.log(result); // => ['a', 'b', 'c', 'd']
This allows you to use the following query builder instead:
const q = query(
colRef,
...(school?.code ? [where("school", "==", school.code)] : []), // include school code filter, only if provided
...(category && category !== "TODOS" ? [where("category", "==", category)] : []), // include category filter, only if provided and not "TODOS"
where("inactive", "==", false),
orderBy("updateDate", "desc"),
limit(7),
...(lastDocData?.updateDate ? [startAfter(lastDocData.updateDate)] : []) // Assumes lastDocData is a { updateDate: number }
);
Bringing this all together gives:
// methods/getPosts.ts
import { School } from '../../types';
import app from '../../firebaseconfig';
import { getFirestore, collection, query, where, getDocs, limit, orderBy, startAfter } from 'firebase/firestore/lite';
const db = getFirestore(app);
const getPosts = ({category, school, dataSavingMode=false, lastDocData}: {category: string, school: School, dataSavingMode: boolean, lastDocData?: { updateDate: number }}) => {
const colRef = collection(db, dataSavingMode ? "dataSavingPosts" "posts");
const q = query(
colRef,
...(school?.code ? [where("school", "==", school.code)] : []),
...(category && category !== "TODOS" ? [where("category", "==", category)] : []),
where("inactive", "==", false),
orderBy("updateDate", "desc"),
limit(7),
...(lastDocData?.updateDate ? [startAfter(lastDocData.updateDate)] : []),
);
const querySnap = await getDocs(q),
postDocsArr = querySnap.docs,
lastDoc = postDocsArr[postDocsArr.length - 1],
noMorePosts = postDocsArr.length < 7;
const postDataArr = postDocsArr
.filter(docSnap => !docSnap.get("inactive")) // shouldn't be needed with above filter
.map(docSnap => ({ ...docSnap.data(), id: docSnap.id }));
return {
posts: postDataArr,
noMorePosts,
lastDocData: { updateDate: lastDoc.get("updateDate") }
}
}
export default getPosts;
// routes/getPosts.ts
import express from 'express';
import { School } from '../../types';
const router = express.Router();
// Methods
import getPosts from '../methods/getPosts';
router.post('/', async (req, res) => {
const reqBody: {
school: School,
category: string,
lastDocData?: any,
dataSavingMode?: boolean
} = req.body;
const {posts, lastDocData, noMorePosts} = await getPosts({
category: reqBody.category,
school: reqBody.school,
dataSavingMode: reqBody.dataSavingMode,
lastDocData: reqBody.lastDocData
});
res.json({posts, lastDocData, noMorePosts});
})
export default router;

Firebase query() with condition ( Version 9 web ) [duplicate]

For example I have dynamic filter for my list of books where I can set specific color, authors and categories.
This filter can set multiple colors at once and multiple categories.
Book > Red, Blue > Adventure, Detective.
How can I add "where" conditionally?
firebase
.firestore()
.collection("book")
.where("category", "==", )
.where("color", "==", )
.where("author", "==", )
.orderBy("date")
.get()
.then(querySnapshot => {...

As you can see in the API docs, the collection() method returns a CollectionReference. CollectionReference extends Query, and Query objects are immutable. Query.where() and Query.orderBy() return new Query objects that add operations on top of the original Query (which remains unmodified). You will have to write code to remember these new Query objects so you can continue to chain calls with them. So, you can rewrite your code like this:
var query = firebase.firestore().collection("book")
query = query.where(...)
query = query.where(...)
query = query.where(...)
query = query.orderBy(...)
query.get().then(...)
Now you can put in conditionals to figure out which filters you want to apply at each stage. Just reassign query with each newly added filter.
if (some_condition) {
query = query.where(...)
}

Firebase Version 9
The docs do not cover this but here is how to add conditional where clauses to a query
import { collection, query, where } from 'firebase/firestore'
const queryConstraints = []
if (group != null) queryConstraints.push(where('group', '==', group))
if (pro != null) queryConstraints.push(where('pro', '==', pro))
const q = query(collection(db, 'videos'), ...queryConstraints)
The source of this answer is a bit of intuitive guesswork and help from my best friend J-E^S^-U-S

With Firebase Version 9 (Jan, 2022 Update):
You can filter data with multiple where clauses:
import { query, collection, where, getDocs } from "firebase/firestore";
const q = query(
collection(db, "products"),
where("category", "==", "Computer"),
where("types", "array-contains", ['Laptop', 'Lenovo', 'Intel']),
where("price", "<=", 1000),
);
const docsSnap = await getDocs(q);
docsSnap.forEach((doc) => {
console.log(doc.data());
});

In addition to #Doug Stevenson answer. When you have more than one where it is necessary to make it more dynamic as in my case.
function readDocuments(collection, options = {}) {
let {where, orderBy, limit} = options;
let query = firebase.firestore().collection(collection);
if (where) {
if (where[0] instanceof Array) {
// It's an array of array
for (let w of where) {
query = query.where(...w);
}
} else {
query = query.where(...where);
}
}
if (orderBy) {
query = query.orderBy(...orderBy);
}
if (limit) {
query = query.limit(limit);
}
return query
.get()
.then()
.catch()
}
// Usage
// Multiple where
let options = {where: [["category", "==", "someCategory"], ["color", "==", "red"], ["author", "==", "Sam"]], orderBy: ["date", "desc"]};
//OR
// A single where
let options = {where: ["category", "==", "someCategory"]};
let documents = readDocuments("books", options);

Note that a multiple WHERE clause is inherently an AND operation.

If you're using angular fire, you can just use reduce like so:
const students = [studentID, studentID2,...];
this.afs.collection('classes',
(ref: any) => students.reduce(
(r: any, student: any) => r.where(`students.${student}`, '==', true)
, ref)
).valueChanges({ idField: 'id' });
This is an example of multiple tags...
You could easily change this for any non-angular framework.
For OR queries (which can't be done with multiple where clauses), see here.

For example, there's an array look like this
const conditionList = [
{
key: 'anyField',
operator: '==',
value: 'any value',
},
{
key: 'anyField',
operator: '>',
value: 'any value',
},
{
key: 'anyField',
operator: '<',
value: 'any value',
},
{
key: 'anyField',
operator: '==',
value: 'any value',
},
{
key: 'anyField',
operator: '==',
value: 'any value',
},
]
Then you can just put the collection which one you want to set query's conditions into this funcion.
function* multipleWhere(
collection,
conditions = [{ field: '[doc].[field name]', operator: '==', value: '[any value]' }],
) {
const pop = conditions.pop()
if (pop) {
yield* multipleWhere(
collection.where(pop.key, pop.operator, pop.value),
conditions,
)
}
yield collection
}
You will get the collection set query's conditions.

async yourFunction(){
const Ref0 = firebase.firestore().collection("your_collection").doc(doc.id)
const Ref1 = appointmentsRef.where('val1', '==',condition1).get();
const Ref2 = appointmentsRef.where("val2", "!=", condition2).get()
const [snapshot_val1, snapshot_val2] = await Promise.all([Ref1, Ref2]);
const val1_Array = snapshot_val1.docs;
const val2_Array = snapshot_val2.docs;
const globale_val_Array = val1_Array .concat(val2_Array );
return globale_val_Array ;
}
/*Call you function*/
this.checkCurrentAppointment().then(docSnapshot=> {
docSnapshot.forEach(doc=> {
console.log("Your data with multiple code query:", doc.data());
});
});

As CollectionRef does not have query method in firebase web version 9,
I modified #abk's answer.
async getQueryResult(path, options = {}) {
/* Example
options = {
where: [
["isPublic", "==", true],
["isDeleted", "==", false]
],
orderBy: [
["likes"],
["title", "desc"]
],
limit: 30
}
*/
try {
let { where, orderBy, limit } = options;
let collectionRef = collection(<firestore>, path);
let queryConstraints = [];
if (where) {
where = where.map((w) => firestore.where(...w));
queryConstraints = [...queryConstraints, ...where];
}
if (orderBy) {
orderBy = orderBy.map((o) => firestore.orderBy(...o));
queryConstraints = [...queryConstraints, ...orderBy];
}
if (limit) {
limit = firestore.limit(limit);
queryConstraints = [...queryConstraints, limit];
}
const query = firestore.query(collectionRef, ...queryConstraints);
const querySnapshot = await firestore.getDocs(query);
const docList = querySnapshot.docs.map((doc) => {
const data = doc.data();
return {
id: doc.id,
...data,
};
});
return docList;
} catch (error) {
console.log(error);
}
}

Simple function where you can specify the path and an array of filters that you can pass and get you documents, hope it helps.
async function filterDoc(path, filters) {
if (!path) return [];
//define the collection path
let q = db.collection(path);
//check if there are any filters and add them to the query
if (filters.length > 0) {
filters.forEach((filter) => {
q = q.where(filter.field, filter.operator, filter.value);
});
}
//get the documents
const snapshot = await q.get();
//loop through the documents
const data = snapshot.docs.map((doc) => doc.data());
//return the data
return data;
}
//call the function
const data = await filterDoc(
"categories_collection",
[
{
field: "status",
operator: "==",
value: "active",
},
{
field: "parent_id",
operator: "==",
value: "kSKpUc3xnKjtpyx8cMJC",
},
]
);

Get all records from collection with all refrences in Firestore

Hi I'm currently blocked because I can't get all records from a collection with references values.
I would like to get all records from collection events (it works) but when I wanna merge the category information associated with categoryId my code doesn't work anymore.
Events collection
Categories collection
export const getEventsRequest = async () => {
const output = [];
const data = await firebase.firestore().collection('events').get();
data.forEach(async (doc) => {
const {
name,
address,
city,
duration,
level,
startDate,
maxPeople,
categoryId,
} = doc.data();
const { name: categoryName, color } = (
await firebase.firestore().collection('categories').doc(categoryId).get()
).data();
output.push({
name,
address,
city,
duration,
level,
startDate,
maxPeople,
category: { name: categoryName, color },
});
});
return output;
};
Example testing in a React Native project
const [events, setEvents] = useState([]);
const [isEventsLoading, setIsEventsLoading] = useState(false);
const getEvents = async () => {
setEvents([]);
setIsEventsLoading(true);
try {
const evts = await getEventsRequest();
setEvents(evts);
setIsEventsLoading(false);
} catch (e) {
console.error(e);
}
};
useEffect(() => {
getEvents();
}, []);
console.log('events', events);
Output
events Array []
Expected
events Array [
{
name : "blabla",
address: "blabla",
city: "blabla",
duration: 60,
level: "hard",
startDate: "13/04/2021",
maxPeople: 7,
category: {
name: "Football",
color: "#fff"
},
},
// ...
]
I don't know if there is a simpler method to retrieve this kind of data (for example there is populate method on mongo DB).
Thank you in advance for your answers.

When you use CollectionReference#get, it returns a Promise containing a QuerySnapshot object. The forEach method on this class is not Promise/async-compatible which is why your code stops working as you expect.
What you can do, is use QuerySnapshot#docs to get an array of the documents in the collection, then create a Promise-returning function that processes each document and then use it with Promise.all to return the array of processed documents.
In it's simplest form, it would look like this:
async function getDocuments() {
const querySnapshot = await firebase.firestore()
.collection("someCollection")
.get();
const promiseArray = querySnapshot.docs
.map(async (doc) => {
/* do some async work */
return doc.data();
});
return Promise.all(promiseArray);
}
Applying it to your code gives:
export const getEventsRequest = async () => {
const querySnapshot = await firebase.firestore()
.collection('events')
.get();
const dataPromiseArray = querySnapshot.docs
.map(async (doc) => {
const {
name,
address,
city,
duration,
level,
startDate,
maxPeople,
categoryId,
} = doc.data();
const { name: categoryName, color } = (
await firebase.firestore().collection('categories').doc(categoryId).get()
).data();
return {
name,
address,
city,
duration,
level,
startDate,
maxPeople,
category: { name: categoryName, color },
};
});
// wait for each promise to complete, returning the output data array
return Promise.all(dataPromiseArray);
};

How i can avoid the loop on my querysnapshot.foreach?

My User has 2 teams :
Danse
Judo
On my subcollection "membersList" the team Danse had 1 friend request and Judo had none.
So I'm suppose to have just one request on my screen. But when I have 2 or more teams, the while continue to loop and the request appear with the numbers of team.
I think the problem is on my querySnaphost.forEach but on my console he return me the doc not empty (so the team danse ) and an other one with document not found.
let fetch = async () => {
firestore()
.collection("Teams")
.where("uid", "==", await AsyncStorage.getItem("userID"))
.get()
.then((querySnapshot) => {
if (querySnapshot.empty) {
console.log("no documents found");
} else {
querySnapshot.forEach((doc) => {
let Teams = doc._data;
console.log(Teams);
updateActivity((arr) => [...arr, Teams]);
console.log(Activity);
doc.ref
.collection("membersList")
.where("statut", "==", "en attente")
.get()
.then((querySnapshot) => {
if (querySnapshot.empty) {
console.log("no documents found cc");
} else {
querySnapshot.forEach((doc) => {
let members = doc._data;
console.log("aa", members);
updateMembersList((arr) => [...arr, members]);
console.log("cc", MembersList);
});
}
});
});
}
});
};
useEffect(() => {
fetch();
}, []);
Here is what is logged when fetch() is called:
{"Activity": "Danse", "Adress": "Plage", "City": "Nice", "Owner": true, "members": "3", "text": "bouh", "tokenTeam": "n5ounxsf2bq", "uid": "PTbEn2fba0QudXI8JE8RioQ9of53"}
[]
no documents found cc

You should not name your function fetch as this is a reserved global function which you should treat like undefined - don't assign anything to it. I recommend also using const over let where applicable.
Note: This answer makes use of the same strategy described in this answer.
const fetchMemberRequests = async () => {
const userTeamsQuerySnapshot = await firestore()
.collection("Teams")
.where("uid", "==", await AsyncStorage.getItem("userID"))
.get();
if (userTeamsQuerySnapshot.empty) {
console.log("User owns no teams.");
// empty Activity & MembersLists
updateActivity([]);
updateMembersList([]);
return;
}
// for each team, fetch the pending requests and return a { team, memberRequests } object
const fetchMemberRequestsPromises = userTeamsQuerySnapshot.docs
.map(async (teamDocSnapshot) => {
const teamData = teamDocSnapshot.data();
const memberRequestsQuerySnapshot = await teamDocSnapshot.ref
.collection("membersList")
.where("statut", "==", "en attente")
.get();
if (memberRequestsQuerySnapshot.empty) {
console.log(`Activity ${teamData.Activity} has no pending requests.`);
return {
team: teamData,
memberRequests: []
}
}
const memberRequestsArray = memberRequestsQuerySnapshot.docs
.map((memberRequestDocSnapshot) => memberRequestDocSnapshot.data());
console.log(`Activity ${teamData.Activity} has ${memberRequestsArray.length} pending requests.`);
return {
team: teamData,
memberRequests: memberRequestsArray
}
});
const memberRequests = await Promise.all(fetchMemberRequestsPromises);
// memberRequests is an array of { team: teamData, memberRequests: arrayOfMembers }
// which could be used to show the requests in groups
// these lines replicate your current code:
const allTeams = [];
const allMemberRequests = [];
for (const request of memberRequests) {
allTeams.push(request.team);
allMemberRequests.push(...request.memberRequests); // note: spreads array
}
console.log(`User owns ${allTeams.length} teams, with a total of ${allMemberRequests.length} pending requests.`);
// replace Activity and MembersList rather than append to them
updateActivity(allTeams);
updateMembersList(allMemberRequests);
}

You are reusing the variable you are looping in (querySnapshot and doc) inside the loop, so I think this is messing up everything. If you loop over querySnapshot, then you should not reuse querySnapshot again inside the loop.
Also, to avoid headaches and callback hells, use await instead of .then( .then( .then( .then()))). You can't use await inside of a method (.filter(), .map(), .forEach() etc) but you can use it inside a for loop :
let fetch = async () => {
const querySnapshot = await firestore()
.collection("Teams")
.where("uid", "==", await AsyncStorage.getItem("userID"))
.get();
if (querySnapshot.empty) {
console.log("no documents found");
return;
}
for( let doc of querySnapshot.docs){
let Teams = doc._data;
console.log(Teams);
updateActivity((arr) => [...arr, Teams]);
console.log(Activity);
let querySnapshot2 = await doc.ref
.collection("membersList")
.where("statut", "==", "en attente")
.get();
if (querySnapshot2.empty) {
console.log("no documents found");
continue;
}
for(let doc2 of querySnapshot2.docs){
let members = doc2._data;
console.log("aa", members);
updateMembersList( arr => [...arr, members]);
console.log("cc", MembersList);
}
}
}

How to insert array into a column in spanner database using javascript es6

I have the following data:
data = {
id: 10012,
name: "abc",
hobby: ["cricket", "football"]
}
My table structure is:
id int64,
name string(25),
petName string(20),
hobby string(25)
How to insert an array of data into the spanner table?
async insert(data) {
const result;
const params = {
id: data.id,
name: data.name,
petName: null
}
await data.hobby.map(async h => {
params.hobby = h;
const query = 'INSERT INTO my_table (id, name, petName, hobby) VALUES (#id, #name, #petName, #hobby)';
result = await db.RunTrans(query, params);
});
return result;
}
The above code doesn't insert the values into the table. Kindly help me with this..

Destructure out the wanted values (to avoid unnecessary repeated dot notation) then use template literals:
async insert(data) {
const result;
const params = {
id: data.id,
name: data.name,
petName: null
}
await data.hobby.map(async h => {
params.hobby = h;
const { id, name, petName, hobby } = params;
const query = `INSERT INTO my_table (id, name, petName, hobby) VALUES (${id}, ${name}, ${petName}, ${hobby})`;
result = await db.RunTrans(query, params);
});
return result;
}

You aren't handling any promise issues, so it is possible a few different things.
I am not certain your columns are nullable. If they aren't petName being NULL isn't valid for the schema.
#google-cloud/spanner doesn't have a RunTrans method. it has a runTransaction and a runTransactionAsync method though.
The way this is authored, the code would result in multiple rows with the same id. Typically id would be set to a primary key. Since I assume this isn't in error, I made a primary key that is the composite of hobby and id as this would be unique.
The documentation for runTransactionAsync is here:
https://cloud.google.com/nodejs/docs/reference/spanner/3.1.x/Database.html#runTransactionAsync
const {Spanner} = require('#google-cloud/spanner');
const spanner = new Spanner();
const instance = spanner.instance('testinstance');
const db = instance.database('people');
async function insert(data) {
const params = {
id: data.id,
name: data.name,
petName: null
}
const query = 'INSERT INTO my_table (id, name, petName, hobby) VALUES (#id, #name, #petName, #hobby)';
await Promise.all(data.hobby.map(async h => {
await db.runTransactionAsync(async (transaction) => {
params.hobby = h;
await transaction.run({
sql: query,
params: params,
types: {
petName: {
type: 'string'
},
}
});
await transaction.commit();
});
}));
};
insert({id: 0, name: 'foo', hobby:['a','b','c'], extra: 'blah'}).catch((r) => console.log(r))