Firestore - Get pagination cursor from denormalized data - javascript

I have a paginated list of posts, where my cursor is re-calculated on each fetch, assigning the last QuerySnapshot document to it:
// Calculate the new startAfter
if (querySnapshot.size) {
startAfter = querySnapshot.docs[querySnapshot.docs.length - 1];
}
Somehow, using doc snapshots as cursors is one of the most "secure" ways to avoid breaking paginations.
I am thinking about optimizing some parts of my app with denormalizations. For example, as each user of my app can upload infinite posts that are shown on their profiles 10 by 10, it might be a good idea to add the last 10 user posts (denormalization) to the user's document (an array field).
With this, I avoid making the first 10 post fetch of each visited profile. But... how do I assign the first startAfter cursor? I mean, is it possible to write a doc snapshot in a document or something like that to get the same behavior?

If you look at the documentation for startAfter, you'll see there are two overloads:
startAfter ( snapshot : DocumentSnapshot < any > ) : Query < T >
Creates and returns a new Query that starts after the provided document (exclusive). The starting position is relative to the order of the query. The document must contain all of the fields provided in the orderBy of this query.
Parameters
snapshot: DocumentSnapshot<any>
The snapshot of the document to start after.
And
startAfter ( ... fieldValues : any [] ) : Query < T >
Creates and returns a new Query that starts after the provided fields relative to the order of the query. The order of the field values must match the order of the order by clauses of the query.
Parameters
Rest ...fieldValues: any[]
The field values to start this query after, in order of the query's order by.
Since you don't have a DocumentSnapshot, you can use the second overload and pass the relevant field values.

Related

react-native-firebase/firestore: query limit from result x to y [duplicate]

I would like to create two queries, with pagination option. On the first one I would like to get the first ten records and the second one I would like to get the other all records:
.startAt(0)
.limit(10)
.startAt(9)
.limit(null)
Can anyone confirm that above code is correct for both condition?
Firestore does not support index or offset based pagination. Your query will not work with these values.
Please read the documentation on pagination carefully. Pagination requires that you provide a document reference (or field values in that document) that defines the next page to query. This means that your pagination will typically start at the beginning of the query results, then progress through them using the last document you see in the prior page.
From CollectionReference:
offset(offset) → {Query}
Specifies the offset of the returned results.
As Doug mentioned, Firestore does not support Index/offset - BUT you can get similar effects using combinations of what it does support.
Firestore has it's own internal sort order (usually the document.id), but any query can be sorted .orderBy(), and the first document will be relative to that sorting - only an orderBy() query has a real concept of a "0" position.
Firestore also allows you to limit the number of documents returned .limit(n)
.endAt(), .endBefore(), .startAt(), .startBefore() all need either an object of the same fields as the orderBy, or a DocumentSnapshot - NOT an index
what I would do is create a Query:
const MyOrderedQuery = FirebaseInstance.collection().orderBy()
Then first execute
MyOrderedQuery.limit(n).get()
or
MyOrderedQuery.limit(n).get().onSnapshot()
which will return one way or the other a QuerySnapshot, which will contain an array of the DocumentSnapshots. Let's save that array
let ArrayOfDocumentSnapshots = QuerySnapshot.docs;
Warning Will Robinson! javascript settings is usually by reference,
and even with spread operator pretty shallow - make sure your code actually
copies the full deep structure or that the reference is kept around!
Then to get the "rest" of the documents as you ask above, I would do:
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).get()
or
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).onSnapshot()
which will start AFTER the last returned document snapshot of the FIRST query. Note the re-use of the MyOrderedQuery
You can get something like a "pagination" by saving the ordered Query as above, then repeatedly use the returned Snapshot and the original query
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).limit(n).get() // page forward
MyOrderedQuery.endBefore(ArrayOfDocumentSnapshots[0]).limit(n).get() // page back
This does make your state management more complex - you have to hold onto the ordered Query, and the last returned QuerySnapshot - but hey, now you're paginating.
BIG NOTE
This is not terribly efficient - setting up a listener is fairly "expensive" for Firestore, so you don't want to do it often. Depending on your document size(s), you may want to "listen" to larger sections of your collections, and handle more of the paging locally (Redux or whatever) - Firestore Documentation indicates you want your listeners around at least 30 seconds for efficiency. For some applications, even pages of 10 can be efficient; for others you may need 500 or more stored locally and paged in smaller chucks.

Ignoring DocID in Firestore

I have a simple Query which is there to recommend Users under a Post related Posts.
await this.$fireStore.collection('posts')
.limit(6).where('tags', 'array-contains', this.post.tags[0]).get().then(querySnapshot => {
Problem is that this also Reads out the Post itself (since it contains the Tag too), is there any way to query for Posts that dont have a specific DocID without wasting a Where ergo needing a complete new Query?
Firestore doesn't provide the possibility to query with inequality (i.e. not equals), see for example this SO post: Firestore: how to perform a query with inequality / not equals
I understand that you want to get a maximum of 6 "sibling" posts. You could make the same query, with a limit of 7 docs and, in your front-end, filter the result:
By looping over the list of docs and compare their Id to the current doc Id you will encounter two cases:
The current doc is within the 7 docs returned by the query: remove it from the list
The current doc is NOT within the 7 docs returned by the query: remove one doc randomly.

Why does this firestore query require an index?

I have a query with a where() method with an equality operator and then an orderBy() method and I can't figure out why it requires an index.
The where method checks for a value in an object (a map) and the order by is with a number.
The documentation says
If you have a filter with a range comparison (<, <=, >, >=), your first ordering must be on the same field
So I would have thought that an equality filter would be fine.
Here is my query code:
this.afs.collection('posts').ref
.where('tags.' + this.courseID,'==',true)
.orderBy("votes")
.limit(5)
.get().then(snap => {
snap.forEach(doc => {
console.log(doc.data());
});
});
Here is an example of the database structure
Why does this Firestore query require an index?
As you probably noticed, queries in Cloud Firestore are very fast and this is because Firestore automatically creates an index for any field you have in your document. So when you simply filter with a range comparison, Firestore creates the required index automatically. If you also try to order your results, another index is required. This kind of index is not created automatically. You should create it yourself. This can be done, by creating it manually in your Firebase Console or you'll find in your logs a message that sounds like this:
FAILED_PRECONDITION: The query requires an index. You can create it here: ...
You can simply click on that link or copy and paste the URL into a web browser and your index will be created automatically.
So Firestore requires an index so you can have very fast queries.
An index is simply a database inventory or a record of what is where. And each index is a specific inventory of a specific thing—for example, how many propertyX fields exist in a collection and what their values are, sorted (the fact that they are sorted is critical).
If this inventory didn't exist, to query for documents where propertyX is someValue, the machine would have to iterate over the entire collection to determine (1) which documents contain propertyX and (2) which documents contain propertyX equal to someValue. By keeping an inventory (or index) of queried properties, when a query is performed on propertyX, the machine can go straight to the propertyX index and gather the locations of all the documents that equal someValue and then fetch those documents from the collection and return them. Not only does the machine not need to touch the collection to know where the documents are but it doesn't even need to iterate over the entire index because it's always in order.
Indexes are why collection sizes have no impact on the performance of Firestore queries and why we only need to index properties that are ever queried.

Test for unset values in firebase firestore

How does one query for documents where the test is for a missing key in the document data? I'd like to catch cases where the key is absent primarily, and -- if there's a way to store undefined or null in a doc (and I'm not sure there is) -- I'd like to catch those too.
Shouldn't it be one of these...
let query = db.collection('myCollection').where('someProp','==',null);
let query = db.collection('myCollection').where('someProp','==',undefined);
But in my test with these, neither of these are finding the doc with the missing key. I'm new to firestore, so maybe I've got something else wrong with the queries.
I can't find it in the docs, but maybe special object in firestore that means "not there"?
Cloud Firestore relies upon indexes to answer queries and cannot answer any query without an index.
When a field doesn't exist, there can't be index entry for it, so there isn't a way to query for it.
Workaround #1
Query for all documents in the collection and check the returned document. Note, this can get more expensive for large collections as it results in your reading all documents
Workaround #2
If it is planned, you can have a field called schema_version. Whenever you add a new field, increment the value stored in this field (say from 12 to 13). Now if you know a field didn't exist under schema_version == 13, you can simply query for every document with a version < 13.

How to perform dynamic where queries with firestore and add indexes

In my site i am conducting a survey like tests, each test has attendies sub collection look like this
When someone finishes a test i also add their uid to completed field like i drawn in the box. Now i want to query tests based on status == completed.
Here is what i tried
this.completedModulesRef$ = this.afs.collection('tests', ref =>
ref.orderBy('moduleNum', 'desc')
.where('completed.'+auth.uid+'.status','==','completed'));
this.completedModules$ = this.completedModulesRef$.valueChanges();
Then firestore asked me to add indexes, when i follow the generated link to add indexes i got this
which is pointing to completed.CurrentUserId.status. I believe this only work for current user.
I have few question
1) .where('completed.'+auth.uid+'.status','==','completed') Is this a valid query?
2) If yes how can i index it?
3) Any way to query the top collections based on sub collection value?. (this is what i really want)
Any help appreciated.
You could just instead use an array of objects (each object represents one attendie) to track all the completed attendies. Arrays are indexed in firestore.
The data structure of the array would be:
compeleted: [{}] = [
{id: string, points: number, status: string}
]
This might not be the most optimal database model depending on how and where you want to fetch the data but it will be indexed and you will be able to query it. I would consider storing the points and status in the subcollection of attendies. Have a look at grouped collection queries in firestore - new feature where you can query across all attendies subcollections at once - for any attendie with a certain id and a status of complete if you want to fetch all the tests that one attendie completed.

Categories