I am completely new to Firebase, maybe I just cannot search for the right keywords, but here's what I'd like to do:
I would like to read a document from the database by id, but only if that document has a specific property set. To be more specific, this would be something like a tiny blog server, where I want to read a post's data, but only if its "published" property is set to true. I don't want unpublished posts to end up on the client side.
I am using the JavaScript SDK, but can only find examples of lookup by either ID or properties...
Is this even possible?
UPDATE:
What I have now is (this is Firestore):
firebase.firestore().collection('blog-posts').doc(id).get()
and then later check for the property on the client side:
if (data && data.published === "true")
I would like all the filtering to happen on the server side.
If you want to get a document only if a field has a certain value, you won't be able to use get() on its DocumentReference. You will have to perform a query on the collection using both the document ID and field as filters on that query like this, using FieldPath.documentId():
const qsnap = await firebase.firestore()
.collection('blog-posts')
.where(firebase.firestore.FieldPath.documentId(), "==", id)
.where("published", "==" true)
.get()
This yields a QuerySnapshot, which you will have to check if the document was provided.
if (qsnap.docs.length > 0) {
const doc = qsnap.docs[0];
}
Note that it still costs 1 document read to perform this query even if the document isn't returned.
You can use where function to filter data on the server-side.
firebase.firestore().collection('blog-posts').where('published', '==', 'true').where(firebase.firestore.FieldPath.documentId(), '==', id).get()
Firestore does not easily let you filter/select on documentID; among other issues, FieldPath.documentId() returns the FULL PATH of the document, NOT just the id; see this question/answer for a fairly lengthy explanation of what is happening:
Firestore collection group query on documentId
it is possible to also copy the Id into a field in the document, and query by that (and I have my Firestore wrapper automatically helping with that) - BUT - this quickly become what I've seen called an "X-Y" question - asking "How do I do Y?" only because you already assume it's the answer to "How do I do X?"
The question is why do you want to query by the document ID? Is this ID a proxy for some other information (such as a way to identify the user)? If so, use the User ID as a field. In general, the documentId is more about Google Firestore efficiently sharding their database for performance than as a real identifier. Since the documentId is assumed to be unique (or, as the answer referenced explains, the full path to the document is unique), it's most often less useful for finding.
An analogy: using a documentId to find a document is like using a Municipal Parcel Number to designate an address. Sure it's unique, but "Division 46675 Tract 32567 Parcel 12344" is a lot less useful than "450 Lazy Deer Road, Winnetka, New Jersey".
So: in your case, ask yourself: where did you get that documentId from? What does it represent? Can I store that in my document instead so I can use a compound query?
firebase.firestore().collection('blog-posts').where([whatever query identifies the document or documents]).where('published', '==', 'true').get()
Note whether your first condition identifies one or identifies multiple documents, the response object will include an ARRAY, which may have one or more documents in it.
The best general advice for Firestore and other NoSQL databases is to spend the time identifying exactly how you want to use your data - i.e. your queries - and build your structure to make that easier. Remember, Firestore should be a tool to make your life easier, not to make you a slave to Firestore.
Related
I would like to create two queries, with pagination option. On the first one I would like to get the first ten records and the second one I would like to get the other all records:
.startAt(0)
.limit(10)
.startAt(9)
.limit(null)
Can anyone confirm that above code is correct for both condition?
Firestore does not support index or offset based pagination. Your query will not work with these values.
Please read the documentation on pagination carefully. Pagination requires that you provide a document reference (or field values in that document) that defines the next page to query. This means that your pagination will typically start at the beginning of the query results, then progress through them using the last document you see in the prior page.
From CollectionReference:
offset(offset) → {Query}
Specifies the offset of the returned results.
As Doug mentioned, Firestore does not support Index/offset - BUT you can get similar effects using combinations of what it does support.
Firestore has it's own internal sort order (usually the document.id), but any query can be sorted .orderBy(), and the first document will be relative to that sorting - only an orderBy() query has a real concept of a "0" position.
Firestore also allows you to limit the number of documents returned .limit(n)
.endAt(), .endBefore(), .startAt(), .startBefore() all need either an object of the same fields as the orderBy, or a DocumentSnapshot - NOT an index
what I would do is create a Query:
const MyOrderedQuery = FirebaseInstance.collection().orderBy()
Then first execute
MyOrderedQuery.limit(n).get()
or
MyOrderedQuery.limit(n).get().onSnapshot()
which will return one way or the other a QuerySnapshot, which will contain an array of the DocumentSnapshots. Let's save that array
let ArrayOfDocumentSnapshots = QuerySnapshot.docs;
Warning Will Robinson! javascript settings is usually by reference,
and even with spread operator pretty shallow - make sure your code actually
copies the full deep structure or that the reference is kept around!
Then to get the "rest" of the documents as you ask above, I would do:
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).get()
or
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).onSnapshot()
which will start AFTER the last returned document snapshot of the FIRST query. Note the re-use of the MyOrderedQuery
You can get something like a "pagination" by saving the ordered Query as above, then repeatedly use the returned Snapshot and the original query
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).limit(n).get() // page forward
MyOrderedQuery.endBefore(ArrayOfDocumentSnapshots[0]).limit(n).get() // page back
This does make your state management more complex - you have to hold onto the ordered Query, and the last returned QuerySnapshot - but hey, now you're paginating.
BIG NOTE
This is not terribly efficient - setting up a listener is fairly "expensive" for Firestore, so you don't want to do it often. Depending on your document size(s), you may want to "listen" to larger sections of your collections, and handle more of the paging locally (Redux or whatever) - Firestore Documentation indicates you want your listeners around at least 30 seconds for efficiency. For some applications, even pages of 10 can be efficient; for others you may need 500 or more stored locally and paged in smaller chucks.
In my scenario a user can "like" the profile of another user. As a result I have a subcollection called "likedBy" for each user, where I create a document for a specific user, e.g.:
users(col) -> User A(doc) -> likedBy(col) -> User B (doc), User C (doc)
So in the rare scenario of a user being deleted, I want to delete all the likes issues by that user.
I am just not sure if this is even possible (a workaround could be to just save the userId again in said document and query for that).
What I am basically looking for is something like this:
db.collectionGroup('likedBy').where(firebase.firestore.FieldPath.documentId(), '==', "User A").get();
The problem is, that I can not create an index for the documentId in the Firestore console.
UPDATE 2020-01-23:
For updates on this, see a conversation with Sam Stern on the group board:https://groups.google.com/d/msgid/google-cloud-firestore-discuss/e1b47358-b106-43a0-91fb-83c97d6244de%40googlegroups.com
Much of the discussion comes from the apparent fact that there is a SINGLE "master index" of ALL records in a database based on the FULLY QUALIFIED path to the document (hence only needing to be unique "within a collection").
To "accelerate" document references, the JS SDK actually "pre-pends" the collection path onto whatever info is passed in .doc to "conveniently" use that master index
i.e. all of these are exactly equivalent
db.doc('collection/docId/collection/docId/collection/docId")
db.collection('collection").doc("docId/collection/docId/collection/docId")
db.collection('collection").doc("docId").collection("collection").doc("docId/collection/docId")
db.collection('collection").doc("docId").collection("collection").doc("docId").collection("collection").doc("docId")
db.doc("collection/docId").collection("collection").doc("docId").collection("collection").doc("docId")
db.collection("collection/docId/collection").doc("docId").collection("collection").doc("docId")
db.doc("collection/docId/collection/docId").collection("collection").doc("docId")
db.collection("collection/docId/collection/docId/collection").doc("docId")
--- they ALL create the same index reference 'collection/docId/collection/docId/collection/docId" to find the document in the "master index".
(in my not-at-all-humble-opinion) FieldPath.documentId() was implemented (incorrectly) to "conveniently" match this behavior thus requiring the fully-qualified path, not the docId, when it should have been implemented like any other query, and required creating and maintaining a NEW INDEX to accommodate the query.
The code for this behavior was written BEFORE collectionGroups were implemented - and never documented the hack used didn't match the METHOD NAME used.
The work around is to require the Coder to copy the docId as a field in each document, and write your queries on that. I already wrote my own layer between Firestore.js and my application to abstract the behavior, and will probably simply implement this as a basic feature of the library.
But this is clearly a MISTAKE, and so far everybody keeps trying to tell me it makes sense, and that they'll change the documentation to match the existing behavior (but not the method name).
As I wrote previously, I keep getting handed a ratty bunch of daisies, and being told "See? these are roses!! The documentation calls them roses!! Roses by any other name smell as sweet, and all that!!"
No Update Expected Unless They Get Embarrassed Enough
UPDATE 2020-01-10: I have built a demo app showing the exact bug, and have sent it to Firebase support as requested. For some dang reason, the support critter considers it a "feature request", in spite of it clearly a bug. When a URL is called in the form "/ShowInfo/showID", the App signs in to Firebase Auth anonymously; then calls a query on the collectionGroup (3 levels deep) using FieldPath.documentId() "==" showID
It makes the query 3 ways:
1) Once with only the showID- which fails with the familiar "Invalid query. When querying a collection group by FieldPath.documentId(), the value provided must result in a valid document path, but 'pqIPV5I7UWne9QjQMm72'(the actual showID) is not because it has an odd number of segments (1)."
2) Once with a "Relative Path" (/Shows/showID), which doesn't have the error, but returns no document.
3) Finally with the "Full Path" (/Artists/ArtistID/Tour/tourID/Shows/showID). This doesn't have an error, and does return a document - but if I have the full path, why do I need the query on the collectionGroup? And I don't have the full path - the showID (above) comes in as part of a URL (a link to the show data, obviously) - I hand-faked it for the test.
Waiting for response.
UPDATE 2019-12-02: Firebase support reached out to ask if I still wanted this solved. Duh.
UPDATE 2019-09-27: Firebase Support has acknowledged this is a bug. No word on when it will be fixed. documentId() should, indeed, be able to be used directly against only the document Id.
documentID can be used as part of a query but, as #greg-ennis notes above, it needs an even number of segments. As it turns out, if you truly need a collectionGroup (I do), then the "Id" to be compared needs to redundantly add the collectionGroup ID/name as a segment:
db.collectionGroup('likedBy')
.where(firebase.firestore.FieldPath.documentId(), '==', "likedBy" + "/" + "User A")
.get();
I've used it and it (*sorta) works. (admittedly, it took me 2 hours to figure it out, and the above question helped focus the search)
On the other hand, this specific case is not where you want to use collectionGroup. Remember what collection group is - a way to refer to a a set of separate collections as if they were one. In this case the "collection" that holds "User A"s likes exists as a collection under "User A"s doc. Simply delete that single collection before deleting "User A"s doc. No need to bring everybody else's likes into it.
Sorta: the field path compared apparently has to be the complete path to the document. If you know the documentId, but for "reasons" you do not know the complete path, including which document the sub-collection is a part of (kinda why you were using the collectionGroup approach in the first place), this now seems a tadly dead-end. Continuing working on it.
Verified and Bug Report filed: FieldPath.documentID() does not compare against the documentId; it compares against the fully segmented document path, exactly as you would have to provide to .doc(path):
To wit: to find a document at "TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want"
using the collectionGroup "Travesty"
db.collectionGroup("Travesty")
.where(firebase.firestore.FieldPath.documentId(), '==', "document_id_I_want")
...will fail, but...
db.collectionGroup("Travesty")
.where(firebase.firestore.FieldPath.documentId(), '==', "TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want")
.get()
...will succeed. Which makes this useless, since if we had all that info, we would just use:
db.doc("TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want")
.get()
There is no way you can use the following query:
db.collectionGroup('likedBy').where(firebase.firestore.FieldPath.documentId(), '==', "User A").get();
And this is because collection group queries work only on document properties and not on document ids. According to the official documentation regarding collection group queries:
db.collectionGroup('landmarks').where('type', '==', 'museum');
You query the landmarks subcollection where the type property holds the value of museum.
A workaround could be to store the id of the user in an array and use array-contains but remember, for each collection group query you use, you need an index and unfortunately you cannot create such an index programmatically. Even if you can create an index in the Firebase console, it won't help you since you get those ids dynamically. So is not an option to create an index for each user separately because you'll reach the maximim number of indexes very quickly.
Maximum number of composite indexes for a database: 200
To solve this kind of problems, you should consider adding an array under each user object and use a query like this:
usersRef.where("usersWhoLikedMe", "array-contains", "someUserId")
Where usersWhoLikedMe is a property of type array.
If you add User A and B ids to the doc itself:
users(col) -> User A(doc) -> likedBy(col) -> User B ({user_id: B, profile_liked_id: A})
Then you can query using:
db.collectionGroup('likedBy').where('user_id', '==', 'B');
if the reference is a collection, the value you compare to needs to be document id (the last segment of a full path)
this is a error message if you violated the rule
Invalid query. When querying a collection by documentId(), you must
provide a plain document ID, but 'a/b/c' contains a '/' character
if the reference is a collectonGroup, the value you compare to needs to be a full document path <-- which is very redundant in my opinion
Invalid query. When querying a collection group by documentId(), the
value provided must result in a valid document path, but
'a/b/c' is not because it has an odd number of
segments (3)
tested recently
I am building a list with pagination using data coming from the firestore.
I have read the docs at https://firebase.google.com/docs/firestore/query-data/query-cursors and it's stated that if you want to use the pagination you should use startAt with a document reference.
However, my list as objects it's not using document references, because my data are not firestore documents They are processed as in most apps and all db metadata are stripped out. Such as the document references.
So in order to find the last object in my list I need todo a get call to the firestore to get the document reference.
Is there any other way I can use startAt without having to pull (get) a complete document from the firestore , since I do have the ID of the referenced doc I need?
PS. I would not like to store any DB(firestore) data on my objects/classes. They are not tight coupled to the DB as they shouldn't.
The documentation for startAt says:
startAt
startAt(snapshotOrVarArgs) returns firebase.firestore.Query
Creates a new query where the results start at the provided document (inclusive). The starting position is relative to the order of the query. The document must contain all of the fields provided in the orderBy of the query.
Parameter
snapshotOrVarArgs (non-null firebase.firestore.DocumentSnapshot or repeatable any type)
The snapshot of the document you want the query to start at or the field values to start this query at, in order of the query's order by.
So you can either pass in a DocumentSnapshot or a value for each field that you're filtering/ordering on. Just having the document ID is not enough, unless your query is only filtered/ordered by document ID.
How does one query for documents where the test is for a missing key in the document data? I'd like to catch cases where the key is absent primarily, and -- if there's a way to store undefined or null in a doc (and I'm not sure there is) -- I'd like to catch those too.
Shouldn't it be one of these...
let query = db.collection('myCollection').where('someProp','==',null);
let query = db.collection('myCollection').where('someProp','==',undefined);
But in my test with these, neither of these are finding the doc with the missing key. I'm new to firestore, so maybe I've got something else wrong with the queries.
I can't find it in the docs, but maybe special object in firestore that means "not there"?
Cloud Firestore relies upon indexes to answer queries and cannot answer any query without an index.
When a field doesn't exist, there can't be index entry for it, so there isn't a way to query for it.
Workaround #1
Query for all documents in the collection and check the returned document. Note, this can get more expensive for large collections as it results in your reading all documents
Workaround #2
If it is planned, you can have a field called schema_version. Whenever you add a new field, increment the value stored in this field (say from 12 to 13). Now if you know a field didn't exist under schema_version == 13, you can simply query for every document with a version < 13.
What's the fastest way to check for the existence of a mongodb doc?
Should I just use find and if it returns nothing?
EDIT:
collection.findOne {#attribute}, (err, doc) ->
if err then console.log err
if interaction
#exists
else
#does not
If you are just testing for a single document, use findOne (or the equivalent in your driver); most drivers implement this in the most efficient possible way (by setting a negative limit of 1 on the request, which asks mongo to return immediately after finding one document, even if more might match, and not to create a cursor that won't ever be used by the client).
If you have an index that can serve your query, you can use field selection to select (a subset of) the fields in the index; this will make use of Mongo's "covered index" functionality to avoid a lookup to the underlying collection data. Be sure to set {_id: 0} in your field selector unless _id is in your index.
Covered Index is what you want. If the info you know about the document is indexed, then you can use this facility to query and retrieve info from the index only (in RAM) and will not go to disk to get the reference document. Explained in the mongo docs here.
http://www.mongodb.org/display/DOCS/mongo%20wire%20protocol#MongoWireProtocol-OPQUERY
numberToReturn : Limits the number of documents in the first CONTRIB:OP_REPLY message to the query. However, the database will still establish a cursor and return the cursorID to the client if there are more results than numberToReturn. If the client driver offers 'limit' functionality (like the SQL LIMIT keyword), then it is up to the client driver to ensure that no more than the specified number of document are returned to the calling application. If numberToReturn is 0, the db will used the default return size. If the number is negative, then the database will return that number and close the cursor. No futher results for that query can be fetched. If numberToReturn is 1 the server will treat it as -1 (closing the cursor automatically).