CouchDB docs seem to have a key attached; it does not show up when retrieving a single document but you can use them to retrieve ranges of documents such as :
wget "http://localhost:5984/monitor20n/_all_docs?startkey=1111&endkey=2222
However, apparently that key is always the same as the document id, so that all you obtain is stuff like this
{"total_rows":14269,"offset":0,"rows":[
{"id":"128127896626798592","key":"128127896626798592","value":{"rev":"1-4e07e2c4b4eddfad5846ddf905337197"}},
{"id":"128128575021907970","key":"128128575021907970","value":{"rev":"1-43d983af1e837a4415b6167cae3b5de8"}},
... and so on }}
(see here key == id ). However, you can use more complex keys in views, including vectors which allow for much more complex interaction; at least, you can set the keys of views so you can now in advance what to search without looking up document ids.
The question is now: Can you set those keys up when creating a document? Or maybe after creating it?
An obvious workaround is to create a view like this
function (doc) {
emit(doc.key,doc)
}
however, I would like to know if there's a more direct way of obtaining the same effect.
Keys are an important part of CouchDB views. With a view, the key does not have to be the document ID. But the only way to produce a key is to use the emit function from within a view. There is no property that you can set that will automatically become the key.
Think of _all_docs like a built in view. To be consistent it follows the same output as a regular view, and it uses the id as the key. But you can't change the _all_docs view. If you wanted to provide your own _id when you save a document, that will end up being the key.
So if you wanted custom 'keys' in the '_all_docs' view you could create docs like this:
{ _id: 'Sample1' }, {_id: 'My2'}. and after they are saved, when you request the '_all_docs' view you would get:
{"total_rows":2,"offset":0,"rows":[
{"id":"Sample1","key":"Sample1","value":{"rev":"1-4e07e2c4b4eddfad5846ddf905337197"}},
{"id":"My2","key":"My2","value":{"rev":"1-43d983af1e837a4415b6167cae3b5de8"}},
... and so on }}
Here is a link about what makes a documentID:
http://wiki.apache.org/couchdb/HTTP_Document_API#Special_Fields
While it does not say explicitly, you can't use objects or arrays as DocumentIDs.
Hope that helps.
Related
I am completely new to Firebase, maybe I just cannot search for the right keywords, but here's what I'd like to do:
I would like to read a document from the database by id, but only if that document has a specific property set. To be more specific, this would be something like a tiny blog server, where I want to read a post's data, but only if its "published" property is set to true. I don't want unpublished posts to end up on the client side.
I am using the JavaScript SDK, but can only find examples of lookup by either ID or properties...
Is this even possible?
UPDATE:
What I have now is (this is Firestore):
firebase.firestore().collection('blog-posts').doc(id).get()
and then later check for the property on the client side:
if (data && data.published === "true")
I would like all the filtering to happen on the server side.
If you want to get a document only if a field has a certain value, you won't be able to use get() on its DocumentReference. You will have to perform a query on the collection using both the document ID and field as filters on that query like this, using FieldPath.documentId():
const qsnap = await firebase.firestore()
.collection('blog-posts')
.where(firebase.firestore.FieldPath.documentId(), "==", id)
.where("published", "==" true)
.get()
This yields a QuerySnapshot, which you will have to check if the document was provided.
if (qsnap.docs.length > 0) {
const doc = qsnap.docs[0];
}
Note that it still costs 1 document read to perform this query even if the document isn't returned.
You can use where function to filter data on the server-side.
firebase.firestore().collection('blog-posts').where('published', '==', 'true').where(firebase.firestore.FieldPath.documentId(), '==', id).get()
Firestore does not easily let you filter/select on documentID; among other issues, FieldPath.documentId() returns the FULL PATH of the document, NOT just the id; see this question/answer for a fairly lengthy explanation of what is happening:
Firestore collection group query on documentId
it is possible to also copy the Id into a field in the document, and query by that (and I have my Firestore wrapper automatically helping with that) - BUT - this quickly become what I've seen called an "X-Y" question - asking "How do I do Y?" only because you already assume it's the answer to "How do I do X?"
The question is why do you want to query by the document ID? Is this ID a proxy for some other information (such as a way to identify the user)? If so, use the User ID as a field. In general, the documentId is more about Google Firestore efficiently sharding their database for performance than as a real identifier. Since the documentId is assumed to be unique (or, as the answer referenced explains, the full path to the document is unique), it's most often less useful for finding.
An analogy: using a documentId to find a document is like using a Municipal Parcel Number to designate an address. Sure it's unique, but "Division 46675 Tract 32567 Parcel 12344" is a lot less useful than "450 Lazy Deer Road, Winnetka, New Jersey".
So: in your case, ask yourself: where did you get that documentId from? What does it represent? Can I store that in my document instead so I can use a compound query?
firebase.firestore().collection('blog-posts').where([whatever query identifies the document or documents]).where('published', '==', 'true').get()
Note whether your first condition identifies one or identifies multiple documents, the response object will include an ARRAY, which may have one or more documents in it.
The best general advice for Firestore and other NoSQL databases is to spend the time identifying exactly how you want to use your data - i.e. your queries - and build your structure to make that easier. Remember, Firestore should be a tool to make your life easier, not to make you a slave to Firestore.
In my scenario a user can "like" the profile of another user. As a result I have a subcollection called "likedBy" for each user, where I create a document for a specific user, e.g.:
users(col) -> User A(doc) -> likedBy(col) -> User B (doc), User C (doc)
So in the rare scenario of a user being deleted, I want to delete all the likes issues by that user.
I am just not sure if this is even possible (a workaround could be to just save the userId again in said document and query for that).
What I am basically looking for is something like this:
db.collectionGroup('likedBy').where(firebase.firestore.FieldPath.documentId(), '==', "User A").get();
The problem is, that I can not create an index for the documentId in the Firestore console.
UPDATE 2020-01-23:
For updates on this, see a conversation with Sam Stern on the group board:https://groups.google.com/d/msgid/google-cloud-firestore-discuss/e1b47358-b106-43a0-91fb-83c97d6244de%40googlegroups.com
Much of the discussion comes from the apparent fact that there is a SINGLE "master index" of ALL records in a database based on the FULLY QUALIFIED path to the document (hence only needing to be unique "within a collection").
To "accelerate" document references, the JS SDK actually "pre-pends" the collection path onto whatever info is passed in .doc to "conveniently" use that master index
i.e. all of these are exactly equivalent
db.doc('collection/docId/collection/docId/collection/docId")
db.collection('collection").doc("docId/collection/docId/collection/docId")
db.collection('collection").doc("docId").collection("collection").doc("docId/collection/docId")
db.collection('collection").doc("docId").collection("collection").doc("docId").collection("collection").doc("docId")
db.doc("collection/docId").collection("collection").doc("docId").collection("collection").doc("docId")
db.collection("collection/docId/collection").doc("docId").collection("collection").doc("docId")
db.doc("collection/docId/collection/docId").collection("collection").doc("docId")
db.collection("collection/docId/collection/docId/collection").doc("docId")
--- they ALL create the same index reference 'collection/docId/collection/docId/collection/docId" to find the document in the "master index".
(in my not-at-all-humble-opinion) FieldPath.documentId() was implemented (incorrectly) to "conveniently" match this behavior thus requiring the fully-qualified path, not the docId, when it should have been implemented like any other query, and required creating and maintaining a NEW INDEX to accommodate the query.
The code for this behavior was written BEFORE collectionGroups were implemented - and never documented the hack used didn't match the METHOD NAME used.
The work around is to require the Coder to copy the docId as a field in each document, and write your queries on that. I already wrote my own layer between Firestore.js and my application to abstract the behavior, and will probably simply implement this as a basic feature of the library.
But this is clearly a MISTAKE, and so far everybody keeps trying to tell me it makes sense, and that they'll change the documentation to match the existing behavior (but not the method name).
As I wrote previously, I keep getting handed a ratty bunch of daisies, and being told "See? these are roses!! The documentation calls them roses!! Roses by any other name smell as sweet, and all that!!"
No Update Expected Unless They Get Embarrassed Enough
UPDATE 2020-01-10: I have built a demo app showing the exact bug, and have sent it to Firebase support as requested. For some dang reason, the support critter considers it a "feature request", in spite of it clearly a bug. When a URL is called in the form "/ShowInfo/showID", the App signs in to Firebase Auth anonymously; then calls a query on the collectionGroup (3 levels deep) using FieldPath.documentId() "==" showID
It makes the query 3 ways:
1) Once with only the showID- which fails with the familiar "Invalid query. When querying a collection group by FieldPath.documentId(), the value provided must result in a valid document path, but 'pqIPV5I7UWne9QjQMm72'(the actual showID) is not because it has an odd number of segments (1)."
2) Once with a "Relative Path" (/Shows/showID), which doesn't have the error, but returns no document.
3) Finally with the "Full Path" (/Artists/ArtistID/Tour/tourID/Shows/showID). This doesn't have an error, and does return a document - but if I have the full path, why do I need the query on the collectionGroup? And I don't have the full path - the showID (above) comes in as part of a URL (a link to the show data, obviously) - I hand-faked it for the test.
Waiting for response.
UPDATE 2019-12-02: Firebase support reached out to ask if I still wanted this solved. Duh.
UPDATE 2019-09-27: Firebase Support has acknowledged this is a bug. No word on when it will be fixed. documentId() should, indeed, be able to be used directly against only the document Id.
documentID can be used as part of a query but, as #greg-ennis notes above, it needs an even number of segments. As it turns out, if you truly need a collectionGroup (I do), then the "Id" to be compared needs to redundantly add the collectionGroup ID/name as a segment:
db.collectionGroup('likedBy')
.where(firebase.firestore.FieldPath.documentId(), '==', "likedBy" + "/" + "User A")
.get();
I've used it and it (*sorta) works. (admittedly, it took me 2 hours to figure it out, and the above question helped focus the search)
On the other hand, this specific case is not where you want to use collectionGroup. Remember what collection group is - a way to refer to a a set of separate collections as if they were one. In this case the "collection" that holds "User A"s likes exists as a collection under "User A"s doc. Simply delete that single collection before deleting "User A"s doc. No need to bring everybody else's likes into it.
Sorta: the field path compared apparently has to be the complete path to the document. If you know the documentId, but for "reasons" you do not know the complete path, including which document the sub-collection is a part of (kinda why you were using the collectionGroup approach in the first place), this now seems a tadly dead-end. Continuing working on it.
Verified and Bug Report filed: FieldPath.documentID() does not compare against the documentId; it compares against the fully segmented document path, exactly as you would have to provide to .doc(path):
To wit: to find a document at "TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want"
using the collectionGroup "Travesty"
db.collectionGroup("Travesty")
.where(firebase.firestore.FieldPath.documentId(), '==', "document_id_I_want")
...will fail, but...
db.collectionGroup("Travesty")
.where(firebase.firestore.FieldPath.documentId(), '==', "TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want")
.get()
...will succeed. Which makes this useless, since if we had all that info, we would just use:
db.doc("TopCollection/document_this/NextCollection/document_that/Travesty/document_Id_I_want")
.get()
There is no way you can use the following query:
db.collectionGroup('likedBy').where(firebase.firestore.FieldPath.documentId(), '==', "User A").get();
And this is because collection group queries work only on document properties and not on document ids. According to the official documentation regarding collection group queries:
db.collectionGroup('landmarks').where('type', '==', 'museum');
You query the landmarks subcollection where the type property holds the value of museum.
A workaround could be to store the id of the user in an array and use array-contains but remember, for each collection group query you use, you need an index and unfortunately you cannot create such an index programmatically. Even if you can create an index in the Firebase console, it won't help you since you get those ids dynamically. So is not an option to create an index for each user separately because you'll reach the maximim number of indexes very quickly.
Maximum number of composite indexes for a database: 200
To solve this kind of problems, you should consider adding an array under each user object and use a query like this:
usersRef.where("usersWhoLikedMe", "array-contains", "someUserId")
Where usersWhoLikedMe is a property of type array.
If you add User A and B ids to the doc itself:
users(col) -> User A(doc) -> likedBy(col) -> User B ({user_id: B, profile_liked_id: A})
Then you can query using:
db.collectionGroup('likedBy').where('user_id', '==', 'B');
if the reference is a collection, the value you compare to needs to be document id (the last segment of a full path)
this is a error message if you violated the rule
Invalid query. When querying a collection by documentId(), you must
provide a plain document ID, but 'a/b/c' contains a '/' character
if the reference is a collectonGroup, the value you compare to needs to be a full document path <-- which is very redundant in my opinion
Invalid query. When querying a collection group by documentId(), the
value provided must result in a valid document path, but
'a/b/c' is not because it has an odd number of
segments (3)
tested recently
I want to query object from Parse DB through javascript, that has only 1 of some specific relation object. How can this criteria be achieved?
So I tried something like this, the equalTo() acts as a "contains" and it's not what I'm looking for, my code so far, which doesn't work:
var query = new Parse.Query("Item");
query.equalTo("relatedItems", someItem);
query.lessThan("relatedItems", 2);
It seems Parse do not provide a easy way to do this.
Without any other fields, if you know all the items then you could do the following:
var innerQuery = new Parse.Query('Item');
innerQuery.containedIn('relatedItems', [all items except someItem]);
var query = new Parse.Query('Item');
query.equalTo('relatedItems', someItem);
query.doesNotMatchKeyInQuery('objectId', 'objectId', innerQuery);
...
Otherwise, you might need to get all records and do filtering.
Update
Because of the data type relation, there are no ways to include the relation content into the results, you need to do another query to get the relation content.
The workaround might add a itemCount column and keep it updated whenever the item relation is modified and do:
query.equalTo('relatedItems', someItem);
query.equalTo('itemCount', 1);
There are a couple of ways you could do this.
I'm working on a project now where I have cells composed of users.
I currently have an afterSave trigger that does this:
const count = await cell.relation("members").query().count();
cell.put("memberCount",count);
This works pretty well.
There are other ways that I've considered in theory, but I've not used
them yet.
The right way would be to hack the ability to use select with dot
notation to grab a virtual field called relatedItems.length in the
query, but that would probably only work for me because I use PostGres
... mongo seems to be extremely limited in its ability to do this sort
of thing, which is why I would never make a database out of blobs of
json in the first place.
You could do a similar thing with an afterFind trigger. I'm experimenting with that now. I'm not sure if it will confuse
parse to get an attribute back which does not exist in its schema, but
I'll find out, by the end of today. I have found that if I jam an artificial attribute into the objects in the trigger, they are returned
along with the other data. What I'm not sure about is whether Parse will decide that the object is dirty, or, worse, decide that I'm creating a new attribute and store it to the database ... which could be filtered out with a beforeSave trigger, but not until after the data had all been sent to the cloud.
There is also a place where i had to do several queries from several
tables, and would have ended up with a lot of redundant data. So I wrote a cloud function which did the queries, and then returned a couple of lists of objects, and a few lists of objectId strings which
served as indexes. This worked pretty well for me. And tracking the
last load time and sending it back when I needed up update my data allowed me to limit myself to objects which had changed since my last query.
I hope I can expain myself what I mean.
Let's say I have a car resource. The car has the attributes color, name or whatever.
I get the list of cars using a service, something like cars.index().
But in the interface I have all the cars and when I click on one car, a little popup appears showing the inputs to edit the color and the name.
And here comes the issue. Where do I save the displayingInputs attribute?
Should I save it directly in the car resource and then just send the original attributes when submitting to the updated resource?
Should I create a new service called carWidget or something along the lines that each one has something like:
{
car: cars.get(carId),
displayingInputs: false
}
Should I store the displayingInputs inside a carWidget directive scope? What happens if I need to change the `displayingInputs from the parent scope? (for example when making a "display all"/"hide all" button)
Something else?
My best bet is #3, but I'm not sure how should I access the displayingInputs from outside the widget.
If you want to keep your car model clean, you can have a separate variable editedCar and display each popup with ng-show="car == editedCar".
If multiple cars can be edited at the same time, editedCars can be an associative array (car ID => boolean) and display popups using ng-show="editedCars[car.id]".
Another way not to send certain car properties is to prefix their name with a $ sign. This was simply use car.$displayingInputs. Be careful for name collisions as Angular uses this prefix to store internal data.
From the angular.toJson doc:
Serializes input into a JSON-formatted string. Properties with leading
$ characters will be stripped since angular uses this notation
internally.
I'm interested in using the visualsearch.js control for my website but, having read through the documentation, I am still unclear regarding how to effectively obtain the output search collection data. Based on the example, the output string is constructed through serialization of the search collection. However, I was wondering if there is a way to access the search collection in a more array-like fashion (so that for/in loops can be used) rather than having to parse a single serialized string. Ultimately, I need to construct SQL queries from the search collection data.
If there is an even more efficient or appropriate way of accessing the search collection data, please let me know!
Thanks!
as far as i know there are 2 ways to fetch data from visual search
it is also directly explained in their documentation in usage #4
like you said, the stringified version of the search.
visualSearch.searchBox.value();
// returns: 'country: "United States" state: "New York" account: 5-samuel title: "Pentagon Papers"'
or the facetted object to loop over
visualSearch.searchQuery.facets();
// returns: [{"country":"United States"},{"state":"New York"},{"account":"5-samuel"},{"title":"Pentagon Papers"}]
as you can see, this option gives you an array, per facet that was filtered on, and for each asset the value that was entered.
mhmmm.. ok, the answer is not so straightforward. I would suggest you to get some practice with backbone structure just making some modification to the todo-list app. It is a great startpoint. So you get familiar with some of the wonderful backbone.js methods for collections
The Basic idea is the following:
With visualsearch you can obtain a list of "facets", that is to say an array of key/values objects.
var myFacets = visualSearch.searchQuery.facets();
//my facets is then something like [{"field1":"value1-a"},{"field2":"value2-c"}]
after this you can use myFacets elements to iterativrely filter you collection with the WONDERFUL filter method hinerithed from _underscore lib.
How to do it? You can use the _.each method in the underscore lib
_.each(myFacets,function(facet){
myCollection=myCollection.filter(function(item){
return item.get(facet.get('category')) == facet.get('value');
});
});
}
Here you use the filter method of backbone.js, which returns only the values are true according to your clause. So, you filter your collection once for each single facet. It is like telling to javascript: "Return me only the elements of the collection which match with this facets (value)", and you do it iteratively for all the different facets you got.
Hope this helps.
Ah.. one last thing, just to mess ideas up :-) :Visualsearch is built on backbone.js, and the searchQuery object is nothing but a backbone Collection, so you can use the methods and the properties of the basic backbone collection. Read this line again if this is not clear, because this can be a key point for future implementations! :-)
I suggest you to have a look at the search_jquery.js file in the lib/js/models folder. It's very interesting...