I have a firestore collection with a bunch of documents, each with plenty subfields. On a web page I need a list of a specific subfields from each document.
Currently I load the the entire database when you load the page and then loop through and get the wanted values. This uses way to many reads to get very little data.
Is there a way to solve this e.g. a autogenerated a collection that contains field from other collection in an array or something.
Many thanks in advance
Auto-creating such a subcollection with just the fields you need is a great way to reduce the bandwidth needed to load the data.
There is nothing built into Firestore to create those derived documents, but it's fairly easy to build something using Cloud Functions. Create a function that responds to a Firestore onWrite trigger, and write the subset of the data to its destination there. It's common to have a separate Cloud Function for each such use-case, and I regularly see projects with 100+ such functions.
I expect we'll also start seeing Firebase Extensions for this type of thing, but right now no-one seems to have built one.
Related
I want to read data from firebase, in the ref ("locations/{{someLocation}}/logs/{{someDevice}}")
but I do not know if exists a correct form to do, because I need to use ".on" to hear constantly all the devices changes, the problem is that I need to read all devices in logs in all locations, but I do not want another information, I just need information in logs or in the specific device, I know that I could reach that invoking one callback ".on" for each device, but I want a cleaner form to make it and in the documentation, there is no help for this.
maybe I could invoke a method like that
firebase.database("locations/{eachLocation}/logs/{eachDevice}").ref().on...
because I do not want all the devices, the problem is that I do not know if there is a form to make it thank you.
It sounds like you've nested your data too much. The Firebase documentation has explicit sections on avoiding nesting data and flattening data structures with hints on how to prevent this.
At first glance you'll need at least two top-level lists: locations and locationLogs. Under each you have the same keys as you have now, but the logs are now under /locationLogs/$key instead of under /locations/$key/logs. With that change you can get the logs for a location without getting the other data for that location.
If you don't know the location key, but don't want to get the logs for all locations, it sounds like you've nested another level too deep. Firebase queries work on a flat list of nodes, and can't search across multiple levels of unknown keys. If you want to search across all logs across all locations, you will need to keep a flat list of all logs. You can then tie each log back to its location, by adding the location ID to each log.
So that might lead to a structure of:
logs: {
"adjustableLight....1": {
locationId: "DTZB35",
date: 156...,"
...
}
}
What you're trying to do isn't supported by Realtime Database. There are no wildcard queries or placeholders. You must be able to build the full path to the node whose data you want to get.
Consider changing the structure of your data so that you can more easily find the nodes you want. It is common in nosql type databases to duplicate data into structures that are easier to query for a particular use case.
I want to store the comma separated ids on a child node & how can I filter data as in sql we can use IN clause to fetch data any possibility in firebase to perform this kind of operation in firebase database.
Please suggest any possible solution for this.
Firebase Realtime Database doesn't have the equivalent of SQLs IN clause. It also doesn't have a way to find a substring in a value. So the data model you are looking to use, doesn't allow the use-case you want. As usual with NoSQL databases, the solution is to pick a data model that does allow your use-case..
The most likely cause I know for the structure you describe is to associate the child node with a bunch of categories. If that is your case, read my answer here for a proper data structure: Firebase query if child of child contains a value
This is one of the cases where the new Cloud Firestore database offers better querying support, since it recently added a feature to efficiently test if an array contains a certain value (video). If you're only just getting started with your project, you might want to check if Firestore is a better fit for your use-cases.
I have a question about Collections - specifically, I want to have a large collection on a server, and load only small bits of it a piece at a time, in an unpredictable order, where I might stop wanting to have a local copy of any given piece at any time. Should I make a new subscription for each piece of data, and then stop it when I no longer want that piece of data? Or should I use some other method? Or should I just load large chunks of it that I won't use and leave them sitting around in my local copy of the collection?
Edit: Or should I have one subscription with a list of the ID's for each piece of data I want, and have the publication function specifically find each of those? Seems complicated, but it does keep me with only having to deal with one subscription.
Edit: Or maybe I should just skip using publications and subscriptions, and just use Methods to pass my data to the client? Loses a lot of functionality, and requires some extra work, but it does dodge most of the problems and should work just fine for my purposes.
Suppose Mongo collection ="items"
{
name:'item1',
type:'basic',
qty:40
}
you define collections on the Meteor server with
Items= new Mongo.Collection('items')
1.These collections contain all the data from the MongoDB collections, and you can run Items.find({...}) on them, which will return a cursor (a set of records, with methods to iterate through them and return them).
Meteor.publish('itemOver30', function itemPublication() {
return(Items.find({qty:{gte:10},{name:1,qty:1}));
});
This will return cursor to all the records with item qty over 30 in items collection(subset of total records, not whole collection).
2.Cursor is used to publish (send) a set of records (called a "record set"). You can optionally publish only some fields from those records. It is record sets (not collections) that clients subscribe to.
Meteor.subscribe('itemOver30');
On the client, you have Minimongo collections that partially mirror some of the records from the server. "Partially" because they may contain only some of the fields, and "some of the records" because you usually want to send to the client only the records it needs, to speed up page load, and only those it needs and has permission to access.
I'm using Meteor JS for a project so inherently I'm using MongoDB. I'm storing a user's check in and out actions. I'm currently storing them as individual docs in the collection. Each action contains 3 fields; in or out, time of action and userid. Is the best way to go though? Should I just have one doc per members and then store each action in an array? Is there another way? I anticipate several hundred members, but hopefully several thousands of members in the future. Thanks.
From experience, I can say that storing records instead of arrays is a better choice in the long run.
As far as Meteor is concerned, its reactivity handles collection records, but not individual fields in arrays. In other words, if one element gets added to the checkins array of a user object, the entire user object needs to be synchronized with the clients. If you store records instead, only the newly added record will be sent by the publication.
As far as MongoDB is concerned, there is a document size limit of 16MB. Not sure how frequent your checkins and checkouts are, but if you store them in an array, you might run into that limitation at some point.
Records are also easier to access than arrays.
For more details, see MongoDB data modeling and Database modeling in Bulletproof Meteor.
As far as I understand, I "publish" named collection server-side, on the client side I "subscribe" to that collection and pass them to Template, where I work only with the data, provided by "published" collection.
Please comment, have I got it right, and what should I do if I have a big collection that I don't want to retreive at once?
That is correct. When using large collections you should add parameters to the publish and/or limit the amount of records being published.
Theres a good explanation about publish/subscribe on Meteorpedia