I wanna pull whole data from the firebase database and access that without change data.
firebase.database().ref(‘someBigNode’).once(‘value’, (snapshot) => {
//do something
}
But it will cost many memory.
May I ask how to pull whole data partially and save memory?
When you read data from Firebase Database with the JavaScript (or iOS or Android) SDK, it will always read complete nodes. So the only way to retrieve less data is to retrieve a node lower in the JSON tree.
If you find you need to retrieve a part of each node under someBigNode, you should split that part of each node out into a top-level node of their own importantBitsOfSomeBigNode.
It's unfortunately hard to be more helpful without a more concrete example of your data structure and the bits you're trying to retrieve (and the reason why those bits are special).
Related
I have a firestore collection with a bunch of documents, each with plenty subfields. On a web page I need a list of a specific subfields from each document.
Currently I load the the entire database when you load the page and then loop through and get the wanted values. This uses way to many reads to get very little data.
Is there a way to solve this e.g. a autogenerated a collection that contains field from other collection in an array or something.
Many thanks in advance
Auto-creating such a subcollection with just the fields you need is a great way to reduce the bandwidth needed to load the data.
There is nothing built into Firestore to create those derived documents, but it's fairly easy to build something using Cloud Functions. Create a function that responds to a Firestore onWrite trigger, and write the subset of the data to its destination there. It's common to have a separate Cloud Function for each such use-case, and I regularly see projects with 100+ such functions.
I expect we'll also start seeing Firebase Extensions for this type of thing, but right now no-one seems to have built one.
exist any way to know if this function collects information from an accesible collection?
For example these registres:
db.currentOp(true).inprog.forEach(function(d){if(d.client)print(d.client, d.connectionId)})
I was looking at the data in the 'oplog.rs' collection but it didn't seem contain what I am looking for.
There's no database collection mentioned in the documentation which means that currentOps are most likely held in an internal MongoDB data structure.
Browsing the github MongoDB repository and the body of currentOp method leads to CurOp C++ class which is simple in-memory stack implementation so there's no persistent storage for current ops.
You can also take a look at system.profile if you're trying to build some sort of diagnostic solution which requires persistent storage.
I have a table with a thousand records in it and I want to do a google like search full-text/fuzzy search.
I read about MySQL v8's Full-Text search and let's say we don't have that functionality yet.
There is this JavaScript library called Fuse.js that do fuzzy-search which is what I need.
I can combine it by creating a API that returns the table data in JSON format and then pass it to Fuse.js to do a fuzzy-search.
Now, I think it's not recommended to load all data from table every time someone wants to search.
I read about Redis, and the first thing that came in my mind is to save all table data in Redis using JSON.stringify and just call it every time instead of querying the database. Then whenever a data is added in the table, I will also update the contents of the data in Redis.
Is there a better way to do this?
That is a very common caching pattern.
If you need a more efficient way to store and retrieve your JSON to/from Redis you might want to consider one of the available Redis Modules.
e.g.
RedisJSON allows you to efficiently store, retrieve, project (jsonpath) and update in place.
RediSearch allows you to have full text search over Redis Hash and efficiently retrieve data according to the user's query.
Last
RedisJSON2 (aka RedisDoc) combines both modules above, meaning efficient JSON store and retrieve with Full Text support
While working with Firebase (the realtime database) I'd like to know the following about the objects in the database:
Whether an object exists at a path
When an object is created at a path
When an object is destroyed at a path
I can figure out some hacks for these. For example:
Subscribe to on('value') events and download the whole (often huge) object.
Create a separate "flag" value for each object in the database that gets modified when objects are created and destroyed.
Listen for 'child_added', 'child_removed', etc. on a particular field inside of an object, that has the same lifetime as the object.
These get the job done in some specific circumstances but are pretty arbitrary and don't scale well.
Is there a way to achieve the above without having to either download the whole object or else creating a bunch of "book-keeping" entries in the database?
It's common to duplicate data and add additional to your database to satisfy your expected queries, without accessing too much data. There aren't any hidden or secret operations - what you see in the documentation is your toolset. It sounds like you're doing the right things already.
Assuming the web SDK (browser perspective), realtime updates (.on) are the proper way to get this type of data. The data should not be huge - it's called the "Realtime Database" bcs it's not intended to store large filetypes such as images and videos.
Listen for value events
var starCountRef = firebase.database().ref('posts/' + postId + '/starCount');
starCountRef.on('value', function(snapshot) {
updateStarCount(postElement, snapshot.val());
});
I want to store the comma separated ids on a child node & how can I filter data as in sql we can use IN clause to fetch data any possibility in firebase to perform this kind of operation in firebase database.
Please suggest any possible solution for this.
Firebase Realtime Database doesn't have the equivalent of SQLs IN clause. It also doesn't have a way to find a substring in a value. So the data model you are looking to use, doesn't allow the use-case you want. As usual with NoSQL databases, the solution is to pick a data model that does allow your use-case..
The most likely cause I know for the structure you describe is to associate the child node with a bunch of categories. If that is your case, read my answer here for a proper data structure: Firebase query if child of child contains a value
This is one of the cases where the new Cloud Firestore database offers better querying support, since it recently added a feature to efficiently test if an array contains a certain value (video). If you're only just getting started with your project, you might want to check if Firestore is a better fit for your use-cases.