I have a Firebase (Real-time Database) node with a large number of children. I want to list all child keys, so I used this code:
reference.once('value', snapshot=>{
snapshot.forEach(child => {
console.log(child.key);
})
})
then Firebase will throw an exception:
The specified payload is too large, please request a location with less data.
So I don't know how to get the child keys (only keys) without getting that error. Please help!
Use Firebase's shallow query https://firebase.google.com/docs/database/rest/retrieve-data#shallow
For example, if your data is structured like this
parent: {
child_0: {...<large data>...},
child_1: {...<large data>...},
child_2: "abc",
child_3: 123
}
The shallow query's result will be:
parent: {
child_0: true,
child_1: true,
child_2: "abc",
child_3: 123
}
Some unexpected errors can send a cancel event and terminate the connection. The cause is described in the data provided for this event. Some potential causes are as follows: 1. The Firebase Realtime Database Rules no longer allow a read at the requested location. The data description for this cause is "Permission denied." 2. A write triggered an event streamer that sent a large JSON tree that exceeds our limit, 512MB. The data for this cause is "The specified payload is too large, please request a location with less data."
Since realtime database only limit 500Mb per request so in order to execute this request you should try to paging a query with or try to iterate over children using child_added listener instead of querying all of the childs.
Related
i am getting an error of
Using an unspecified index. Your data will be downloaded and filtered on the client. Consider adding ".indexOn": "WkJymEhTtvgtIzQZZxs3VUTbmLh2quan" at /Products to your security rules for better performance.
this is my code:
firebase.database().ref("Products").orderByChild(user.uid + "quan").startAt(0).on('value', function (s) {
var cartNum = 0;
s.forEach(function (d) {
console.log(d.child(user.uid + "quan").val());
cartNum += d.child(user.uid + "quan").val();
});
$("#cartCount").text(cartNum);
});
am trying to query products that has user.uid+ 'quan' in my firebase database
and this is the structure of my JSON ---->>>
many thanks if someone can help me out
As described in the documentation on indexing data and in the error message, you will need to add an index to the Firebase rules for your database in the console:
{
"rules": {
"Products": {
".indexOn": "WkJymEhTtvgtIzQZZxs3VUTbmLh2quan"
}
}
}
This will solve the error message for the current user. The problem is that you will need to declare such an index explicitly for each UID, which is not feasible unless you have a specific set of users in mind.
The underlying problem is that your current data structure makes it easy to find a user for a given product, but it does not make it easy to find the products for a specific user. To allow the latter use-case, you'll want to add an additional data structure for that inverted mapping, sometimes referred to as a reverse or inverted index:
"Users": {
"uid1": {
"productId1": true,
"productId2": true
},
"uid2": {
"productId3": true,
"productId4": true
}
}
While this duplicates some data, it allows you to look up the related data in both directions. This type of data duplication to allow use-cases is quite common in NoSQL databases.
Also see:
Many to Many relationship in Firebase
Firebase Query Double Nested
Firebase query if child of child contains a value
I have tried to use .indexOn of firebase on field 'status' which is inside the child of 'BOUGHT_PRODUCT'.
But, when i run following query.
db.ref().child('BOUGHT_PRODUCT')
.orderByChild('status')
.equalTo('Service')
.on("value", function (snapshot) {
console.log(snapshot.val());
});
I get null snapshots.
Even, Firebase warning of index :
"#firebase/database: FIREBASE WARNING: Using an unspecified index. Your data will be downloaded and filtered on the client. Consider adding ".indexOn": "status" at /BOUGHT_PRODUCT to your security rules for better performance."
Firebase Data:
Index firebase:
Your help would be appreciated.
Firebase Database queries run against each child under the location where you run them. The field you order/filter on must be at a fixed path under each child. Since you run the query on /BOUGHT_PRODUCT, the database searches for /BOUGHT_PRODUCT/$uid/status. And since that property doesn't exist, there are no results matching the query.
In other words: your current data structure allows you to query a specific user for the status of their products, but not across all users. If you want to implement that use-case, you will need to create a data model that allows it, e.g. a single top-level list of product statuses.
Statuses
product1: "Service"
product2: "Service"
product3: "Something else"
Also see:
Firebase Query Double Nested
Firebase query if child of child contains a value
I want to send push notifications every time the value of a single key of my object changes in a parse cloud code afterSave hook.
Parse.Cloud.afterSave("Channel", function(request) {
var channel = request.object
// TODO: check if value of channel key "state" was changed
});
How can I check if the value of the key state was updated?
This is all data I can get from the request object: http://parseplatform.org/Parse-SDK-JS/api/v1.11.0/Parse.Cloud.html#.TriggerRequest
The solution suggested in this thread feels wrong: Parse Javascript API Cloud Code afterSave with access to beforeSave values
I know I can do this via the dirty method in the beforeSave hook. However this does not work for me. Why? If I do send push notifications to many users this takes some time. The clients receiving the push notifications start requesting the updated channel object from the server. However they might receive an old version of the object because as long as beforeSave has not finished sending all pushes the channel object is not persisted in the database.
You can use request.original. For example:
Parse.Cloud.afterSave("Channel", function(request) {
var channel = request.object;
var channel_orig = request.original;
if (channel.get("status") != channel_orig.get("status")) {
// Send push notification.
}
});
The documentation states about request.original, that: "If set, the object, as currently stored." I'm not sure in what cases it would be set, though. In my use cases it works as provided in the code snippet above.
In my user collection, I have an object that contains an array of contacts.
The object definition is below.
How can this entire object, with the full array of contacts, be written to the user database in Meteor from the server, ideally in a single command?
I have spent considerable time reading the mongo docs and meteor docs, but can't get this to work.
I have also tried a large number of different commands and approaches using both the whole object and iterating through the component parts to try to achieve this, unsuccessfully. Here is an (unsuccessful) example that attempts to write the entire contacts object using $set:
Meteor.users.update({ _id: this.userId }, {$set: { 'Contacts': contacts}});
Thank you.
Object definition (this is a field within the user collection):
"Contacts" : {
"contactInfo" : [
{
"phoneMobile" : "1234567890",
"lastName" : "Johnny"
"firstName" : "Appleseed"
}
]
}
This update should absolutely work. What I suspect is happening is that you're not publishing the Contacts data back to the client because Meteor doesn't publish every key in the current user document automatically. So your update is working and saving data to mongo but you're not seeing it back on the client. You can check this by doing meteor mongo on the command line then inspecting the user document in question.
Try:
server:
Meteor.publish('me',function(){
if (this.userId) return Meteor.users.find(this.userId, { fields: { profile: 1, Contacts: 1 }});
this.ready();
});
client:
Meteor.subscribe('me');
The command above is correct. The issue is schema verification. Simple Schema was defeating the ability to write to the database while running 'in the background'. It doesn't produce an error, it just fails to produce the expected outcome.
I'm creating a mock application with JSON server as the backend and I'm wondering if it is possible to get the total number of records contained at an end point without loading all the records themselves? Assuming the db.json file looks like the JSON snippet below, how would I find out that the end point only has one record without fetching the record itself, provided it's possible?
{
"books": [{
"title": "The Da Vinci Code",
"rating": "0"}]
}
You can simply retrieve the X-Total-Count header
This is a screen-shot of a response headers returned by JSON Server when enabling pagination i.e using the _page parameter (e.g. localhost:3000/contacts?_page=1)
Whenever you fetch the data, json-server actually returns the total count by default (it has an x-total-count property:
Example:
axios
.get("http://localhost:3001/users", {
params: {
_page: 1,
_limit: 10
}
})
.then(res => {
console.log(res.data); // access your data which is limited to "10" per page
console.log(res.headers["x-total-count"]); // length of your data without page limit
});
You've three options. I'd recommend the 3rd one to you:
Return all the records and count them. This could be slow and send a lot of data over the wire but probably is the smallest code change for you. It also opens you up to attacks where people can hammer your server by requesting many records repeatedly.
Add a new endpoint. You could add a new endpoint that simply returns the count. It's simple but slightly annoying having a 2nd endpointime to document and maintain.
Modify the existing endpoint. Return something like
{
count: 157,
rows: [...data]
}
The benefit of 3 is its all in one endpoint. It also nears you toward a point where you can add a skip and take parameter in future to allow pagination of the resultant data.
You will write another end point that returns number of records. Usually also you may want end point for limit and offset to be used with pagination.
let response = await fetch("http://localhost:3001/books?_page=1");
let total = response.headers.get('X-Total-Count');