We Keep Coding

firebase query with order by not returning the data in correct order [duplicate]

I am trying to query my database such that it retrieves an ordered list based on a child key. I do it as follows (see below), but nothing happens, meaning that it returns an object ordered exactly in the same way as it is stored in the Firebase database. What is going on?
self.getAllProfiles = function () {
var qProfile = $q.defer();
var ref = new Firebase(FBURL);
ref.child("users").orderByChild('last_update').on("value", function (snapshot) {
console.log(snapshot.val()) // HERE IS WHERE IT SHOULD BE ORDERED
qProfile.resolve(snapshot.val());
}, function (errorObject) {
qProfile.reject(errorObject);
});
return qProfile.promise;
};
To add, my users node looks as follows:
users
/$username
/last_update
/id
/data
/profile_image
/display_name
Here is a snapshot:
Tester: Object
github: Object
last_update: 1447732462170
userId: "github:12345"

When you call snapshot.val(), you are getting back a JSON object. The order of keys in a JSON object is determined by your browser and not by Firebase.
To get the children in order use the built-in forEach method of the snapshot:
self.getAllProfiles = function () {
var qProfile = $q.defer();
var ref = new Firebase(FBURL);
ref.child("users").orderByChild('last_update').on("value", function (snapshot) {
snapshot.forEach(function(child) {
console.log(child.val()) // NOW THE CHILDREN PRINT IN ORDER
});
qProfile.resolve(snapshot.val());
}, function (errorObject) {
qProfile.reject(errorObject);
});
return qProfile.promise;
};
You can leave the q.resolve() call where it is: snapshot.forEach() is not an asynchronous call.

I know this question has been answered and is more than 1 year old, but since there are still some confusion in the comment section, I would like to add some information.
The problem
The original problem is that the OP want to retrieve an ordered list based on a child key from Firebase realtime database, but the .orderByChild('arg') does not work as expected.
But what didn't work as expected is not .orderByChild('arg'), but .on("value", callback). Because .on("value", callback) works a bit of different from other eventTypes like .on("child_added", callback).
Example
Say we have a firebase realtime database as below:
{
myData: {
-KYNMmYHrzLcL-OVGiTU: {
NO: 1,
image: ...
},
-KYNMwNIz4ObdKJ7AGAL: {
NO: 2,
image: ...
},
-KYNNEDkLpuwqQHSEGhw: {
NO: 3,
image: ...
},
}
}
--
If we use .on("value", callback), the callback() will be called 1 time, and return an Object Array of 3 objects.
ref.on("value", function(snapshot) {
console.log(snapshot.val());
// Please see Frank van Puffelen's answer
}
--
If we use .on("child_added", callback), the callback() will be called 3 times, each time returns an Object, and they are returned in order.
ref.once("child_added", function(snapshot) {
console.log(snapshot.val());
// The objects are returned in order, do whatever you like
}
Conclusion
If you only need to fetch ordered data from firebase (e.g. to initialize UI.) Then ref.orderByChild('arg').once("child_added", callback) suits you well, it is simple and easy to use.
However, if for some reason you need to use ref.orderByChild('arg').on("value", callback), then please see Frank van Puffelen's answer.
Reference
Please read Firebase Document for more information about on(eventType, callback, cancelCallbackOrContext, context), their arguments and their return values.
Another useful document: Work with Lists of Data on the Web

For ordering using the value event listener:
firebase.database().ref('/invoices').orderByChild('name').on('value', snapshot => {
snapshot.forEach(child => {
console.log(child.key, child.val());
});
}
If you want to reverse the order, try:
function reverseSnapshotOrder (snapshot) {
let reversed = [];
snapshot.forEach(child => {
reversed.unshift(child);
});
return reversed;
}
reverseSnapshotOrder(snapshot).forEach(child => {
console.log(child.key, child.val());
});
Please see: https://firebase.google.com/docs/database/web/lists-of-data#listen_for_value_events

I struggled with the same problem in iOS. If you convert snapshot to NSDictionary object, it converts to unordered list. Objective-c version in case of need:
[[self.refChild queryOrderedByChild:#"date_created"]
observeEventType:FIRDataEventTypeValue
withBlock:^(FIRDataSnapshot * _Nonnull snapshot) {
for (FIRDataSnapshot *child in snapshot.children) {
NSDictionary *anObject = child.value;
NSString *aKey = child.key;
}
}];

Publishing changes to a single field of a subdocument

I have a complicated data structure being built by queries on multiple collections and published.
It is working great for the initial creation, and on my local machine all the changes observed are reflected in the client as expected. However, in my staging environment I get the following error from mini-mongo when a change is observed
Uncaught Error: When replacing document, field name may not contain '.'(…)
The publishing code looks like this, where pub is the this from a Meteor.publish and rootObj is a reference to an Object in memory which gets properties modified but never has it's reference destoryed.
function _republish(pub, rootId, rootObj, handles, startup) {
// cleanup handles
if (handles.foo) {
handles.foo.stop();
}
// some query which could depend on rootObj/other calculated values
let cursor = SubColl.find({_id: {$in: bar}});
handles.foo = cursor.observeChanges({
removed(_id) {
rootObj.bar = rootObj.bar.filter(o => o._id !== _id);
pub.changed('foobarbaz', rootId, {bar: rootObj.bar})
},
changed(_id, fields) {
const index = rootObj.bar.findIndex(line => line._id === _id);
const changed = {};
_.each(fields, (value, field) => {
rootObj.bar[index][field] = value;
changed[`bar.${index}.${field}`] = value;
});
pub.changed('foobarbaz', rootId, changed);
},
added(_id, fields) {
rootObj.bar.push(_.extend({}, fields, {_id}));
if (!startup) {
// deeper children stuff
pub.changed('foobarbaz', rootId, {bar: rootObj.bar});
}
}
});
// deeper children stuff
startup = false;
// if startup was true, expect caller to publish this
}
As we can see, the publish works fine when I'm pub.changeding on just bar, but attempting to update a specific subdocument field (e.g. bar.0.prop) results in the inconsistent behaviour
If possible I want to avoid re-publishing the whole of bar as it is huge compared to updating a simple property.
How can I publish the change to a single field of a subdocument?

orderByChild not working in Firebase

I am trying to query my database such that it retrieves an ordered list based on a child key. I do it as follows (see below), but nothing happens, meaning that it returns an object ordered exactly in the same way as it is stored in the Firebase database. What is going on?
self.getAllProfiles = function () {
var qProfile = $q.defer();
var ref = new Firebase(FBURL);
ref.child("users").orderByChild('last_update').on("value", function (snapshot) {
console.log(snapshot.val()) // HERE IS WHERE IT SHOULD BE ORDERED
qProfile.resolve(snapshot.val());
}, function (errorObject) {
qProfile.reject(errorObject);
});
return qProfile.promise;
};
To add, my users node looks as follows:
users
/$username
/last_update
/id
/data
/profile_image
/display_name
Here is a snapshot:
Tester: Object
github: Object
last_update: 1447732462170
userId: "github:12345"

When you call snapshot.val(), you are getting back a JSON object. The order of keys in a JSON object is determined by your browser and not by Firebase.
To get the children in order use the built-in forEach method of the snapshot:
self.getAllProfiles = function () {
var qProfile = $q.defer();
var ref = new Firebase(FBURL);
ref.child("users").orderByChild('last_update').on("value", function (snapshot) {
snapshot.forEach(function(child) {
console.log(child.val()) // NOW THE CHILDREN PRINT IN ORDER
});
qProfile.resolve(snapshot.val());
}, function (errorObject) {
qProfile.reject(errorObject);
});
return qProfile.promise;
};
You can leave the q.resolve() call where it is: snapshot.forEach() is not an asynchronous call.

I know this question has been answered and is more than 1 year old, but since there are still some confusion in the comment section, I would like to add some information.
The problem
The original problem is that the OP want to retrieve an ordered list based on a child key from Firebase realtime database, but the .orderByChild('arg') does not work as expected.
But what didn't work as expected is not .orderByChild('arg'), but .on("value", callback). Because .on("value", callback) works a bit of different from other eventTypes like .on("child_added", callback).
Example
Say we have a firebase realtime database as below:
{
myData: {
-KYNMmYHrzLcL-OVGiTU: {
NO: 1,
image: ...
},
-KYNMwNIz4ObdKJ7AGAL: {
NO: 2,
image: ...
},
-KYNNEDkLpuwqQHSEGhw: {
NO: 3,
image: ...
},
}
}
--
If we use .on("value", callback), the callback() will be called 1 time, and return an Object Array of 3 objects.
ref.on("value", function(snapshot) {
console.log(snapshot.val());
// Please see Frank van Puffelen's answer
}
--
If we use .on("child_added", callback), the callback() will be called 3 times, each time returns an Object, and they are returned in order.
ref.once("child_added", function(snapshot) {
console.log(snapshot.val());
// The objects are returned in order, do whatever you like
}
Conclusion
If you only need to fetch ordered data from firebase (e.g. to initialize UI.) Then ref.orderByChild('arg').once("child_added", callback) suits you well, it is simple and easy to use.
However, if for some reason you need to use ref.orderByChild('arg').on("value", callback), then please see Frank van Puffelen's answer.
Reference
Please read Firebase Document for more information about on(eventType, callback, cancelCallbackOrContext, context), their arguments and their return values.
Another useful document: Work with Lists of Data on the Web

For ordering using the value event listener:
firebase.database().ref('/invoices').orderByChild('name').on('value', snapshot => {
snapshot.forEach(child => {
console.log(child.key, child.val());
});
}
If you want to reverse the order, try:
function reverseSnapshotOrder (snapshot) {
let reversed = [];
snapshot.forEach(child => {
reversed.unshift(child);
});
return reversed;
}
reverseSnapshotOrder(snapshot).forEach(child => {
console.log(child.key, child.val());
});
Please see: https://firebase.google.com/docs/database/web/lists-of-data#listen_for_value_events

I struggled with the same problem in iOS. If you convert snapshot to NSDictionary object, it converts to unordered list. Objective-c version in case of need:
[[self.refChild queryOrderedByChild:#"date_created"]
observeEventType:FIRDataEventTypeValue
withBlock:^(FIRDataSnapshot * _Nonnull snapshot) {
for (FIRDataSnapshot *child in snapshot.children) {
NSDictionary *anObject = child.value;
NSString *aKey = child.key;
}
}];

What are MongoDB Modifiers and Operators?

I am working on a Meteor application and one of the features I'm building is a form that inserts a new document into an array (inserts a shipping address to a user's profile where a user can have multiple addresses). The error I keep getting is:
Exception while invoking method 'addAddress' Error: When the modifier option is true, validation object must have at least one operator
I have been unsuccessfully trying to figure out the answer on Stackoverflow, Github, etc. but could not find a solution. I now want to take the approach of understanding exactly what the error means - so my question is what exactly are modifier options and operators in MongoDB? From what I understand, modifiers provide constraints on what type of data is returned from a query, and operators are used to modify data. Are these definitions correct?
Does anyone know what the error I'm getting might mean? Here is my sample code:
My click event to capture data on a form and call a method to add an address:
Template.editAddress.events({
'click .addAddress': function(e, tmpl) {
e.preventDefault();
var currentUserId = Meteor.userId();
console.log(currentUserId);
var addressDetails = {
address: {
streetAddress: $('#streetAddress').val()
}
};
console.log(addressDetails);
Meteor.call('addAddress', addressDetails, currentUserId, function(error) {
if (error) {
alert(error.reason);
} else {
console.log('success!');
Router.go('Admin');
}
});
}
});
My method to insert the address:
Meteor.methods({
'addAddress': function(addressDetails, currUserId) {
var currentUserId = currUserId;
console.log('user to add address to is ' + currUserId);
Meteor.users.update(currentUserId, {$addToSet:
{
'address.streetAddress': addressDetails.streetAddress
}
});
}
});
Note that when I type that query in the console, it works:
db.users.update({_id: 'Qdf89k3fd93jfdk'}, {$addToSet: {'address.streetAddress': '12345 fake st'}});
Thank you in advance!

Your addressDetails object doesn't have a field streetAddress, so addressDetails.streetAddress returns undefined. Use addressDetails.address.streetAddress instead in the update. And also, like Joshua pointed out, use an object as selector with { _id: currUserId }. So the whole function should be:
Meteor.users.update( { _id: currentUserId }, {$addToSet:
{
'address.streetAddress': addressDetails.address.streetAddress
}
});
}
One more thing, you should not pass the userId from the client. Any method you define is callable from the client and like that, I would be able to call your method 'addAddress' from the browser console with any userId to update their address. Instead, use the this.userId object in method calls (see here) and check that it is not null, i.e. user is logged in.
if (! this.userId)
throw new Meteor.Error(401, "You must be logged in!");
var currentUserId = this.userId;

It looks like you're passing in the document id directly into the MongoDB query method, rather than constructing an object with an _id property with a value of said document id.
i.e try
var currentUserId = { _id: currUserId };

Mongoose - RangeError: Maximum Call Stack Size Exceeded

I am trying to bulk insert documents into MongoDB (so bypassing Mongoose and using the native driver instead as Mongoose doesn't support bulk insert of an array of documents). The reason I'm doing this is to improve the speed of writing.
I am receiving the error "RangeError: Maximum Call Stack Size Exceeded" at console.log(err) in the code below:
function _fillResponses(globalSurvey, optionsToSelectRegular, optionsToSelectPiped, responseIds, callback) {
Response.find({'_id': {$in: responseIds}}).exec(function(err, responses) {
if (err) { return callback(err); }
if (globalSurvey.questions.length) {
responses.forEach(function(response) {
console.log("Filling response: " + response._id);
response.answers = [];
globalAnswers = {};
globalSurvey.questions.forEach(function(question) {
ans = _getAnswer(question, optionsToSelectRegular, optionsToSelectPiped, response);
globalAnswers[question._id] = ans;
response.answers.push(ans);
});
});
Response.collection.insert(responses, function(err, responsesResult) {
console.log(err);
callback()
});
} else {
callback();
}
});
}
So similar to: https://stackoverflow.com/questions/24356859/mongoose-maximum-call-stack-size-exceeded
Perhaps it's something about the format of the responses array that Mongoose returns that means I can't directly insert using MongoDB natively? I've tried .toJSON() on each response but no luck.
I still get the error even with a very small amount of data but looping through and calling the Mongoose save on each document individually works fine.
EDIT: I think it is related to this issue: http://howtosjava.blogspot.com.au/2012/05/nodejs-mongoose-rangeerror-maximum-call.html
My schema for responses is:
var ResponseSchema = new Schema({
user: {
type: Schema.ObjectId,
ref: 'User'
},
randomUUID: String,
status: String,
submitted: Date,
initialEmailId: String,
survey: String,
answers: [AnswerSchema]
});
So, answers are a sub-document within responses. Not sure how to fix it though....

I was having this same issue and I started digging through the mongoose source code (version 3.8.14). Eventually it led me to this line within
mongoose/node_modules/mongodb/lib/mongodb/collection/core.js -> insert(...) -> insertWithWriteCommands(...) ->
mongoose/node_modules/mongodb/lib/mongodb/collection/batch/ordered.js -> bulk.insert(docs[i]) -> addToOperationsList(...) -> bson.calculateObjectSize(document, false);
var bsonSize = bson.calculateObjectSize(document, false);
Apparently, this calls BSON.calculateObjectSize, which calls calculateObjectSize which then infinitely recurses. I wasn't able to dig that far in to what caused it, but figured that it may have something to do with the mongoose wrapper binding functions to the Schema. Since I was inserting raw data into mongoDB, once I decided to change the bulk insert in mongoose to a standard javascript object, the problem went away and bulk inserts happened correctly. You might be able to do something similar.
Essentially, my code went from
//EDIT: mongoose.model needs lowercase 'm' for getter method
var myModel = mongoose.model('MyCollection');
var toInsert = myModel();
var array = [toInsert];
myModel.collection.insert(array, {}, function(err, docs) {});
to
//EDIT: mongoose.model needs lowercase 'm' for getter method
var myModel = mongoose.model('MyCollection');
var toInsert = { //stuff in here
name: 'john',
date: new Date()
};
var array = [toInsert];
myModel.collection.insert(array, {}, function(err, docs) {});

Confirmed, but not a bug. Model.collection.insert() bypasses Mongoose and so you're telling the node driver to insert an object that contains mongoose internals like $__, etc. The stack overflow is probably because bson is trying to compute the size of an object that references itself indirectly.
Long story short, use Document.toObject(), that's what its for: http://mongoosejs.com/docs/api.html#document_Document-toObject
Response.find({}).exec(function(err, responses) {
if (err) {
return callback(err);
}
if (true) {
var toInsert = [];
responses.forEach(function(response) {
console.log("Filling response: " + response._id);
response.answers = [];
[{ name: 'test' }].forEach(function(ans) {
response.answers.push(ans);
});
toInsert.push(response.toObject());
});
Response.collection.insert(toInsert, function(err, responsesResult) {
console.log(err);
});
} else {
callback();
}
});
Also, the code you specified won't work even if you fix the stack overflow. Since you're trying to insert() docs that are already in the database, all the inserts will fail because of _id conflicts. You'd really be much better off just using a stream() to read the results one at a time and then save() them back into the db.

guys! I've faced that weird error today. It happened because of I had a Schema with ref properties and tried to pass in create/update whole related document. I've changed argument to _id only and that did the trick. Works like a charm. I found the answer here (scroll down to February 21, 2013, 8:05 pm gustavohenke comment).

I have faced similar issue.
//manyvalues is array of objects
schema.methods.somemethod = function(manyvalues,callback) {
this.model(collection).collection.insertMany(manyvalues,callback);
}
But this caused error [RangeError: Maximum call stack size exceeded].
So I have created new model from manyvalues and used it as below and it worked.
schema.methods.somemethod = function(manyvalues,callback){
var list = JSON.parse(JSON.stringify(manyvalues));//created a new object.
this.model(collection).collection.insertMany(list,callback);
}
The problem may be caused if manyvalues is changed internally.

This also happens if there's a duplication of of the _id value. Most situations will be when you might create an new record from an existing record.
Deleting the _id and inserting the record and letting Mongoose/MongoDb take care of the creation of the id.

I had the same issue. Mongoose version is 5.13.14. My stack trace is:
RangeError: Maximum call stack size exceeded
at minimize (...\node_modules\mongoose\lib\document.js:3564:18)
at minimize (...\node_modules\mongoose\lib\document.js:3576:18)
at minimize (...\node_modules\mongoose\lib\document.js:3576:18)
at minimize (...\node_modules\mongoose\lib\document.js:3576:18)
at minimize (...\node_modules\mongoose\lib\document.js:3576:18)
at minimize (...\node_modules\mongoose\lib\document.js:3576:18)
at minimize (...\node_modules\mongoose\lib\document.js:3576:18)
I found 2 ways to fix the issue:
Using toObject() method:
const model = await MyModel.findOne(conditions);
return model?.toObject();
Using minimize: false in toJSON option of the schema:
export const MySchema = new Schema({
...
}, {
...
toJSON: {
getters: true,
// !!! HERE !!!
minimize: false,
},
...
});

Check for circular references in the responses object. I Faced a similar issue due to circular references.

I had a similar problem, it was that I was querying a field that didn't exist in the schema using the $ne(other query operators may have a similar problem)
var TestSchema = new Schema({
test:[]
});
...
models.Test.findOne({"test2": {$ne: "t"} })...
In the example above I am testing for test2 instead of test