We Keep Coding

Query firebase to return if value more than number

I want to get data from Firebase.
This is more or less my data structure:
"Reports" : {
"N06Jrz5hx6Q9bcVDBBUrF3GKSTp2" : 2,
"eLLfNlWLkTcImTRqrYnU0nWuu9P2" : 2
},
"Users":{
"N06Jrz5hx6Q9bcVDBBUrF3GKSTp2" : {
"completedWorks" : {
...
},
"reports" : {
"-LHs0yxUXn-TQC7z_MJM" : {
"category" : "Niewyraźne zdjęcie",
"creatorID" : "z8DxcXyehgMhRyMqmf6q8LpCYfs1",
"reportedID" : "N06Jrz5hx6Q9bcVDBBUrF3GKSTp2",
"resolved" : false,
"text" : "heh",
"workID" : "-LHs-aZJkAhEf1RHVasg"
},
"-LHs1hzlL4roUJfMlvyA" : {
"category" : "Zdjęcie nie przedstawia zadania",
"creatorID" : "z8DxcXyehgMhRyMqmf6q8LpCYfs1",
"reportedID" : "N06Jrz5hx6Q9bcVDBBUrF3GKSTp2",
"resolved" : false,
"text" : "",
"workID" : "-LHs-aZJkAhEf1RHVasg"
}
},
"userType" : "company",
"verified" : true
},
}
So as you can see the number of reports is listed in the Reports part. How can I make Firebase return only the ids of the users where the report number is over or equal 3?
Something like this (this will not work, but I hope kind of shows what I was thinking about):
firebase.database().ref('Reports').orderBy(whatHere?).moreThen(2).on('value', snap => {
Is this even doable like this? If yes how could I do it? I want to grab the IDs of the users where reports are >= 3

You're looking for orderByValue():
firebase.database().ref('Reports').orderByValue().startAt(3).on('value', snapshot => {
snapshot.forEach(reportSnapshot => {
console.log(reportSnapshot.key);
})
})
Also check out the Firebase documentation on ordering data.

There are two options for doing that but not exactly the way you wants. You have to use javascript for further processing. One is to use limitToLast after using order by. which will give the last numbers from the result.
firebase.database().ref('Reports').orderBy(reportid).limitToLast(2).on('value', snap => {
Or use startAt and endAt to skip and fetch the result as offset which can provide the data between two reportId.
firebase.database().ref('Reports').orderBy(reportid).
.startAt(reportIdStart)
.endAt(reportIdLast)
.limitToLast(15)
According Firebase documentation:
Using startAt(), endAt(), and equalTo() allows you to choose arbitrary
starting and ending points for your queries
To filter data, you can combine any of the limit or range methods with an order-by method when constructing a query.
Unlike the order-by methods, you can combine multiple limit or range
functions. For example, you can combine the startAt() and endAt()
methods to limit the results to a specified range of values.
For more information go through documentation on filtering data

Firebase filter with pagination

I'm doing an opensource Yelp with Firebase + Angular.
My database:
{
"reviews" : {
"-L0f3Bdjk9aVFtVZYteC" : {
"comment" : "my comment",
"ownerID" : "Kug2pR1z3LMcZbusqfyNPCqlyHI2",
"ownerName" : "MyName",
"rating" : 2,
"storeID" : "-L0e8Ua03XFG9k0zPmz-"
},
"-L0f7eUGqenqAPC1liYj" : {
"comment" : "me second comment",
"ownerID" : "Kug2pR1z3LMcZbusqfyNPCqlyHI2",
"ownerName" : "MyName",
"rating" : 3,
"storeID" : "-L0e8Ua03XFG9k0zPmz-"
},
},
"stores" : {
"-L0e8Ua03XFG9k0zPmz-" : {
"description" : "My good Store",
"name" : "GoodStore",
"ownerID" : "39UApyo0HIXmKPrTOi8D0nWLi6n2",
"tags" : [ "good", "health", "cheap" ],
}
},
"users" : {
"39UApyo0HIXmKPrTOi8D0nWLi6n2" : {
"name" : "First User"
},
"Kug2pR1z3LMcZbusqfyNPCqlyHI2" : {
"name" : "MyName",
"reviews" : {
"-L0f3Bdjk9aVFtVZYteC" : true,
"-L0f7eUGqenqAPC1liYj" : true
}
}
}
}
I use this code below to get all store's reviews (using AngularFire2)
getReviews(storeID: string){
return this.db.list('/reviews', ref => {
return ref.orderByChild('storeID').equalTo(storeID);
});
}
Now, I want to make a server-side review pagination, but I think I cannot do it with this database structure. Am I right? Tried:
getReviews(storeID: string){
return this.db.list('/reviews', ref => {
return ref.orderByChild('storeID').equalTo(storeID).limitToLast(10) //How to make pagination without retrive all data?
});
}
I thought that I could put all reviews inside stores, but (i) I don't want to retrieve all reviews at once when someone ask for a store and (ii) my review has a username, so I want to make it easy to change it (that why I have a denormalized table)

For the second page you need to know two things:
the store ID that you want to filter on
the key of the review you want to start at
You already have the store ID, so that's easy. As the key to start at, well use the key of the last item on the previous page, and then just request one item extra. Then finally, you'll need to use start() (and possibly endAt() for this:
return this.db.list('/reviews', ref => {
return ref.orderByChild('storeID')
.startAt(storeID, lastKeyOnPreviousPage)
.limitToLast(11)
});

Refer this and this documentation.
For the first page:
snapshot = await ref.orderByChild('storeID')
.equalTo(store_id) //store_id is the variable name.
.limitToLast(10)
.once("value")
Store the firstKey (NOT the lastKey) from the above query. (Since you are using limitToLast())
firstKey = null
snapshot.forEach(snap => {
if (!firstKey)
firstKey = snap.key
// code
})
For the next page:
snapshot = await ref.orderByChild('storeID') //storeID is the field name in the database
.startAt(store_id) //store_id is the variable name which has the desired store ID
.endAt(store_id, firstKey)
.limitToLast(10 + 1) //1 is added because you will also get value for the firstKey
.once("value")
The above query will fetch 11 list data which will contain one redundant data from the first page's query.
How it works:
startAt ( value : number | string | boolean | null , key ? : string ) : Query
The starting point is inclusive, so children with exactly the specified value will be included in the query. The optional key argument can be used to further limit the range of the query. If it is specified, then children that have exactly the specified value must also have a key name greater than or equal to the specified key.
endAt ( value : number | string | boolean | null , key ? : string ) : Query
The ending point is inclusive, so children with exactly the specified value will be included in the query. The optional key argument can be used to further limit the range of the query. If it is specified, then children that have exactly the specified value must also have a key name less than or equal to the specified key.
So the query will try to fetch:
storeID >= store_id && storeID <= store_id (lexicographically)
which will equal to
storeID == store_id

What is the addressing issue with my complex document update() in MongoDB?

I have been unable to reach into my MongoDB collection and change a value in a complex document. I have tried more variations than the one example shown below, all sorts of variations, but they fail.
I want to change the Value of the Key "air" from "rain" to "clear". In real life, I will not know that the current Value of the Key "air" is "rain".
Note, I am not using the MongoDB _id Object and would like to accomplish this without using it.
3 documents in the weatherSys collection:
{
"_id" : ObjectId("58a638fb1831c61917f921c5"),
"SanFrancisco" : [
{ "sky" : "grey" },
{ "air" : "rain" },
{ "ground" : "wet" }
]
}
{
"_id" : ObjectId("58a638fb1831c61917f921c6"),
"LosAngeles" : [
{ "sky" : "grey" },
{ "air" : "rain" },
{ "ground" : "wet" }
]
}
{
"_id" : ObjectId("58a638fb1831c61917f921c7"),
"SanDiego" : [
{ "sky" : "grey" },
{ "air" : "rain" },
{ "ground" : "wet" }
]
}
var docKey = "LosAngeles";
var subKey = "air";
var newValue = "clear";
var query = {};
//var queryKey = docKey + ".$";
query[query] = subKey; // query = { }
var set = {};
var setKey = docKey + ".0." + subKey;
set[setKey] = newValue; // set = { "weather.0.air" : "clear" }
db.collection('weatherSys').update(query, { $set: set }, function(err, result) {
if (err) throw err;
});
UPDATE-1:
Ok, so I was hoping I could find a layout a bit simpler than you had suggested but I failed. Everything I tried was not addressable at the "air" Key level. So I copy and pasted your exact JSON into my collection and ran it. I'm using MongoChef to manipulate and test the collection.
Here is my new layout drived from pasting your JSON in 3 times to create 3 documents:
When I then attempted to update the "San Francisco" document's "air" key I got an unexpected result. Rather than updating "air":"dry" it created a new "air" key in the "San Francisco" Object:
So I thought ok, lets try the update again and see what happens:
As you can see it updated the "air" key that it had previously created. I could fight this out and try to make it work "my" way but I just want it to work so I reconfigure my collection layout again, along the lines of what is "working":
And run the update again:
Then I verify it by running the update again:
It works, I am updating properly in a multi-document environment. So this is my current working collection layout:
I have a couple of questions about this-
I am using the top level Key "weather" in every document. It adds nothing to the information within the document. Is there a layout design change that would not necessitate that Key and the overhead it brings along?
Lets say I have to use the "weather" key. Its value is an array, but that array only has one element, the Object which contains the Keys: city, sky, air, and ground. Does addressing necessitate the use of an array with only one element? Or could I get rid of it. Instead of "weather":[{}] could the design be "weather":{} or would I get into non addressability issues again?
It appears I can now update() any of the Values for the Keys: air, sky, and ground, but what is the find() structure to say READ the Value of the Key "ground" in one of the documents?
----> OK, I think I've got this question #3-
db.weatherSys.find({ "weather.city" : "San Francisco" }, { "weather.ground": 1 })
In the original collection layout that you had suggested, could you explain to me why it did not update as you and I had expected but instead created a new the "city" object?
A lot here. I appreciate your sticking with it.

You can't use positional operator for querying the array by its key.
You can access the weather array by index, but that means you know the array index.
For example if you want to update air element value in weather array.
db.collection('weatherSys').update( {}, { $set: { "weather.1.air" : "clear"} } );
Update:
Unfortunately, I can't see any way to update the values without knowing the array index for key.
You don't need query object as your keys are unique .
db.collection('weatherSys').update( {}, { $set: { "SanFrancisco.1.air" : "clear"} } );
or
Other variant if you want to make sure the key exists.
db.collection('weatherSys').update( { "SanFrancisco": { $exists: true } }, { $set: { "SanFrancisco.1.air" : "clear"} } );
Not sure if you can but if you can update your structure to below.
{
"_id" : ObjectId("58a638fb1831c61917f921c5"),
"weather" : [
{
"city": "LosAngeles",
"sky" : "grey" ,
"air" : "rain" ,
"ground" : "wet"
}
]
}
You can now use $positional operator for update.
db.collection('weatherSys').update( {"weather.city":"LosAngeles"}, { $set: { "weather.$.air" : "clear"} } );
I am using the top level Key "weather" in every document. It adds
nothing to the information within the document. Is there a layout
design change that would not necessitate that Key and the overhead it
brings along?
The only layout that I can think of is promoting all the embedded properties to the top level. Sorry, not sure why I didn't think of this the first time around. Sometimes you just need a right question to get the right answer.
{
"_id" : ObjectId("58a638fb1831c61917f921c5"),
"city": "LosAngeles",
"sky" : "grey",
"air" : "rain",
"ground" : "wet"
}
All the updates will be simply top level updates.
db.collection('weatherSys').update( {"city":"LosAngeles"}, { $set: { "air" : "clear"} } );
Lets say I have to use the "weather" key. Its value is an array, but
that array only has one element, the Object which contains the Keys:
city, sky, air, and ground. Does addressing necessitate the use of an
array with only one element? Or could I get rid of it. Instead of
"weather":[{}] could the design be "weather":{} or would I get into
non addressability issues again?
N/A if you are okay with first suggestion.
It appears I can now update() any of the Values for the Keys: air,
sky, and ground, but what is the find() structure to say READ the
Value of the Key "ground" in one of the documents?
db.weatherSys.find({ "city" : "San Francisco" }, { "ground": 1 })
In the original collection layout that you had suggested, could you
explain to me why it did not update as you and I had expected but
instead created a new the "city" object?
That is a copy paste error. I meant to suggest the working layout you have right now. Updated my previous layout.

Meteor: Return only single object in nested array within collection

I'm attempting to filter returned data sets with Meteor's find().fetch() to contain just a single object, it doesn't appear very useful if I query for a single subdocument but instead I receive several, some not even containing any of the matched terms.
I have a simple mixed data collection that looks like this:
{
"_id" : ObjectId("570d20de3ae6b49a54ee01e7"),
"name" : "Entertainment",
"items" : [
{
"_id" : ObjectId("57a38b5f2bd9ac8225caff06"),
"slug" : "this-is-a-long-slug",
"title" : "This is a title"
},
{
"_id" : ObjectId("57a38b835ac9e2efc0fa09c6"),
"slug" : "mc",
"title" : "Technology"
}
]
}
{
"_id" : ObjectId("570d20de3ae6b49a54ee01e8"),
"name" : "Sitewide",
"items" : [
{
"_id" : ObjectId("57a38bc75ac9e2efc0fa09c9"),
"slug" : "example",
"name" : "Single Example"
}
]
}
I can easily query for a specific object in the nested items array with the MongoDB shell as this:
db.categories.find( { "items.slug": "mc" }, { "items.$": 1 } );
This returns good data, it contains just the single object I want to work with:
{
"_id" : ObjectId("570d20de3ae6b49a54ee01e7"),
"items" : [
{
"_id" : ObjectId("57a38b985ac9e2efc0fa09c8")
"slug" : "mc",
"name" : "Single Example"
}
]
}
However, if a similar query within Meteor is directly attempted:
/* server/publications.js */
Meteor.publish('categories.all', function () {
return Categories.find({}, { sort: { position: 1 } });
});
/* imports/ui/page.js */
Template.page.onCreated(function () {
this.subscribe('categories.all');
});
Template.page.helpers({
items: function () {
var item = Categories.find(
{ "items.slug": "mc" },
{ "items.$": 1 } )
.fetch();
console.log('item: %o', item);
}
});
The outcome isn't ideal as it returns the entire matched block, as well as every object in the nested items array:
{
"_id" : ObjectId("570d20de3ae6b49a54ee01e7"),
"name" : "Entertainment",
"boards" : [
{
"_id" : ObjectId("57a38b5f2bd9ac8225caff06")
"slug" : "this-is-a-long-slug",
"name" : "This is a title"
},
{
"_id" : ObjectId("57a38b835ac9e2efc0fa09c6")
"slug" : "mc",
"name" : "Technology"
}
]
}
I can then of course filter the returned cursor even further with a for loop to get just the needed object, but this seems unscalable and terribly inefficient while dealing with larger data sets.
I can't grasp why Meteor's find returns a completely different set of data than MongoDB's shell find, the only reasonable explanation is both function signatures are different.
Should I break up my nested collections into smaller collections and take a more relational database approach (i.e. store references to ObjectIDs) and query data from collection-to-collection, or is there a more powerful means available to efficiently filter large data sets into single objects that contain just the matched objects as demonstrated above?

The client side implementation of Mongo used by Meteor is called minimongo. It currently only implements a subset of available Mongo functionality. Minimongo does not currently support $ based projections. From the Field Specifiers section of the Meteor API:
Field operators such as $ and $elemMatch are not available on the client side yet.
This is one of the reasons why you're getting different results between the client and the Mongo shell. The closest you can get with your original query is the result you'll get by changing "items.$" to "items":
Categories.find(
{ "items.slug": "mc" },
{ "items": 1 }
).fetch();
This query still isn't quite right though. Minimongo expects your second find parameter to be one of the allowed option parameters outlined in the docs. To filter fields for example, you have to do something like:
Categories.find(
{ "items.slug": "mc" },
{
fields: {
"items": 1
}
}
).fetch();
On the client side (with Minimongo) you'll then need to filter the result further yourself.
There is another way of doing this though. If you run your Mongo query on the server, you won't be using Minimongo, which means projections are supported. As a quick example, try the following:
/server/main.js
const filteredCategories = Categories.find(
{ "items.slug": "mc" },
{
fields: {
"items.$": 1
}
}
).fetch();
console.log(filteredCategories);
The projection will work, and the logged results will match the results you see when using the Mongo console directly. Instead of running your Categories.find on the client side, you could instead create a Meteor Method that calls your Categories.find on the server, and returns the results back to the client.

MongoDB aggregation: How to extract the field in the results

all!
I'm new to MongoDB aggregation, after aggregating, I finally get the result:
"result" : [
{
"_id" : "531d84734031c76f06b853f0"
},
{
"_id" : "5316739f4031c76f06b85399"
},
{
"_id" : "53171a7f4031c76f06b853e5"
},
{
"_id" : "531687024031c76f06b853db"
},
{
"_id" : "5321135cf5fcb31a051e911a"
},
{
"_id" : "5315b2564031c76f06b8538f"
}
],
"ok" : 1
The data is just what I'm looking for, but I just want to make it one step further, I hope my data will be displayed like this:
"result" : [
"531d84734031c76f06b853f0",
"5316739f4031c76f06b85399",
"53171a7f4031c76f06b853e5",
"531687024031c76f06b853db",
"5321135cf5fcb31a051e911a",
"5315b2564031c76f06b8538f"
],
"ok" : 1
Yes, I just want to get all the unique id in a plain string array, is there anything I could do? Any help would be appreciated!

All MongoDB queries produce "key/value" pairs in the result document. All MongoDB content is basically a BSON document in this form, which is just "translated" back to native code form by the driver to the language it is implemented in.
So the aggregation framework alone is never going to produce a bare array of just the values as you want. But you can always just transform the array of results, as after all it is only an array
var result = db.collection.aggregate(pipeline);
var response = result.result.map(function(x) { return x._id } );
Also note that the default behavior in the shell and a preferred option is that the aggregation result is actually returned as a cursor from MongoDB 2.6 and onwards. Since this is in list form rather than as a distinct document you would process differently:
var response = db.collection.aggregate(pipeline).map(function(x) {
return x._id;
})