i have data that looks like this in my database
> db.whocs_up.find()
{ "_id" : ObjectId("52ce212cb17120063b9e3869"), "project" : "asnclkdacd", "users" : [ ] }
and i tried to add to the 'users' array like thus:
> db.whocs_up.update({'users.user': 'usex', 'project' : 'asnclkdacd' },{ '$addToSet': { 'users': {'user':'userx', 'lastactivity' :2387843543}}},true)
but i get the following error:
Cannot apply $addToSet modifier to non-array
same thing happens with push operator, what im i doing wrong?
im on 2.4.8
i tried to follow this example from here:
MongoDB - Update objects in a document's array (nested updating)
db.bar.update( {user_id : 123456, "items.item_name" : {$ne : "my_item_two" }} ,
{$addToSet : {"items" : {'item_name' : "my_item_two" , 'price' : 1 }} } ,
false ,
true)
the python tag is because i was working with python when i ran into this, but it does nto work on the mongo shell as you can see
EDIT ============================== GOT IT TO WORK
apparently if i modify the update from
db.whocs_up.update({'users.user': 'usex', 'project' : 'asnclkdacd' },{ '$addToSet': { 'users': {'user':'userx', 'lastactivity' :2387843543}}},true)
to this:
db.whocs_up.update({'project' : 'asnclkdacd' },{ '$addToSet': { 'users': {'user':'userx', 'lastactivity' :2387843543}}},true)
it works, but can anyone explain why the two do not achieve the same thing, in my understanding they should have referenced the same document and hence done the same thing,
What does the addition of 'users.user': 'userx' change in the update? does it refer to some inner document in the array rather than the document as a whole?
This is a known bug in MongoDB (SERVER-3946). Currently, an update with $push/$addToSet with a query on the same field does not work as expected.
In the general case, there are a couple of workarounds:
Restructure your update operation to not have to query on a field that is also to be updated using $push/$addToSet (as you have done above).
Use the $all operator in the query, supplying a single-value array containing the lookup value. e.g. instead of this:
db.foo.update({ x : "a" }, { $addToSet : { x : "b" } }, true)
do this:
db.foo.update({ x : { $all : ["a"] } }, { $addToSet : { x : "b" } } , true)
In your specific case, I think you need to re-evaluate the operation you're trying to do. The update operation you have above will add a new array entry for each unique (user, lastactivity) pair, which is probably not what you want. I assume you want a unique entry for each user.
Consider changing your schema so that you have one document per user:
{
_id : "userx",
project : "myproj",
lastactivity : 123,
...
}
The update operation then becomes something like:
db.users.update({ _id : "userx" }, { $set : { lastactivity : 456 } })
All users in a given project may still be looked up efficiently by adding a secondary index on project.
This schema also avoids the unbounded document growth of the above schema, which is better for performance.
Related
I want to get data from Firebase.
This is more or less my data structure:
"Reports" : {
"N06Jrz5hx6Q9bcVDBBUrF3GKSTp2" : 2,
"eLLfNlWLkTcImTRqrYnU0nWuu9P2" : 2
},
"Users":{
"N06Jrz5hx6Q9bcVDBBUrF3GKSTp2" : {
"completedWorks" : {
...
},
"reports" : {
"-LHs0yxUXn-TQC7z_MJM" : {
"category" : "Niewyraźne zdjęcie",
"creatorID" : "z8DxcXyehgMhRyMqmf6q8LpCYfs1",
"reportedID" : "N06Jrz5hx6Q9bcVDBBUrF3GKSTp2",
"resolved" : false,
"text" : "heh",
"workID" : "-LHs-aZJkAhEf1RHVasg"
},
"-LHs1hzlL4roUJfMlvyA" : {
"category" : "Zdjęcie nie przedstawia zadania",
"creatorID" : "z8DxcXyehgMhRyMqmf6q8LpCYfs1",
"reportedID" : "N06Jrz5hx6Q9bcVDBBUrF3GKSTp2",
"resolved" : false,
"text" : "",
"workID" : "-LHs-aZJkAhEf1RHVasg"
}
},
"userType" : "company",
"verified" : true
},
}
So as you can see the number of reports is listed in the Reports part. How can I make Firebase return only the ids of the users where the report number is over or equal 3?
Something like this (this will not work, but I hope kind of shows what I was thinking about):
firebase.database().ref('Reports').orderBy(whatHere?).moreThen(2).on('value', snap => {
Is this even doable like this? If yes how could I do it? I want to grab the IDs of the users where reports are >= 3
You're looking for orderByValue():
firebase.database().ref('Reports').orderByValue().startAt(3).on('value', snapshot => {
snapshot.forEach(reportSnapshot => {
console.log(reportSnapshot.key);
})
})
Also check out the Firebase documentation on ordering data.
There are two options for doing that but not exactly the way you wants. You have to use javascript for further processing. One is to use limitToLast after using order by. which will give the last numbers from the result.
firebase.database().ref('Reports').orderBy(reportid).limitToLast(2).on('value', snap => {
Or use startAt and endAt to skip and fetch the result as offset which can provide the data between two reportId.
firebase.database().ref('Reports').orderBy(reportid).
.startAt(reportIdStart)
.endAt(reportIdLast)
.limitToLast(15)
According Firebase documentation:
Using startAt(), endAt(), and equalTo() allows you to choose arbitrary
starting and ending points for your queries
To filter data, you can combine any of the limit or range methods with an order-by method when constructing a query.
Unlike the order-by methods, you can combine multiple limit or range
functions. For example, you can combine the startAt() and endAt()
methods to limit the results to a specified range of values.
For more information go through documentation on filtering data
I have been unable to reach into my MongoDB collection and change a value in a complex document. I have tried more variations than the one example shown below, all sorts of variations, but they fail.
I want to change the Value of the Key "air" from "rain" to "clear". In real life, I will not know that the current Value of the Key "air" is "rain".
Note, I am not using the MongoDB _id Object and would like to accomplish this without using it.
3 documents in the weatherSys collection:
{
"_id" : ObjectId("58a638fb1831c61917f921c5"),
"SanFrancisco" : [
{ "sky" : "grey" },
{ "air" : "rain" },
{ "ground" : "wet" }
]
}
{
"_id" : ObjectId("58a638fb1831c61917f921c6"),
"LosAngeles" : [
{ "sky" : "grey" },
{ "air" : "rain" },
{ "ground" : "wet" }
]
}
{
"_id" : ObjectId("58a638fb1831c61917f921c7"),
"SanDiego" : [
{ "sky" : "grey" },
{ "air" : "rain" },
{ "ground" : "wet" }
]
}
var docKey = "LosAngeles";
var subKey = "air";
var newValue = "clear";
var query = {};
//var queryKey = docKey + ".$";
query[query] = subKey; // query = { }
var set = {};
var setKey = docKey + ".0." + subKey;
set[setKey] = newValue; // set = { "weather.0.air" : "clear" }
db.collection('weatherSys').update(query, { $set: set }, function(err, result) {
if (err) throw err;
});
UPDATE-1:
Ok, so I was hoping I could find a layout a bit simpler than you had suggested but I failed. Everything I tried was not addressable at the "air" Key level. So I copy and pasted your exact JSON into my collection and ran it. I'm using MongoChef to manipulate and test the collection.
Here is my new layout drived from pasting your JSON in 3 times to create 3 documents:
When I then attempted to update the "San Francisco" document's "air" key I got an unexpected result. Rather than updating "air":"dry" it created a new "air" key in the "San Francisco" Object:
So I thought ok, lets try the update again and see what happens:
As you can see it updated the "air" key that it had previously created. I could fight this out and try to make it work "my" way but I just want it to work so I reconfigure my collection layout again, along the lines of what is "working":
And run the update again:
Then I verify it by running the update again:
It works, I am updating properly in a multi-document environment. So this is my current working collection layout:
I have a couple of questions about this-
I am using the top level Key "weather" in every document. It adds nothing to the information within the document. Is there a layout design change that would not necessitate that Key and the overhead it brings along?
Lets say I have to use the "weather" key. Its value is an array, but that array only has one element, the Object which contains the Keys: city, sky, air, and ground. Does addressing necessitate the use of an array with only one element? Or could I get rid of it. Instead of "weather":[{}] could the design be "weather":{} or would I get into non addressability issues again?
It appears I can now update() any of the Values for the Keys: air, sky, and ground, but what is the find() structure to say READ the Value of the Key "ground" in one of the documents?
----> OK, I think I've got this question #3-
db.weatherSys.find({ "weather.city" : "San Francisco" }, { "weather.ground": 1 })
In the original collection layout that you had suggested, could you explain to me why it did not update as you and I had expected but instead created a new the "city" object?
A lot here. I appreciate your sticking with it.
You can't use positional operator for querying the array by its key.
You can access the weather array by index, but that means you know the array index.
For example if you want to update air element value in weather array.
db.collection('weatherSys').update( {}, { $set: { "weather.1.air" : "clear"} } );
Update:
Unfortunately, I can't see any way to update the values without knowing the array index for key.
You don't need query object as your keys are unique .
db.collection('weatherSys').update( {}, { $set: { "SanFrancisco.1.air" : "clear"} } );
or
Other variant if you want to make sure the key exists.
db.collection('weatherSys').update( { "SanFrancisco": { $exists: true } }, { $set: { "SanFrancisco.1.air" : "clear"} } );
Not sure if you can but if you can update your structure to below.
{
"_id" : ObjectId("58a638fb1831c61917f921c5"),
"weather" : [
{
"city": "LosAngeles",
"sky" : "grey" ,
"air" : "rain" ,
"ground" : "wet"
}
]
}
You can now use $positional operator for update.
db.collection('weatherSys').update( {"weather.city":"LosAngeles"}, { $set: { "weather.$.air" : "clear"} } );
I am using the top level Key "weather" in every document. It adds
nothing to the information within the document. Is there a layout
design change that would not necessitate that Key and the overhead it
brings along?
The only layout that I can think of is promoting all the embedded properties to the top level. Sorry, not sure why I didn't think of this the first time around. Sometimes you just need a right question to get the right answer.
{
"_id" : ObjectId("58a638fb1831c61917f921c5"),
"city": "LosAngeles",
"sky" : "grey",
"air" : "rain",
"ground" : "wet"
}
All the updates will be simply top level updates.
db.collection('weatherSys').update( {"city":"LosAngeles"}, { $set: { "air" : "clear"} } );
Lets say I have to use the "weather" key. Its value is an array, but
that array only has one element, the Object which contains the Keys:
city, sky, air, and ground. Does addressing necessitate the use of an
array with only one element? Or could I get rid of it. Instead of
"weather":[{}] could the design be "weather":{} or would I get into
non addressability issues again?
N/A if you are okay with first suggestion.
It appears I can now update() any of the Values for the Keys: air,
sky, and ground, but what is the find() structure to say READ the
Value of the Key "ground" in one of the documents?
db.weatherSys.find({ "city" : "San Francisco" }, { "ground": 1 })
In the original collection layout that you had suggested, could you
explain to me why it did not update as you and I had expected but
instead created a new the "city" object?
That is a copy paste error. I meant to suggest the working layout you have right now. Updated my previous layout.
This code uses a loop to $unset the "checked" property of all embedded documents in the "value" array, then $set the one when a condition evaluates to true.
But when the condition is true, the update block failed to update the embedded document by setting a checked: "checked", I know that because meteor:PRIMARY> db.radioOptions.find({}).pretty(); gives the same results before and after.
What am I doing wrong? and how to fix it? Thanks
meteor:PRIMARY> db.radioOptions.find({}).pretty();
{
"_id" : "jXQcsXtedQYotKQXG",
"name" : "optionType",
"value" : [
{
"name" : "1stOption",
"caption" : "1st Option"
},
{
"name" : "2ndOption",
"caption" : "2nd Option"
}
]
}
var doc = RadioOptions.findOne({name: obj.name});
if (typeof doc != 'undefined') {
doc.value.forEach(function (embdoc) {
console.log(embdoc);
RadioOptions.update(
{name: obj.name, 'value.name': obj.value},
{$unset: {'value.$.checked': ""}}
);
if (embdoc.name == obj.value) {
console.log(obj.name + " " + obj.value); //obj.value = 1stOption for example
RadioOptions.update(
{name: obj.name, 'value.name': obj.value}, //obj.name = "optionType"
{$set: {'value.$.checked': "checked"}}
);
}
})
}
Let's say that is was your objective to set the array element with the name "2ndOption" to "checked" and $unset all other array elements. You would then instead do:
var doc = RadioOptions.findOne({name: obj.name});
if (typeof doc != 'undefined') {
// You have to update every element
doc.value.forEach(function (embdoc) {
RadioOptions.update(
{ "_id": doc._id, "value.name": embdoc.name },
{ "$unset": { "value.$.checked": "" } }
)
});
// Then just set the one you want
RadioOptions.update(
{ "_id": doc._id, "value.name": "2ndOption" }, // <-- actually using a variable of course
{ "$set": { "value.$.checked": "checked" } }
)
}
Now if you had actually read all the responses on the duplicate question you were given for your original question:
How to Update Multiple Array Elements in mongodb
Then you would have seen the response there that mentioned the best way to handle all these mutliple updates was using the "Bulk" API methods in the underlying driver. And also has some useful tips on the general process of updating multiple elements.
On the "server" (not in minimongo) is the best place to to this, and all meteor collections have a .rawCollection() method which returns the collection object from the underlying node native driver.
So the general logic is:
Loop all array elements and update to "turn off"
Match the element you want to "turn on"
And best done in Bulk rather than responding back and forth for each update.
I am trying to find duplicates in a Mongo version 2.4 database that is being used for production and therefore cannot be updated. Since aggregate does not exist in 2.4, I cannot use the aggregate pipeline to find duplicates, therefore I am trying to find a solution using MapReduce.
I have tried the following set of map, reduce, and finalize functions, through MongoVUE's Map Reduce interface, and they returned nothing after running for less than a second on a 3,000,000 record collection that definitely has duplicates on the indicated fields. Obviously something went wrong, but MongoVUE did not show any error messages or helpful indications.
function Map() {
emit(
{name: this.name, LocationId: this.LocationId,
version: this.version},
{count:1, ScrapeDate: this.ScrapeDate}
);
}
function Reduce(key, values) {
var reduced = {count:0, ScrapeDate:''2000-01-01''};
values.forEach(function(val) {
reduced.count += val.count;
if (reduced.ScrapeDate.localeCompare(val.ScrapeDate) < 0)
reduced.ScrapeDate=val.ScrapeDate;
});
return reduced;
return values[0];
}
function Finalize(key, reduced) {
if (reduced.count > 1)
return reduced;
}
I just need to find any instance of multiple records that share the same name, LocationId, and version, and ideally display the most recent ScrapeDate of such a record.
Your map-reduce code worked without any issues, though for a very small dataset. I think return values[0]; in the reduce function would be a copy paste error. You could try the same through the mongo shell.
Since aggregate does not exist in 2.4, I cannot use the aggregate pipeline to find duplicates, therefore I am trying to find a solution
using MapReduce.
You got it wrong here, db.collection.aggregate(pipeline, options) was introduced in the version 2.2.
Here is how it could be done with the aggregation framework, but it would not be preferred since your dataset is very huge, and the $sort operator has memory limit of 10% of RAM, in v2.4.
db.collection.aggregate(
[
// sort the records, based on the 'ScrapeDate' field, in descending order.
{$sort:{"ScrapeDate":-1}},
// group by the key fields, and take the 'ScrapeDate' of the first document,
// Since it is in sorted order, the first document would contain the
// highest field value.
{$group:{"_id":{"name":"$name","LocationId":"$LocationId","version":"$version"}
,"ScrapeDate":{$first:"$ScrapeDate"}
,"count":{$sum:1}}
},
// output only the group, having documents greater than 1.
{$match:{"count":{$gt:1}}}
]
);
Coming to your Map-reduce functions, it ran without issues on my test data.
db.collection.insert({"name":"c","LocationId":1,"version":1,"ScrapeDate":"2000-01-01"});
db.collection.insert({"name":"c","LocationId":1,"version":1,"ScrapeDate":"2001-01-01"});
db.collection.insert({"name":"c","LocationId":1,"version":1,"ScrapeDate":"2002-01-01"});
db.collection.insert({"name":"d","LocationId":1,"version":1,"ScrapeDate":"2002-01-01"});
running the map-reduce,
db.collection.mapReduce(Map,Reduce,{out:{"inline":1},finalize:Finalize});
o/p:
{
"results" : [
{
"_id" : {
"name" : "c",
"LocationId" : 1,
"version" : 1
},
"value" : {
"count" : 3,
"ScrapeDate" : "2002-01-01"
}
},
{
"_id" : {
"name" : "d",
"LocationId" : 1,
"version" : 1
},
"value" : null
}
],
"timeMillis" : 0,
"counts" : {
"input" : 4,
"emit" : 4,
"reduce" : 1,
"output" : 2
},
"ok" : 1,
}
Notice that the output contains value:null for a record which doesn't have any duplicates.
This is due to your finalize function:
function Finalize(key, reduced) {
if (reduced.count > 1)
return reduced; // returned null by default for keys with single value,
// i.e count=1
}
The finalize function do not filter out keys. So you can't get only the keys that are duplicates. You will get all the keys, in the map-reduce output. In your finalize functions, you can just not show their values, which is what you are doing.
all!
I'm new to MongoDB aggregation, after aggregating, I finally get the result:
"result" : [
{
"_id" : "531d84734031c76f06b853f0"
},
{
"_id" : "5316739f4031c76f06b85399"
},
{
"_id" : "53171a7f4031c76f06b853e5"
},
{
"_id" : "531687024031c76f06b853db"
},
{
"_id" : "5321135cf5fcb31a051e911a"
},
{
"_id" : "5315b2564031c76f06b8538f"
}
],
"ok" : 1
The data is just what I'm looking for, but I just want to make it one step further, I hope my data will be displayed like this:
"result" : [
"531d84734031c76f06b853f0",
"5316739f4031c76f06b85399",
"53171a7f4031c76f06b853e5",
"531687024031c76f06b853db",
"5321135cf5fcb31a051e911a",
"5315b2564031c76f06b8538f"
],
"ok" : 1
Yes, I just want to get all the unique id in a plain string array, is there anything I could do? Any help would be appreciated!
All MongoDB queries produce "key/value" pairs in the result document. All MongoDB content is basically a BSON document in this form, which is just "translated" back to native code form by the driver to the language it is implemented in.
So the aggregation framework alone is never going to produce a bare array of just the values as you want. But you can always just transform the array of results, as after all it is only an array
var result = db.collection.aggregate(pipeline);
var response = result.result.map(function(x) { return x._id } );
Also note that the default behavior in the shell and a preferred option is that the aggregation result is actually returned as a cursor from MongoDB 2.6 and onwards. Since this is in list form rather than as a distinct document you would process differently:
var response = db.collection.aggregate(pipeline).map(function(x) {
return x._id;
})