We Keep Coding

find query to find retrieve the data of specific element in mongodb

I have a document in my mongo instance in below format,
{
"_id" : "08d4a242-08fb-07f7-46e5-8717a81d5b70",
"fname" : "john",
"created_date" : ISODate("2017-05-24T01:13:06.829Z"),
"customProp" : [
[
"customX","{\"some data related to X \"}"
],
[
"customY","{\"some data related to Y \"}"
],
[
"customZ","{\"some data related to Z \"}"
]
]
}
the elements/values like "customX","customY" & "customZ" are not necessarily be in all documents. How to retrieve all the values in second element of "customProp" array, in this document it contains "customZ"?
I'm able to use following query to filter & find all the documents which are having "customZ" element,
db.getCollection('col1').find({$and : [{"customProp":{$elemMatch:{0:"customZ"}}}, {"created": { $gte: ISODate("2017-05-22T00:00:00.000Z") }}] },{"created":1}).limit(1) .pretty()
output:
{
"_id" : "08d4a242-08fb-07f7-46e5-8717a81d5b45",
"created" : ISODate("2017-05-24T01:13:06.829Z")
}
but finding a way to retrieve all the values in second element of array where the first value is "customZ".
expected result:
{
"_id" : "08d4a242-08fb-07f7-46e5-8717a81d5b45",
"created" : ISODate("2017-05-24T01:13:06.829Z"),
"customPro": ["customZ","{\"some data related to Z \"}"]
}
I'm fine if my query just returns
{
"{\"some data related to Z \"}"
}

Well it is a nested array, which is not a great idea but you are in fact matching the element with the $elemMatch expression, so you do get the position in the "outer" array of customProp, which allows you to project with the positional $ operator:
db.getCollection('coll1').find(
{
"customProp":{ "$elemMatch": { "0": "customZ" } },
"created_date": { "$gte": ISODate("2017-05-22T00:00:00.000Z") }
},
{ "created_date": 1, "customProp.$": 1 }
)
That yields the result:
{
"_id" : "08d4a242-08fb-07f7-46e5-8717a81d5b70",
"created_date" : ISODate("2017-05-24T01:13:06.829Z"),
"customProp" : [
[
"customZ",
"{\"some data related to Z \"}"
]
]
}
Where customProp is of course still in a nested array, but when processing the individual documents in python you can just access the property at the array index:
doc['customProp'][0][1]
Which of course returns the value:
'{"some data related to Z "}'
Same goes for JavaScript really, which is basically identical in syntax. As a shell example:
db.getCollection('coll1').find(
{
"customProp":{ "$elemMatch": { "0": "customZ" } },
"created_date": { "$gte": ISODate("2017-05-22T00:00:00.000Z") }
},
{ "created_date": 1, "customProp.$": 1 }
).map(function(doc) {
doc['customProp'] = doc['customProp'][0][1];
return doc;
})
And the output:
{
"_id" : "08d4a242-08fb-07f7-46e5-8717a81d5b70",
"created_date" : ISODate("2017-05-24T01:13:06.829Z"),
"customProp" : "{\"some data related to Z \"}"
}
And the positional $ projection here ensures there is only one element in the returned array, so the notation is always the same to extract from all document results. So you get the matched element from the database, and you extract the property through the code.
Also note that you do not need $and here since all the query arguments are already AND conditions. This is the MongoDB default, so you do not need to explicitly express it. See how much nicer this looks without it.

How to exclude a document if two fields are the same?

After performing some aggregation magic, I have arrived at this data:
{ "_id" : "5700edfe03fcdb000347bebb", "size" : 3, "count" : 2 }
{ "_id" : "5700edfe03fcdb000347bebf", "size" : 2, "count" : 2 }
Now, I want to eliminate all the entries where size is equal to count.
So I ran this aggregation instruction:
match3 = { "$match" : { "size" : { "$ne" : "count"} } }
But it doesn't eliminate anything and returns the two lines as it is.
I want the result to be just this one line as it is the only one where size is not equal to count:
{ "_id" : "5700edfe03fcdb000347bebb", "size" : 3, "count" : 2 }

You need to add a $redact stage to your aggregation pipeline:
{ "$redact": {
"$cond": [
{ "$eq": [ "$size", "$count" ] },
"$$PRUNE",
"$$KEEP"
]
}}

You can use the $where operator for this
db.collection.find({ $where: "this.size != this.count" })
db.collection.remove({ $where: "this.size != this.count" })
UPDATE:
After I got downvoted I decided to compare the 2 solutions.
Both use a COLLSCAN and both return the same results.
So please enlighten me what is so wrong about my solution? :)

Only return some of the fields from list of embedded documents

Here is my document:
{
"_id" : "2",
"account" : "1234",
"positions" : {
"APPL" : { "quantity" : "13", "direction" : "long" },
"GOOG" : { "quantity" : "24", "direction" : "long" }
}
}
I would like to get the whole positions object, but only the quantity field and ignore the direction field. Is it possible to do that? Or should I consider this other schema (array of objects):
{
"_id" : "2",
"account" : "1234",
"positions" : [
{ "name" : "APPL", "quantity" : "13", "direction" : "long" },
{ "name" : "GOOG", "quantity" : "24", "direction" : "long" }
]
}
Many thanks!

For the "array" form, all you really need to do is specify the field using "dot notation" to the array member:
db.collection.find({}, { "positions.quantity": 1, })
Which would return:
{
"_id" : "2",
"positions" : [
{ "quantity" : "13" },
{ "quantity" : "24" }
]
}
Or for multiple fields but excluding the "direction" just use both in projection:
db.collection.find({},{ "positions.name": 1, "positions.quantity": 1 })
Which returns the named fields still:
{
"_id" : "2",
"positions" : [
{
"name" : "APPL",
"quantity" : "13"
},
{
"name" : "GOOG",
"quantity" : "24"
}
]
}
For the "named keys" form you need to specify each path:
db.collection.find({},{ "positions.APPL.quantity": 1, "positions.GOOG.quantity": 1 })
Which would return of course:
{
"_id" : "2",
"positions" : {
"APPL" : {
"quantity" : "13"
},
"GOOG" : {
"quantity" : "24"
}
}
}
And that kind of "nastiness" is pervasive with basically ALL MongoDB operations, query or projection or otherwise. When you used "named keys" the "database" has no sane option other than to require you to "name the path". Doing that is of course not really a practical exercise, when the names of keys are likely to differ between documents in the collection.
Traversing keys can only really be done in JavaScript evaluation from a MongoDB standpoint. Since JavaScript evaluation requires interpreter cost in launching and translating data from BSON to a workable JavaScript format, and not to mention the actual cost of evaluating the coded expressions themselves, that is not an ideal approach.
Moreover, from a "query" perspective such handling requires the use of $where to evaluate such an expression where you just want to look for things under each "key" of the "positions" data. This is a "bad" thing, since such an expression cannot possible use an "index" to optimize the query search. Only with a "directly named path" can you actually use or even "create" an index under those conditions.
From a "projection" perspective, the usage of "named keys" means that by similar "traversal" concepts, you actually need JavaScript processing again to do so. And the only mechanism in which MongoDB can use a JavaScript expression to "alter" the output document is by using mapReduce so again this is "super horrible" and you would be using this "aggregation method" for nothing more than document manipulation in this case:
db.collection.mapReduce(
function() {
var id = this._id;
delete this._id;
Object.keys(this.positions).forEach(function(el) {
delete el.direction;
});
emit(id,this);
},
function() {}, // reducer never gets called when all are unique
{ "out": { "inline": 1 } }
)
Even after you did that to just avoid naming paths, the output of mapReduce cannot be a "cursor". So this limits you to either the size of a BSON document in response or actually outputting to a "collection" instead. So this is as far from "practical" as you can get.
There are numerous reasons "why" using an array with "common paths" is so much better than a "named keys" structure that are also far too broad to go into here. The one thing you should accept is that "named keys are bad, okay!" and just move forward with consistent object naming that actually makes quite a lot of sense.

How to extract an array of fields from an array of JSON documents?

I have 2 mongodb collections, stu_creds and stu_profile. I first want to retrieve all the student records from stu_creds where stu_pref_contact is the email and then for those stu_ids I want to retrieve the complete profile from stu_profile. The problem is, the first query returns an array of JSON documents, with each document holding one field, the stu_id. Here is my query and the result:
db.stu_creds.find({"stu_pref_contact" : "email"}, {'_id' : 1})
Result:
[{ "_id" : ObjectId("51927cc93080baac04000001") },
{ "_id" : ObjectId("51927d7b3080baac04000002") },
{ "_id" : ObjectId("519bb011c5c5035b2a000002") },
{ "_id" : ObjectId("519ce3d09f047a192b000010") },
{ "_id" : ObjectId("519e6dc0f919cfdc66000003") },
{ "_id" : ObjectId("51b39be0c74f0e3d23000012") },
{ "_id" : ObjectId("51b39ca9c74f0e3d23000014") },
{ "_id" : ObjectId("51b39cb7c74f0e3d23000016") },
{ "_id" : ObjectId("51b39e87c74f0e3d23000018") },
{ "_id" : ObjectId("51b39f2fc74f0e3d2300001a") },
{ "_id" : ObjectId("51b39f47c74f0e3d2300001c") },
{ "_id" : ObjectId("518d454deb1e3a525e000009") },
{ "_id" : ObjectId("51bc8381dd10286e5b000002") },
{ "_id" : ObjectId("51bc83f7dd10286e5b000004") },
{ "_id" : ObjectId("51bc85cbdd10286e5b000006") },
{ "_id" : ObjectId("51bc8630dd10286e5b000008") },
{ "_id" : ObjectId("51bc8991dd10286e5b00000a") },
{ "_id" : ObjectId("51bc8a43dd10286e5b00000c") },
{ "_id" : ObjectId("51bc8a7ddd10286e5b00000e") },
{ "_id" : ObjectId("51bc8acadd10286e5b000010") }]
The thing is, I don't think I can use the above array as part of an $in clause for my second query to retrieve the student profiles. I have to walk through the array and and create a new array which is just an array of object ids rather than an array of JSON docs.
Is there an easier way to do this?

Use Array.map (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map). This allows you to perform a transform on each element of the array, returning you a new array of the transformed items.
var arrayOfIds = result.map(function(item){ return item._id; });
Array.map was introduced in ECMAScript 5. If you're using node.js, a modern browser, or an Array polyfill, it should be available to use.

Ummm, am I missing something or is all you want the following:
var results = [];
for(var i = 0; i < yourArray.length; i++) {
results.push(yourArray[i]._id);
}

You could use $or:
db.stu_profile.find({ $or : results }) // `results` is your list of ObjectId's
But it's considerably slower than $in, so I would suggest using one of the other answers ;)

Ordering JSON Objects using jQuery

I have the following JSON Object being loaded into my application and stored into a var called obj:
{
"items" : [
{
"name" : "item-1",
"group" : [
{
"groupName" : "name-1",
"groupPosition" : 2
},
{
"groupName" : "name-2",
"groupPosition" : 1
}]
},
{
"name" : "item-2",
"group" : [
{
"groupName" : "name-1",
"groupPosition" : 1
},
{
"groupName" : "name-2",
"groupPosition" : 2
}]
}]
}
I then do the following to go through it:
var groups = new Array();
var items = new Array();
$.each(obj.items, function(i,r){
var itemName = r.name;
$.each(r.group, function(index, record){
if ($.inArray(record.groupName) == -1) {
groups.push(record.groupName);
$('body').append('<div id="record.groupName"></div>');
}
$('#'+record.groupName).append('<div id="itemName">itemName</div>');
// At this point I'm stuck as the items get added in order of iteration,
// not according to their record.groupPosition value.
});
});
There will eventually be several hundred "items" each contained within an unset number of "groups".
The trouble I'm having is how to iterate through the JSON object using jQuery or good ol'JavaScript and display the items in the correct position within each group as the items and groups won't be listed inside the JSON object in sequential order.
Any help would be greatly appreciated!
Thank you.

Why not just give the group items the position index like this:
{
"items" : [
{
"name" : "item-1",
"group" : {
2:{
"groupName" : "name-1",
"groupPosition" : 2
},
1:{
"groupName" : "name-2",
"groupPosition" : 1
}}
},
{
"name" : "item-2",
"group" : {
1:{
"groupName" : "name-1",
"groupPosition" : 1
},
2:{
"groupName" : "name-2",
"groupPosition" : 2
}}
}]
}

Assuming you have a variable which is assigned to this:
var data = ...
you could use the $.each() method:
$.each(data.items, function(index, item) {
// here item.name will contain the name
// and if you wanted to fetch the groups you could loop through them as well:
$.each(item.group, function(i, group) {
// here you can use group.groupName and group.groupPosition
});
});
Arrays ([]) in javascript are 0 index based and preserve their order when you are iterating over them.

If I understood correctly your problem it is not about the sorting it self but how to link them to your dom nodes, solution: use classes with numbers.
For example:
$(".group"+items[1].group[0].grouposition").append(items[1].group[0].name);
// this will give append to the element with class="group1"
If you join this with having the html structure that is being generated to match the same names, then it won't be a problem and you don't have to sort them