We Keep Coding

How to delete all documents except most recent for each group in mongo? [SOLVED]

I have a collection with this data registered
{
_id: 0000120210903, iid: 00001, date: 20210903 }, {
_id: 0000220210903, iid: 00002, date: 20210903 }, {
_id: 0000120210101, iid: 00001, date: 20210101 }
I want to delete all except the document with the most recent date for each iid.
My idea is to group by the date, select the _id of the register with the max(date) and then delete all except this array of _ids. But I can't figure out how to do it.
db.getCollection('testing_data').aggregate(
{ $sort:{ _id:1 }},
{ $group:{
_id:"$iid",
lastId:{ "$last":"$_id" },
}},
{ $project:{ _id: 0, lastId: 1 } }
)
But I don't know where to go from here. Any help is greatly appreciated.
[Solution]
To fix the problem I used an aggregation to recover the combination of the field iid (the identifier shared between documents) and the unique _id as an array.
Then for each element on the array it performs a deleteMany operation on the iid but letting out the most recent _id. In this case I sort by _id because it includes the date but could also sort by the field date.
Due to the high volume of data { allowDiskUse: true } had to be put in the aggregate.
var ids = db.getCollection('testing_data').aggregate([
{ $sort:{ _id:1 }},
{ $group:{
_id:"$iid",
lastId:{ "$last":"$_id" },
}},
{ $project:{ _id: 1, lastId: 1 } }
], { allowDiskUse: true } ).toArray();
ids.forEach(function(x){
db.getCollection('testing_data').deleteMany({ "iid": x._id, "_id": {$ne:x.lastId} })
});

Mine Idea is just stock all _ids at some array that you want to delete, and then use deleteMany with $or filter
db.getCollection("testing_data").find({}).toArray((err,data)=>{
let to_elim = [];
let filtering ={};
for(let el of data){
if(!filtering[el.iid]) filtering[el.iid] = el;
else {
if(filtering[el.iid].date>el.date) to_elim.push({_id:new ObjectID(el._id)})
}
}
db.getCollection("testing_data").deleteMany({$or:to_elim})
})
I hope that all is written rightly, cause wrote all that down on mobile
There is missing some checking if something more recent...

[Solution]
To fix the problem I used an aggregation to recover the combination of the field iid (the identifier shared between documents) and the unique _id as an array.
Then for each element on the array it performs a deleteMany operation on the iid but letting out the most recent _id. In this case I sort by _id because it includes the date but could also sort by the field date.
Due to the high volume of data { allowDiskUse: true } had to be put in the aggregate.
var ids = db.getCollection('testing_data').aggregate([
{ $sort:{ _id:1 }},
{ $group:{
_id:"$iid",
lastId:{ "$last":"$_id" },
}},
{ $project:{ _id: 1, lastId: 1 } }
], { allowDiskUse: true } ).toArray();
ids.forEach(function(x){
db.getCollection('testing_data').deleteMany({ "iid": x._id, "_id": {$ne:x.lastId} })
});

MongoDB/Mongoose - Adding an object to an array of objects only if a certain field is unique

So I have a nested array of objects in my MongoDB document and I would like to add a new object to the array only if a certain field (in this case, eventId) is unique. My question is very similar to this post, only I cannot seem to get that solution to work in my case.
Here is what the documents (UserModel) look like:
{
"portal" : {
"events" : [
{
"important" : false,
"completed" : false,
"_id" : ObjectId("5c0c2a93bb49c91ef8de0b21"),
"eventId" : "5bec4a7361853025400ee9e9",
"user_notes" : "My event note"
},
...and so on
]
}
}
And here is my (so far unsuccessful) Mongoose operation:
UserModel.findByIdAndUpdate(
userId,
{ "portal.events.eventId": { $ne: req.body.eventId } },
{ $addToSet: { "portal.events": req.body } },
{ new: true }
);
Basically I am trying to use '$ne' to check if the field is unique, and then '$addToSet' (or '$push', I believe they are functionally equivalent in this case) to add the new object.
Could anyone point me in the right direction?
Cheers,
Gabe

If you look into the documentation on your method you will see that the parameters passed are not in the proper order.
findByIdAndUpdate(id, update, options, callback)
I would use update instead and have your id and portal.events.eventId": { $ne: req.body.eventId } part of the initial filter followed by $addToSet: { "portal.events": req.body }
Something among these lines:
UserModel.update(
{
"_id": mongoose.Types.ObjectId(userId),
"portal.events.eventId": { $ne: req.body.eventId }
},
{ $addToSet: { "portal.events": req.body } },
{ new: true }
);

You need to include your eventId check into condition part of your query. Because you're usig findByIdAndUpdate you can only pass single value matched against _id as a condition. Therefore you have to use findOneAndUpdate to specify custom filtering condition, try:
UserModel.findOneAndUpdate(
{ _id: userId, "portal.events.eventId": { $ne: req.body.eventId } },
{ $addToSet: { "portal.events": req.body } },
{ new: true }
);

Mongoose update previously fetched collection

I would like to update a collection. Docs seem unclear on this.
I am wondering how to achieve the following:
Order.find({ _id: { $in: ids }}).exec(function(err, items, count) {
// Following gives error - same with save()
items.update({ status: 'processed'}, function(err, docs) {
});
});
I know how to batch save like this:
Model.update({ _id: id }, { $set: { size: 'large' }}, { multi: true }, callback);
But that requires setting my query again.
I've also tried:
Order.collection.update(items...
But that throws a max call stack error.

In mongoose, model.find(callback), return an Array of Document via callback. You can call save on a Document but not on an Array. So you can use for loop or forEach on the Array.
Order
.find({ _id: { $in: ids}})
.exec(function(err, items, count) {
items.forEach(function (it) {
it.save(function () {
console.log('you have saved ', it)
});
})
});

Multiple populates - mongoosejs

Just a simple query, for example with a double ref in the model.
Schema / Model
var OrderSchema = new Schema({
user: {
type : Schema.Types.ObjectId,
ref : 'User',
required: true
},
meal: {
type : Schema.Types.ObjectId,
ref : 'Meal',
required: true
},
});
var OrderModel = db.model('Order', OrderSchema);
Query
OrderModel.find()
.populate('user') // works
.populate('meal') // dont works
.exec(function (err, results) {
// callback
});
I already tried something like
.populate('user meal')
.populate(['user', 'meal'])
In fact only one of the populates works.
So, how do is get two populates working ?

You're already using the correct syntax of:
OrderModel.find()
.populate('user')
.populate('meal')
.exec(function (err, results) {
// callback
});
Perhaps the meal ObjectId from the order isn't in the Meals collection?

UPDATE:
This solution remains for the version 3.x of Mongoose http://mongoosejs.com/docs/3.8.x/docs/populate.html but is no longer documented for >= 4.x versions of Mongoose and so the answer from #JohnnyHK is the only valid one for now on.
ORIGINAL POST
If you're using Mongoose >= 3.6, you can pass a space delimited string of the path names to populate:
OrderModel.find()
.populate('user meal')
.exec(function (err, results) {
// callback
});
http://mongoosejs.com/docs/populate.html

This has probably been resolved already, but this is my take on multiple & deep population in Mongodb > 3.6:
OrderModel.find().populate([{
path: 'user',
model: 'User'
}, {
path: 'meal',
model: 'Meal'
}]).exec(function(err, order) {
if(err) throw err;
if(order) {
// execute on order
console.log(order.user.username); // prints user's username
console.log(order.meal.value); // you get the idea
}
});
There are probably other ways to do this, but this makes very readable code for beginners (like me)

The best solution in my opinion is arrays when you are populating more than one foreign field on the same level. My code shows that I have multiple populates for different levels.
const patients = await Patient.find({})
.populate([{
path: 'files',
populate: {
path: 'authorizations',
model: 'Authorization'
},
populate: {
path: 'claims',
model: 'Claim',
options: {
sort: { startDate: 1 }
}
}
}, {
path: 'policies',
model: 'Policy',
populate: {
path: 'vobs',
populate: [{
path: 'benefits'
}, {
path: 'eligibility',
model: 'Eligibility'
}]
}
}]);
As you can see, wherever I needed more than one field of a document populated, I encased the populate key in an array and provided an array of objects, each object having a different path. Most robust and concise way to do it, in my opinion.

You can use array syntax:
let results = await OrderModel.find().populate(['user', 'meal']);
You can also select which properties you want from each populate:
let results = await OrderModel.find().populate([{path: 'user', select: 'firstname'}, {path: 'meal', select: 'name'}]);

Latest mongoose v5.9.15
has ability to take array of populate fields
so you can do,
.populate([ 'field1', 'field2' ])

You can try:
OrderModel.find()
.populate('user')
.populate('meal')
.exec(function (err, results) {
// callback
});
or with array options
OrderModel.find()
.populate([
{
path: "path1",
select: "field",
model: Model1
},
{
path: "path2",
select: "field2",
model: Model2
}
])
.exec(function (err, results) {
// callback
});

In model file do something like:-
doctorid:{
type:Schema.Types.ObjectId, ref:'doctor'
},
clinicid:{
type:Schema.Types.ObjectId, ref:'baseClinic'
}
In js file for adding operator use Something like:-
const clinicObj = await BaseClinic.findOne({clinicId:req.body.clinicid})
const doctorObj = await Doctor.findOne({ doctorId : req.body.doctorid}) ;
**and add data as:-**
const newOperator = new Operator({
clinicid:clinicObj._id,
doctorid: doctorObj._id
});
Now, while populating
apiRoutes.post("/operator-by-id", async (req, res) => {
const id = req.body.id;
const isExist = await Operator.find({ _id: id }).populate(['doctorid','clinicid'])
if (isExist.length > 0) {
res.send(isExist)
} else {
res.send("No operator found");
}
});

i have same problem , but my mistake not in populate , i have an error in Model
if you do this
uncorrected
user: {
type: [Schema.Types.ObjectId],
ref: 'User'
}
correct
user: [{
type: Schema.Types.ObjectId,
ref: 'User'
}]
you must put array around of object like this

To populate multiple fields with array of objects in controller/action function, model of both is already referred in schema of post
post.find({}).populate('user').populate('comments').exec(function (err,posts)
{
if(err)
{
console.log("error in post");
}
return res.render('home',{
h1:"home Page",
posts:posts,
});
});

I think you are trying to the nested population you can visit official docs
User.
findOne({ name: 'Val' }).
populate({
path: 'friends',
// Get friends of friends - populate the 'friends' array for every friend
populate: { path: 'friends' }
});

Updating a property in an array after querying with $ne

A document in my collection rooms has the following structure:
{
"_id": { "$oid": "4edaaa4a8d285b9311eb63ab" },
"users": [
{
"userId": { "$oid": "4edaaa4a8d285b9311eb63a9" },
"unreadMessages": 0
},
{
"userId": { "$oid": "4edaaa4a8d285b9311eb63aa" },
"unreadMessages": 0
},
]
}
I'd like to increment unreadMessages of the second user (the one at index 1 in the array), if I know the ObjectId of the first user. Knowing the _id of the document is sufficient for selecting, but for updating I need to be able to reference to the other user, so I'm also selecting by users. I'm doing so with the following update call:
collections.rooms.update(
{
_id: ObjectId("4edaaa4a8d285b9311eb63ab"),
users: { // not first user
$elemMatch: { $ne: { userId: ObjectId("4edaaa4a8d285b9311eb63a9") } }
}
},
{
// this should increment second user's unreadMessages,
// but does so for unreadMessages of first user
$inc: { "users.$.unreadMessages": 1 }
}
);
It increments unreadMessages of the first user, not of the second user. I guess this is because $ne actually matches on the first user, and apparently that's what users.$ is referring to.
How should I modify this call so that the second user's unreadMessages is incremented?

I think you're doing the right thing, except for a little syntax dyslexia around the $ne query constraint.
So where you have
users: { // not first user
$elemMatch: { $ne: { userId: ObjectId("4edaaa4a8d285b9311eb63a9") } }
}
instead you want
users: { // not first user
$elemMatch: { userId: { $ne: ObjectId("4edaaa4a8d285b9311eb63a9") } }
}
If I had a nickel for every time I've done the same kind of swap ... I'd have a lot of nickels. :)
Actually, if you want to tighten things up, the $elemMatch is not really needed, since you're only constraining one property of the matched array element. I think you can boil the whole query criteria down to:
{
_id: ObjectId("4edaaa4a8d285b9311eb63ab"),
users.userId: { $ne : ObjectId("4edaaa4a8d285b9311eb63a9") }
}