We Keep Coding

How to delete all documents except most recent for each group in mongo? [SOLVED]

I have a collection with this data registered
{
_id: 0000120210903, iid: 00001, date: 20210903 }, {
_id: 0000220210903, iid: 00002, date: 20210903 }, {
_id: 0000120210101, iid: 00001, date: 20210101 }
I want to delete all except the document with the most recent date for each iid.
My idea is to group by the date, select the _id of the register with the max(date) and then delete all except this array of _ids. But I can't figure out how to do it.
db.getCollection('testing_data').aggregate(
{ $sort:{ _id:1 }},
{ $group:{
_id:"$iid",
lastId:{ "$last":"$_id" },
}},
{ $project:{ _id: 0, lastId: 1 } }
)
But I don't know where to go from here. Any help is greatly appreciated.
[Solution]
To fix the problem I used an aggregation to recover the combination of the field iid (the identifier shared between documents) and the unique _id as an array.
Then for each element on the array it performs a deleteMany operation on the iid but letting out the most recent _id. In this case I sort by _id because it includes the date but could also sort by the field date.
Due to the high volume of data { allowDiskUse: true } had to be put in the aggregate.
var ids = db.getCollection('testing_data').aggregate([
{ $sort:{ _id:1 }},
{ $group:{
_id:"$iid",
lastId:{ "$last":"$_id" },
}},
{ $project:{ _id: 1, lastId: 1 } }
], { allowDiskUse: true } ).toArray();
ids.forEach(function(x){
db.getCollection('testing_data').deleteMany({ "iid": x._id, "_id": {$ne:x.lastId} })
});

Mine Idea is just stock all _ids at some array that you want to delete, and then use deleteMany with $or filter
db.getCollection("testing_data").find({}).toArray((err,data)=>{
let to_elim = [];
let filtering ={};
for(let el of data){
if(!filtering[el.iid]) filtering[el.iid] = el;
else {
if(filtering[el.iid].date>el.date) to_elim.push({_id:new ObjectID(el._id)})
}
}
db.getCollection("testing_data").deleteMany({$or:to_elim})
})
I hope that all is written rightly, cause wrote all that down on mobile
There is missing some checking if something more recent...

[Solution]
To fix the problem I used an aggregation to recover the combination of the field iid (the identifier shared between documents) and the unique _id as an array.
Then for each element on the array it performs a deleteMany operation on the iid but letting out the most recent _id. In this case I sort by _id because it includes the date but could also sort by the field date.
Due to the high volume of data { allowDiskUse: true } had to be put in the aggregate.
var ids = db.getCollection('testing_data').aggregate([
{ $sort:{ _id:1 }},
{ $group:{
_id:"$iid",
lastId:{ "$last":"$_id" },
}},
{ $project:{ _id: 1, lastId: 1 } }
], { allowDiskUse: true } ).toArray();
ids.forEach(function(x){
db.getCollection('testing_data').deleteMany({ "iid": x._id, "_id": {$ne:x.lastId} })
});

Add to an array - sub-document without duplicate field values

I am trying to add an object to an array in MongoDB. I don't want it to be duplicated.
I am trying to update the user read array by using $addToset in findOneAndUpdate. However, it is inserting duplicate because of timestamp; the timestamp is an important property. I can't negate it. Can I insert based on key like userId? Please let me know.
{
_id: 'ddeecd8b-79b5-437d-9026-d0663b53ad8d',
message: 'hello world notification',
deliverToUsersList: [ '123-xxx-xx', '124-xxx-xx']
userRead: [
{
isOpened: true,
userId: '123-xxx-xx'
updatedOn: new Date(Date.now()).toISOString()
},
{
isOpened: true,
userId: '124-xxx-xx'
updatedOn: new Date(Date.now()).toISOString()
}
]
}

Add an index to the field userId and enable 'Avoid duplicates' in index settings.
I use Robo3T client to do that.

To add new objects without duplicate information into the userRead array, you have check for the duplicate information in the update method's query filter. For example, the following code will not allow adding new object with duplicate userId field value.
new_userId = "999-xxx-xx"
new_doc = { userId: new_userId, isOpened: true, updatedOn: ISODate() }
db.test_coll.findOneAndUpdate(
{ _id: 'ddeecd8b-79b5-437d-9026-d0663b53ad8d', "userRead.userId": { $ne: new_userId } },
{ $push: { "userRead" : new_doc } },
)

What is the best way to keep track of changes of a document's property in MongoDB?

I would like to know how to keep track of the values of a document in MongoDB.
It's a MongoDB Database with a Node and Express backend.
Say I have a document, which is part of the Patients collection.
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Burn fat"
}
Then I edit the "objective" property, so the document results like this:
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Gain muscle"
}
What's the best/most efficient way to keep track of that change? In other words, I would like to know that the "objective" property had the value "Burn fat" in the past, and access it in the future.
Thanks a lot!

Maintaining/tracking history in the same document is not all recommended. As the document size will keep on increasing leading to
probably if there are too many updates, 16mb document size limit
Performance degrades
Instead, you should maintain a separate collection for history. You might have use hibernates' Javers or envers for auditing for your relational databases. if not you can check how they work. A separate table (xyz_AUD) is maintained for each table (xyz). For each row (with primary key abc) in xyz table, there exist multiple rows in xyz_AUD table, where each row is version of that row.
Moreover, Javers also support MongoDB auditing. If you are using java you can directly use it. No need to write your own logic.
Refer - https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/
One more thing, Javers Envers Hibernate are java libraries. But I'm sure for other programming languages also, similar libraries will be present.
There is a mongoose plugin as well -
https://www.npmjs.com/package/mongoose-audit (quite oudated 4 years)
https://github.com/nassor/mongoose-history#readme (better)

Maybe you can change the type of "objective" to array and track the changes in it. the last one of the array is the latest value.

Maintain it as a sub-document like below
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": {
obj1: "Gain muscle",
obj2: "Burn fat"
}
}
You can also maintain it as an array field but remember, mongodb doesn't allow you to maintain uniqueness in an array field and if you plan to index the "objective" field, you'll have to create a multi key index

I think the simplest solution would be to use and update an array:
const patientSchema = new Schema({
firstName: { type: String, required: true },
lastName: { type: String, required: true },
objective: { type: String, required: true }
notes: [{
date: { type: Date, default: Date.now() },
note: { type: String, required: true }
}],
});
Then when you want to update the objective...
const updatePatientObjective = async (req, res) => {
try {
// check if _id and new objective exist in req.body
const { _id, objective, date } = req.body;
if (!_id || !objective) throw "Unable to update patient's objective.";
// make sure provided _id is valid
const existingPatient = await Patient.findOne({ _id });
if (!existingPatient) throw "Unable to locate that patient.";
// pull out objective as previousObjective
const { objective: previousObjective } = existingPatient;
// update patient's objective while pushing
// the previous objective into the notes sub document
await existingPatient.updateOne({
// update current objective
$set { objective },
// push an object with a date and note (previouseObjective)
// into a notes array
$push: {
notes: {
date,
note: previousObjective
},
},
}),
);
// send back response
res
.status(201)
.json({ message: "Successfully updated your objective!" });
} catch (err) {
return res.status(400).json({ err: err.toString() });
}
};
Document will look like:
firstName: "John",
lastName: "Smith",
objective: "Lose body fat.",
notes: [
{
date: 2019-07-19T17:45:43-07:00,
note: "Gain muscle".
},
{
date: 2019-08-09T12:00:38-07:00,
note: "Work on cardio."
}
{
date: 2019-08-29T19:00:38-07:00,
note: "Become a fullstack web developer."
}
...etc
]
Alternatively, if you're worried about document size, then create a separate schema for patient history and reference the user's id (or just store the patient's _id as a string instead of referencing an ObjectId, whichever you prefer):
const patientHistorySchema = new Schema({
_id: { type: Schema.Types.ObjectId, ref: "Patient", required: true },
objective: { type: String, required: true }
});
Then create a new patient history document when the objective is updated...
PatientHistory.create({ _id, objective: previousObjective });
And if you need to access to the patient history documents...
PatientHistory.find({ _id });

Update an array of objects mongoose

I know that this question might be beginner level but I haven't find anything yet.
I would like to update an array of objects with mongoose. I am interested in updating one object from the users array according to the index.
Usually one user is getting changed at a time.
Here is my schema:
_id: Schema.Types.ObjectId,
name: { type: String, required: true },
gm: {
type: Schema.Types.ObjectId,
ref: 'User',
required: true
},
users: [],
I want to update an object in the users array which is like this:
{
id:"5bcb7c7ff9c5c01b9482d244",
gm:"5bcb7c7ff9c5c01b9482d246",
name:"room 1"
users: [
{
id:"5bcb7c7ff9c5c01b9482d243",
stats:{
power:10,
mobility: 5,
vitality: 20
},
bag:{itemSlot1: "Knife",itemSlot2:"Sword" }
},
{
id:"5bcb7c7ff9c5c01b9482d241",
stats:{
power:10,
mobility: 5,
vitality: 20
},
bag:{itemSlot1: "Knife",itemSlot2:"Sword" }
]
}
I want to perform a patch or a post request to update one user each time from the user array. i am getting the id of the user from req.body to match it with my db.
My request is like this:
I would like to update based on a request like this:
data = {
stats={
power:"10",
vitality:"20"
}
}
Thanks in advance,
Cheers

You can do an update like this:
YourSchema.update({
'users.id': '5bcb7c7ff9c5c01b9482d243'
}, {
$set: {
'users.$.stats': data.stats
}
})
Which would update the first user with id 5bcb7c7ff9c5c01b9482d243 power stats to 20
This is using the update with the $ positional operator to update the element in the array.
Just have it set up in your post/patch request.

Using Meteor to do $push and $set at the same time

Is it possible to use Meteor to do both a $push and a $set in the same operation?
I want it to be in the same operation so a cursor.observe will not be triggered twice by splitting up into 2 different MongoDB operations.
Currently, the following fails to insert into the array:
Animals.update(
{_id: animal_id},
{
$set: {
driver_id: '',
status: 'feeding'
},
$push: {
feeder: Meteor.user()._id
}
}
)
For $push to work, it needs to be split into 2:
Animals.update(
{_id: animal_id},
{ $set:
{
driver_id: '',
status: 'feeding'
}
}
Animals.update(
{_id: animal_id},
{
$push: {
feeder: Meteor.user()._id
}
}

You can combine both into a $set by extracting the array first, pushing onto it, then including it.
const animal = Animals.findOne(animal_id);
let feeder = [] && animal.feeder;
feeder.push(Meteor.userId);
Animals.update(animal_id,
{
$set: {
driver_id: '',
status: 'feeding'
feeder: feeder
}
}
)

Multiple updates may be performed on the same document, as long as those updates do not conflict (hence the "have conflicting mods in update" error).
Because "$push" : {"bugs" : [{"name":"bug1", "count":1}]} and "$inc" : {"bugs.0.count" : 1} are both attempting to modify the same portion of the document (namely the "bugs" array), they conflict.
Multiple updates may be combined if each affects a different part of the document: