We Keep Coding

Mongoose - Deleting documents is unresponsive

I'm trying to use Mongoose (MongoDB JS library) to create a basic database, but I can't figure out how to delete the documents / items, I'm not sure what the technical term for them is.
Everything seems to work fine, when I use Item.findById(result[i].id), it returns a valid id of the item, but when I use Item.findByIdAndDelete(result[i].id), the function doesn't seem to start at all.
This is a snippet the code that I have: (Sorry in advance for bad indentation)
const testSchema = new schema({
item: {
type: String,
required: true
},
detail: {
type: String,
required: true
},
quantity: {
type: String,
required: true
}
})
const Item = mongoose.model("testitems", testSchema)
Item.find()
.then((result) => {
for (i in result) {
Item.findByIdAndDelete(result[i].id), function(err, result) {
if (err) {
console.log(err)
}
else {
console.log("Deleted " + result)
}
}
}
mongoose.connection.close()
})
.catch((err) => {
console.log(err)
})
I'm not sure what I'm doing wrong, and I haven't been able to find anything on the internet.
Any help is appreciated, thanks.

_id is a special field on MongoDB documents that by default is the type ObjectId. Mongoose creates this field for you automatically. So a sample document in your testitems collection might look like:
{
_id: ObjectId("..."),
item: "xxx",
detail: "yyy",
quantity: "zzz"
}
However, you retrieve this value with id. The reason you get a value back even though the field is called _id is because Mongoose creates a virtual getter for id:
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString. If you don't want an id getter added to your schema, you may disable it by passing this option at schema construction time.
The key takeaway is that when you get this value with id it is a string, not an ObjectId. Because the types don't match, MongoDB will not delete anything.
To make sure the values and types match, you should use result[i]._id.

Query builder how to use "where" clause on populated documents

I'm having some issues using the Mongoose queries. I'm just trying to find documents in a collection using the where clause.
Unfortunately, it seems you can't use the where clause on populated documents.
This is the collection schema
const schema: Schema = new Schema({
game: { type: Schema.Types.ObjectId, ref: 'Game', required: true, index: true },
players: [{ type: Schema.Types.ObjectId, ref: 'Player', required: true, index: true }],
scores: [{ type: String }],
resultDate: { type: Date}
});
I'm trying to find all games where type = 1v1, so I tried this
let query = Matchs.find()
.populate('game')
.populate('players')
.where('game.name').equals('Trackmania')
const matchs: Match[] = await query.exec();
This returns an empty array.
Notes
Removing the where clause returns the correct results (all Matchs)
Any where clause on game returns an empty array
I'd like to use query builder instead of passing a json because I use some parameters to define what I am querying
I read that where clause didn't work on nested documents, but there must be a way to do this right ? Am I missing something ?

You can Use Lookup and Match instead of Where.
MatchesModel.aggregate([
const GameName = req.query;
{"$lookup":{
"from":"games", // name of the foreign collection
"localField":"game",
"foreignField":"_id",
"as":"game"
}},
{"$lookup":{
"from":"players",
"localField":"players",
"foreignField":"_id",
"as":"players"
}},
{"$match":{
"game.name":{
"$eq": GameName
}
}}
])

Why $ne: null returning the empty arrays?

I`m trying to return the arrays are that have contents but $ne: null or $exsists: true still returning the empty arrays..
const userSchema = new mongoose.Schema({
email: String,
password: String,
googleId: String,
facebookId: String,
secret: Array
});
app.get("/secrets", function(req, res) {
User.find({secret: {$ne: null} }, function(err, secrets) {
if (err) {
console.log(err);
} else {
if (secrets) {
console.log(secrets);
res.render("secrets", {
secrets: secrets
});
};
};
});
});
I googled quite a bit about it and I do understand that $ne: null would return every document where the secret array doesn't exists, but if its an empty array then why? Any suggestion how to overcome this rookie problem? Im new here be kind! :)

This is because null != [].
You will have to explicitly write your condition to handle both.
$nin is for not in. So find users where secret value is not in the given array. The given array can hold your multiple values.
User.find({secret: {$nin: [null, [] ] } },...
Also, with $exists you will even get docs where the field is null. The only docs you will not get are the ones where the secret field does not exist at all.

If the documents contain values for the secret field that include missing/undefined, null, empty array, and populated array, you have a few options to only match populated arrays:
{$eval:{$gt:[0,{$size:"$secret"}]}}- get the size for thesecrets` array, and only match arrays that are not empty.
{"secret.0":{$exists:true}} - only match the document if there is a first element in the secret array (implicitly doesn't match non-array fields)
If the elements in the secret array are all the same type, a type-based query can be used. For example, if the elements in the secret array are strings, {"secret.0":{$gte:""}} will match any array whose first element is a string. This could also be optimized by creating an index on {"secret.0":1}

What is the best way to keep track of changes of a document's property in MongoDB?

I would like to know how to keep track of the values of a document in MongoDB.
It's a MongoDB Database with a Node and Express backend.
Say I have a document, which is part of the Patients collection.
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Burn fat"
}
Then I edit the "objective" property, so the document results like this:
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Gain muscle"
}
What's the best/most efficient way to keep track of that change? In other words, I would like to know that the "objective" property had the value "Burn fat" in the past, and access it in the future.
Thanks a lot!

Maintaining/tracking history in the same document is not all recommended. As the document size will keep on increasing leading to
probably if there are too many updates, 16mb document size limit
Performance degrades
Instead, you should maintain a separate collection for history. You might have use hibernates' Javers or envers for auditing for your relational databases. if not you can check how they work. A separate table (xyz_AUD) is maintained for each table (xyz). For each row (with primary key abc) in xyz table, there exist multiple rows in xyz_AUD table, where each row is version of that row.
Moreover, Javers also support MongoDB auditing. If you are using java you can directly use it. No need to write your own logic.
Refer - https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/
One more thing, Javers Envers Hibernate are java libraries. But I'm sure for other programming languages also, similar libraries will be present.
There is a mongoose plugin as well -
https://www.npmjs.com/package/mongoose-audit (quite oudated 4 years)
https://github.com/nassor/mongoose-history#readme (better)

Maybe you can change the type of "objective" to array and track the changes in it. the last one of the array is the latest value.

Maintain it as a sub-document like below
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": {
obj1: "Gain muscle",
obj2: "Burn fat"
}
}
You can also maintain it as an array field but remember, mongodb doesn't allow you to maintain uniqueness in an array field and if you plan to index the "objective" field, you'll have to create a multi key index

I think the simplest solution would be to use and update an array:
const patientSchema = new Schema({
firstName: { type: String, required: true },
lastName: { type: String, required: true },
objective: { type: String, required: true }
notes: [{
date: { type: Date, default: Date.now() },
note: { type: String, required: true }
}],
});
Then when you want to update the objective...
const updatePatientObjective = async (req, res) => {
try {
// check if _id and new objective exist in req.body
const { _id, objective, date } = req.body;
if (!_id || !objective) throw "Unable to update patient's objective.";
// make sure provided _id is valid
const existingPatient = await Patient.findOne({ _id });
if (!existingPatient) throw "Unable to locate that patient.";
// pull out objective as previousObjective
const { objective: previousObjective } = existingPatient;
// update patient's objective while pushing
// the previous objective into the notes sub document
await existingPatient.updateOne({
// update current objective
$set { objective },
// push an object with a date and note (previouseObjective)
// into a notes array
$push: {
notes: {
date,
note: previousObjective
},
},
}),
);
// send back response
res
.status(201)
.json({ message: "Successfully updated your objective!" });
} catch (err) {
return res.status(400).json({ err: err.toString() });
}
};
Document will look like:
firstName: "John",
lastName: "Smith",
objective: "Lose body fat.",
notes: [
{
date: 2019-07-19T17:45:43-07:00,
note: "Gain muscle".
},
{
date: 2019-08-09T12:00:38-07:00,
note: "Work on cardio."
}
{
date: 2019-08-29T19:00:38-07:00,
note: "Become a fullstack web developer."
}
...etc
]
Alternatively, if you're worried about document size, then create a separate schema for patient history and reference the user's id (or just store the patient's _id as a string instead of referencing an ObjectId, whichever you prefer):
const patientHistorySchema = new Schema({
_id: { type: Schema.Types.ObjectId, ref: "Patient", required: true },
objective: { type: String, required: true }
});
Then create a new patient history document when the objective is updated...
PatientHistory.create({ _id, objective: previousObjective });
And if you need to access to the patient history documents...
PatientHistory.find({ _id });

Simple Mongodb embedded database structure

i want to build a Mongodb database (Mongoose/Node.js) structure but i face a problem right now. i have two entities. Users and Books and i want to use embedded system(because of lack of joins in mongodb). And my problem is that which of this entities shpuld be an inner value to other.
For example i wiil face this two type of query in my app:
1- Books of an specific user
2- Users of an specific book
Now, Books should be a inner value for Users or contrariwise?
i can do this two:
Users schema:
var schema = new mongoose.Schema({
use_name: String,
user_family: String,
user_books: { type: Schema.Types.ObjectId, ref: 'books' }
});
Or this:
Books Schema:
var schema = new mongoose.Schema({
book_name: String,
book_lang: String,
book_user: { type: Schema.Types.ObjectId, ref: 'users' }
});
which is better? which is standard approach?
if i use both of them, when saving i have to do two save operation. if i has a large database with a lots of collections its gets worse that this...
after a lot of research i find out i have to use embedded system rather that using relation like collections to connect entities to each other, because Mongodb doesn't support joins and has poor support of things like this. embedded system is the correct way for a NoSql database like Mongodb?

Firstly, there's a minor correction that your user_books needs to be an array [].
Secondly, you should only reference one schema into another, otherwise you'll have to add unnecessary complexity in keeping them in sync.
So here's what your schemas should look like:
var UserSchema = new mongoose.Schema({
use_name: String,
user_family: String,
user_books: [{ type: Schema.Types.ObjectId, ref: 'books' }]
});
var BookSchema = new mongoose.Schema({
book_name: String,
book_lang: String,
});
Now "to fetch users that reads an specific book", you'll query like this:
UserSchema.find({ user_books: book._id })
which will give you all users that have BOOK_ID as (at least) one of their books.
If that's all, I guess you don't need population at all then.
Updated on the issue with $elemMatch query not working:
So as it turned out, we don't actually need $elemMatch with referenced docs array, since it's a simple array of _ids.
user // =>
{
_id: 56351c611ca0d2e81274100a
name: ...
books: [56351c611ca0d2e81274100b, 56351c611ca0d2e81274100c, ...]
}
$elemMatch works with array of objects, and would've been in the case of embedded doc:
var BookSchema = new mongoose.Schema({
book_name: String,
book_lang: String,
});
var UserSchema = new mongoose.Schema({
use_name: String,
user_family: String,
user_books: [BookSchema]
});
Since in this case the document would be like this:
user // =>
{
_id: 56351c611ca0d2e81274100a
name: ...
books: [
{ // each book here has an _id:
_id: 56351c611ca0d2e81274100b,
// this is what `$elemMatch: {_id:` would match for
name: ...
// you could do `$elemMatch: {name:`
lang: ...
// or `$elemMatch: {lang:`
}, {
_id: 56351c611ca0d2e81274100c,
name: ...
lang: ...
}, ...
]
}
This is where $elemMatch query would be needed.
UserSchema.find({ user_books: {$elemMatch: {_id: book._id } } })