How to process deep update with subdocument in Mongodb/Mongoose? - javascript

I would like to use mongoose in this case. Let's say we have a Schema like this:
const userSchema = new Schema({
name: {
first: { type: String, required: true },
last: { type: String, required: true },
},
email: { type: String, required: true, unique: true, lowercase: true },
});
Let's say we already have document with first and last name. I need to update only first name and email with the following args object:
const updateUser = {
name: {
first: Eddy,
},
email: 'eddy#gordo.io'
};
If it will be used with mongoose update methods in most cases it would also change last name to null. That's because JavaScript doesn't support deep object merge and so on. So what's the best way to gain possibility to merge objects properly?
I've found the way to do it below but it doesn't seem to be the best solution:
User.findById(args.id, (error, user) => {
if (error) throw err;
user.name.first = args.name.first ? args.name.first : user.name.first;
user.name.last = args.name.last ? args.name.last : user.name.last;
user.email = args.email ? args.email : user.email;
user.save((err, updatedUser) => {
if (err) throw err;
return updatedUser;
});
});
I and Eslint don't like this with reassigning and unmaintainable code. Have somebody better idea? I'm not sure that I need to use lodash and other libs only for this capability.

Here is what I've done with this issue.
Thanks to Neil Lunn with this idea.
We're need to map all object keys to dot.notation and pass to the $set.
Here is the gist with this function. If we need two level nesting we can add another condition and mapping.
Anyway I hope that Mongoose API will resolve us to make deep merge with nested objects, but for now Neil's solution is the best in my opinion.

Related

Mongoose - Deleting documents is unresponsive

I'm trying to use Mongoose (MongoDB JS library) to create a basic database, but I can't figure out how to delete the documents / items, I'm not sure what the technical term for them is.
Everything seems to work fine, when I use Item.findById(result[i].id), it returns a valid id of the item, but when I use Item.findByIdAndDelete(result[i].id), the function doesn't seem to start at all.
This is a snippet the code that I have: (Sorry in advance for bad indentation)
const testSchema = new schema({
item: {
type: String,
required: true
},
detail: {
type: String,
required: true
},
quantity: {
type: String,
required: true
}
})
const Item = mongoose.model("testitems", testSchema)
Item.find()
.then((result) => {
for (i in result) {
Item.findByIdAndDelete(result[i].id), function(err, result) {
if (err) {
console.log(err)
}
else {
console.log("Deleted " + result)
}
}
}
mongoose.connection.close()
})
.catch((err) => {
console.log(err)
})
I'm not sure what I'm doing wrong, and I haven't been able to find anything on the internet.
Any help is appreciated, thanks.
_id is a special field on MongoDB documents that by default is the type ObjectId. Mongoose creates this field for you automatically. So a sample document in your testitems collection might look like:
{
_id: ObjectId("..."),
item: "xxx",
detail: "yyy",
quantity: "zzz"
}
However, you retrieve this value with id. The reason you get a value back even though the field is called _id is because Mongoose creates a virtual getter for id:
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString. If you don't want an id getter added to your schema, you may disable it by passing this option at schema construction time.
The key takeaway is that when you get this value with id it is a string, not an ObjectId. Because the types don't match, MongoDB will not delete anything.
To make sure the values and types match, you should use result[i]._id.

Mongoose gives errors when trying to save object into array [duplicate]

I wrote a service that analyses videos with Google Cloud Video Intelligence
And I save the analysis results to the MongoDB with mongoose
This is the model I use (I've simplified everything to avoid confusion):
// Video.js
const mongoose = require('mongoose');
const videoSchema = new mongoose.Schema({
analysis_progress: {
percent: { type: Number, required: true },
details: {}
},
status: {
type: String,
enum: ['idle', 'processing', 'done', 'failed'],
default: 'idle'
}
});
module.exports = mongoose.model('Video', videoSchema);
When analyse operation ends, I call the function below and run update like this:
function detectFaces(video, results) {
//Build query
let update = {
$set: {
'analysis_results.face_annotations': results.faceDetectionAnnotations // results is the the test result
}
};
Video.findOneAndUpdate({ _id: video._id }, update, { new: true }, (err, result) => {
if (!err)
return console.log("Succesfully saved faces annotiations:", video._id);
throw err // This is the line error thrown
});
}
And this is the error I get:
Error: cyclic dependency detected
at serializeObject (C:\Users\murat\OneDrive\Masaüstü\bycape\media-analysis-api\node_modules\bson\lib\bson\parser\serializer.js:333:34)
at serializeInto (C:\Users\murat\OneDrive\Masaüstü\bycape\media-analysis-api\node_modules\bson\lib\bson\parser\serializer.js:947:17)
...
Solutions I tried:
Added {autoIndex: false} inside db config.
mongoose.connect(process.env.DB_CONNECTION, {useNewUrlParser: true, useUnifiedTopology: true, useFindAndModify: false, autoIndex: false });
Removing retryWrites=true from Mongo URI structure. (I didn't have that parameter in my connection URI already)
So, I think the source of the problem is that I am saving the whole test result but I don't have any other option to do that. I need to save as it is.
I am open to all kinds of suggestions.
Just as I guessed, the problem was that there was a cyclic dependency in the object that came to me from google.
With help of my colleague:
Then since JSON.stringify() changes an object into simple types:
string, number, array, object, boolean it is not capable of storing
references to objects therefor by using stringify and then parse you
destroy the information that stringify cannot convert.
Another way would be knowing which field held the cyclic reference and
then unsetting, or deleting that field.
I couldn't find which field has cycylic dependency so I used I JSON.stringfy() and JSON.parse() to remove it.
let videoAnnotiations = JSON.stringify(operationResult.annotationResults[0]);
videoAnnotiations = JSON.parse(videoAnnotiations);

What is the best way to keep track of changes of a document's property in MongoDB?

I would like to know how to keep track of the values of a document in MongoDB.
It's a MongoDB Database with a Node and Express backend.
Say I have a document, which is part of the Patients collection.
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Burn fat"
}
Then I edit the "objective" property, so the document results like this:
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Gain muscle"
}
What's the best/most efficient way to keep track of that change? In other words, I would like to know that the "objective" property had the value "Burn fat" in the past, and access it in the future.
Thanks a lot!
Maintaining/tracking history in the same document is not all recommended. As the document size will keep on increasing leading to
probably if there are too many updates, 16mb document size limit
Performance degrades
Instead, you should maintain a separate collection for history. You might have use hibernates' Javers or envers for auditing for your relational databases. if not you can check how they work. A separate table (xyz_AUD) is maintained for each table (xyz). For each row (with primary key abc) in xyz table, there exist multiple rows in xyz_AUD table, where each row is version of that row.
Moreover, Javers also support MongoDB auditing. If you are using java you can directly use it. No need to write your own logic.
Refer - https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/
One more thing, Javers Envers Hibernate are java libraries. But I'm sure for other programming languages also, similar libraries will be present.
There is a mongoose plugin as well -
https://www.npmjs.com/package/mongoose-audit (quite oudated 4 years)
https://github.com/nassor/mongoose-history#readme (better)
Maybe you can change the type of "objective" to array and track the changes in it. the last one of the array is the latest value.
Maintain it as a sub-document like below
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": {
obj1: "Gain muscle",
obj2: "Burn fat"
}
}
You can also maintain it as an array field but remember, mongodb doesn't allow you to maintain uniqueness in an array field and if you plan to index the "objective" field, you'll have to create a multi key index
I think the simplest solution would be to use and update an array:
const patientSchema = new Schema({
firstName: { type: String, required: true },
lastName: { type: String, required: true },
objective: { type: String, required: true }
notes: [{
date: { type: Date, default: Date.now() },
note: { type: String, required: true }
}],
});
Then when you want to update the objective...
const updatePatientObjective = async (req, res) => {
try {
// check if _id and new objective exist in req.body
const { _id, objective, date } = req.body;
if (!_id || !objective) throw "Unable to update patient's objective.";
// make sure provided _id is valid
const existingPatient = await Patient.findOne({ _id });
if (!existingPatient) throw "Unable to locate that patient.";
// pull out objective as previousObjective
const { objective: previousObjective } = existingPatient;
// update patient's objective while pushing
// the previous objective into the notes sub document
await existingPatient.updateOne({
// update current objective
$set { objective },
// push an object with a date and note (previouseObjective)
// into a notes array
$push: {
notes: {
date,
note: previousObjective
},
},
}),
);
// send back response
res
.status(201)
.json({ message: "Successfully updated your objective!" });
} catch (err) {
return res.status(400).json({ err: err.toString() });
}
};
Document will look like:
firstName: "John",
lastName: "Smith",
objective: "Lose body fat.",
notes: [
{
date: 2019-07-19T17:45:43-07:00,
note: "Gain muscle".
},
{
date: 2019-08-09T12:00:38-07:00,
note: "Work on cardio."
}
{
date: 2019-08-29T19:00:38-07:00,
note: "Become a fullstack web developer."
}
...etc
]
Alternatively, if you're worried about document size, then create a separate schema for patient history and reference the user's id (or just store the patient's _id as a string instead of referencing an ObjectId, whichever you prefer):
const patientHistorySchema = new Schema({
_id: { type: Schema.Types.ObjectId, ref: "Patient", required: true },
objective: { type: String, required: true }
});
Then create a new patient history document when the objective is updated...
PatientHistory.create({ _id, objective: previousObjective });
And if you need to access to the patient history documents...
PatientHistory.find({ _id });

Exclude a field in Mongoose wile using assignment operator for Query

I am aware that this question has been asked before. Sites often give the solution using .select('-queryData')however I do not know how to use it for my scenario.
I have a DB with the following schema
let TestSchema = new Schema ({
..
test: {type: String, required: true},
arrayField: {type: Array, required: true},
..
});
I have to query the arrayField where it should not include 'data1'. If it includes this value then that object should not be shown.
However I am using assignment operator for query object, instead of calling the function '.select()'. Hence I tried adding it in the query JSON itself, but it did not work.
index.js
let query = {};
query.test = "Hello";
query.arrayField = '-data1'; //Change needed here
TestSchema.find(query, function(err, result){
if(err){
console.log(err);
}else{
console.log(result);
}
});
I tried checking various sites for a specific solution to such a scenario, however I could not get any conclusive solution.
You can use $exists mongodb operator for your query.
TestModel.find({ test: "Hello", "arrayField.data1": { $exists: false } })
Select method has a different use case which is to select or limit the fields returned as a result of a query. And you need to execute find method on TestModel not TestSchema.
const TestModel = mongoose.model("TestModel", TestSchema);
You should use the $exists operator, here's an example:
TestModel.find({
test: "Hello",
"arrayField.$.data1": {
$exists: false
}
}, function(err, result){
if(err){
console.log(err);
}else{
console.log(result);
}
});

Recreate models using Mongoose

I want to recreate the models in database after dropping everything in it.
Mongoose (or Mongo itself )actually recreates the documents but not the indices. So is there a way to reset Mongoose so that it can recreate indices as if running the first time?
The reason why I'm using dropDatabase is because it seems easier while testing. Otherwise I would have to remove all collections one by one.
While not recommended for production use, depending on your scenario, you can add the index property to a field definition to specify you want an index created:
var animalSchema = new Schema({
name: String,
type: String,
tags: { type: [String], index: true } // field level
});
animalSchema.index({ name: 1, type: -1 }); // schema level
Or,
var s = new Schema({ name: { type: String, sparse: true })
Schema.path('name').index({ sparse: true });
Or, you can call ensureIndex on the Model (docs):
Animal.ensureIndexes(function (err) {
if (err) return handleError(err);
});

Categories