In mongoose we are deeply searching inside a nested schema, without much success. Every time we run this function we always get an empty array returned.
function findAlarms(lastUpdate = new Date(0), record = Record) {
// For docs on find http://mongoosejs.com/docs/queries.html
return record
.find({
// Date due must be less than "now"
'documents.alarm.date_due': {
$lte: Date.now(),
},
// Must be greater than the last update and less than "now"
'documents.alarm.date_reminder.reminder': {
$gte: lastUpdate,
$lte: Date.now(),
},
})
.populate('documents')
.exec();
}
Our schemas, greatly summarized, look like this:
const RecordSchema = new mongoose.Schema({
documents: [
{
type: Schema.Types.ObjectId,
ref: 'Document',
},
],
});
And our documents schema, similarly summarized looks like this:
const DocumentSchema = new mongoose.Schema({
alarm: {
date_due: { type: Date },
date_reminder: [
{
reminder: { type: Date },
},
],
},
});
This search returns no matching elements, even though we know there are documents that match. If we modify our findAlarms method to use the documents schema:
function findAlarms(lastUpdate = new Date(0), document = Document) {
// For docs on find http://mongoosejs.com/docs/queries.html
return document
.find({
// Date due must be less than "now"
'alarm.date_due': {
$lte: Date.now(),
},
// Must be greater than the last update and less than "now"
'alarm.date_reminder.reminder': {
$gte: lastUpdate,
$lte: Date.now(),
},
})
.exec();
}
It will return all of our matching documents. However, having records is essential for our needs. Now, I could use a hack and then find records using the array of document._ids that return.
Nonetheless, I would love to know if there's an approach where we can find using the records directly, since adding that extra step feels really hacky, and this operation runs every 5 minutes so I'd love to be more efficient wherever posible.
Related
I have a Mongoose collection called WorkoutUser with the following simplified schema:
const WorkoutUserSchema = new Schema({
user: {
type: Schema.Types.ObjectId,
ref: 'User',
required: true,
},
exercises: [{
exercise: {
type: Schema.Types.ObjectId,
ref: 'Exercise',
},
sets: [// not important],
date: Date, // <- important
}],
});
For reasons I won't get into here, I'd like to query the WorkoutUser collection to find all instances of a workout tied to the current user that is prior to the current workout. The date is not defined on the WorkoutUser for other reasons that aren't relevant here, but the date on each exercise will be the same.
I'd like to do something like the following:
const workouts = await db.WorkoutUser
.find({
_id: { $ne: workout._id },
user: workout.user,
'exercises.exercise': exercise.exercise,
'exercises.exercise.date': { $lt: userWorkout.date }, // <- relevant line
});
This does not work currently. Is there a way to basically search on the date for exercises.exercise[0].date?
Yes, actually exercise and date both are at the same level so you can modify your query like this
const workouts = await db.WorkoutUser
.find({
_id: { $ne: workout._id },
user: workout.user,
'exercises.exercise': exercise.exercise,
'exercises.date': { $lt: userWorkout.date }, // <- relevant line
});
I'm trying to get my query working correctly. It's supposed to return 5 items which have a utcTime that falls between two dates. I've verified that documents in my mongoDB do have correct dates and should be returned. This query works if I remove the .where line. In fact it works if I remove the gte and lt lines. The date gets parsed correctly so I'm not sure why this query is failing me.
I've also tried the alternative which is to put a filter in the find function like this
{
"utcTime": {
"$gte": new Date(req.params.startdate),
"$lt": new Date(req.params.enddate)
}
}
But that doesn't work either.
// Get the top 5 items sorted in decreasing order by karma in between
// the specified dates
listItemRoute.route('/top5/:startdate/:enddate').get((req, res) => {
console.log(req.params.startdate) //2020-05-13T07:00:00Z
console.log(req.params.enddate) //2020-05-20T07:00:00Z
ListItem
.find()
.limit(5)
.where('utcTime').gte( new Date(req.params.startdate) ).lt( new Date(req.params.enddate) )
.sort({karma: -1})
.exec((error, data) => {
if (error) {
return next(error)
} else {
res.json(data)
}
})
})
Here's my model
// Define collection and schema
let ListItemSchema = new Schema({
name: {
type: String,
required: true
},
karma: {
type: Number
},
utcTime: {
type: Date
}
})
Everything looks correct and I've tried many other stackoverflow suggestions to no avail. Any help would be awesome.
Edit: example of DB data
{
"name": "TITLE BLAH",
"karma": 11502,
"utcTime": "2020-05-17T09:50:21Z"
}
I would like to know how to keep track of the values of a document in MongoDB.
It's a MongoDB Database with a Node and Express backend.
Say I have a document, which is part of the Patients collection.
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Burn fat"
}
Then I edit the "objective" property, so the document results like this:
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Gain muscle"
}
What's the best/most efficient way to keep track of that change? In other words, I would like to know that the "objective" property had the value "Burn fat" in the past, and access it in the future.
Thanks a lot!
Maintaining/tracking history in the same document is not all recommended. As the document size will keep on increasing leading to
probably if there are too many updates, 16mb document size limit
Performance degrades
Instead, you should maintain a separate collection for history. You might have use hibernates' Javers or envers for auditing for your relational databases. if not you can check how they work. A separate table (xyz_AUD) is maintained for each table (xyz). For each row (with primary key abc) in xyz table, there exist multiple rows in xyz_AUD table, where each row is version of that row.
Moreover, Javers also support MongoDB auditing. If you are using java you can directly use it. No need to write your own logic.
Refer - https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/
One more thing, Javers Envers Hibernate are java libraries. But I'm sure for other programming languages also, similar libraries will be present.
There is a mongoose plugin as well -
https://www.npmjs.com/package/mongoose-audit (quite oudated 4 years)
https://github.com/nassor/mongoose-history#readme (better)
Maybe you can change the type of "objective" to array and track the changes in it. the last one of the array is the latest value.
Maintain it as a sub-document like below
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": {
obj1: "Gain muscle",
obj2: "Burn fat"
}
}
You can also maintain it as an array field but remember, mongodb doesn't allow you to maintain uniqueness in an array field and if you plan to index the "objective" field, you'll have to create a multi key index
I think the simplest solution would be to use and update an array:
const patientSchema = new Schema({
firstName: { type: String, required: true },
lastName: { type: String, required: true },
objective: { type: String, required: true }
notes: [{
date: { type: Date, default: Date.now() },
note: { type: String, required: true }
}],
});
Then when you want to update the objective...
const updatePatientObjective = async (req, res) => {
try {
// check if _id and new objective exist in req.body
const { _id, objective, date } = req.body;
if (!_id || !objective) throw "Unable to update patient's objective.";
// make sure provided _id is valid
const existingPatient = await Patient.findOne({ _id });
if (!existingPatient) throw "Unable to locate that patient.";
// pull out objective as previousObjective
const { objective: previousObjective } = existingPatient;
// update patient's objective while pushing
// the previous objective into the notes sub document
await existingPatient.updateOne({
// update current objective
$set { objective },
// push an object with a date and note (previouseObjective)
// into a notes array
$push: {
notes: {
date,
note: previousObjective
},
},
}),
);
// send back response
res
.status(201)
.json({ message: "Successfully updated your objective!" });
} catch (err) {
return res.status(400).json({ err: err.toString() });
}
};
Document will look like:
firstName: "John",
lastName: "Smith",
objective: "Lose body fat.",
notes: [
{
date: 2019-07-19T17:45:43-07:00,
note: "Gain muscle".
},
{
date: 2019-08-09T12:00:38-07:00,
note: "Work on cardio."
}
{
date: 2019-08-29T19:00:38-07:00,
note: "Become a fullstack web developer."
}
...etc
]
Alternatively, if you're worried about document size, then create a separate schema for patient history and reference the user's id (or just store the patient's _id as a string instead of referencing an ObjectId, whichever you prefer):
const patientHistorySchema = new Schema({
_id: { type: Schema.Types.ObjectId, ref: "Patient", required: true },
objective: { type: String, required: true }
});
Then create a new patient history document when the objective is updated...
PatientHistory.create({ _id, objective: previousObjective });
And if you need to access to the patient history documents...
PatientHistory.find({ _id });
I need to get the average value of a whole mongo collection. To be more specific, I have two date fields, let's call them beginning and end.
First of all, I need to do something like end - beginning time to get the elapsed time. After that, I want to sum all the elapsed time and get the average time.
I need this data to plot a chart. I've seen that mongoDB has some built in functions like subtract, sum and average as well. I don't know how to use them, and I also need the data in minutes so I may use some Javascript to convert it, I don't know yet but this is not the problem.
Mongoose schema:
module.exports = mongoose.model('atendimento', {
id: String,
id_atendimento: { type: Number, default: 0 },
id_cliente: { type: Number, default: 0 },
id_user: mongoose.Schema.Types.ObjectId,
user_nome: String,
cliente_nome: String,
id_atendente: { type: Number, default: 0 },
atendente_nome: String,
atendente_imagem: String,
setor: Number,
descricao: String,
status: String,
date: { type: Date, default: Date.now },
inicio: { type: Date, default: Date.now },
fim: { type: Date, default: Date.now },
update: { type: Date, default: Date.now }
});
The begin is the variable called inicio and the end is the variable called fim.
At the moment I need help with these mongo functions or any other suggestions will be welcome.
Thanks in advance, hope I can get some help!
Sounds like you need to run an aggregation pipeline which aggregates the whole collection using a $group pipeline.
Within the $group, you need to calculate the timestamp difference with $subtract operator and divide the result by the number of milliseconds in a minute (60 * 1000) with $divide operator.
You will then apply the $avg operator to the above expression so that you will have your collection average.
For the $group pipeline, you can specify an _id value of null to calculate accumulated values for all the input documents as a whole.
Following example shows the above:
Atendimento.aggregate([
{ '$group': {
'_id': null,
'average_duration': {
'$avg': {
'$divide': [
{ '$subtract': ['$fim', '$inicio'] },
60*1000
]
}
}
} }
]).exec((err, results) => console.log(results))
You need to user aggregate function with $subtract function.Subtracts two dates to return the difference in milliseconds
db.sales.aggregate( [ { $project: { item: 1, dateDifference: { $subtract: [ new Date(), "$date" ] } } } ] )
this will give you data like
{ "_id" : 1, "item" : "abc", "dateDifference" : NumberLong("11713985194") }
{ "_id" : 2, "item" : "jkl", "dateDifference" : NumberLong("11710385194") }
You can change the new date to another date field you have in the database.
Then you need to use $substract and $group in aggregate function to get the desired result as you haven't posted exact schema so it's hard to write your query so you can try with following
https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/
https://docs.mongodb.com/manual/reference/operator/aggregation/subtract/
https://docs.mongodb.com/manual/reference/operator/aggregation/group/
This is my Mongoose model:
var sessionSchema = new Schema({
_id: { type: String, required: true, index: { unique: true } },
user: { type: Schema.Types.ObjectId },
expire: { type: Date, index: { expireAfterSeconds: 21600 } }
})
module.exports = mongoose.model('Session', sessionSchema)
I need to be able to set a date object into expire (usually it's something like Date.now plus a few minutes) and have the object removed from the collection after 6 hours past the expiration.
However, I'm not able to have Mongoose to create the index. When I run db.sessions.getIndexes() in the mongo console, here's the output:
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "dev.sessions"
}
]
I've tried also with different syntaxes, like
expire: { type: Date, expires: 21600 } (Mongoose's short-hand version).
I tried also defining the index at the schema level:
sessionSchema.index({ expire: 1 }, { expireAfterSeconds: 21600 })
None is working.
Unlike others who asked questions on SO, my index is simply not created. I've tried also removing the collection and the database as well, and when they're recreated they still don't contain the index.
Versions: Mongoose 3.8.19, MongoDB 2.6.5 (OSX) and Node.js 0.10.33
Edit
More info: I tried creating the index directly from the mongo console, with:
db.sessions.ensureIndex({"expire":1}, {expireAfterSeconds: 21600})
That appears to be working (the index is created).
However, it's not working with Mongoose in any way.
Apparently the problem was that I created an index on the custom _id field. MongoDB creates an index on that field by itself, so when Mongoose was calling ensureIndex to create also the TTL index, it failed for both.
See https://github.com/LearnBoost/mongoose/issues/2459