We Keep Coding

Average value of a whole collection on mongoDB

I need to get the average value of a whole mongo collection. To be more specific, I have two date fields, let's call them beginning and end.
First of all, I need to do something like end - beginning time to get the elapsed time. After that, I want to sum all the elapsed time and get the average time.
I need this data to plot a chart. I've seen that mongoDB has some built in functions like subtract, sum and average as well. I don't know how to use them, and I also need the data in minutes so I may use some Javascript to convert it, I don't know yet but this is not the problem.
Mongoose schema:
module.exports = mongoose.model('atendimento', {
id: String,
id_atendimento: { type: Number, default: 0 },
id_cliente: { type: Number, default: 0 },
id_user: mongoose.Schema.Types.ObjectId,
user_nome: String,
cliente_nome: String,
id_atendente: { type: Number, default: 0 },
atendente_nome: String,
atendente_imagem: String,
setor: Number,
descricao: String,
status: String,
date: { type: Date, default: Date.now },
inicio: { type: Date, default: Date.now },
fim: { type: Date, default: Date.now },
update: { type: Date, default: Date.now }
});
The begin is the variable called inicio and the end is the variable called fim.
At the moment I need help with these mongo functions or any other suggestions will be welcome.
Thanks in advance, hope I can get some help!

Sounds like you need to run an aggregation pipeline which aggregates the whole collection using a $group pipeline.
Within the $group, you need to calculate the timestamp difference with $subtract operator and divide the result by the number of milliseconds in a minute (60 * 1000) with $divide operator.
You will then apply the $avg operator to the above expression so that you will have your collection average.
For the $group pipeline, you can specify an _id value of null to calculate accumulated values for all the input documents as a whole.
Following example shows the above:
Atendimento.aggregate([
{ '$group': {
'_id': null,
'average_duration': {
'$avg': {
'$divide': [
{ '$subtract': ['$fim', '$inicio'] },
60*1000
]
}
}
} }
]).exec((err, results) => console.log(results))

You need to user aggregate function with $subtract function.Subtracts two dates to return the difference in milliseconds
db.sales.aggregate( [ { $project: { item: 1, dateDifference: { $subtract: [ new Date(), "$date" ] } } } ] )
this will give you data like
{ "_id" : 1, "item" : "abc", "dateDifference" : NumberLong("11713985194") }
{ "_id" : 2, "item" : "jkl", "dateDifference" : NumberLong("11710385194") }
You can change the new date to another date field you have in the database.
Then you need to use $substract and $group in aggregate function to get the desired result as you haven't posted exact schema so it's hard to write your query so you can try with following
https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/
https://docs.mongodb.com/manual/reference/operator/aggregation/subtract/
https://docs.mongodb.com/manual/reference/operator/aggregation/group/

Getting MongoDb document to expire at a certain time using mongoose [duplicate]

Below is the command that can be used via the mongo terminal to set an expiry time for collections (a TTL):
db.log.events.ensureIndex( { "status": 1 }, { expireAfterSeconds: 3600 } )
How do I do this from my code in Node.js using mongoose?

In Mongoose, you create a TTL index on a Date field via the expires property in the schema definition of that field:
// expire docs 3600 seconds after createdAt
new Schema({ createdAt: { type: Date, expires: 3600 }});
Note that:
MongoDB's data expiration task runs once a minute, so an expired doc might persist up to a minute past its expiration.
This feature requires MongoDB 2.2 or later.
It's up to you to set createdAt to the current time when creating docs, or add a default to do it for you as suggested here.
{ createdAt: { type: Date, expires: 3600, default: Date.now }}

this code is working for me.
may it help
let currentSchema = mongoose.Schema({
id: String,
name: String,
packageId: Number,
age: Number
}, {timestamps: true});
currentSchema.index({createdAt: 1},{expireAfterSeconds: 3600});

Providing a string to expires also works nicely with Mongoose if you do not want to deal with the expire time calculation and improve the overall readability of the schema.
For example here we are setting the expires to 2m (2 minutes) and mongoose would convert to 120 seconds for us:
var TestSchema = new mongoose.Schema({
name: String,
createdAt: { type: Date, expires: '2m', default: Date.now }
});
Mongoose would create an index in the background and auto set the expireAfterSeconds to in this case 120 seconds (specified by the 2m).
It is important to note that the TTL process runs once every 60 seconds so it is not perfectly on time always.

If you are working with Mongodb Atlas Replica Sets - try:
import * as mongoose from 'mongoose';
let currentSchema = new mongoose.Schema({
createdAt: { type: Date, expires: 10000, default: Date.now },
id: String,
name: String,
packageId: Number,
age: Number
});
currentSchema.index({"lastModifiedDate": 1 },{ expireAfterSeconds: 10000 });

new Scehma({
expireAt: {
type: Date,
expires: 11,
default: Date.now
}
)}
This is the solution that worked for me according to this in the current Mongoose docs.

There is a npm library - 'mongoose-ttl'.:
var schema = new Schema({..});
schema.plugin(ttl, { ttl: 5000 });
you can see all the options of this library:
https://www.npmjs.com/package/mongoose-ttl

const Schema = new mongoose.Schema({id: {
type: Number},
createdAt: {
type: Date, expires: '4h', index: true,
default: Date.now}});
You need to add index: true while creating you schema

9/2022 Working Solution using Mongoose 6.5.4
None of the answers here worked for me, but I was able to finally get it working using the latest version of Mongoose currently available, 6.5.4.
Say our Schema looks like this:
const MySchema = new mongoose.Schema({
id: { type: Number },
myCustomTTLField: { type: Date }
});
myCustomTTLField is the field you want to index and have control the expiration. To achieve this, we add the following under our schema definition:
MySchema.path('myCustomTTLField').index({ expires: 60 });
The argument in MySchema.path is the name of the field you want to index for TTL. The expires option should be the number of seconds that will elapse from the Date represented in myCustomTTLField before the document is deleted. In the example above, the document will be deleted 60 seconds after whatever date is saved in myCustomTTLField. The full example:
const MySchema = new mongoose.Schema({
id: { type: Number },
myCustomTTLField: { type: Date }
});
MySchema.path('myCustomTTLField').index({ expires: 60 });
Please let me know if this works for you, I hope this helps. Mongoose TTL has been a thorn in my side for a long time, as their docs are notoriously tough to navigate. I found this solution via a small example buried in the docs here.
IMPORTANT NOTE:
TTL is not guaranteed to happen at exactly the time specified by your date + expiration seconds. This is due to how MongoDB's background delete process works. It runs every 60 seconds, so you may theoretically wait up to 60 seconds past expected TTL before seeing your document deleted. More info on that from the MongoDB docs.

FWIW I could only get the expires feature to work on a field called expiresAt. Here's my interface, and schema for implementing this in Typescript.
import { model, Schema, Types } from 'mongoose';
export interface ISession {
sessionId: string;
userId: Types.ObjectId;
role: string;
expiresAt?: Date;
}
const sessionSchema = new Schema<ISession>({
sessionId: { type: String, required: true, indexes: { unique: true} },
userId: { type: Schema.Types.ObjectId, required: true, ref: 'users'},
role: { type: String, required: true, enum: [ 'ADMIN', 'BASIC_USER' ]},
expiresAt: { type: Date, expires: '1h', default: Date.now }
}, { versionKey: false });
Reading the Mongoose documentation it seems like all the other proposed solutions should work too. I don't know why they were not for me. You can read the official Mongoose docs on expiresAt here.

Mongoose find inside nested schema

In mongoose we are deeply searching inside a nested schema, without much success. Every time we run this function we always get an empty array returned.
function findAlarms(lastUpdate = new Date(0), record = Record) {
// For docs on find http://mongoosejs.com/docs/queries.html
return record
.find({
// Date due must be less than "now"
'documents.alarm.date_due': {
$lte: Date.now(),
},
// Must be greater than the last update and less than "now"
'documents.alarm.date_reminder.reminder': {
$gte: lastUpdate,
$lte: Date.now(),
},
})
.populate('documents')
.exec();
}
Our schemas, greatly summarized, look like this:
const RecordSchema = new mongoose.Schema({
documents: [
{
type: Schema.Types.ObjectId,
ref: 'Document',
},
],
});
And our documents schema, similarly summarized looks like this:
const DocumentSchema = new mongoose.Schema({
alarm: {
date_due: { type: Date },
date_reminder: [
{
reminder: { type: Date },
},
],
},
});
This search returns no matching elements, even though we know there are documents that match. If we modify our findAlarms method to use the documents schema:
function findAlarms(lastUpdate = new Date(0), document = Document) {
// For docs on find http://mongoosejs.com/docs/queries.html
return document
.find({
// Date due must be less than "now"
'alarm.date_due': {
$lte: Date.now(),
},
// Must be greater than the last update and less than "now"
'alarm.date_reminder.reminder': {
$gte: lastUpdate,
$lte: Date.now(),
},
})
.exec();
}
It will return all of our matching documents. However, having records is essential for our needs. Now, I could use a hack and then find records using the array of document._ids that return.
Nonetheless, I would love to know if there's an approach where we can find using the records directly, since adding that extra step feels really hacky, and this operation runs every 5 minutes so I'd love to be more efficient wherever posible.

sequelize interprets date with hour = 2

I'm trying to use sequelize to search for a record in my database based in a date field:
date_from2 is this value:
2017-01-09
and sequelize interprets its as a date with hour = 2:
SELECT `id`, `user_id`, `date`, `total_visits`, `created_at`, `updated_at` FROM `table1` AS `table1` WHERE `table1`.`user_id` = 123 AND `table1`.`date` = '2017-01-09 02:00:00' LIMIT 1;
And it creates a new record everytime, instead of updating it.
When it inserts, the date is inserted with this value:
2017-01-09 00:00:00
This is my code:
where = { user_id: user_id,
date: date_from2
};
values = {
user_id: user_id,
date: date_from2,
total_visits: total_visits
};
Model.findOne({where: where}).then(function (record) {
if (!record) {
// Item not found, create a new one
Model.create(values)
.then(function () {
console.log('created!');
}).error(function (err) {
console.log('error on create');
});
} else {
// Found an item, update it
Model.update(values, {where: where})
.then(function () {
console.log('updated!');
})
.catch(function (err) {
console.log('error on update');
});
}
});

Query by just using the user's user_id and not the date if you are trying to get a specific user:
Model.findOne({where: {user_id: user_id}})
The issue is that it's storing the date differently then what you are passing in. Because of this you were never able to find the existing user and you kept creating new ones. There is nothing wrong with using UTC timezone and you should be able to convert this to any other time zone.

How do I update an embedded document with a nested hierarchy?

I have an embedded document that needs to be updated. I have managed to get everything working except for one scenario: I am unable to get an embedded document with a nested hierarchy to get updated. Here is an example of what I'm attempting to do:
console.log('Phone type: ' + req.body.phone.type); // prints, e.g., 'Phone type: Work'
console.log('Phone #: ' + req.body.phone.number); // prints, e.g., 'Phone #: 555-555-5555'
var updateData = {
"user.$.contact_info": {
email: req.body.email,
phone: {
type: req.body.phone.type,
number: req.body.phone.number
}
}
};
Group.update(
{ "user._id" : req.params.user_id },
{ $push : updateData },
function(err,data) {
console.log('Success!'); // I see the "Success!" message every time I save a record
}
);
Schema:
var contactInfoSchema = mongoose.Schema({
created : {
type: Date,
default: Date.now
},
email : String
phone: {
type: String,
number: String
}
});
var userSchema = mongoose.Schema({
created : {
type: Date,
default: Date.now
},
contact_info : [contactInfoSchema]
});
var GroupSchema = mongoose.Schema({
created : {
type: Date,
default: Date.now
},
users : [userSchema]
});
What I find is that I can create records, but only the email address is stored, not the phone information. When inspecting the console I can see that the phone type and phone number info is being sent, but it just isn't updating in the database. What am I doing wrong?

The problem in in the "contactInfoSchema" with the use of the "type" keyword. You need this:
var contactInfoSchema = mongoose.Schema({
created : {
type: Date,
default: Date.now
},
email : String
phone: {
"type": { "type": String },
number: String
}
});
So basically mongoose is confused since you tried to call a "sub-document" field "type", and it thinks that is the data "type" for the field "phone". Declared as above all works fine.