Average value of a whole collection on mongoDB - javascript

I need to get the average value of a whole mongo collection. To be more specific, I have two date fields, let's call them beginning and end.
First of all, I need to do something like end - beginning time to get the elapsed time. After that, I want to sum all the elapsed time and get the average time.
I need this data to plot a chart. I've seen that mongoDB has some built in functions like subtract, sum and average as well. I don't know how to use them, and I also need the data in minutes so I may use some Javascript to convert it, I don't know yet but this is not the problem.
Mongoose schema:
module.exports = mongoose.model('atendimento', {
id: String,
id_atendimento: { type: Number, default: 0 },
id_cliente: { type: Number, default: 0 },
id_user: mongoose.Schema.Types.ObjectId,
user_nome: String,
cliente_nome: String,
id_atendente: { type: Number, default: 0 },
atendente_nome: String,
atendente_imagem: String,
setor: Number,
descricao: String,
status: String,
date: { type: Date, default: Date.now },
inicio: { type: Date, default: Date.now },
fim: { type: Date, default: Date.now },
update: { type: Date, default: Date.now }
});
The begin is the variable called inicio and the end is the variable called fim.
At the moment I need help with these mongo functions or any other suggestions will be welcome.
Thanks in advance, hope I can get some help!

Sounds like you need to run an aggregation pipeline which aggregates the whole collection using a $group pipeline.
Within the $group, you need to calculate the timestamp difference with $subtract operator and divide the result by the number of milliseconds in a minute (60 * 1000) with $divide operator.
You will then apply the $avg operator to the above expression so that you will have your collection average.
For the $group pipeline, you can specify an _id value of null to calculate accumulated values for all the input documents as a whole.
Following example shows the above:
Atendimento.aggregate([
{ '$group': {
'_id': null,
'average_duration': {
'$avg': {
'$divide': [
{ '$subtract': ['$fim', '$inicio'] },
60*1000
]
}
}
} }
]).exec((err, results) => console.log(results))

You need to user aggregate function with $subtract function.Subtracts two dates to return the difference in milliseconds
db.sales.aggregate( [ { $project: { item: 1, dateDifference: { $subtract: [ new Date(), "$date" ] } } } ] )
this will give you data like
{ "_id" : 1, "item" : "abc", "dateDifference" : NumberLong("11713985194") }
{ "_id" : 2, "item" : "jkl", "dateDifference" : NumberLong("11710385194") }
You can change the new date to another date field you have in the database.
Then you need to use $substract and $group in aggregate function to get the desired result as you haven't posted exact schema so it's hard to write your query so you can try with following
https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/
https://docs.mongodb.com/manual/reference/operator/aggregation/subtract/
https://docs.mongodb.com/manual/reference/operator/aggregation/group/

Related

Mongoose findOne sending null

I am trying to edit a discord bot made in python (I stored data initially in python) and transferring it to javascript (node.js) and can't feature out while connecting to my old db why findOne giving me null while providing proper discord id.
Without anything inside
Code
anifarm.findOne();
Output
{
_id: 707876147324518400,
farmed: 17,
ordered: 5,
pimage: 'https://media.tenor.com/images/e830217a5d9926788ef25119955edc7f/tenor.gif',
pstatus: 'I want you to be happy. I want you to laugh a lot. I don’t know what exactly I’ll be able to do for you, but I’ll always be by your side.',
avg: 184,
speed: 2,
badges: [
'https://cdn.discordapp.com/attachments/856137319149207563/856137435696332800/Black-and-Yellow-Gaming-Badge--unscreen.gif',
'https://cdn.discordapp.com/attachments/856137319149207563/862219383866523688/Front-removebg-preview.png', 'https://cdn.discordapp.com/attachments/856137319149207563/862240758768599100/download-removebg-preview.png'
],
setBadges: 'https://cdn.discordapp.com/attachments/856137319149207563/862240758768599100/download-removebg-preview.png'
}
With id inside
Code
anifarm.findOne({
_id: 707876147324518400
});
Output
null
anifarm in the schema.
Decleared Schema
module.exports = mongoose.model('anifarm', new mongoose.Schema({
_id: Number,
farmed: {
type: Number,
default: 0
},
ordered: {
type: Number,
default: 0
},
pimage: {
type: String,
default: ""
},
pstatus: {
type: String,
default: ""
},
avg: {
type: Number,
default: 200
},
speed: {
type: Number,
default: 2
},
badges: {
type: Array,
default: []
},
setBadges: {
type: String,
default: ""
}
},
{
collection: 'anifarm',
versionKey: false
})
);
I cannot figure out what am I doing wrong. This problem also happens with .find()
Nothing inside find fetches everything by if I provide id it sends a empty array.
A Little help would be appreciated
For you problem use mongoose-long that should fix your problem.
This library will handle all long type data for mongoose since mongoose cannot handle long type data
you can't pass an id as a number, you will have to use ObjectId to convert the id to an instanceof ObjectId
Change your code like this
anifarm.findOne({
_id: mongoose.Types.ObjectId(707876147324518400);
});
If you're querying by _id, use findById() instead.
anifarm.findById("707876147324518400")
Official docs here

Mongoose find inside nested schema

In mongoose we are deeply searching inside a nested schema, without much success. Every time we run this function we always get an empty array returned.
function findAlarms(lastUpdate = new Date(0), record = Record) {
// For docs on find http://mongoosejs.com/docs/queries.html
return record
.find({
// Date due must be less than "now"
'documents.alarm.date_due': {
$lte: Date.now(),
},
// Must be greater than the last update and less than "now"
'documents.alarm.date_reminder.reminder': {
$gte: lastUpdate,
$lte: Date.now(),
},
})
.populate('documents')
.exec();
}
Our schemas, greatly summarized, look like this:
const RecordSchema = new mongoose.Schema({
documents: [
{
type: Schema.Types.ObjectId,
ref: 'Document',
},
],
});
And our documents schema, similarly summarized looks like this:
const DocumentSchema = new mongoose.Schema({
alarm: {
date_due: { type: Date },
date_reminder: [
{
reminder: { type: Date },
},
],
},
});
This search returns no matching elements, even though we know there are documents that match. If we modify our findAlarms method to use the documents schema:
function findAlarms(lastUpdate = new Date(0), document = Document) {
// For docs on find http://mongoosejs.com/docs/queries.html
return document
.find({
// Date due must be less than "now"
'alarm.date_due': {
$lte: Date.now(),
},
// Must be greater than the last update and less than "now"
'alarm.date_reminder.reminder': {
$gte: lastUpdate,
$lte: Date.now(),
},
})
.exec();
}
It will return all of our matching documents. However, having records is essential for our needs. Now, I could use a hack and then find records using the array of document._ids that return.
Nonetheless, I would love to know if there's an approach where we can find using the records directly, since adding that extra step feels really hacky, and this operation runs every 5 minutes so I'd love to be more efficient wherever posible.

Mongoose doesn't create TTL indexes

This is my Mongoose model:
var sessionSchema = new Schema({
_id: { type: String, required: true, index: { unique: true } },
user: { type: Schema.Types.ObjectId },
expire: { type: Date, index: { expireAfterSeconds: 21600 } }
})
module.exports = mongoose.model('Session', sessionSchema)
I need to be able to set a date object into expire (usually it's something like Date.now plus a few minutes) and have the object removed from the collection after 6 hours past the expiration.
However, I'm not able to have Mongoose to create the index. When I run db.sessions.getIndexes() in the mongo console, here's the output:
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "dev.sessions"
}
]
I've tried also with different syntaxes, like
expire: { type: Date, expires: 21600 } (Mongoose's short-hand version).
I tried also defining the index at the schema level:
sessionSchema.index({ expire: 1 }, { expireAfterSeconds: 21600 })
None is working.
Unlike others who asked questions on SO, my index is simply not created. I've tried also removing the collection and the database as well, and when they're recreated they still don't contain the index.
Versions: Mongoose 3.8.19, MongoDB 2.6.5 (OSX) and Node.js 0.10.33
Edit
More info: I tried creating the index directly from the mongo console, with:
db.sessions.ensureIndex({"expire":1}, {expireAfterSeconds: 21600})
That appears to be working (the index is created).
However, it's not working with Mongoose in any way.
Apparently the problem was that I created an index on the custom _id field. MongoDB creates an index on that field by itself, so when Mongoose was calling ensureIndex to create also the TTL index, it failed for both.
See https://github.com/LearnBoost/mongoose/issues/2459

Weird value returned for sum aggregation in Elasticsearch Javascript client

I'm trying to perform a sum aggregation on a numeric field (with type double) using the Javascript client for Elasticsearch.
Here's my code:
this.client.search({
index: "customers",
body: {
aggs: {
counts_in_range: {
filter: {
range: {
timestamp : {
gte : startDate,
lt : endDate
}
}
},
aggs: {
counts: {
sum : {
field : "price"
}
}
}
}
}
}
}).then(function (resp) {
cb(resp.aggregations, null);
}, function (err) {
cb(null, err);
});
Example document:
{
_index: "customers",
_type: "purchase",
_id: "98cb1066-057b-48e1-adff-eb32d9ed75a5",
_score: 1,
_source: {
timestamp: "2014-06-11T18:14:36+03:00",
itemId: 1,
price: 0.54
}
}
What I get back from the aggregation is a very long number e.g. 27549779928520990000 instead of a decimal number. The problem seems to be that in my document I store decimal numbers and not integers. If I store an integer in the price field the aggregation works just fine.
Not sure if this is a parsing issue with the Javascript client.
When you're first indexing a document, the type for each field is decided by elasticsearch if none is specified. In your case, elasticsearch thinks you're storing integers in your document, even if you will later store decimal numbers. So when it's computing the sum, it will try to work only with integers, but it came across to decimal numbers, hence the long number returned.
To avoid that from happening, map each field to a core value when creating a new type of document.

Use of $avg function within MongoDB aggregation framework does not appear to alter returned value

I'm sure there's something relatively obvious that I'm missing. Basically, these two queries return the exact same result despite the use of the $avg and $sum:1 functions, from what I understand the first query should return an average of the previously grouped row count?
db.bbservicedata.aggregate(
{
$match:{"accesstime" : {"$gte" : ISODate('2012-02-09T01:45:32.962Z') }}
},
{
$match:{"requestModel.serviceName" : "ContentItem"}},
{
$unwind: "$requestModel.methodParams.ContentItemLoggingListModel.items"},
{
$group: {
_id:{
myYear:{$year:"$accesstime"},
myMonth:{$month:"$accesstime"},
myDay:{$dayOfMonth:"$accesstime"}},
count:{$sum:1}}},
{
$group: {
_id: {
year: "$_id.myYear",
month: "$_id.myMonth",
day: "$_id.myDay"},
averagecount : {$avg : "$count"}}},
//{averagecount : {$sum: "$count"}}}***, -- Returns the same result*
{$sort: {averagecount:-1}}
);
Any assistance would be very much appreciated, thanks in advance!
Your second $group is using the same _id terms as the first one, so you'll always have a single doc per _id. In that situation $avg is always going to equal $sum.

Categories