MongoDB Aggregation Framework - How To Do Multiple $group Queries - javascript

I have the following MongoDB aggregation query that finds all records within a specified month, $groups the records by day, and then returns an average price for each day. I would also like to return a price average for the entire month. Can I do this by using multiple $groups, if so, how?
PriceHourly.aggregate([
{ $match: { date: { $gt: start, $lt: end } } },
{ $group: {
_id: "$day",
price: { $avg: '$price' },
system_demand: { $avg: '$system_demand'}
}}
], function(err, results){
results.forEach(function(r) {
r.price = Helpers.round_price(r.price);
r.system_demand = Helpers.round_price(r.system_demand);
});
console.log("Results Length: "+results.length, results);
res.jsonp(results);
}); // PriceHourly();
Here is my model:
// Model
var PriceHourlySchema = new Schema({
created: {
type: Date,
default: Date.now
},
day: {
type: String,
required: true,
trim: true
},
hour: {
type: String,
required: true,
trim: true
},
price: {
type: Number,
required: true
},
date: {
type: Date,
required: true
}
},
{
autoIndex: true
});

The short answer is "What is wrong with just expanding your date range to include all the days in a month?", and therefore that is all you need to change in order to get your result.
And could you "nest" grouping stages? Yes you can add additional stages to the pipeline, that is what the pipeline is for. So if you first wanted to "average" per day and then take the average over all the days of the month, you can form like this:
PriceHourly.aggregate([
{ "$match": {
"date": {
"$gte": new Date("2014-03-01"), "$lt": new Date("2014-04-01")
}
}},
{ "$group": {
"_id": "$day",
"price": { "$avg": "$price" },
"system_demand": { "$avg": "$system_demand" }
}},
{ "$group": {
"_id": null,
"price": { "$avg": "$price" },
"system_demand": { "$avg": "$system_demand" }
}}
])
Even though that is likely to be reasonably redundant as this can arguably be done with one single group statement.
But there is a longer commentary on this schema. You do not actually state much of the purpose of what you are doing other than obtaining an average, or what the schema is meant to contain. So I want to describe something that is maybe a bit different.
Suppose you have a collection that includes the "product", "type" the "current price" and the "timestamp" as a date when that "price" was "changed". Let us call the collection "PriceChange". So every time this event happens a new document is created.
{
"product": "ABC",
"type": 2,
"price": 110,
"timestamp": ISODate("2014-04-01T00:08:38.360Z")
}
This could change many times in an hour, a day or whatever the case.
So if you were interested in the "average" price per product over the month you could do this:
PriceChange.aggregate([
{ "$match": {
"timestamp": {
"$gte": new Date("2014-03-01"), "$lt": new Date("2014-04-01")
}
}},
{ "$group": {
"_id": "$product",
"price_avg": { "$avg": "$price" }
}}
])
Also, without any additional fields you can get the average price per product for each day of the month:
PriceChange.aggregate([
{ "$match": {
"timestamp": {
"$gte": new Date("2014-03-01"), "$lt": new Date("2014-04-01")
}
}},
{ "$group": {
"_id": {
"day": { "$dayOfMonth": "$timestamp" },
"product": "$product"
},
"price_avg": { "$avg": "$price" }
}}
])
Or you can even get the last price for each month over a whole year:
PriceChange.aggregate([
{ "$match": {
"timestamp": {
"$gte": new Date("2013-01-01"), "$lt": new Date("2014-01-01")
}
}},
{ "$group": {
"_id": {
"date": {
"year": { "$year" : "$timestamp" },
"month": { "$month": "$timestamp" }
},
"product": "$product"
},
"price_last": { "$last": "$price" }
}}
])
So those are some things you can do using the build in Date Aggregation Operators to achieve various results. These can even aid in collection of this information for writing into new "pre-aggregated" collections, to be used for faster analysis.
I suppose there would be one way to combine a "running" average against all prices using mapReduce. So again from my sample:
PriceHourly.mapReduce(
function () {
emit( this.timestamp.getDate(), this.price )
},
function (key, values) {
var sum = 0;
values.forEach(function(value) {
sum += value;
});
return ( sum / values.length );
},
{
"query": {
"timestamp": {
"$gte": new Date("2014-03-01"), "$lt": new Date("2014-04-01")
}
},
"out": { "inline": 1 },
"scope": { "running": 0, "counter": 0 },
"finalize": function(key,value) {
running += value;
counter++;
return { "dayAvg": value, "monthAvg": running / counter };
}
}
)
And that would return something like this:
{
"results" : [
{
"_id" : 1,
"value" : {
"dayAvg" : 105,
"monthAvg" : 105
}
},
{
"_id" : 2,
"value" : {
"dayAvg" : 110,
"monthAvg" : 107.5
}
}
],
}
But if you are otherwise expecting to see discrete values for both the day and the month, then that would not be possible without running separate queries.

Related

Get count of documents matching different conditions

I have collection: bookSchema as:
[
{
_id: ObjectId("637d05dc32428ed75ea08d09"),
book_details: {
book_name: "random123",
book_auth: "Amber"
}
},
{
_id: ObjectId("637d0673ce0f17f6c473dee2"),
book_details: {
book_name: "random321",
book_auth: "Amber"
}
},
{
_id: ObjectId("637d069a3d597c8458ebe4ec"),
book_details: {
book_name: "random676",
book_auth: "Amber"
}
},
{
_id: ObjectId("637d06c05b32d503007bcb54"),
book_details: {
book_name: "random999",
book_auth: "Saurav"
}
}
]
Desired O/P to show as:
{
score_ambr: 3,
score_saurabh: 1
}
For this I tried as:
db.bookSchema.aggregate([
{
"$group": {
"_id": {
"$eq": [
"$book_details.book_auth",
"Amber"
]
},
"score_ambr": {
"$sum": 1
}
},
},
{
"$group": {
"_id": {
"$eq": [
"$book_details.book_auth",
"Saurav"
]
},
"score_saurabh": {
"$sum": 1
}
},
}
])
I tried using $group to as I want to group all the matching documents in one and use $count to give the number of count for the matching documents but it doesn't seem to be working and gives the O/P as
O/P:
[
{
"_id": false,
"score_sau": 2
}
]
MongoDB Playground: https://mongoplayground.net/p/cZ64KwAmwlv
I don't know what mean 3 and 1 in your example but if I've understood correctly you can try this query:
The trick here is to use $facet to create "two ways" in the aggregation. One option will filter by Amber and the other one by Saurav.
And then, as values are filtered, you only need yo know the size of the array generated.
db.collection.aggregate([
{
"$facet": {
"score_ambr": [
{
"$match": {
"book_details.book_auth": "Amber"
}
}
],
"score_saurabh": [
{
"$match": {
"book_details.book_auth": "Saurav"
}
}
]
}
},
{
"$project": {
"score_ambr": {
"$size": "$score_ambr"
},
"score_saurabh": {
"$size": "$score_saurabh"
}
}
}
])
Example here
Note that in this way you avoid to use $group.
It looks like what you want is two group twice and create a dynamic key from the book_details.book_auth:
db.bookSchema.aggregate([
{$group: {_id: "$book_details.book_auth", count: {$sum: 1}}},
{$group: {
_id: 0,
data: {$push: {
k: {$concat: ["score_", {$toLower: "$_id"}]},
v: {$sum: "$count"}
}}
}},
{$replaceRoot: {newRoot: {$arrayToObject: "$data"}}}
])
See how it works on the playground example

Script that returns _id in MongoDB with JavaScript

I need a javascript code that saves in a variable the _id I'm having trouble getting that _id because it’s inside an array.
I have looked everywhere but I can't find a solution for this.
{
"_id": {
"$oid": "626bacea1847f675e47b2bd8"
},
"tipo": "conta",
"data_abertura": {
"$date": "2013-02-19T00:00:00Z"
},
"observacoes": "Sem observações",
"iban": "PT50000506515926456299903",
"saldo": 1456.23,
"bic": "BBPIPTPL001",
"tipo_conta": "Conta Ordenado",
"cartoes": [
{
"_id": {
"$oid": "626bacea1847f675e47b2bd7"
},
"num_cartao": 4908509005925727,
"nome_cartao": "João Correia",
"validade": {
"$date": "2025-10-03T00:00:00Z"
},
"pin": 5609,
"cvc": 975,
"estado": "Ativo",
"tipo_cartao": "Crédito"
}
],
"permissoes": {
"levantamentos": true,
"depositos": true,
"pagamentos": true,
"transferencias": true,
"creditos": true,
"acoes": true
},
"titulares": [
{
"$oid": "626bacea1847f675e47b2bd6"
}
]
}
I want the cartoes _id
"cartoes": [
{
"_id": {
"$oid": "626bacea1847f675e47b2bd7"
},
I want something like this:
let conta = db.banco.findOne({"tipo": "conta"});
idConta = conta._id;
console.log("id: " + idConta);//id: 626bacea1847f675e47b2bd8
Edit:
According to the comments, I understand you want to input num_cartao value and to get an output of the _id of this cartao. You can use an aggregation for this:
const cartao_id = await db.collection.aggregate([
{
$match: {cartoes: {$elemMatch: {"num_cartao": 4908509005925727}}}
},
{
$project: {
cartoes: {
$filter: {
input: "$cartoes",
as: "item",
cond: {$eq: ["$$item.num_cartao", 4908509005925727]}
}
}
}
},
{
$project: {
data: {"$arrayElemAt": ["$cartoes", 0]}
}
},
{
$project: {
_id: {$toString: "$data._id"}
}
}
])
console.log(`cartao_id: ${cartao_id}`);//id: 626bacea1847f675e47b2bd7
That will provide you what you want, as you can see on this playground
You first $match the documents with the right num_cartao, then $filter the right carto from the cartoes array, and then you format it to string.

Query access variable outside query based on condition in NodeJS and MongoDB

I have a schema like below:
[
{
"_id": 1,
"showResult": true,
"subject": "History",
},
{
"_id": 2,
"showResult": false,
"subject": "Math",
}
]
and an object in JS like below:
result = {
"History": 22,
"Math": 18
}
I am using aggregate to process query, in between i need to find score based on subject field in the document if showResult field is true i.e to access result variable inside query as map result[$subject]
My query:
db.collection.aggregate([
{
"$project": {
_id: 1,
"score":{$cond: { if: { $eq: [ "$showResult", true ] }, then: subjectObj[$subject], else: null }}
}
}
])
can this be done in MongoDB, i want result like below:
{
_id: 1,
score: 22
}
I think query is little costly than JS code, but i am adding the query if it will help you as per your question,
$match showResult is true
$project to show required fields, $reduce to iterate loop of result after converting from object to array using $objectToArray, check condition if subject match then return matching score
let result = {
"History": 22,
"Math": 18
};
db.collection.aggregate([
{ $match: { showResult: true } },
{
$project: {
_id: 1,
score: {
$reduce: {
input: { $objectToArray: result },
initialValue: 0,
in: {
$cond: [{ $eq: ["$$this.k", "$subject"] }, "$$this.v", "$$value"]
}
}
}
}
}
])
Playground

mongodb/javascript: Group data by date, perform operations on it and then display it

This is a sample JSON object, among 1000 like them, stored in my MongoDB collection.
{
"_id": ObjectId("5b1bb74ffc7ee601c6915939"),
"groupId": "-abcde",
"applicationId": "avcvcvc",
"integration": "web",
"Category": "message",
"Action": "message",
"Type": "newMessage",
"Id": "activity",
"data": {
"test": "good morning"
},
"timestamp": 1528543055858.0,
"createdAt": ISODate("2018-06-09T11:17:35.868+0000"),
"updatedAt": ISODate("2018-06-09T11:17:35.868+0000"),
"__v": NumberInt(0)
}
This object is an example of data with date as 2018-06-09. There are objects with successive dates in db. I have to fetch data with a specific date and perform this operation on all objects with that date
db.collection.aggregate([{
$match: {
$or: [{
Type: "on mouse hover click"
}, {
Type: "on mouse out"
},
{
Type: "on chat start"
}, {
Type: "Load Event"
}
]
}
},
{
$group: {
_id: null,
count: {
$sum: 1
}
}
}
]);
The api, when called, should show a specific date and the calculation performed in the upper query alongwith it. this should happen on all objects with diff. dates. For this, i have made a sample schema
let { Schema } = require("mongoose");
let mongoose = require("mongoose");
let dataSchema = Schema({
date: { type: Date },
calculation: { type: Number }
});
let NewData = mongoose.model('dataSchema', dataSchema);
module.exports = { NewData };
How can i achieve this? I am new to MongoDB so i cant quite figure out how to do this.
You cannot group by a date cause are differents minutes seconds millis etc etc
you need make a project for date instead datetime, those can have same date but different time
Then you can perform grouping and what you want to do
db.getCollection("collection")
.aggregate([
{
$project : {
"groupId":1,
"applicationId":1,
"integration":1,
"Category":1,
"Action":1,
"Type":1,
"Id":1,
"data":1,
"timestamp":1,
"createdAt":1,
"updatedAt":1,
"__v":1
createdDate: { $dateToString: {format: "%G-%m-%d", date: "$createdAt"}},
}
},
{
$group: { _id: "$createdDate", count: { $sum: 1 } },
}
]);

How can I sort array by a field inside a MongoDB document

I have document called question
var QuestionSchema = new Schema({
title: {
type: String,
default: '',
trim: true
},
body: {
type: String,
default: '',
trim: true
},
user: {
type: Schema.ObjectId,
ref: 'User'
},
category: [],
comments: [{
body: {
type: String,
default: ''
},
root: {
type: String,
default: ''
},
user: {
type: Schema.Types.ObjectId,
ref: 'User'
},
createdAt: {
type: Date,
default: Date.now
}
}],
tags: {
type: [],
get: getTags,
set: setTags
},
image: {
cdnUri: String,
files: []
},
createdAt: {
type: Date,
default: Date.now
}
});
As a result, I need to sort comments by root field, like this
I tried to sort the array of comments manually at backend and tried to use aggregation, but I was not able to sort this. Help please.
Presuming that Question is a model object in your code and that of course you want to sort your "comments by "date" from createdAt then using .aggregate() you would use this:
Question.aggregate([
// Ideally match the document you want
{ "$match": { "_id": docId } },
// Unwind the array contents
{ "$unwind": "comments" },
// Then sort on the array contents per document
{ "$sort": { "_id": 1, "comments.createdAt": 1 } },
// Then group back the structure
{ "$group": {
"_id": "$_id",
"title": { "$first": "$title" },
"body": { "$first": "$body" },
"user": { "$first": "$user" },
"comments": { "$push": "$comments" },
"tags": { "$first": "$tags" },
"image": { "$first": "$image" },
"createdAt": { "$first": "$createdAt" }
}}
],
function(err,results) {
// do something with sorted results
});
But that is really overkill since you are not "aggregating" between documents. Just use the JavaScript methods instead. Such as .sort():
Quesion.findOneById(docId,function(err,doc) {
if (err) throw (err);
var mydoc = doc.toObject();
mydoc.questions = mydoc.questions.sort(function(a,b) {
return a.createdAt > b.createdAt;
});
console.log( JSON.stringify( doc, undefined, 2 ) ); // Intented nicely
});
So whilst MongoDB does have the "tools" to do this on the server, it makes the most sense to do this in client code when you retrieve the data unless you actually need to "aggregate" accross documents.
But both example usages have been given now.

Categories