MongoDB Get Average of each Filed-Child - javascript

I am using javascript - mongoose. (beginner)
Trying to calculate average of 'timeSpend' for each fields
{
"_id" : ObjectId("5a2cca1c44ac67e8d5b8d2ff"),
"total" : {
"timeSpend" : "20"
},
"name_1" : {
"timeSpend" : 9,
"test" : "fail"
},
"name_2" : {
"timeSpend" : "11",
"test" : "fail"
},
"name_3" : {
"timeSpend" : "8",
"test" : "fail"
}
"__v" : 0
},
{
"_id" : ObjectId("5a2cca1c44ac67e8d5b8d2ff"),
"total" : {
"timeSpend" : "10"
},
"name_1" : {
"timeSpend" : 10,
"test" : "fail"
},
"name_2" : {
"timeSpend" : "5",
"test" : "fail"
},
"name_3" : {
"timeSpend" : "2",
"test" : "fail"
}
"__v" : 0
}
.... more documents...(every 10 minutes, new document will be saved)
I am trying to calculate average 'timeSpend' for each fields
Example :
{
total : 15,
name_1 : 9.5,
name_2 : 8,
name_3 : 5
}
I tried
(mongoose Schema).find({}).select('name_1' : 1, '_id':0)
then calculate average using loop, but I think it will be too expensive computing since document will be updated every 10 minutes (have to update average every 10 mins)

You can achieve that with a simple mongodb aggregation.
You basically have to group by null to get one document in the end, and use the $avg aggregation operator to get all averages.
Following would work
collection.aggregate([{
$group: {
_id: null,
total: {
$avg: "$total.timeSpend"
},
name_1: {
$avg: "$name_1.timeSpend"
},
name_2: {
$avg: "$name_2.timeSpend"
},
name_3: {
$avg: "$name_3.timeSpend"
}
}
}])

Related

Aggregate a collection of timestamps in MongoDB using the Aggregation Pipeline

I have a collection of timestamps which record what actions are performed by users at which time. For now, the collection consists of only two actions start and end. There can only be a single end action, while there can be multiple start actions per user.
Now I want a generate a list of users where the time difference between the last start action and the end action is - for example - less than a minute.
The simplified documents in my collection timestamps look like this:
document #1
{
id: 123,
user: "user1",
type: "start",
date: 2019-09-10
}
document #2
{
id: 234,
user: "user1",
type: "end",
date: 2019-09-11
}
Now the result I want should look like this:
{
id: null,
list: ["user1, user2"]
}
The field list should contain every user, where the time difference between the start and end action is less than a minute.
I am having trouble combining the documents which contain the start and end attribute. I was trying to combine them into documents that looks like this:
{
id: 345
user: "user1"
date_start: 2019-09-10
date_end: 2019-09-11
}
I don't know where to start with the aggregation pipeline and how to split and combine the different types of timestamps. Furthermore, I still need to add a field that contains the difference between both dates.
The following query can get us the expected output:
db.collection.aggregate([
{
$sort:{
"date":-1
}
},
{
$group:{
"_id":{
"id":"$id",
"type":"$type"
},
"id":{
$first:"$id"
},
"user":{
$first:"$user"
},
"type":{
$first:"$type"
},
"date":{
$first:"$date"
}
}
},
{
$group:{
"_id":"$id",
"user":{
$first:"$user"
},
"info":{
$push:{
"k":"$type",
"v":"$date"
}
}
}
},
{
$addFields:{
"info":{
$arrayToObject:"$info"
}
}
},
{
$match:{
$expr:{
$lt:[
{
$subtract:[
{
$toDate:"$info.end"
},
{
$toDate:"$info.start"
}
]
},
60000
]
}
}
},
{
$group:{
"_id":null,
"users":{
$push:"$user"
}
}
},
{
$project:{
"_id":0
}
}
]).pretty()
Data set:
{
"_id" : ObjectId("5d77a117bd4e75c58d598214"),
"id" : 123,
"user" : "user1",
"type" : "start",
"date" : "2019-09-10T13:01:14.242Z"
}
{
"_id" : ObjectId("5d77a117bd4e75c58d598215"),
"id" : 123,
"user" : "user1",
"type" : "start",
"date" : "2019-09-10T13:04:14.242Z"
}
{
"_id" : ObjectId("5d77a117bd4e75c58d598216"),
"id" : 123,
"user" : "user1",
"type" : "start",
"date" : "2019-09-10T13:09:02.242Z"
}
{
"_id" : ObjectId("5d77a117bd4e75c58d598217"),
"id" : 123,
"user" : "user1",
"type" : "end",
"date" : "2019-09-10T13:09:14.242Z"
}
{
"_id" : ObjectId("5d77a117bd4e75c58d598218"),
"id" : 234,
"user" : "user2",
"type" : "start",
"date" : "2019-09-10T13:02:02.242Z"
}
{
"_id" : ObjectId("5d77a117bd4e75c58d598219"),
"id" : 234,
"user" : "user2",
"type" : "end",
"date" : "2019-09-10T13:09:14.242Z"
}
{
"_id" : ObjectId("5d77a117bd4e75c58d59821a"),
"id" : 345,
"user" : "user3",
"type" : "start",
"date" : "2019-09-10T13:08:55.242Z"
}
{
"_id" : ObjectId("5d77a117bd4e75c58d59821b"),
"id" : 345,
"user" : "user3",
"type" : "end",
"date" : "2019-09-10T13:09:14.242Z"
}
Output:
{ "users" : [ "user3", "user1" ] }
Query analysis:
Stage I: Sorting the documents in descending order of the date
Stage II: Grouping on [id, type] and picking the first date for
each type i.e. the latest date for each type
Stage III: Grouping only on id and pushing the type and associated date into an array as key-value pairs
Stage IV: Converting the array of key-value pairs into an object
Stage V: Filtering documents which has the difference between end and start date less than 60000 ms. (milliseconds equivalent of 1 minute)
Stage VI: Pushing all filtered names into an array

Merge arrays and group to produce a count for each combined array value

I have a dataset like this:
{
"_id" : ObjectId("5a4c6fb6993a721b3479a27e"),
"score" : 8.3,
"page" : "message",
"lastmodified" : ISODate("2018-01-03T06:49:19.232Z"),
"createdate" : ISODate("2018-01-03T05:52:54.446Z"),
"slug" : [
"#APPLE"
],
"__v" : 0
},
{
"_id" : ObjectId("5a4c6fb6993a721b3479a27e"),
"score" : 9.3,
"page" : "#BANANA",
"lastmodified" : ISODate("2018-01-03T06:49:19.232Z"),
"createdate" : ISODate("2018-01-03T05:52:54.446Z"),
"slug" : [
"#APPLE"
],
"__v" : 0
}
{
"_id" : ObjectId("5a4c6fb6993a721b3479a27e"),
"score" : 5.3,
"page" : "#BANANA",
"lastmodified" : ISODate("2018-01-03T06:49:19.232Z"),
"createdate" : ISODate("2018-01-03T05:52:54.446Z"),
"slug" : [
"#BANANA"
],
"__v" : 0
}
Now I want to calculate the sum of score according to my Filter Like this:
#APPLE: 8.3+9.3 = 17.6 i.e #APPLE: 17.6,
#BANANA: 9.3+5.3 = 14.6 i.e #BANANA: 14.6
So for this I have to pick only last 1 hour data rather than picking the whole database
. So my query is like this
var newTime = new Date();
newTime.setHours( newTime.getHours() - 1 );
db.Test.find({"lastmodified":{$gt: newTime}})
so By this I can get only last 1 hour value. Now I am confuse that how i can do sum with filter. I also attached filter query i.e
db.Test.find({"lastmodified":{$gt: newTime}}, {$or: [{slug: {$in: ['#APPLE']}}, {page: '#APPLE'}]})
But it does not give anything. any help is appreciated
Try this aggregate query...
db.tests.aggregate([{
"$unwind": "$slug"
},
{
"$group": {
"_id": "$slug",
"totalScore": {
"$sum": "$score"
}
}
}
]);
Result:
{
"_id" : "#BANANA",
"totalScore" : 5.3
}
{
"_id" : "#APPLE",
"totalScore" : 17.6
}

Mongoose (MongoDB) - Error: Can't use $each with Number

I've to push a given array of Number values into a selected Document inside my MongoDB database.
The Document that I'm going to update as the following structure:
{
"_id" : {
"id" : 17,
"type" : "f"
},
"__v" : 0,
"created_at" : ISODate("2017-03-22T11:16:21.403Z"),
"token" : {
"expDate" : ISODate("2017-12-31T00:00:00Z"),
"token" : "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJlbWFpbCI6ImFsYWRpbkBjb25zb3J6aW9jZXIuaXQiLCJleHAiOjE1MTQ2Nzg0MDB9.QvbT146bA_KH5XA7MH8ASXm9cr3sPZChJ3prYyDireI"
},
"updated_at" : ISODate("2017-07-24T09:42:33.741Z"),
"plots" : {
"idPlot" : [
23570,
23475
]
},
"machines" : [
{
"idPlotBind" : 1,
"ip" : "",
"mac" : "18-5F-00-4A-FE-F4",
"irrId" : 31,
"_id" : ObjectId("59084f527d634d301338aac6"),
"addr" : "pialadin.ddns.net"
},
{
"idPlotBind" : null,
"ip" : "",
"mac" : "12-01-02-FE-AB-B2",
"irrId" : 35,
"_id" : ObjectId("59084f7d7d634d301338aac7")
}
]
}
I'm using the Mongoose library for JS, and the accused query is this one:
userSchema.findOneAndUpdate({$and:[{ '_id.id': resData.PlotRows.IdUser}, {'_id.type': 'f'}]},{$addToSet:{'plots.$.idPlot': {$each: plotData}}}, {upsert: false}, function(err, usr){
if(err){
console.log(err);
return;
}
});
But when I try to execute it, gives me back:
Error: Can't use $each with Number

$multiply only supports numeric types, not array

I am trying to use the $multiply operator in MongoDB.
STEP 1
db.message1064sd_00011_3744.aggregate([
{$project : {
"prices.00026" : 1,
priceReal : {
price : "$prices.00026",
}
}},
{ $match : { "priceReal.price" : {$gt : 30 } } },
{ $limit : 1 }
])
I am getting the results
{
"result" : [
{
"_id" : "54884_00011_001",
"prices" : {
"00026" : 34.43
},
"priceReal" : {
"price" : 34.43
}
}
],
"ok" : 1
}
STEP 2
But, when I use $multiply, I get
$multiply only supports numeric types, not array
db.message1064sd_00011_3744.aggregate([
{$project : {
"prices.00026" : 1,
priceReal : {
price : { $multiply : ["$prices.00026", 1 ] },
}
}},
{ $match : { "priceReal.price" : {$gt : 30 } } },
{ $limit : 1 }
])
Help me anybody
Example document which I can get from db.message1064sd_00011_3744.findOne()
{
"_id" : "25906_00011_001",
"Message_ID" : 25906,
"Subdivision_ID" : 3747,
"Sub_Class_ID" : 6300,
"Checked" : 1,
"Nomencl_ID" : "10000014597",
"manufacturer_ID" : "П1170",
"disableIfZero" : 0,
"Discontinued" : 0,
"New" : 0,
"Nomencl_Group_ID" : 28,
"Nalichie" : "Мало",
"sort" : 99,
"Warehouse_ID" : "00011",
"ParentWarehouse_ID" : "00011",
"Kachestvo" : "001",
"Svobod_Nalichie" : "10",
"Svobod_sort" : 10,
"character" : [],
"prices" : {
"00014" : 1.51,
"00015" : 1.45,
"00016" : 1.41,
"00017" : 1.38,
"00018" : 1.35,
"00019" : 1.33,
"00021" : 1.31,
"00022" : 1.29,
"00023" : 1.28,
"00024" : 1.27,
"00025" : 1.25,
"00026" : 1.24
},
"price" : {
"Curr_ID" : 840,
"ChangePriceTime" : "2017-01-22 19:18:21",
"PriceUpDown" : "up",
"callPrice" : 0,
"Price_Value_RODP" : 1.24,
"Price_Value_RUR" : 72.04000000000001
},
"sName" : "чип epson m2300{m2400{mx20 8k (elp, китай)",
"sNomencl_ID" : "10000014597",
"sNomencl_Articul_Proizvod" : "elp-ch-e2300-8k",
"sItemID" : "elp-ch-e2300-8k",
"EnglishName" : "cZ277",
"begin_vl" : 121,
"Hidden_URL" : "/netshop/cZ079/cZ270/cZ277/",
"Checked_Subdivision" : 1
}
In case you want to ensure that only data from a certain data type enters your result set, for example in an aggregate query, you can add the following filter to the match:
{$match: {
// ... your query
, "priceReal.price": { $type : "double" }
}

Meteor Mongodb count fields in nested object

I am trying to create a dashboard where I show summaries of order data within the app. In this case I am simply wanting count the number of items in a given category in my Orders collection. My code so far looks like this:
Collection data
{
"_id" : "a6wHiXxyM5DwSAsfq",
"orderNumber" : 1234,
"createdAt" : "11/01/2016, 14:43:49",
"productsInOrder" : [
{
"category" : "ambient",
"item" : 50818,
"desc" : "Tasty Rubber Chicken",
"quantity" : "44",
"price" : "0.92",
"lineprice" : "40.48",
"_id" : "FFNxG8vujs6NGN69r"
},
{
"category" : "frozen",
"item" : 71390,
"desc" : "Generic Granite Fish",
"quantity" : "11",
"price" : "1.00",
"lineprice" : "11.00",
"_id" : "LcRtpyLxkWyh39kkB"
}
]
}
{
"_id" : "PdpywXCvfew7qojmA",
"orderNumber" : 1234,
"createdAt" : "11/01/2016, 14:44:15",
"productsInOrder" : [
{
"category" : "frozen",
"item" : 71390,
"desc" : "Generic Granite Fish",
"quantity" : "44",
"price" : "1.00",
"lineprice" : "44.00",
"_id" : "dAscx4R8pcBgbzoZs"
},
{
"category" : "frozen",
"item" : 66940,
"desc" : "Gorgeous Granite Bike",
"quantity" : "55",
"price" : "4.21",
"lineprice" : "231.55",
"_id" : "xm3mFRmPmmdPxjfP9"
},
{
"category" : "frozen",
"item" : 96029,
"desc" : "Gorgeous Plastic Fish",
"quantity" : "1234",
"price" : "4.39",
"lineprice" : "5417.26",
"_id" : "7u7SHnpTf7PWcrhGA"
}
]
}
{
"_id" : "xcHZ25qwvyDpDJtAZ",
"orderNumber" : 1234,
"createdAt" : "11/01/2016, 14:44:47",
"productsInOrder" : [
{
"category" : "frozen",
"item" : 31104,
"desc" : "Handcrafted Rubber Keyboard",
"quantity" : "11",
"price" : "4.78",
"lineprice" : "52.58",
"_id" : "LMMwbKFEgnCbgCt9c"
},
{
"category" : "frozen",
"item" : 77832,
"desc" : "Practical Rubber Shirt",
"quantity" : "21",
"price" : "0.62",
"lineprice" : "13.02",
"_id" : "63otkkXWGrTJkwEgX"
},
{
"category" : "frozen",
"item" : 66940,
"desc" : "Gorgeous Granite Bike",
"quantity" : "111",
"price" : "4.21",
"lineprice" : "467.31",
"_id" : "rbPSujey8CFeMPjza"
}
]
}
JS
So far I have tried:
Orders.find({ 'productsInOrder': ['ambient']}).count();
Orders.find({ productsInOrder: { category: 'ambient' }}).count();
Orders.find({ productsInOrder: { $all: [ 'frozen' ] }}).count();
I am having a hard time understanding Mongo queries when the data is nested in this manner. Please can you help point me in the right direction? Many thanks in advance.
* SOLUTION *
I have accomplished the desired result thanks to the contributions below. To make this work I created a method on the server as the query cannot be run on the client using an existing collection. This is done as follows:
Meteor.methods({
'byCategory': function() {
var result = Orders.aggregate([
{ "$unwind": "$productsInOrder" },
{
"$group": {
"_id": null,
"ambient_count": {
"$sum": {
"$cond": [ { "$eq": [ "$productsInOrder.category", "ambient" ] }, 1, 0 ]
}
},
"frozen_count": {
"$sum": {
"$cond": [ { "$eq": [ "$productsInOrder.category", "frozen" ] }, 1, 0 ]
}
},
"other_category_count": {
"$sum": {
"$cond": [ { "$eq": [ "$productsInOrder.category", "other_category" ] }, 1, 0 ]
}
}
}
}
]);
return result;
}
})
and then on the client:
Meteor.call('byCategory', function( error, result ) {
if( error ) {
console.log( error.reason );
} else {
console.log( result[0].ambient_count );
console.log( result[0].frozen_count );
etc....
}
});
Thanks and credit to #chridam and #Brett.
An alternative approach is to use the aggregation framework. Consider the following aggregation pipeline which as the first stage of the aggregation pipeline, the $unwind operator denormalizes the productsInOrder array to output for each input document, n documents where n is the number of array elements. The next pipeline stage has the $group operator which groups all the documents into a single document and stores the counts for each category with the help of the $sum and $cond operators.
In Meteor, you can then use meteorhacks:aggregate package to implement the aggregation:
Add to your app with
meteor add meteorhacks:aggregate
Note, this only works on server side and there is no oberserving support or reactivity built in. Then simply use .aggregate function like below.
var coll = new Mongo.Collection('orders');
var pipeline = [
{ "$unwind": "$productsInOrder" },
{
"$group": {
"_id": null,
"ambient_count": {
"$sum": {
"$cond": [ { "$eq": [ "$productsInOrder.category", "ambient" ] }, 1, 0 ]
}
},
"frozen_count": {
"$sum": {
"$cond": [ { "$eq": [ "$productsInOrder.category", "frozen" ] }, 1, 0 ]
}
},
"other_category_count": {
"$sum": {
"$cond": [ { "$eq": [ "$productsInOrder.category", "other_category" ] }, 1, 0 ]
}
}
}
}
];
var result = coll.aggregate(pipeline);
Running the same pipeline in mongo shell using the sample data will yield:
{
"result" : [
{
"_id" : null,
"ambient_count" : 1,
"frozen_count" : 7,
"other_category_count" : 0
}
],
"ok" : 1
}
You can access the native mongo collection and publish the aggregation results to the orders collection on the client side:
Meteor.publish('categoryCounts', function() {
var self = this,
db = MongoInternals.defaultRemoteCollectionDriver().mongo.db;
orders = db.collection("orders").aggregate(pipeline, // Need to wrap the callback so it gets called in a Fiber.
Meteor.bindEnvironment(
function(err, result) {
// Add each of the results to the subscription.
_.each(result, function(e) {
self.added("orders", e._id, e);
});
self.ready();
},
function(error) {
Meteor._debug( "Error doing aggregation: " + error);
}
)
);
});
If you don't want to do this within Meteor, you will need to use mongo aggregation. Minimongo doesn't include aggregation though, so you will need this package to accomplish it:
https://docs.mongodb.org/manual/core/aggregation-introduction/
I only tested this in mongo itself, so you will have to adapt it to the way that the aggregation package does it:
db.orders.aggregate([
{
$unwind: "$productsInOrder"
},
{
$match: {
"productsInOrder.category": "frozen"
}
},
{
$group: {
_id: null,
count: {
$sum: 1
}
}
}
]);
The first part is unwinding the collection. It will basically make an "order" entry for every instance of $productsInOrder. Once you have the array flattened out, we match on the category you care about; in this case, the "frozen" category. Next we group it up so we can count the number of documents returned. $group is simply constructing the final object that will be output from the query. You can modify this to be whatever you want, or you could group by productsInOrder.category and not even $match on "frozen".

Categories