Technologies: Mongoose, NodeJS, MongoDB
Question
The following document will save on the MongoDB cluster every 10 seconds.
{
"_id": "6003fafc04cb3727e40812b2",
"currentRound": 39300,
"current": 4.131929,
"voltage": 245.855,
"power": 956.5797,
"frequency": 50,
"totalPower": 1167.862,
"importPower": 1167.862,
"exportPower": 0,
"powerFactor": 0.998356,
"rssi": -59,
"deviceId": "EC:FA:BC:63:02:C1",
"slaveId": 201,
"timestamp": 1610873596543,
"__v": 0
}
There is two types of slave ids (101, 201) documents saving on the same collection and each slave id is having a specific device id. I want to retrieve the most latest and oldest 101 and 201 containing document by using yesterday's timestamps, starting 0:00 AM to 11:59 PM.
Attempt
I have tried the following solution. but distinct('slaveId') returns only the distinct specific field attribute only.
const latestPGStats = await PGStat
.find({
deviceId: { $in: deviceIds },
timestamp: { $lte: endTimestamp, $gte: startTimestamp }
})
.sort({ timestamp: -1 })
.distinct('slaveId')
.limit(2);
I have seen some peoples suggest using mongo aggregation. but I don't have knowledge about that domain.
You can try aggregate(),
$match your conditions
$facet to separate results, first is latest and second is oldest
$sort by timestamp in descending order
$group by slaveId and get $first document in latest and $last document in oldest
$limit` to get single document
const latestPGStats = await PGStat.aggregate([
{
$match: {
deviceId: { $in: deviceIds },
timestamp: { $lte: endTimestamp, $gte: startTimestamp }
}
},
{ $sort: { timestamp: -1 } },
{
$facet: {
latest: [
{
$group: {
_id: "$slaveId",
root: { $first: "$$ROOT" }
}
},
{ $limit: 1 }
],
oldest: [
{
$group: {
_id: "$slaveId",
root: { $last: "$$ROOT" }
}
},
{ $limit: 1 }
]
}
}
])
Related
I have a schema like below:
[
{
"_id": 1,
"showResult": true,
"subject": "History",
},
{
"_id": 2,
"showResult": false,
"subject": "Math",
}
]
and an object in JS like below:
result = {
"History": 22,
"Math": 18
}
I am using aggregate to process query, in between i need to find score based on subject field in the document if showResult field is true i.e to access result variable inside query as map result[$subject]
My query:
db.collection.aggregate([
{
"$project": {
_id: 1,
"score":{$cond: { if: { $eq: [ "$showResult", true ] }, then: subjectObj[$subject], else: null }}
}
}
])
can this be done in MongoDB, i want result like below:
{
_id: 1,
score: 22
}
I think query is little costly than JS code, but i am adding the query if it will help you as per your question,
$match showResult is true
$project to show required fields, $reduce to iterate loop of result after converting from object to array using $objectToArray, check condition if subject match then return matching score
let result = {
"History": 22,
"Math": 18
};
db.collection.aggregate([
{ $match: { showResult: true } },
{
$project: {
_id: 1,
score: {
$reduce: {
input: { $objectToArray: result },
initialValue: 0,
in: {
$cond: [{ $eq: ["$$this.k", "$subject"] }, "$$this.v", "$$value"]
}
}
}
}
}
])
Playground
I have documents of a collection that look like following:
{
items: [
{
price: 80
},
{
price: 70
}
]
},
{
items: [
{
price: 100
},
{
price: 85
}
]
},
{
items: [
{
price: 200
},
{
price: 85
}
]
},
...
I want to be able to query all documents with MyObj.find(query) where the sum of all item prices is between a given range. For example, $gte 160 and $lte 200 which would only return me the document the second document in the example.
const query = ???
const filteredDocs = await MyObj.find(query).exec()
How would the query for this look like?
In this case, you can use aggregation. First, add field represent the sum of items price with $addFields and $sum then find document has that field match your conditions. Something like:
MyObj.aggregate([
{
$addFields: {
total: {
$sum: "$items.price"
}
}
},
{
$match: {
total: {
$gte: 160,
$lte: 200
}
}
},
])
Mongo Playground
EDIT: If you only want to use find(), you can try $expr:
MyObj.find({
$expr: {
$and: [
{
$gte: [
{
$sum: "$items.price"
},
160
]
},
{
$lte: [
{
$sum: "$items.price"
},
200
]
}
]
}
})
Mongo Playground
I am new to MongoDB and I am stuck in the below scenario.
I have a collection that contains duplicate docs.
I just want to get the sum of the property in each doc excluding the duplicate docs.
My Docs looks like this:
{"_id":"5dd629461fc50b782479ea90",
"referenceId":"5dd581f10859d2737965d23a",
"sellingId":"319723fb80b1a297cf0803abad9bc60787537f14a6a37d6e47",
"account_name":"mrfsahas1234",
"vendor_name":"testaccount2",
"action_type":"purchase",
"product_name":"Bottle",
"product_quantity":10,
"transactionId":"319723fb80b1a297cf0803abad9bc60787537f14a6a37d6e47",
"uid":"2019-11-20T17:39:17.405Z",
"createdAt":"2019-11-21T08:56:56.589+00:00",
"updatedAt":"2019-11-21T08:56:56.589+00:00","__v":0
},
{
"_id":"5dd629461fc50b782479ea90",
"referenceId":"5dd581f10859d2737965d23a",
"sellingId":"320a9a2f814a45e01eb98344c9af708fa2864d81587e5914",
"account_name":"mrfsahas1234",
"vendor_name":"testaccount2",
"action_type":"purchase",
"product_name":"Bottle",
"product_quantity":50,
"transactionId":"320a9a2f814a45e01eb98344c9af708fa2864d81587e5914",
"uid":"2019-11-20T17:39:17.405Z",
},
{
"_id":"5dd629461fc50b782479ea90",
"referenceId":"5dd581f10859d2737965d23a",
"sellingId":"320a9a2f814a45e01eb98344c9af708fa2864d81587e5914",
"account_name":"mrfsahas1234",
"vendor_name":"testaccount2",
"action_type":"purchase",
"product_name":"Bottle",
"product_quantity":50,
"transactionId":"320a9a2f814a45e01eb98344c9af708fa2864d81587e5914",
"uid":"2019-11-20T17:39:17.405Z",
},
Currently, I am doing this:
MaterialsTrack.aggregate([
{
$match: {
$and: [
{product_name: product_name},
{account_name: account_name},
{action_type: 'purchase'},
{uid:uid}
]
}
},
{
$group: {_id: "$sellingId", PurchseQuantity: {$sum: "$product_quantity"}}
},
])
It returns the sum of product_quantity all the matching docs (including the duplicate docs).
Current Output:
{_id: "320a9a2f814a45e01eb98344c9af708fa2864d81587e5914", PurchseQuantity:110}
Expected Output:
{_id: "320a9a2f814a45e01eb98344c9af708fa2864d81587e5914", PurchseQuantity:60}
I want to get the sum of only unique docs. How can I achieve it?
Thanks in advance!
You need to sum inside of the $group _id field, and then use the replaceRoot to achieve the the result you wanted.
MaterialsTrack.aggregate([
{
$match: {
$and: [
{
product_name: "Bottle"
},
{
account_name: "mrfsahas1234"
},
{
action_type: "purchase"
},
{
uid: "2019-11-20T17:39:17.405Z"
}
]
}
},
{
$group: {
_id: {
sellingId: "$sellingId",
PurchaseQuantity: {
$sum: "$product_quantity"
}
}
}
},
{
$replaceRoot: {
newRoot: {
_id: "$_id.sellingId",
PurchaseQuantity: "$_id.PurchaseQuantity"
}
}
}
]);
Sample Output:
[
{
"PurchaseQuantity": 50,
"_id": "320a9a2f814a45e01eb98344c9af708fa2864d81587e5914"
}
]
Playground:
https://mongoplayground.net/p/MOneCRiSlO0
What about adding $addToSet to your aggregations pipeline
MaterialsTrack.aggregate([
{
$match: {
$and: [
{product_name: product_name},
{account_name: account_name},
{action_type: 'purchase'},
{uid:uid}
]
}
},
{
$group: {_id: "$sellingId", PurchseQuantity: {$sum: "$product_quantity"},"list" : {$addToSet : "$list"}}
},
])
I've an analytics API written using MongoDB.
This is my sessions model
const sessionSchema = new Schema(
{
user: { id: Number, name: String, email: String },
},
{ timestamps: true },
);
I want to get the unique users count by the date.
for the date 2019-10-24 there maybe 10 sessions from two users (id 1, id 2)
and
for the date 2019-10-25 there maybe 20 sessions from two users (id 3, id 8)
So my expected results is
2019-10-24 2 users
2019-10-25 2 users
I tried this
db.Session.aggregate([
{
$group: {
_id: { user: '$user.id', day: { $dayOfYear: '$createdAt' } },
count: { $sum: 1 },
},
},
])
and this doesn't seem to work.
My createdAt field's type is Date (eg:- createdAt: 2019-10-16T13:11:17.935Z) That is why I used $dayOfYear: '$createdAt'
db.Session.aggregate([
{
$project: {
date: { $dateToString: { format: "%Y-%m-%d", date: "$createdAt" } }
}
},
{
$group: {
_id: "$date" ,
count: { $sum: 1 }
}
}
]);
At first group according to createdAt, as we want final result according to createdAt.
In this $group stage just push the userIds into the array, use $addToSet to keep it unique.
Then get the count of userIds with $size operator in $project stage.
You get the result.
Here is the query.
(convert the date as you want, its just a format, and you have done this, so I am skipping this task).
db.sessions.aggregate({$match:{}},
{$group:{_id: "$createdAt", userId : {$addToSet: "$user.id"}}},
{$project:{_id: 1, noOfUsers:{$size:"$userId"}}}).pretty()
Hope this helps!
on my project i have users that complete combinations (called sessions) of courses. the fact of playing a course is called an attempt. During the attempt they can close it and come back later (so we keep a timelog object).
I have a request from the client which needs to return for each session, the users (and their attempts) that have played whole or part of their session during a certain timeframe.
During a certain timeframe means that the client sends a begin and end date and we count a user for a specific session if:
- the first attempt has begun before the end of the timeframe => the started of the first timelog of the first < ending date
- the last attempt has been finished after the begining of the timeframe => the end of the last timelog of the last attempt > starting date
Here is an example of an attempt object (the only one we need to use here):
{
"_id" : ObjectId("5b9148650ab5f43b5e829a4b"),
"index" : 0,
"author" : ObjectId("5acde2646055980a84914b6b"),
"timelog" : [
{
"started" : ISODate("2018-09-06T15:31:49.163Z"),
"ended" : ISODate("2018-09-06T15:32:03.935Z")
},
...
],
"session" : ObjectId("5b911d31e58dc13ab7586f9b")}
My idea was to make an aggregate on the attempts, to group those using author and session as an _id for the $group stage, and to push all the attempts of the user for this particular session into an array userAttempts.
Then to make an $addField stage to retrieve the started field of the first timelog of the first attempt and the last ended of the last attempt.
And finally to $filter or $match using those new fields.
Here is my aggregate:
const newDate = new Date()
_db.attempts.aggregate([
{ $match: {
author: { $in: programSessionsData.users },
$or: [{ programSession: { $in: programSessionIds } }, { oldTryFor: { $in: programSessionIds } }],
globalTime: $ex,
timelog: $ex }
},
{
$group: {
_id: {
user: "$author",
programSession: "$programSession"
},
userAttempts: { $push: { attemptId: "$_id", lastTimelog: { $arrayElemAt: ["$timelog", -1] }, timelog: "$timelog" } }
}
},
{
$addFields: { begin: { $reduce: {
input: "$userAttempts",
initialValue: newDate,
in: {
$cond: {
if: { $lt: ["$$this.timelog.0.started", "$$value"] },
then: "$$this.timelog.0.started",
else: "$$value"
} }
} } }
}
I also tried this for the addFields stage:
{
$addFields: { begin: { $reduce: {
input: "$userAttempts",
initialValue: newDate,
in: { $min: ["$$this.timelog.0.started", "$$value] }
} } }
}
However everytime begin is an empty array.
I do not really know how i can extract those two date, or compare dates between them.
To Note: the end one is more difficult that is why i have to first extract lastTimelog. If you an other method i would gladly take it.
Also this code is on a node server so i cannot use ISODate. and the mongo version used is 3.6.3.
After playing with aggregate a bit i came up with 2 solutions:
Solution 1
_db.attempts.aggregate([
{ $match: {
query
},
{
$group: {
_id: {
user: "$author",
programSession: "$programSession"
},
userAttempts: { $push: { attemptId: "$_id", timelog: "$timelog" } }
}
}, {
$addFields: {
begin: { $reduce: {
input: "$userAttempts",
initialValue: newDate,
in: { $min: [{ $reduce: {
input: "$$this.timelog",
initialValue: newDate,
in: { $min: ["$$this.started", "$$value"] }
} }, "$$value"] }
} },
end: { $reduce: {
input: "$userAttempts",
initialValue: oldDate,
in: { $max: [{ $reduce: {
input: "$$this.timelog",
initialValue: oldDate,
in: { $max: ["$$this.ended", "$$value"] }
} }, "$$value"] }
} }
}
},
{
$match: {
begin: { $lt: req.body.ended },
end: { $gt: req.body.started }
}
}
], { allowDiskUse: true });
newDate is today and oldDate is an arbitrary date in the past.
I had to chain 2 reduce because "$$this.timelog.0.started" would always return nothing. Don't really know why though.
Solution 2
_db.attempts.aggregate([
{ $match: {
query
},
{
$addFields: {
firstTimelog: { $arrayElemAt: ["$timelog", 0] },
lastTimelog: { $arrayElemAt: ["$timelog", -1] }
}
},
{
$group: {
_id: {
user: "$author",
programSession: "$programSession"
},
begin: { $min: "$firstTimelog.started" },
end: { $max: "$lastTimelog.ended" },
userAttempts: { $push: { attemptId: "$_id", timelog: "$timelog"} }
}
},
{
$match: {
begin: { $lt: req.body.ended },
end: { $gt: req.body.started }
}
}
], { allowDiskUse: true });
This one is a lot more straight forward and seems simpler, but oddly enough, from my testing, Solution 1 is always quicker at least in the object distribution for my project.