average per day mongo in lookup - javascript

I have a collection (sensordatas) with data every minute. When I use this code I get all the data from de collections from sensordatas. This data is way too much. I like to have the average data per day, otherwise it will send to much data to the client. My code is
Sensor.aggregate([
{
$match: { _id: ObjectId(req.params.id) },
},
{
$lookup: {
from: "sensordatas",
as: "data",
let: { device_id: "$device_id", datum: "$datum" },
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: ["$device_id", "$$device_id"],
},
{ $gte: ["$datum", moment(startdate).toDate(),] },
{ $lte: ["$datum", moment(enddate).toDate(),] },
],
},
},
},
{ $sort: { datum: -1 } },
],
},
},
How can I print not every item but have a average per day. I hope you can help me

Related

MongoDB - Generating dynamic $or using pipeline variable?

hoping someone can help as I am truly stuck!
I have this query
SwapModel.aggregate([
{
$match: {
organisationId: mongoose.Types.ObjectId(organisationId),
matchId: null,
matchStatus: 0,
offers: {
$elemMatch: {
from: { $lte: new Date(from) },
to: { $gte: new Date(to) },
locations: { $elemMatch: { $eq: location } },
types: { $elemMatch: { $eq: type } },
},
},
//problem is HERE
$or: {
$map: {
input: "$offers",
as: "offer",
in: {
from: { $gte: new Date("$$offer.from") },
to: { $lte: new Date("$$offer.to") },
location: { $in: "$$offer.locations" },
type: { $in: "$$offer.types" },
},
},
},
},
},
{ ...swapUserLookup },
{ $unwind: "$matchedUser" },
{ $sort: { from: 1, to: 1 } },
]);
I'm trying to use the results of the $match document to generate an array for $or. My data looks like this:
[{
_id: ObjectId("id1"),
from: ISODate("2023-01-21T06:30:00.000Z"),
to: ISODate("2023-01-21T18:30:00.000Z"),
matchStatus: 0,
matchId: null,
userId: ObjectId("ddbb8f3c59cf13467cbd6a532"),
organisationId: ObjectId("246afaf417be1cfdcf55792be"),
location: "Chertsey",
type: "DCA",
offers: [{
from: ISODate("2023-01-23T05:00:00.000Z"),
to: ISODate("2023-01-24T07:00:00.000Z"),
locations: ["Chertsey", "Walton"],
types: ["DCA", "SRV"],
}]
}, {
_id: ObjectId("id2"),
from: ISODate("2023-01-23T06:30:00.000Z"),
to: ISODate("2023-01-23T18:30:00.000Z"),
matchStatus: 0,
matchId: null,
userId: ObjectId("d6f10351dd8cf3462e3867f56"),
organisationId: ObjectId("246afaf417be1cfdcf55792be"),
location: "Chertsey",
type: "DCA",
offers: [{
from: ISODate("2023-01-21T05:00:00.000Z"),
to: ISODate("2023-01-21T07:00:00.000Z"),
locations: ["Chertsey", "Walton"],
types: ["DCA", "SRV"],
}]
}]
I want the $or to match all documents that have the corresponding from/to/location/type as the current document - the idea is two shifts that could be swapped
If the offers are known (passed as an array to the function calling aggregate), I can do this with:
$or: offers.map((x) => ({
from: { $gte: new Date(x.from) },
to: { $lte: new Date(x.to) },
location: { $in: x.locations },
type: { $in: x.types },
}))
BUT I want to be able to do this in an aggregation pipeline when the offers will only be known from the current document, $offers
Is this possible? I've tried $in, $map, $lookup, $filter, $getField but can't get it right and can't get anything from Google as it thinks I want $in (which is the opposite of what I need).
I'm pretty new to MongoDB and am probably approaching this completely wrong but I'd really appreciate any help!
Edit: expected output is simply an array of matching documents, so passing document id1 to the function would return an array with id2 in, because each document is compatible with the other
///expected output, from and to are between an offer in id1's from and to, similarly types/locations are compatible
{
_id: ObjectId("id2"),
from: ISODate("2023-01-23T06:30:00.000Z"),
to: ISODate("2023-01-23T18:30:00.000Z"),
matchStatus: 0,
matchId: null,
userId: ObjectId("d6f10351dd8cf3462e3867f56"),
organisationId: ObjectId("246afaf417be1cfdcf55792be"),
location: "Chertsey",
type: "DCA",
offers: [{
from: ISODate("2023-01-21T05:00:00.000Z"),
to: ISODate("2023-01-21T07:00:00.000Z"),
locations: ["Chertsey", "Walton"],
types: ["DCA", "SRV"],
}]
You can perform self-lookup with your criteria set in the sub-pipeline.
db.collection.aggregate([
{
$match: {
organisationId: "organisationId1",
matchId: null,
matchStatus: 0
}
},
{
$unwind: "$offers"
},
{
"$lookup": {
"from": "collection",
"let": {
offersFrom: "$offers.from",
offersTo: "$offers.to",
offersLocation: "$offers.locations",
offersType: "$offers.types"
},
"pipeline": [
{
$match: {
$expr: {
$and: [
{
$gte: [
"$from",
"$$offersFrom"
]
},
{
$lte: [
"$to",
"$$offersTo"
]
},
{
"$in": [
"$location",
"$$offersLocation"
]
},
{
"$in": [
"$type",
"$$offersType"
]
},
]
}
}
}
],
"as": "selfLookup"
}
},
{
"$unwind": "$selfLookup"
},
{
"$replaceRoot": {
"newRoot": "$selfLookup"
}
}
])
Mongo Playground

Mongodb findOneAndUpdate() compare sums of date and timestamps

I'm working on a MongoDB (+mongoose) based scheduler where tasks have the following type
TaskSchema {
running: Boolean
executedAt: Date,
next: Number (millisecond)
}
I wish to fetch Tasks which should be executed meaning the sum of executedAt + next < now
Since the scheduler should lock the Task, the running flag should be flipped to true in the same operation, hence I'm using findOneAndUpdate()
I'm stuck at dealing with the sum of executedAt and next. How would one compare the sum of these to new Date/Date.now() ?
When doing an aggregation one could use $dateAdd from what I understand so in a find query could be something. like the following:
Task.find({
$and: [
{ running: { $ne: null } },
{ running: false },
{ next: { $ne: null } },
{ executedAt: { $ne: null } },
{
$expr: {
$lt: [
{
$dateAdd: {
startDate: '$executedAt',
unit: 'millisecond',
amount: '$next',
},
},
new Date().toUTCString(),
],
},
},
],
})
However this does not work.
Apparently, my initial attempt works and the above query is nearly correct. Since I didn't set the $addDate timezone explicitly I tried making the new Date() a UTC string. Considering #Wernfrieds comment this complies to my requirements:
Task.find({
$and: [
{ running: { $ne: null } },
{ running: false },
{ next: { $ne: null } },
{ executedAt: { $ne: null } },
{
$expr: {
$lt: [
{
$dateAdd: {
startDate: '$executedAt',
unit: 'millisecond',
amount: '$next',
},
},
new Date(),
],
},
},
],
})

pass mongo dollar sign aggregation var to date object in javascript

im trying to perform this aggregation in a mongo $lookup operation where i want to pass a variable down to a date object.
Wondering if this is possible
deal_cliff: new Date(new Date("$integrated").getTime() + 540 * 86400000),
as you can tell the problem is that "$integrated" is being used as a string and not an actual date since its a mongo dollar sign var.
any kind of help would be appreciated.
full code:
$lookup: {
from: 'deals',
// set variables that can be used in the pipeline below
let: {
group_id: '$parentGroupId',
internal_comm: { $toDecimal: '$internalCommission.amount' },
dealer_id: dealerId,
deal_cliff: new Date(new Date('$integrated').getTime() + 540 * 86400000), <---- Problem here
booking_provider: '$provider',
booking_booked_on: '$bookedOn'
},
pipeline: [
{
$match: {
$expr: {
$eq: ['$groupId', '$$group_id']
}
}
},
{ $unwind: '$dealers' },
...(userType === USER_TYPES.BDM && dealerId
? [
{
$match: {
$expr: {
$eq: ['$dealers.dealerId', '$$dealer_id']
}
}
}
]
: []),
{
$addFields: {
dealerComm: {
$cond: [
// If booking provider not Trivago & booking <= cliff date
{
$and: [
{ $ne: ['$$booking_provider', "Trivago"] },
{ $lte: ['$$booking_booked_on', "$$deal_cliff"] }
]
},
// do
{
$multiply: [
'$dealers.effort',
'$$internal_comm',
'$dealers.repCommission'
]
},
// else return
0
]
}
}
},
{
$group: {
_id: '$_id',
totalDealerComm: {
$sum: {
$cond: {
if: '$dealerComm',
then: { $toDecimal: '$dealerComm' },
else: 0
}
}
}
}
}
],
as: 'deal'
}

MongoDB lookup and unwind

I'm trying to perform a look up which works fine and in the correct document as 'metrics'. The lookup document has an array inside of its object called 'history'. I'm trying to unwind that history and perform a facet on it, an aggregation query that I have directly on the lookup collection that works fine.
However when using it here it's not returning anything. Am I unwinding this incorrectly? should it be $metrics.history ?
{
from: 'historicprices',
localField: 'collectibleId',
foreignField: 'collectibleId',
pipeline: [
{$set: {"target-date": "$$NOW"}},
{$unwind: {path: "$history"}},
{$facet: {
"one_day": [
{ $match: { $expr: { $lte: [{$subtract: ["$target-date", "$history.date" ]}, {$multiply: [24,60,60,1000] }] } } },
{ $group: { _id: null, "first": { $first: "$history.value" }, "last": { $last: "$history.value" }, "min-price": {"$min": "$history.value"}, "max-price": {"$max": "$history.value"} } },
{ $unset: ["_id"]}
]
"one_week": [
{ $match: { $expr: { $lte: [{$subtract: ["$target-date", "$history.date" ]}, {$multiply: [7, 24, 60, 60, 1000] }] } } },
{ $group: { _id: null, "min-price": {"$min": "$history.value"}, "first": { $first: "$history.value" }, "last": { $last: "$history.value" }, "max-price": {"$max": "$history.value"} } },
{ $unset: ["_id"]}
]
}}
],
as: 'metrics',
}
Thanks
Maybe you need to add preserveNullAndEmptyArrays: true to your unwind stage. Otherwise no data will be received if one of the docs doesn't return this field.

Aggregate and $lookup by $project field

I am trying to aggregate on my User collection and $project out the fields I need. Then I want to set let variables in $lookup to be able to use the variable to find matching documents.
I want to do it this way, because I will have many $lookups by the variables.
However I am not able to get this working correctly. What am I doing wrong with the variables?
$project: {
_id: 1,
name: 1,
goal: 1
}
},
{
$lookup: {
from: "goals",
let: { user: "$_id" },
pipeline: [
{
$match: {
"user": "$$user"
}
},
{ $project: { _id: 0, leads: 0 } },
],
as: "goal"
}
},
{
$project: {
_id: 1,
name: 1,
goal: 1
}
},
You need to use $expr to use variable name inside the $match stage.
{ $match: { $expr: { $eq: ['$user', '$$user'] }}}

Categories