I'm a hobbyist web programmer who just started learning MongoDB/Mongoose and I can't seem to figure out how to append to a string located in a deep nested array. I am trying to append a string to the end of hours: String. Below is the Schema I'm working with:
const TestSchema = new Schema({
userID: Number,
years: [
{
year: Number,
months: [{
month: Number,
days: [{
day: Number,
hours: String
}]
}]
}
]
});
Here is what I have so far. I tried to extend upon this answer here: https://stackoverflow.com/a/56589089 .But this is giving me a Cannot specify arrayFilters and a pipeline update error.
TestModel.findOneAndUpdate(
{ "years.months.days.day": 12 },
[
{
$set: {
"years.$[index1].months.$[index2].days.$[index3].hours": {
$concat: [
"$years.$[index1].months.$[index2].days.$[index3].hours",
" 44:44:44"
]
}
}
}
],
{
arrayFilters: [
{ "index1.year": 2020 },
{ "index2.month": 7 },
{ "index3.day": 12 }
]
}
).catch(error => {
console.log("error>>" + error);
});
Edit: Below is code with which I created an instance of the model
var test = new TestModel({
userID: 5,
years: [{
year: 2020,
months: [{
month: 7,
days: [{
day: 12,
hours: "4:4:4 5:5:5"
}]
}]
}]
})
test.save().then(function(){
console .log("testSaved>>" + !test.isNew);
});
Here is a screenshot of the data in the db:
Any help would be greatly appreciated.
Update is not supporting both operation together "arrayFilters" and "aggregation pipeline", you need to use only single operation from both,
So here you need to use only update aggregation pipeline, using nested $map,
TestModel.findOneAndUpdate({
years: {
$elemMatch: {
year: 2020,
months: {
$elemMatch: {
month: 7,
days: { $elemMatch: { day: 12 } }
}
}
}
}
},
[{
$set: {
years: {
$map: {
input: "$years",
as: "y",
in: {
$mergeObjects: [
"$$y",
{
months: {
$map: {
input: "$$y.months",
as: "m",
in: {
$mergeObjects: [
"$$m",
{
days: {
$map: {
input: "$$m.days",
as: "d",
in: {
$mergeObjects: [
"$$d",
{
hours: {
$cond: {
if: {
$and: [
{ $eq: ["$$y.year", 2020] },
{ $eq: ["$$m.month", 7] },
{ $eq: ["$$d.day", 12] }
]
},
then: { $concat: ["$$d.hours", " 44:44:44"] },
else: "$$d.hours"
}
}
}
]
}
}
}
}
]
}
}
}
}
]
}
}
}
}
}]
)
Related
How to get the latest documents from a collection using date time?
I have searched in SO for this specific problem, but couldn't find an example that is similar to my data structure. I have this kind of data structure:
[
{
stationId: 'xxxxx',
stationName: 'xxxx',
state: 'xxxx',
lat: 'xxxxx',
long: 'xx.xxxxx',
waterLevel: [
{
wlDateTime: '11/04/2022 11:30',
wlSeverity: 'Danger',
wlLevel: 7.5
},
{
wlDateTime: '11/04/2022 09:00',
wlSeverity: 'Danger',
wlLevel: 7.3
},
{
wlDateTime: '11/04/2022 03:00',
wlSeverity: 'Normal',
wlLevel: 5.2
}
],
rainfallData: [
{
rfDateTime: '11/04/2022 11:30',
rfSeverity: 'Heavy',
rfLevel: 21
},
{
rfDateTime: '11/04/2022 10:30',
rfSeverity: 'Heavy',
rfLevel: 21
},
{
rfDateTime: '11/04/2022 9:30',
rfSeverity: 'Heavy',
rfLevel: 21
}
]
}
]
The question is, how can I get documents that have wlDateTime equal today, with wlSeverity equal to Danger, but I just want the latest record from the waterLevel array. The same case with the rainfallDataarray i.e. to return with the latest reading for today.
Sample expected return will be like this:
[
{
stationId: 'xxxxx',
stationName: 'xxxx',
state: 'xxxx',
lat: 'xxxxx',
long: 'xx.xxxxx',
waterLevelData: [
{
wlDateTime: '11/04/2022 11:30', //latest data compared to the array
wlSeverity: 'Danger',
wlLevel: 7.5
}
],
rainfallData: [
{
rfDateTime: '11/04/2022 11:30', //latest data compared to the array
rfSeverity: 'Heavy',
rfLevel: 21
}
]
}
]
I've tried querying it like this:
Meteor.publish('Alerts', function(){
return AlertLatest.find({
'waterLevelData.wlSeverity':'Danger',
}, {
fields : {
'stationName' : 1,
'state' : 1,
'lat' : 1,
'long' : 1,
'waterLevelData.wlDateTime' : 1,
'waterLevelData.wlSeverity' : 1,
'waterLevelData.wlLevel' : 1,
'rainfallData.rfSeverity' : 1,
}},{sort: { 'waterLevelData.wlDateTime' : -1}});
})
but the query returned data that isn't how I wanted. Any help will be much appreciated.
UPDATE
I've tried the solution provided by #YuTing, which is using aggregate to customise the publication query. I went ahead and read a bit about Mongodb Aggregation, and found a Meteorjs community package (tunguska:reactive-aggregate) which simplifies the process.
This is the sample of a working aggregation so far:
Meteor.publish('PIBDataAlerts', function(){
const start = dayjs().startOf('day'); // set to 12:00 am today
const end = dayjs().endOf('day'); // set to 23:59 pm today
ReactiveAggregate(this, PIBLatest, [
{
$match: {
'stationStatus' : 'ON',
'waterLevelData': { //trying to get only today's docs
"$elemMatch" : {
"wlDateTime" : {
$gte: start.format() , $lt: end.format()
}
}
}
}
},
{
$set: {
waterLevelHFZ: {
$filter: {
input: "$waterLevelData",
as: "w",
cond: {
$and: [
{ $or : [
{ $eq: [ "$$w.wlSeverity", "Alert" ] },
{ $eq: [ "$$w.wlSeverity", "Warning" ] },
{ $eq: [ "$$w.wlSeverity", "Danger" ] },
]},
{ $eq: [ "$$w.wlDateTime", { $max: "$waterLevelData.wlDateTime" } ] }
],
}
}
},
rainfallDataHFZ: {
$filter: {
input: "$rainfallData",
as: "r",
cond: { $eq: [ "$$r.rfDateTime", { $max: "$rainfallData.rfDateTime" } ] }
}
}
}
},
{
$project : {
"stationId": 1,
"stationName" :1,
"state": 1,
"waterLevelHFZ": 1,
"rainfallDataHFZ": 1
}
}
]);
})
I'm struggling to get documents that only have the wlDateTime that equals today. I've tried a query in the $match but it returned empty array. If the $match is set to {}, it'll return all 1548 records even though the wlDateTime is not equals to today.
change your date string to date
filter the array to find the max one
db.collection.aggregate([
{
$match: {
$expr: {
$or: [
{
$ne: [
{
$filter: {
input: "$waterLevel",
as: "w",
cond: {
$eq: [
{
$dateTrunc: {
date: {
$dateFromString: {
dateString: "$$w.wlDateTime",
format: "%d/%m/%Y %H:%M"
}
},
unit: "day"
}
},
{
$dateTrunc: {
date: "$$NOW",
unit: "day"
}
}
]
}
}
},
[]
]
},
{
$ne: [
{
$filter: {
input: "$rainfallData",
as: "r",
cond: {
$eq: [
{
$dateTrunc: {
date: {
$dateFromString: {
dateString: "$$r.rfDateTime",
format: "%d/%m/%Y %H:%M"
}
},
unit: "day"
}
},
{
$dateTrunc: {
date: "$$NOW",
unit: "day"
}
}
]
}
}
},
[]
]
}
]
}
}
},
{
$set: {
waterLevel: {
$map: {
input: "$waterLevel",
as: "w",
in: {
$mergeObjects: [
"$$w",
{
wlDateTime: {
$dateFromString: {
dateString: "$$w.wlDateTime",
format: "%d/%m/%Y %H:%M"
}
}
}
]
}
}
},
rainfallData: {
$map: {
input: "$rainfallData",
as: "r",
in: {
$mergeObjects: [
"$$r",
{
rfDateTime: {
$dateFromString: {
dateString: "$$r.rfDateTime",
format: "%d/%m/%Y %H:%M"
}
}
}
]
}
}
}
}
},
{
$set: {
waterLevel: {
$filter: {
input: "$waterLevel",
as: "w",
cond: {
$and: [
{
$in: [
"$$w.wlSeverity",
[
"Alert",
"Warning",
"Danger"
]
]
},
{
$eq: [
"$$w.wlDateTime",
{
$max: "$waterLevel.wlDateTime"
}
]
},
{
$eq: [
{
$dateTrunc: {
date: "$$w.wlDateTime",
unit: "day"
}
},
{
$dateTrunc: {
date: "$$NOW",
unit: "day"
}
}
]
}
]
}
}
},
rainfallData: {
$filter: {
input: "$rainfallData",
as: "r",
cond: {
$and: [
{
$eq: [
"$$r.rfDateTime",
{
$max: "$rainfallData.rfDateTime"
}
]
},
{
$eq: [
{
$dateTrunc: {
date: "$$r.rfDateTime",
unit: "day"
}
},
{
$dateTrunc: {
date: "$$NOW",
unit: "day"
}
}
]
}
]
}
}
}
}
}
])
mongoplayground
I don't think you can sort by embedded document in an array field. It's not how mongodb works.
but I just want the latest
I you are only interested in the latest docs you can omit the sort and instead use a natural negative cursor:
Meteor.publish('Alerts', function(){
return AlertLatest.find({
'waterLevelData.wlSeverity':'Danger',
}, {
fields : {
'stationName' : 1,
'state' : 1,
'lat' : 1,
'long' : 1,
'waterLevelData.wlDateTime' : 1,
'waterLevelData.wlSeverity' : 1,
'waterLevelData.wlLevel' : 1,
'rainfallData.rfSeverity' : 1,
}},{ hint: { $natural: -1}});
})
It will start counting docs from the end, instead of the beginning.
https://docs.meteor.com/api/collections.html#Mongo-Collection-find
I'm trying to count all and unique events on daily based based on the following data shape:
{
username: "jack",
events: [
{
eventType: "party",
createdAt: "2022-01-23T10:26:11.214Z",
visitorInfo: {
visitorId: "87654321-0ebb-4238-8bf7-87654321"
}
},
{
eventType: "party",
createdAt: "2022-01-23T10:26:11.214Z",
visitorInfo: {
visitorId: "87654321-0ebb-4238-8bf7-87654321"
}
},
{
eventType: "party",
createdAt: "2022-01-23T10:26:11.214Z",
visitorInfo: {
visitorId: "01234567-0ebb-4238-8bf7-01234567"
}
},
{
eventType: "party",
createdAt: "2022-01-30T10:26:11.214Z",
visitorInfo: {
visitorId: "12345678-0ebb-4238-8bf7-12345678"
}
},
{
eventType: "party",
createdAt: "2022-01-30T10:16:11.214Z",
visitorInfo: {
visitorId: "12345678-0ebb-4238-8bf7-12345678"
}
}
]
}
I'm trying to count events (all and unique ones based on visitorId) on date (daily).
This is what I have so far (thanks to #R2D2's guide on the approach):
Event.aggregate([
{ $match: { username: 'jack' } },
{ $unwind: "$events" },
{
$project: {
total: {
$cond: [
{
$eq: ["$events.eventType", "party"],
},
1,
0,
],
},
unique: { // where I'm stuck. I need to count unique events based on visitorId on current date.
$cond: [
{
$eq: ["$events.eventType", "party"],
},
1,
0,
],
},
date: "$events.createdAt",
},
},
{
$group: {
_id: {
$dateToString: { format: "%Y-%m-%d", date: "$date" },
},
total: {
$sum: "$total",
},
uniqueTotal: {
$sum: "$unique",
},
},
},
{
$project: {
date: "$_id",
total: 1,
uniqueTotal: 1,
},
},
{
$group: {
_id: "0",
dateAndEventFrequency: {
$push: "$$ROOT",
},
},
},
{
$project: {
_id: 0,
dateAndEventFrequency: 1,
},
},
]);
I tried using $addToSet but it's not used with $project (it works with $group).
Any new approach is welcome based on the data shape and the desired result I'm expecting. I used $project because I was already using it.
Basically what I'm hoping to get in the end:
dateAndEventFrequency: [
{
_id: "2022-01-23",
uniqueTotal: 2,
total: 3,
date: "2022-01-23",
},
{
_id: "2022-01-30",
uniqueTotal: 1,
total: 2,
date: "2022-01-30",
},
]
Any help or guidance is appreciated. Thanks!
first group by date and visitorId together and then do another group just by date
you can test it here mongo playground
db.collection.aggregate([
{
$match: {
username: "jack"
}
},
{
"$unwind": "$events"
},
{
"$group": {
"_id": {
date: {
"$dateToString": {
format: "%Y-%m-%d",
date: "$events.createdAt"
}
},
"visitorId": "$events.visitorInfo.visitorId",
},
"count": {
"$count": {}
}
}
},
{
"$group": {
"_id": "$_id.date",
"uniqueTotal": {
"$count": {}
},
total: {
"$sum": "$count"
}
}
}
])
What I want to achieve is finding a specific document on that current month based on the provided date. The date is stored as a string, in order for me to compare the date I need to convert the date first. However I have trouble on converting the datestring in a nested array of objects.
My collections:
{
sections: [{
fields: [{
name: 'Date',
value: '2020-11-30T15:59:59.999Z' // this is string
},
{
name: 'Title',
value: 'My book'
},
{
name: 'Author',
value: 'Henry'
}
]
]
}
}
What I have tried:
1)
const existingReport = await Report.find({
$expr: {
$gte: [
{
$dateFromString: {
dateString: "$sections.field[0].value",
},
},
moment(payload.forPeriod).startOf("month").toDate(),
],
$lt: [
{
$dateFromString: {
dateString: "$sections.field[0].value",
},
},
moment(payload.forPeriod).endOf("month").toDate(),
],
},
});
const existingReport1 = await Report.aggregate([
{
$addFields: {
formattedData: {
$cond: {
if: {
$eq: ["$sections.field.value", "Date"],
},
then: {
$dateFromString: {
dateString: "$sections.field.value",
},
},
else: "$sections.field.value",
},
},
},
},
]);
You can simply do a $toDate with the help of 2 $reduce to iterate the sections and fields array.
db.collection.aggregate([
{
"$match": {
$expr: {
$eq: [
true,
{
"$reduce": {
"input": "$sections",
"initialValue": false,
"in": {
"$reduce": {
"input": "$$this.fields",
"initialValue": false,
"in": {
$or: [
"$$value",
{
$and: [
{
$gte: [
{
"$toDate": "$$this.value"
},
new Date("2020-11-01")
]
},
{
$lte: [
{
"$toDate": "$$this.value"
},
new Date("2020-11-30")
]
}
]
}
]
}
}
}
}
}
]
}
}
}
])
Here is the Mongo playground for your reference.
I have an array like this:
const data = [
{
id: '3499934913',
user_data: {
user_data_stays: [
{
value: '30.0',
date: '2021-10-01T12:55:00.000Z',
},
{
value: '30.0',
date: '2021-11-01T12:55:00.000Z',
},
{
value: '30.0',
date: '2021-13-01T12:55:00.000Z',
}
],
}
},
{
id: '43534535',
user_data: {
user_data_stays: [
{
value: '30.0',
date: '2021-11-01T12:55:00.000Z',
},
{
value: '30.0',
date: '2021-12-01T12:55:00.000Z',
},
{
value: '30.0',
date: '2021-13-01T12:55:00.000Z',
}
],
}
},
]
How can I keep the structure of the whole array but filter in user_data_stays only those objects which date is equal to 2021-10-01T12:55:00.000Z?
I tried using filter and some but I am not able to get to the nested element inside.
I think the problem is the amount of nesting, and I can't deal with that. I would appreciate your help.
You could use forEach on the outer array and run filter method on inner array with the help of de-structuring, the object reference will update the outer object in-place.
const data = [
{
id: "3499934913",
user_data: {
user_data_stays: [
{ value: "30.0", date: "2021-10-01T12:55:00.000Z" },
{ value: "30.0", date: "2021-11-01T12:55:00.000Z" },
{ value: "30.0", date: "2021-13-01T12:55:00.000Z" },
],
},
},
{
id: "43534535",
user_data: {
user_data_stays: [
{ value: "30.0", date: "2021-11-01T12:55:00.000Z" },
{ value: "30.0", date: "2021-12-01T12:55:00.000Z" },
{ value: "30.0", date: "2021-13-01T12:55:00.000Z" },
],
},
},
];
data.forEach(({ user_data }) => {
user_data.user_data_stays = user_data.user_data_stays
.filter(({ date }) => date === '2021-10-01T12:55:00.000Z');
});
console.log(data);
I have a Listing collection with the following document:
{
owner: ObjectId('12345678'),
bids: [
{
amount: 36,
date: 2021-06-23T21:00:00.000+00:00,
placedBy: ObjectId('1111111')
},
{
amount: 16,
date: 2021-06-23T21:00:00.000+00:00,
placedBy: ObjectId('22223332')
}
]
}
I need to query the bids array, but remove the field placedBy (for all elements in the array) only if a condition is met.
This is what I have so far:
async getBids(listingId: string, user: UserDocument) {
return this.listingModel.aggregate([
{ $match: { _id: Types.ObjectId(listingId) } },
{
$project: {
'bids.date': 1,
'bids.amount': 1,
'bids.placedBy': {
$cond: {
if: {
'$eq': ['$owner', Types.ObjectId(user.id)]
},
then: '$bids.placedBy',
else: '$$REMOVE'
}
}
}
}
])
}
This works fine for when the condition is false (bids.placedBy is removed), but when the condition is met I get this (bids.placedBy turns into an array of all placedBys):
bids: [
{
amount: 36,
date: 2021-06-23T21:00:00.000+00:00,
placedBy: [
ObjectId('1111111'),
ObjectId('22223332')
]
},
{
amount: 16,
date: 2021-06-23T21:00:00.000+00:00,
placedBy: [
ObjectId('1111111'),
ObjectId('22223332')
]
}
]
How can I fix this?
You can use $map to loop over the array and set the condition
db.collection.aggregate([
{
$project: {
"bids": {
"$map": {
"input": "$bids",
"in": {
amount: "$$this.amount",
date: "$$this.date",
placedBy: {
$cond: [ { "$eq": [ "$owner", 1 ] }, "$$this.placedBy", "$$REMOVE" ]
}
}
}
}
}
}
])
Working Mongo playground