Mongoose sort by array of dates (today + future) - javascript

I need help with a bit more complex version of sorting than usual. So I have times: [Date] field in scheme. Let's say my DB has 2 items with these fields:
a) time: [ "2022-12-28T18:00:00.000+00:00", "2023-01-04T18:00:00.000+00:00" ].
b) time: [ "2022-12-30T18:00:00.000+00:00" ].
My query:
const today = new Date().toISOString().split("T")[0];
...
.find({ times: { $gte: today } }).sort("times")
Because today is 2022-12-29, .sort("times") or .sort({ times: 1 }) will sort events in a way, that item "a)" will be sorted first, because from "now" Dec 28 is closer than 30, so it works correctly.
What I want is to ignore dates that are 1+ days old. It should consider dates that are today and all in the future, but not yesterday or later in the past.
I thought about querying all items once per day to and removing all past dates, but I kind of want to keep them. Is there a solution with aggregation for this problem?
EDIT: added algorithm for sorting array of dates
export const sortDateByClosest = datesArray => {
if (datesArray.length === 1) return datesArray;
const today = new Date();
const yesterday = new Date(new Date().setDate(new Date().getDate() - 1));
const sorted = datesArray.sort((a, b) => {
const distanceA = Math.abs(today - a);
const distanceB = Math.abs(today - b);
return distanceA - distanceB;
});
// remove times that were before yesterday
return sorted.filter(d => d - yesterday > 0);
};

Aggregation be like:
{ $unwind: "$times" },
{
$match: {
times: { "$gte": today }
},
},
{
$sort: {
times: 1,
},
},
{
$group: {
_id: "$_id",
times: { $push: "$times" },
otherStuff: { $first: "$otherStuff" },
...
}
},
But it doesn't show very good results in terms of scalability

I think it works You want to ignore 1+ days older dates then you need to use aggregation like this:
db.collectionName.aggregate([ { $unwind: '$times' },
{
"$match": {
'times': {
'$gte': today,
}
}
}, {
$sort : {
times: 1
}
}
])

Related

Javascript: Get object with date range in array which matches to the current time

Please correct me if this question has already been asked on Stack, but I haven't found any answer which would solve my problem. I want to display only the object of my array, where the date range, consisting of timeStart and timeEnd, matches to the current time. Let's say the current time would be 2020-10-22T16:35:45+01:00, the function should display Steve (see the example given below) it's appointment. How can I search for the object where the current time is in between the given date ranges?
const currentTime = Date.now(); // 2020-10-22T16:35:45+01:00
This is the array I'm working with:
[
{
name: "Thomas",
timeStart: "2020-10-22T16:00:00+0100",
timeEnd: "2020-10-22T16:15:00+01:00"
},
{
name: "Marc",
timeStart: "2020-10-22T16:15:00+0100",
timeEnd: "2020-10-22T16:30:00+01:00"
},
{
name: "Steve",
timeStart: "2020-10-22T16:30:00+0100",
timeEnd: "2020-10-22T16:45:00+01:00"
}
]
Compare the start, end and current dates inside a find loop.
If the comparison returns true then the current object will be returned, otherwise undefined.
const now = new Date("2020-10-22T16:17:00+01:00"); // 16:17. Should be Marc.
const dates = [
{
name: "Thomas",
timeStart: "2020-10-22T16:00:00+0100",
timeEnd: "2020-10-22T16:15:00+01:00"
},
{
name: "Marc",
timeStart: "2020-10-22T16:15:00+0100",
timeEnd: "2020-10-22T16:30:00+01:00"
},
{
name: "Steve",
timeStart: "2020-10-22T16:30:00+0100",
timeEnd: "2020-10-22T16:45:00+01:00"
}
];
const result = dates.find(({ timeStart, timeEnd }) => {
const start = new Date(timeStart);
const end = new Date(timeEnd);
return start <= now && end > now;
});
console.log(result);
Does this work for you? It searches every date and makes sure that the current date is between the start and end.
const currentTime = Date.now(); // 2020-10-22T16:35:45+01:00
const dates = [{
name: "Thomas",
timeStart: "2020-10-22T16:00:00+0100",
timeEnd: "2020-10-22T16:15:00+01:00"
},
{
name: "Marc",
timeStart: "2020-10-22T16:15:00+0100",
timeEnd: "2020-10-22T16:30:00+01:00"
},
{
name: "Steve",
timeStart: "2020-10-22T16:30:00+0100",
timeEnd: "2020-10-22T16:45:00+01:00"
}
];
// If you want only one result, use find instead of filter
const validDates = dates.filter((obj) => {
// Converts strings to Dates
let startDate = new Date(obj.timeStart);
let endDate = new Date(obj.timeEnd);
// Makes sure the current time is after the start date and before the end date
return currentTime >= startDate && currentTime <= endDate;
});
console.log(validDates);
// To get name, use validDates[0].name (for filter, see above) or validDates.name (for find)
It will be something like this:
const currentTime = Date.now();
const resultArray=yourArray.filter(item=>{
// you have to format dates to be able to compare them
return item.timeStart < currentTime && item.timeEnd > currentTime
});
Try something like this
yourArray.filter(item => currentTime >= item.timeStart &&
currenttime <= timeEnd)

Match time only in mongodb aggregation

I am storing opening times as an array of shifts every day like below:
{
Monday: [{
startTime: {
hour: 8,
minute: 50
},
endTime: {
hour: 20,
minute: 30
}
}];
}
I am trying to retrieve documents between the start and end of a shift, using MongoDB aggregate $match operator like below:
{
$match: {
'Monday.startTime.hour': { // ex: 8
$lte: parseInt(now.format('HH'), 10), // now hours: 18
},
'Monday.startTime.minute': { // ex: 50
$lte: parseInt(now.format('mm'), 10), // now minutes: 40
},
'Monday.endTime.hour': { // ex: 20
$gte: parseInt(now.format('HH'), 10), // now hours: 18
},
'Monday.endTime.minute': { // ex: 30
$gte: parseInt(now.format('mm'), 10), // now minutes: 40
},
}
}
BUT the problem is that we have a shift example shown in the image below,
the first match condition:
'Monday.startTime.hour': {
$lte: parseInt(now.format('HH'), 10),
}
will pass as 8 is smaller than 18.
but the second match condition which matches the minutes part:
'Monday.startTime.minute': { // ex: 50
$lte: parseInt(now.format('mm'), 10), // now minutes: 40
},
will fail as 50 is greater than 40
although in real life 08:50 comes before 18:40
I was able to solve the issue by comparing the hours first by using $and and $or operators see explained code below:
{
$and: [ // grouping two conditions that now time needs to be between the start and the end of the shift.
{
$or: [ // compare the start time hour of the shift first
{
'Monday.startTime.hour': {
$lt: parseInt(now.format('HH'), 10),
},
},
{
$and: [ // if the upper condition didn't work will need to compare hours and minutes
{
'Monday.startTime.hour': {
$lte: parseInt(now.format('HH'), 10),
},
},
{
'Monday.startTime.minute': {
$lte: parseInt(now.format('mm'), 10),
},
},
],
},
],
},
{
$or: [ // compare the end time hour of the shift first
{
'Monday.endTime.hour': {
$gt: parseInt(now.format('HH'), 10),
},
},
{
$and: [ // if the upper condition didn't work will need to compare hours and minutes
{
'Monday.endTime.hour': {
$gte: parseInt(now.format('HH'), 10),
},
},
{
'Monday.endTime.minute': {
$gte: parseInt(now.format('mm'), 10),
},
},
],
},
],
},
];
}
Change Everything to Minutes. That will help you to solve this problem.
8:50 should be 530 and 20:30 should be 1230.
Current time will be 1120(18:40). So by changing to minutes you can able to solve this problem.

Mongoose query for "hot" posts

I want to show a list of posts from the database based on likes and date, think of the basic "trending" items page.
I want to use a formula like score = likes / daysSinceCreation and then get the first 10 posts based on this score.
How can I add that sort function with mongoDB/Mongoose?
Posts.find().sort(???).limit(10).then(posts => console.log(posts));
Currently I can get top posts in last week (find if creation date larger than last week and order by score), but how can I implement a more complex sorting function without getting all the items from the DB?
eg:
Today is Friday
ID CREATION_DAY LIKES
1 Monday 4 // score is 5/5 = 0
2 Tuesday 10 // score is 10/4 = 2
3 Wednesday 3 // score is 3/3 = 1
4 Thursday 20 // score is 20/2 = 10
5 Friday 5 // score is 5/1 = 5
Sorted list of IDs is: [4 (Th), 5 (Fr), 2 (Tu), 3 (We), 1(Mo)]
This will create a new document in a "trendingposts" table:
const fiveDaysAgo = new Date(Date.now() - (5 * 24 * 60 * 60 * 1000));
const oid = new ObjectId();
const now = new Date();
Posts.aggregate([
{
$match: {
createdAt: {
$gte: fiveDaysAgo
},
score: {
$gt: 0
}
}
},
{
$project: {
_id: true,
createdAt: true,
updatedAt: true,
title: true,
description: true,
score: true,
trendScore: {
$divide: [ "$score", {$subtract: [new Date(), "$createdAt"]} ]
}
}
},
{
$sort: {
trendScore: -1
}
},
{
$limit: 10
},
{
$group: {
_id: { $min: oid },
evaluatedAt: { $min: now },
posts: { $push: "$$ROOT"}
}
},
{
$out: "trendingposts"
}
])
.then(...)
A few things to note:
If using Mongo 3.4+ the $project stage can also be written as:
{
$addFields: {
trendScore: {
$divide: [ "$score", {$subtract: [new Date(), "$createdAt"]} ]
}
}
},
{ $min: now } is just a hack to grab the minimum value of now on each document, even though it's the same value for all of them.
"$$ROOT" is the entire current document. This means your end result will be a single object with the form:
{
"_id" : ObjectId("5a0a2fe912a325eb331f2759"),
"evaluatedAt" : ISODate("2017-11-13T23:51:56.051Z"),
"posts" : [/*10 `post` documents, sorted by trendScore */]
}
You can then query with:
TrendingPosts.findOne({})
.sort({_id: -1})
.then(trendingPost => console.log(trendingPost));
If your description/title are changing frequently, instead of $pushing the entire document in, you could just push the ids and use them for an $in query on your posts in order to guarantee the latest data.

Group by Date with Local Time Zone in MongoDB

I am new to mongodb. Below is my query.
Model.aggregate()
.match({ 'activationId': activationId, "t": { "$gte": new Date(fromTime), "$lt": new Date(toTime) } })
.group({ '_id': { 'date': { $dateToString: { format: "%Y-%m-%d %H", date: "$datefield" } } }, uniqueCount: { $addToSet: "$mac" } })
.project({ "date": 1, "month": 1, "hour": 1, uniqueMacCount: { $size: "$uniqueCount" } })
.exec()
.then(function (docs) {
return docs;
});
The issue is mongodb stores date in iso timezone. I need this data for displaying area chart.
I want to group by date with local time zone. is there any way to add timeoffset into date when group by?
General Problem of Dealing with "local dates"
So there is a short answer to this and a long answer as well. The basic case is that instead of using any of the "date aggregation operators" you instead rather want to and "need to" actually "do the math" on the date objects instead. The primary thing here is to adjust the values by the offset from UTC for the given local timezone and then "round" to the required interval.
The "much longer answer" and also the main problem to consider involves that dates are often subject to "Daylight Savings Time" changes in the offset from UTC at different times of the year. So this means that when converting to "local time" for such aggregation purposes, you really should consider where the boundaries for such changes exist.
There is also another consideration, being that no matter what you do to "aggregate" at a given interval, the output values "should" at least initially come out as UTC. This is good practice since display to "locale" really is a "client function", and as later described, the client interfaces will commonly have a way of displaying in the present locale which will be based on the premise that it was in fact fed data as UTC.
Determining Locale Offset and Daylight Savings
This is generally the main problem that needs to be solved. The general math for "rounding" a date to an interval is the simple part, but there is no real math you can apply to knowing when such boundaries apply, and the rules change in every locale and often every year.
So this is where a "library" comes in, and the best option here in the authors opinion for a JavaScript platform is moment-timezone, which is basically a "superset" of moment.js including all the important "timezeone" features we want to use.
Moment Timezone basically defines such a structure for each locale timezone as:
{
name : 'America/Los_Angeles', // the unique identifier
abbrs : ['PDT', 'PST'], // the abbreviations
untils : [1414918800000, 1425808800000], // the timestamps in milliseconds
offsets : [420, 480] // the offsets in minutes
}
Where of course the objects are much larger with respect to the untils and offsets properties actually recorded. But that is the data you need to access in order to see if there is actually a change in the offset for a zone given daylight savings changes.
This block of the later code listing is what we basically use to determine given a start and end value for a range, which daylight savings boundaries are crossed, if any:
const zone = moment.tz.zone(locale);
if ( zone.hasOwnProperty('untils') ) {
let between = zone.untils.filter( u =>
u >= start.valueOf() && u < end.valueOf()
);
if ( between.length > 0 )
branches = between
.map( d => moment.tz(d, locale) )
.reduce((acc,curr,i,arr) =>
acc.concat(
( i === 0 )
? [{ start, end: curr }] : [{ start: acc[i-1].end, end: curr }],
( i === arr.length-1 ) ? [{ start: curr, end }] : []
)
,[]);
}
Looking at the whole of 2017 for the Australia/Sydney locale the output of this would be:
[
{
"start": "2016-12-31T13:00:00.000Z", // Interval is +11 hours here
"end": "2017-04-01T16:00:00.000Z"
},
{
"start": "2017-04-01T16:00:00.000Z", // Changes to +10 hours here
"end": "2017-09-30T16:00:00.000Z"
},
{
"start": "2017-09-30T16:00:00.000Z", // Changes back to +11 hours here
"end": "2017-12-31T13:00:00.000Z"
}
]
Which basically reveals that between the first sequence of dates the offset would be +11 hours then changes to +10 hours between the dates in the second sequence and then switches back to +11 hours for the interval covering to the end of the year and the specified range.
This logic then needs to be translated into a structure that will be understood by MongoDB as part of an aggregation pipeline.
Applying the Math
The mathematical principle here for aggregating to any "rounded date interval" essentially relies on using the milliseconds value of the represented date which is "rounded" down to the nearest number representing the "interval" required.
You essentially do this by finding the "modulo" or "remainder" of the current value applied to the required interval. Then you "subtract" that remainder from the current value which returns a value at the nearest interval.
For example, given the current date:
var d = new Date("2017-07-14T01:28:34.931Z"); // toValue() is 1499995714931 millis
// 1000 millseconds * 60 seconds * 60 minutes = 1 hour or 3600000 millis
var v = d.valueOf() - ( d.valueOf() % ( 1000 * 60 * 60 ) );
// v equals 1499994000000 millis or as a date
new Date(1499994000000);
ISODate("2017-07-14T01:00:00Z")
// which removed the 28 minutes and change to nearest 1 hour interval
This is the general math we also need to apply in the aggregation pipeline using the $subtract and $mod operations, which are the aggregation expressions used for the same math operations shown above.
The general structure of the aggregation pipeline is then:
let pipeline = [
{ "$match": {
"createdAt": { "$gte": start.toDate(), "$lt": end.toDate() }
}},
{ "$group": {
"_id": {
"$add": [
{ "$subtract": [
{ "$subtract": [
{ "$subtract": [ "$createdAt", new Date(0) ] },
switchOffset(start,end,"$createdAt",false)
]},
{ "$mod": [
{ "$subtract": [
{ "$subtract": [ "$createdAt", new Date(0) ] },
switchOffset(start,end,"$createdAt",false)
]},
interval
]}
]},
new Date(0)
]
},
"amount": { "$sum": "$amount" }
}},
{ "$addFields": {
"_id": {
"$add": [
"$_id", switchOffset(start,end,"$_id",true)
]
}
}},
{ "$sort": { "_id": 1 } }
];
The main parts here you need to understand is the conversion from a Date object as stored in MongoDB to Numeric representing the internal timestamp value. We need the "numeric" form, and to do this is a trick of math where we subtract one BSON Date from another which yields the numeric difference between them. This is exactly what this statement does:
{ "$subtract": [ "$createdAt", new Date(0) ] }
Now we have a numeric value to deal with, we can apply the modulo and subtract that from the numeric representation of the date in order to "round" it. So the "straight" representation of this is like:
{ "$subtract": [
{ "$subtract": [ "$createdAt", new Date(0) ] },
{ "$mod": [
{ "$subtract": [ "$createdAt", new Date(0) ] },
( 1000 * 60 * 60 * 24 ) // 24 hours
]}
]}
Which mirrors the same JavaScript math approach as shown earlier but applied to the actual document values in the aggregation pipeline. You will also note the other "trick" there where we apply an $add operation with another representation of a BSON date as of epoch ( or 0 milliseconds ) where the "addition" of a BSON Date to a "numeric" value, returns a "BSON Date" representing the milliseconds it was given as input.
Of course the other consideration in the listed code it the actual "offset" from UTC which is adjusting the numeric values in order to ensure the "rounding" takes place for the present timezone. This is implemented in a function based on the earlier description of finding where the different offsets occur, and returns a format as usable in an aggregation pipeline expression by comparing the input dates and returning the correct offset.
With the full expansion of all the details, including the generation of handling those different "Daylight Savings" time offsets would then be like:
[
{
"$match": {
"createdAt": {
"$gte": "2016-12-31T13:00:00.000Z",
"$lt": "2017-12-31T13:00:00.000Z"
}
}
},
{
"$group": {
"_id": {
"$add": [
{
"$subtract": [
{
"$subtract": [
{
"$subtract": [
"$createdAt",
"1970-01-01T00:00:00.000Z"
]
},
{
"$switch": {
"branches": [
{
"case": {
"$and": [
{
"$gte": [
"$createdAt",
"2016-12-31T13:00:00.000Z"
]
},
{
"$lt": [
"$createdAt",
"2017-04-01T16:00:00.000Z"
]
}
]
},
"then": -39600000
},
{
"case": {
"$and": [
{
"$gte": [
"$createdAt",
"2017-04-01T16:00:00.000Z"
]
},
{
"$lt": [
"$createdAt",
"2017-09-30T16:00:00.000Z"
]
}
]
},
"then": -36000000
},
{
"case": {
"$and": [
{
"$gte": [
"$createdAt",
"2017-09-30T16:00:00.000Z"
]
},
{
"$lt": [
"$createdAt",
"2017-12-31T13:00:00.000Z"
]
}
]
},
"then": -39600000
}
]
}
}
]
},
{
"$mod": [
{
"$subtract": [
{
"$subtract": [
"$createdAt",
"1970-01-01T00:00:00.000Z"
]
},
{
"$switch": {
"branches": [
{
"case": {
"$and": [
{
"$gte": [
"$createdAt",
"2016-12-31T13:00:00.000Z"
]
},
{
"$lt": [
"$createdAt",
"2017-04-01T16:00:00.000Z"
]
}
]
},
"then": -39600000
},
{
"case": {
"$and": [
{
"$gte": [
"$createdAt",
"2017-04-01T16:00:00.000Z"
]
},
{
"$lt": [
"$createdAt",
"2017-09-30T16:00:00.000Z"
]
}
]
},
"then": -36000000
},
{
"case": {
"$and": [
{
"$gte": [
"$createdAt",
"2017-09-30T16:00:00.000Z"
]
},
{
"$lt": [
"$createdAt",
"2017-12-31T13:00:00.000Z"
]
}
]
},
"then": -39600000
}
]
}
}
]
},
86400000
]
}
]
},
"1970-01-01T00:00:00.000Z"
]
},
"amount": {
"$sum": "$amount"
}
}
},
{
"$addFields": {
"_id": {
"$add": [
"$_id",
{
"$switch": {
"branches": [
{
"case": {
"$and": [
{
"$gte": [
"$_id",
"2017-01-01T00:00:00.000Z"
]
},
{
"$lt": [
"$_id",
"2017-04-02T03:00:00.000Z"
]
}
]
},
"then": -39600000
},
{
"case": {
"$and": [
{
"$gte": [
"$_id",
"2017-04-02T02:00:00.000Z"
]
},
{
"$lt": [
"$_id",
"2017-10-01T02:00:00.000Z"
]
}
]
},
"then": -36000000
},
{
"case": {
"$and": [
{
"$gte": [
"$_id",
"2017-10-01T03:00:00.000Z"
]
},
{
"$lt": [
"$_id",
"2018-01-01T00:00:00.000Z"
]
}
]
},
"then": -39600000
}
]
}
}
]
}
}
},
{
"$sort": {
"_id": 1
}
}
]
That expansion is using the $switch statement in order to apply the date ranges as conditions to when to return the given offset values. This is the most convenient form since the "branches" argument does correspond directly to an "array", which is the most convenient output of the "ranges" determined by examination of the untils representing the offset "cut-points" for the given timezone on the supplied date range of the query.
It is possible to apply the same logic in earlier versions of MongoDB using a "nested" implementation of $cond instead, but it is a little messier to implement, so we are just using the most convenient method in implementation here.
Once all of those conditions are applied, the dates "aggregated" are actually those representing the "local" time as defined by the supplied locale. This actually brings us to what the final aggregation stage is, and the reason why it is there as well as the later handling as demonstrated in the listing.
End Results
I did mention earlier that the general recommendation is that the "output" should still return the date values in UTC format of at least some description, and therefore that is exactly what the pipeline here is doing by first converting "from" UTC to local by applying the offset when "rounding", but then the final numbers "after the grouping" are re-adjusted back by the same offset that applies to the "rounded" date values.
The listing here gives "three" different output possibilities here as:
// ISO Format string from JSON stringify default
[
{
"_id": "2016-12-31T13:00:00.000Z",
"amount": 2
},
{
"_id": "2017-01-01T13:00:00.000Z",
"amount": 1
},
{
"_id": "2017-01-02T13:00:00.000Z",
"amount": 2
}
]
// Timestamp value - milliseconds from epoch UTC - least space!
[
{
"_id": 1483189200000,
"amount": 2
},
{
"_id": 1483275600000,
"amount": 1
},
{
"_id": 1483362000000,
"amount": 2
}
]
// Force locale format to string via moment .format()
[
{
"_id": "2017-01-01T00:00:00+11:00",
"amount": 2
},
{
"_id": "2017-01-02T00:00:00+11:00",
"amount": 1
},
{
"_id": "2017-01-03T00:00:00+11:00",
"amount": 2
}
]
The one thing of note here is that for a "client" such as Angular, every single one of those formats would be accepted by it's own DatePipe which can actually do the "locale format" for you. But it depends on where the data is supplied to. "Good" libraries will be aware of using a UTC date in the present locale. Where that is not the case, then you might need to "stringify" yourself.
But it is a simple thing, and you get the most support for this by using a library which essentially bases it's manipulation of output from a "given UTC value".
The main thing here is to "understand what you are doing" when you ask such a thing as aggregating to a local time zone. Such a process should consider:
The data can be and often is viewed from the perspective of people within different timezones.
The data is generally provided by people in different timezones. Combined with point 1, this is why we store in UTC.
Timezones are often subject to a changing "offset" from "Daylight Savings Time" in many of the world timezones, and you should account for that when analyzing and processing the data.
Regardless of aggregation intervals, output "should" in fact remain in UTC, albeit adjusted to aggregate on interval according to the locale provided. This leaves presentation to be delegated to a "client" function, just as it should.
As long as you keep those things in mind and apply just like the listing here demonstrates, then you are doing all the right things for dealing with aggregation of dates and even general storage with respect to a given locale.
So you "should" be doing this, and what you "should not" be doing is giving up and simply storing the "locale date" as a string. As described, that would be a very incorrect approach and causes nothing but further problems for your application.
NOTE: The one topic I do not touch on here at all is aggregating to a "month" ( or indeed "year" ) interval. "Months" are the mathematical anomaly in the whole process since the number of days always varies and thus requires a whole other set of logic in order to apply. Describing that alone is at least as long as this post, and therefore would be another subject. For general minutes, hours, and days which is the common case, the math here is "good enough" for those cases.
Full Listing
This serves as a "demonstration" to tinker with. It employs the required function to extract the offset dates and values to be included and runs an aggregation pipeline over the supplied data.
You can change anything in here, but will probably start with the locale and interval parameters, and then maybe add different data and different start and end dates for the query. But the rest of the code need not be changed to simply make changes to any of those values, and can therefore demonstrate using different intervals ( such as 1 hour as asked in the question ) and different locales.
For instance, once supplying valid data which would actually require aggregation at a "1 hour interval" then the line in the listing would be changed as:
const interval = moment.duration(1,'hour').asMilliseconds();
In order to define a milliseconds value for the aggregation interval as required by the aggregation operations being performed on the dates.
const moment = require('moment-timezone'),
mongoose = require('mongoose'),
Schema = mongoose.Schema;
mongoose.Promise = global.Promise;
mongoose.set('debug',true);
const uri = 'mongodb://localhost/test',
options = { useMongoClient: true };
const locale = 'Australia/Sydney';
const interval = moment.duration(1,'day').asMilliseconds();
const reportSchema = new Schema({
createdAt: Date,
amount: Number
});
const Report = mongoose.model('Report', reportSchema);
function log(data) {
console.log(JSON.stringify(data,undefined,2))
}
function switchOffset(start,end,field,reverseOffset) {
let branches = [{ start, end }]
const zone = moment.tz.zone(locale);
if ( zone.hasOwnProperty('untils') ) {
let between = zone.untils.filter( u =>
u >= start.valueOf() && u < end.valueOf()
);
if ( between.length > 0 )
branches = between
.map( d => moment.tz(d, locale) )
.reduce((acc,curr,i,arr) =>
acc.concat(
( i === 0 )
? [{ start, end: curr }] : [{ start: acc[i-1].end, end: curr }],
( i === arr.length-1 ) ? [{ start: curr, end }] : []
)
,[]);
}
log(branches);
branches = branches.map( d => ({
case: {
$and: [
{ $gte: [
field,
new Date(
d.start.valueOf()
+ ((reverseOffset)
? moment.duration(d.start.utcOffset(),'minutes').asMilliseconds()
: 0)
)
]},
{ $lt: [
field,
new Date(
d.end.valueOf()
+ ((reverseOffset)
? moment.duration(d.start.utcOffset(),'minutes').asMilliseconds()
: 0)
)
]}
]
},
then: -1 * moment.duration(d.start.utcOffset(),'minutes').asMilliseconds()
}));
return ({ $switch: { branches } });
}
(async function() {
try {
const conn = await mongoose.connect(uri,options);
// Data cleanup
await Promise.all(
Object.keys(conn.models).map( m => conn.models[m].remove({}))
);
let inserted = await Report.insertMany([
{ createdAt: moment.tz("2017-01-01",locale), amount: 1 },
{ createdAt: moment.tz("2017-01-01",locale), amount: 1 },
{ createdAt: moment.tz("2017-01-02",locale), amount: 1 },
{ createdAt: moment.tz("2017-01-03",locale), amount: 1 },
{ createdAt: moment.tz("2017-01-03",locale), amount: 1 },
]);
log(inserted);
const start = moment.tz("2017-01-01", locale)
end = moment.tz("2018-01-01", locale)
let pipeline = [
{ "$match": {
"createdAt": { "$gte": start.toDate(), "$lt": end.toDate() }
}},
{ "$group": {
"_id": {
"$add": [
{ "$subtract": [
{ "$subtract": [
{ "$subtract": [ "$createdAt", new Date(0) ] },
switchOffset(start,end,"$createdAt",false)
]},
{ "$mod": [
{ "$subtract": [
{ "$subtract": [ "$createdAt", new Date(0) ] },
switchOffset(start,end,"$createdAt",false)
]},
interval
]}
]},
new Date(0)
]
},
"amount": { "$sum": "$amount" }
}},
{ "$addFields": {
"_id": {
"$add": [
"$_id", switchOffset(start,end,"$_id",true)
]
}
}},
{ "$sort": { "_id": 1 } }
];
log(pipeline);
let results = await Report.aggregate(pipeline);
// log raw Date objects, will stringify as UTC in JSON
log(results);
// I like to output timestamp values and let the client format
results = results.map( d =>
Object.assign(d, { _id: d._id.valueOf() })
);
log(results);
// Or use moment to format the output for locale as a string
results = results.map( d =>
Object.assign(d, { _id: moment.tz(d._id, locale).format() } )
);
log(results);
} catch(e) {
console.error(e);
} finally {
mongoose.disconnect();
}
})()
November 2017 saw the release of MongoDB v3.6, which included timezone-aware date aggregation operators. I would encourage anyone reading this to put them to use rather than rely on client-side date manipulation, as demonstrated in Neil's answer, particularly because it is way easier to read and understand.
Depending on the requirements, different operators might come in handy, but I've found $dateToParts to be the most universal/generic. Here's a basic demonstration using OP's example:
project({
dateParts: {
// This will split the date stored in `dateField` into parts
$dateToParts: {
date: "$dateField",
// This can be an Olson timezone, such as Europe/London, or
// a fixed offset, such as +0530 for India.
timezone: "+05:30"
}
}
})
.group({
_id: {
// Here we group by hour! Using these date parts grouping
// by hour/day/month/etc. is trivial - start with the year
// and add every unit greater than or equal to the target
// unit.
year: "$dateParts.year",
month: "$dateParts.month",
day: "$dateParts.day",
hour: "$dateParts.hour"
},
uniqueCount: {
$addToSet: "$mac"
}
})
.project({
_id: 0,
year: "$_id.year",
month: "$_id.month",
day: "$_id.day",
hour: "$_id.hour",
uniqueMacCount: { $size: "$uniqueCount" }
});
Alternatively, one might wish to assemble the date parts back to a date object. This is also very simple with the inverse $dateFromParts operator:
project({
_id: 0,
date: {
$dateFromParts: {
year: "$_id.year",
month: "$_id.month",
day: "$_id.day",
hour: "$_id.hour",
timezone: "+05:30"
}
},
uniqueMacCount: { $size: "$uniqueCount" }
})
The great thing here is that all the underlying dates remain in UTC and any returned dates are also in UTC.
Unfortunately, it seems that grouping by more unusual arbitrary ranges, such as half-day, might be harder. I haven't given it much thought however.
Maybe this will help someone coming to this question.
There is property "timezone" in $dateToString object.
For example:
$dateToString: { format: "%Y-%m-%d %H", date: "$datefield", timezone: "Europe/London" }

Get all objects for a week number

I need to get a list of week ranges for all records in my MongoDB. When I click on a week range, it will display only the records for that week range. Clicking on the week range sends the ID of the week (lets say 42, ie the 42nd week out of year 2015), it should get those results.
Question: How can I query for a set of records given a week number and year? This should work, right?
SCHEMA:
var orderSchema = mongoose.Schema({
date: Date, //ISO date
request: {
headers : {
...
First: Get all week IDs for all Objects:
var query = Order.aggregate(
[
{
$project:
{
week:
{
$week: '$date'
}
}
},
{
$group:
{
_id: null,
distinctDate:
{
$addToSet:
{
week: '$week'
}
}
}
}
]
);
Result:
distinctDate: Array[35]
0: Object
week: 40
1: Object
week: 37
...
Convert to week ranges using MomentJS and display:
data.forEach(function(v, k) {
$scope.weekRanges.push(getWeekRange(v.week));
});
function getWeekRange(weekNum) {
var monday = moment().day("Monday").isoWeek(weekNum).format('MM-DD-YYYY');
var sunday = moment().day("Sunday").isoWeek(weekNum).format('MM-DD-YYYY');
...
Output:
Week
10-12-2015 to 10-18-2015 //week ID 42
10-05-2015 to 10-11-2015 //week ID 41
09-28-2015 to 10-04-2015 ...
...
Second: Click on week range and get Objects Per Week ID:
var year = 2015;
var weekID = weekParamID; //42
if (!Order) {
Order = mongoose.model('Order', orderSchema());
}
var query = Order.aggregate(
{
$project:
{
cust_ID : '$request.headers.custID',
cost : '$response.body.pricing.cost',
year :
{
$year: '$date'
},
month :
{
$month: '$date'
},
week:
{
$week: '$date'
},
day:
{
$dayOfMonth: '$date'
}
}
},
{
$match:
{
year : year, //2015
week : weekID //42
}
}
);
And if I click on Week Range 10-12-2015 to 10-18-2015 (week ID 42), I get results with dates outside of the range (10-19-2015):
10-19-2015 Order info
10-18-2015 Order info
10-19-2015 Order info
Using MongoDB command line:
db.mycollection.aggregate({ $project: { week: { $week: '$date' }, day: { $dayOfMonth: '$date' } } }, { $match: { week: 42 } }
Results:
{ "_id" : "1bd482f6759b", "week" : 42, "day" : 19 } //shouldn't exceed week range
{ "_id" : "b3d38759", "week" : 42, "day" : 19 }
EDIT: Update
So there is a discrepancy with MongoDB ISO weeks (starts on Sunday) and Moment JS ISO (starts on Monday).
This SO post suggests subtracting the dates from the query so the Mongo date starts on Monday:
{
$project:
{
week: { $week: [ "$datetime" ] },
dayOfWeek:{$dayOfWeek:["$datetime"]}}
},
{
$project:
{
week:{$cond:[{$eq:["$dayOfWeek",1]},{$subtract:["$week",1]},'$week']}
}
}
I implemented this with my query, but now it's not returning two fields that I need:
cust_ID : '$request.headers.custID',
cost : '$response.body.pricing.cost'
Query:
db.mycollection.aggregate(
{
$project:
{
cust_ID : '$request.headers.custID',
cost : '$response.body.pricing.cost',
week:
{
$week: ['$date']
},
dayOfWeek:
{
$dayOfWeek: ['$date']
}
}
},
{
$project:
{
week: {
$cond: [
{
$eq: [
"$dayOfWeek", 1
]
},
{
$subtract: [
"$week", 1
]
}, '$week'
]
}
}
},
{
$match:
{
week : 42
}
}
);
Results:
{ "_id" : "387e2", "week" : 42 }
{ "_id" : "ef269f6341", "week" : 42 }
{ "_id" : "17482f6759b", "week" : 42 }
{ "_id" : "7123d38759", "week" : 42 }
{ "_id" : "ff89b1fb", "week" : 42 }
It's not returning the fieldsets I specified in $project
The MongoDB $week operator considers weeks to begin on Sunday, see docs:
Weeks begin on Sundays, and week 1 begins with the first Sunday of the
year... This behavior is the same as the “%U” operator to the strftime
standard library function.
Moment.JS's isoWeekday() uses the ISO week which considers weeks to begin on Monday. It also differs in that it considers the week 1 to be the first week with a Thursday in it.
This discrepancy could explain the behaviour you are seeing.
E.g. if I save this doc in MongoDB, which is a Monday:
db.test.save({ "date" : new ISODate("2015-10-19T10:10:10Z") })
then run your aggregation query above, I get week 42.
But then if I run the following:
console.log(moment().day("Monday").isoWeek(42))
I get the below date which is not the one I originally saved in MongoDB, even though it is Monday of the week MongoDB reported.
Mon Oct 12 2015
How to fix it I guess depends on which definition of week you need.
If you are happy with the MongoDB $week definition, it's probably easy to find/write an alternative implementation to convert the week number to the corresponding date. Here is one library that adds strftime support to Moment.js:
https://github.com/benjaminoakes/moment-strftime
If you want to use the ISO format, it's more complicated. As per your edit above you'll need to account for the week start difference. But you'll also need to account for the week number at start of year difference. This difference means that the strftime week number can have a week 0 while ISO always starts on week 1. For 2015 it looks like you need to add 1 week on to the strftime week to get the ISO week, as well as accounting for the week start day, but that won't be reliable in general.
Starting from MongoDB version 3.4 you can use the $isoWeek aggregation operator.
Returns the week number in ISO 8601 format, ranging from 1 to 53. Week numbers start at 1 with the week (Monday through Sunday) that contains the year’s first Thursday.
You can find more infos on this in the MongoDB docs.

Categories