Mongodb aggregation query by date difference - javascript

I am trying to make a cron job which syncs my documents. It should try to do it x amount of times but only after 2h have passed since the last try. On each document I have "lastSyncAt" field which I can use.
{
"_id" : ObjectId("628d8c4ddb65027a2cfd1019"),
"calculatedAt" : "",
"count" : 0,
"createdAt" : "2022-05-25 01:54:21",
"lastSyncAt" : "2022-05-25 03:54:21"
}
How should I approach this?
Should I get the lastSyncAt value in pipeline and calculate difference between currentDate? How do I get only the hours portion of that in my pipeline?
Should I convert lastSyncAt into unix and get currentDate in unix substract them and divide by 7200 to see if it is greater than 2?
Or should I take another approach?
I'm not even sure what approach to take. Not looking for code but an idea how to handle this.
Thx
Update:
Thanks to #derek-menénedez I managed to get it working as shown below:
[
// Stage 1
{
$addFields: {
lastSyncAt: {
$dateDiff: {
startDate: {$dateFromString: {
dateString: "$lastSyncAt",
timezone: "Europe/Zagreb"
}},
endDate: "$$NOW",
unit: "minute",
timezone: "Europe/Zagreb"
}
}
}
},
// Stage 2
{
$match: {
lastSyncAt: {
$gt: 120
}
}
}
]

You can use the aggregation framework to achieve the things that you want:
https://mongoplayground.net/p/1RzPCYbeHEP
You can try to remove the projection on the example to validate the number of hours.
$dateFromString operator helps you to create a date from a string
$dateDiff operator helps you to extract the diff of two dates

Related

There is no calibration between MongoDB UTC time and local time

Data is stored and inquired through the API on the web page, and the API is using MongoDB.
The server space has been unified to UTC time so that the same results can be achieved in different time zones.
MongoDB uses the Mongoose schema as follows:
const userSchema = new Schema({
userId : {
type : String
},
score : {
type : Number
},
createdAt : {
type : Date,
default : Date.now
}
});
Because Date.now in the schema is in the createdAt field by default, it does not pass the Date value separately when querying the create or update of mongoose.
Considering the case where offset exists based on UTC time, the time is calculated using moment.js as follows:
// -540 offset value for KST 9 hours faster than UTC
const utc= moment.utc().add(offset, 'm').format('YYYY-MM-DDTHH:mm:ssZ');
let beginDate = new Date(utc);
let endDate = null;
let year = beginDate.getFullYear();
let month = beginDate.getMonth();
let date = beginDate.getDate();
// To view the full duration of the day
beginDate = new Date(new Date(year, month, date).setHours(0, 0, 0, 0));
endDate = new Date(new Date(year, month, date).setHours(23, 59, 59, 59));
// Find document
const user = await userSchema.aggregate([
{
$unwind : 'lists'
},
{
$match : {
'lists.createdAt' : {
$gte : beginDate,
$lte : endDate
}
}
},
...
]);
For example, if you make a query in Korea, the data inquiry may differ from the time after midnight and to 9 a.m. the next day.
What is wrong with the above parallax correction logic? I don't exactly understand the current problem.
Why so difficult? Simply use
{
$match : {
'lists.createdAt' : {
$gte : moment().startOf('day').toDate(),
$ltr : moment().endOf('day').toDate()
}
}
}
moment().startOf('day').toDate() returns the begin of current day in your local time zone. I live in Switzerland, thus it returns ISODate("2023-01-17T23:00:00.000Z")
But you can specify also a time zone, e.g.
moment.tz('Asia/Seoul').startOf('day').toDate();
ISODate("2023-01-17T15:00:00.000Z")
The problem is moment.utc().add(...) really modifies the time value. But that is not what you need to do, you like to change only the way how the time is displayed in your local time zone.
For comparison and calculations, the displayed value does not matter. All methods are done in UTC time only.

How to bucket JSON data for a range?

I'm new to programming so appreciate any help and patience here. :)
We have some data that we're trying to sort by buckets (days of the month in this case) to create smaller sets of data like so:
export const Weather = [
{
city: "STL",
month: "july",
daysbucket: "1 to 10",
percentages: {
sunny: "45.5",
rainy: "20.5",
cloudy: "10.8",
partlycloudy: "23.2",
},
},
Is there a better way than using string like daysbucket: "1 to 10"?
The idea is to run some forecast simulations using a probability function that pulls the percentage of the past weather for a certain day without having to list these percentages for each day of the month. So far I planned to get the day of the month and then do some if statements to slot it into a string for 1-10, 11-20, etc. but wanted to see if there was a better way before I get too far. I have several data sets with a variety of buckets stored as strings but I also have control over the data so can change it as needed. All of the data is stored in MongoDB.
Thanks in advance!!!
To be able to make calculations and comparisons with the daysbucket better to defined it like this:
export const Weather = [
{
city: "STL",
month: "july",
daysbucket: {
from: 1,
to: 10
},
percentages: {
sunny: "45.5",
rainy: "20.5",
cloudy: "10.8",
partlycloudy: "23.2",
},
},
Having this structure you can compare it like:
if (day > daysbucket.from && day < days bucket.to) { ... }
And so on, note that the in order to compare numbers the values should be defined as numbers, not strings, or if string you need to convert them to numbers (use parseInt() or Number())
I would use an object to do that. Something like :
daysbucket: {
min:1,
max: 10
}

Elastic Search Date Math Issue, unsure of bounds

I'm trying to query an amount of records from a database but am a little bit confused about date math. Here's an example of one of my blocks:
var searchParams = {
"query": {
"bool" : {
"must" : [
{ "term": { } },
{ "term": { } },
{ "range": {
"dateUTC": {
"gt": "now-7d/d",
"lt": "now/d",
"format": "basic_date_time"
}
}
}
]
}
}
};
Now if I wanted to query all the result from only today, would I use gt: now-1d/d, lt: now/d. Or would I use gt: now-2d/d, lt: now/d, like the edges around one day? The program I'm trying to make is meant to query all the results from today, yesterday, etc. from the second that the day starts to the Midnight of that day, would I have to switch my date math for that?
Thanks
I think this post answers it pretty clearly: https://discuss.elastic.co/t/several-date-math-questions/27453/3
To answer your questions though:
"to query all the result from only today": gt: now/d, lt:now
from today midnight to 2 days ago midnight: gt:now - 2d/d, lt:now/d
The /d here takes the day calculated in the formula, and adjust time to midnight, so if now is the 18th of July 00:31:43, now/d yields the 18th of July 00:00:00

Searching Dates and Ignoring time and date in mongoDB node.js

I am using node.js and monodb database where I want to query records based on date and I want to ignore time and date only month and year will be matched here is my code:-
collection.find({ date: { "$gte": sDate, "$lt": eDate) } }).count(function (e, c) {
});
this is working but matching date and time as well how I can match only month and year? Please help me to solve this problem.
Edit:- some data from collection:-
{
"_id" : ObjectId("5535e76f82a964d011e34fcf"),
"VisitorId" : "5535e72a82a964d011e34fcb",
"date" : ISODate("2015-01-21T06:00:15.761Z"),
}
{
"_id" : ObjectId("5535e75f82a964d011e34fcf"),
"VisitorId" : "5535e72a82a964d011e34fcb",
"date" : ISODate("2015-04-21T06:00:15.761Z"),
}
I will pass two params i.e {month:"1",year:"2015"};
and in output first docs should be display.
Thanks
You could use the $where operator in your query:
collection.find({
"$where": function() {
return this.date.getMonth() == 0 && this.date.getFullYear() == 2015
}
}).count(function (err, data) {
// Handle err
});
However, query performance is rather compromised because using $where alone requires a table scan, it takes a global lock. You should use $where only when you can't express your query using another operator. If you must use $where , try to include at least one other standard query operator to filter the result set.
Other options are to modify your schema and store the month in its own property (if it's a common field in your queries). You are guaranteed better query performance since the field can be indexed.
The other option will be when query a specific month and year, create a query object that only looks for the start and the end of that specific month.
var sDate = new Date(2015, 0, 1);
var eDate = new Date(2015, 1, 1);
collection.find({ date: { "$gte": sDate, "$lt": eDate) } }).count(function (e, c) {
});

MongoDB find document by date range - JavaScript

I want to find a specific document via a date range (beginDate should be greater or equals - and closeDate should be lesser or equals the current date).
The document looks like this:
{ "beginDate" : ISODate("2014-11-03T23:00:00Z"),
"closeDate" : ISODate("2014-11-10T23:00:00Z"),
"desc" : "Test",
"status" : "ok",
"playerId" : "ZLkQzaY7DDvwL8sRj",
"_id" : "kozi9eHcLYa2LbWDG" }
My query looks like this:
var doc = TestData.findOne({
playerId: player._id,
beginDate: { $gte: new Date(new Date().toISOString()) },
closeDate: { $lte: new Date(new Date().toISOString()) }
});
Unfortunately, this does not work. How can I solve it?
Any help would be greatly appreciated.
You've got the logic backwards on the comparisons so you need to swap the $lte and $gte usage. You want to find the docs where beginDate has a value less than or equal to the current time, and a closeDate has a value greater than or equal to the current time. You can also just use new Date() to get the current time.
var doc = TestData.findOne({
playerId: player._id,
beginDate: { $lte: new Date() },
closeDate: { $gte: new Date() }
});

Categories