I'm trying to work out how I can paginate my data efficiently. If I have a url such as: example.com/items?page=5, how can I retrieve the correct records every single time (so that the same record doesn't show up on another page)? I thought to I'd have to first sort the DB records first, then use mongoose to do a range query such as the one below.
How does this fare when the amount of records scales rapidly? I'm worried that the sorting process will take far too long and bottleneck the process. Any advice?
Submission.find({
/* First Case: Hour */
created: { $lt: new Date(), $gt: new Date(year+','+month+','+day+','+hour+','+min+','+sec) } // Get results from start of current hour to current time.
/* Second Case: Day */
created: { $lt: new Date(), $gt: new Date(year+','+month+','+day) } // Get results from start of current day to current time.
/* Third Case: Month */
created: { $lt: new Date(), $gt: new Date(year+','+month) } // Get results from start of current month to current time.
/* Fourth Case: Year */
created: { $lt: new Date(), $gt: new Date(year) } // Get results from start of current year to current time.
})
You can work on following query:
var query = Model.find(conditions).sort(sort).skip(skip).limit(limit);
where
condition will be say { age: { $gt: 20 } }
sort will be say { name: 1}
skip will be say 20
limit will be say 10
then execute the following query to get the records
return query
.exec()
.then(function (cursor) { ...... });
Related
Data is stored and inquired through the API on the web page, and the API is using MongoDB.
The server space has been unified to UTC time so that the same results can be achieved in different time zones.
MongoDB uses the Mongoose schema as follows:
const userSchema = new Schema({
userId : {
type : String
},
score : {
type : Number
},
createdAt : {
type : Date,
default : Date.now
}
});
Because Date.now in the schema is in the createdAt field by default, it does not pass the Date value separately when querying the create or update of mongoose.
Considering the case where offset exists based on UTC time, the time is calculated using moment.js as follows:
// -540 offset value for KST 9 hours faster than UTC
const utc= moment.utc().add(offset, 'm').format('YYYY-MM-DDTHH:mm:ssZ');
let beginDate = new Date(utc);
let endDate = null;
let year = beginDate.getFullYear();
let month = beginDate.getMonth();
let date = beginDate.getDate();
// To view the full duration of the day
beginDate = new Date(new Date(year, month, date).setHours(0, 0, 0, 0));
endDate = new Date(new Date(year, month, date).setHours(23, 59, 59, 59));
// Find document
const user = await userSchema.aggregate([
{
$unwind : 'lists'
},
{
$match : {
'lists.createdAt' : {
$gte : beginDate,
$lte : endDate
}
}
},
...
]);
For example, if you make a query in Korea, the data inquiry may differ from the time after midnight and to 9 a.m. the next day.
What is wrong with the above parallax correction logic? I don't exactly understand the current problem.
Why so difficult? Simply use
{
$match : {
'lists.createdAt' : {
$gte : moment().startOf('day').toDate(),
$ltr : moment().endOf('day').toDate()
}
}
}
moment().startOf('day').toDate() returns the begin of current day in your local time zone. I live in Switzerland, thus it returns ISODate("2023-01-17T23:00:00.000Z")
But you can specify also a time zone, e.g.
moment.tz('Asia/Seoul').startOf('day').toDate();
ISODate("2023-01-17T15:00:00.000Z")
The problem is moment.utc().add(...) really modifies the time value. But that is not what you need to do, you like to change only the way how the time is displayed in your local time zone.
For comparison and calculations, the displayed value does not matter. All methods are done in UTC time only.
Currently I have a timestamp field with value format like 1479664146607.
What I wanted to do is to get all data with timestamp that has a year of let's say 2017.
My current code is non-performant. It gets all the data, and then uses a filter method.
Let's say I got 2000+ records.
const records = []; // all records
const data = records.filter(r => new Date(r).getYear == '2017');
While this code works, it kills the server.
My database is nedb and using feathersjs, I can actually get equality items by
app.service('messages').find({
query: {
timestamp: '2017'
}
});
This code will not work because it will search for the exact year. I am looking for a way to convert the timestamp field to a year before searching it in the database.
Okay, so what I did is to use the $gt and $lt operators.
Let's say we want to get all data in year 2018.
Using momentjs, I did something like this:
const year = '2018';
// Get previous year based on given year
const previousYear = moment(year, 'YYYY').subtract(1, 'year').format('YYYY');
// Get next year based on given year
const nextYear = moment(year, 'YYYY').add(1, 'year').format('YYYY');
// get full ending date of previous year
const endOfPreviousYear = moment(previousYear, 'YYYY').endOf('year').format('x');
// get full starting date of next year
const startOfNextYear = moment(nextYear, 'YYYY').startOf('year').format('x');
// get data where year is greater than `endOfPreviousYear` and less than `startOfNextYear`
const yearQuery = {
$gt: +endOfPreviousYear,
$lt: +startOfNextYear
}
app.service('messages').find({
query: {
timestamp: yearQuery
}
});
I am using node.js and monodb database where I want to query records based on date and I want to ignore time and date only month and year will be matched here is my code:-
collection.find({ date: { "$gte": sDate, "$lt": eDate) } }).count(function (e, c) {
});
this is working but matching date and time as well how I can match only month and year? Please help me to solve this problem.
Edit:- some data from collection:-
{
"_id" : ObjectId("5535e76f82a964d011e34fcf"),
"VisitorId" : "5535e72a82a964d011e34fcb",
"date" : ISODate("2015-01-21T06:00:15.761Z"),
}
{
"_id" : ObjectId("5535e75f82a964d011e34fcf"),
"VisitorId" : "5535e72a82a964d011e34fcb",
"date" : ISODate("2015-04-21T06:00:15.761Z"),
}
I will pass two params i.e {month:"1",year:"2015"};
and in output first docs should be display.
Thanks
You could use the $where operator in your query:
collection.find({
"$where": function() {
return this.date.getMonth() == 0 && this.date.getFullYear() == 2015
}
}).count(function (err, data) {
// Handle err
});
However, query performance is rather compromised because using $where alone requires a table scan, it takes a global lock. You should use $where only when you can't express your query using another operator. If you must use $where , try to include at least one other standard query operator to filter the result set.
Other options are to modify your schema and store the month in its own property (if it's a common field in your queries). You are guaranteed better query performance since the field can be indexed.
The other option will be when query a specific month and year, create a query object that only looks for the start and the end of that specific month.
var sDate = new Date(2015, 0, 1);
var eDate = new Date(2015, 1, 1);
collection.find({ date: { "$gte": sDate, "$lt": eDate) } }).count(function (e, c) {
});
I've got a little problem, which I'm unable to find a solution to. Let's say I want to query a collection for a field value within a range. However, the field value should have a computation done on it before being search for in a range.
For e.g.
itemCollection.find({interval: {
$gte: ISODate("2010-04-29T00:00:00.000Z"),
$lt: ISODate("2010-05-01T00:00:00.000Z")
}}).find(function(err, items){
if (!err){
res.send(items);
} else {
console.log(err);
}
});
Here, interval value is 2. I need to get the current date, add two days to it and get the date it will be in two days (interval value) and then check if the resulting date is in the range.
Is there any way to do this?
Yes, that is quite possible and easy to do (if I didn't fail to understand your question).
var startDate = new Date();
var endDate = new Date(startDate);
endDate.setDate(endDate.getDate()+2);
itemCollection.find({interval: {
$gte: startDate,
$lt: endDate
}}).find(function(err, items){
if (!err){
res.send(items);
} else {
console.log(err);
}
});
I want to find a specific document via a date range (beginDate should be greater or equals - and closeDate should be lesser or equals the current date).
The document looks like this:
{ "beginDate" : ISODate("2014-11-03T23:00:00Z"),
"closeDate" : ISODate("2014-11-10T23:00:00Z"),
"desc" : "Test",
"status" : "ok",
"playerId" : "ZLkQzaY7DDvwL8sRj",
"_id" : "kozi9eHcLYa2LbWDG" }
My query looks like this:
var doc = TestData.findOne({
playerId: player._id,
beginDate: { $gte: new Date(new Date().toISOString()) },
closeDate: { $lte: new Date(new Date().toISOString()) }
});
Unfortunately, this does not work. How can I solve it?
Any help would be greatly appreciated.
You've got the logic backwards on the comparisons so you need to swap the $lte and $gte usage. You want to find the docs where beginDate has a value less than or equal to the current time, and a closeDate has a value greater than or equal to the current time. You can also just use new Date() to get the current time.
var doc = TestData.findOne({
playerId: player._id,
beginDate: { $lte: new Date() },
closeDate: { $gte: new Date() }
});