How to query a timestamp field by year in Feathersjs? - javascript

Currently I have a timestamp field with value format like 1479664146607.
What I wanted to do is to get all data with timestamp that has a year of let's say 2017.
My current code is non-performant. It gets all the data, and then uses a filter method.
Let's say I got 2000+ records.
const records = []; // all records
const data = records.filter(r => new Date(r).getYear == '2017');
While this code works, it kills the server.
My database is nedb and using feathersjs, I can actually get equality items by
app.service('messages').find({
query: {
timestamp: '2017'
}
});
This code will not work because it will search for the exact year. I am looking for a way to convert the timestamp field to a year before searching it in the database.

Okay, so what I did is to use the $gt and $lt operators.
Let's say we want to get all data in year 2018.
Using momentjs, I did something like this:
const year = '2018';
// Get previous year based on given year
const previousYear = moment(year, 'YYYY').subtract(1, 'year').format('YYYY');
// Get next year based on given year
const nextYear = moment(year, 'YYYY').add(1, 'year').format('YYYY');
// get full ending date of previous year
const endOfPreviousYear = moment(previousYear, 'YYYY').endOf('year').format('x');
// get full starting date of next year
const startOfNextYear = moment(nextYear, 'YYYY').startOf('year').format('x');
// get data where year is greater than `endOfPreviousYear` and less than `startOfNextYear`
const yearQuery = {
$gt: +endOfPreviousYear,
$lt: +startOfNextYear
}
app.service('messages').find({
query: {
timestamp: yearQuery
}
});

Related

Issue matching string date to compare with with dates in database?

I am trying to find all users that have a matching birthday stored in the format "1975-01-12T00:00:00.000+00:00"
When I send a date of type String to my route and convert it to a date with new Date(req.body.DOB)
I am getting a date with an additional 4 hours: "1975-01-12T04:00:00.000Z" and a Z at the end.
I am new with dates so not too sure what is going on. How can I convert the string date into a date in the same way it is saved in my database, without the additional 4 hours and Z?
router.post("/api/date", async (req, res) => {
try {
const birthdate = new Date(req.body.DOB) // String as "01-12-1975"
const users = await User.find({
DOB: birthdate,
})
res.status(200).send(users)
} catch (err) {
res.status(400).send()
}
})
Databases typically use iso_8601 date format, which includes timezone. Javascript dates use the system timezone upon instantiation, if you use a format which omits the timezone (like the one you have). This is why there is an offset of 4 (offset from UTC timezone), presumably your server is in Russia or something.
You should convert the string to ISO format before converting to a date object. For example, if you store all birthdays as UTC time (at midnight), then you can just convert it like so:
const dateStr = "01-12-1975"
const [date, month, year] = dateStr.split('-')
const isoStr = `${year}-${month}-${date}T00:00:00.000Z`
const newDate = new Date(isoStr) // 1975-12-01T00:00:00.000Z

Subtract one day from date and find values -Js

i working on small billing application using electronjs and pouchdb
i want to find the opening balance for every day based on date selection on input
if there is no data in particular date i want to find the data by the previous dates
i tried in recursive way but it did not work
// getting date value on change
var TodayDate = document.getElementById('daybook-firstdate').value;
// onchange function
function getday(TodayDate){
daybook.find({
selector: {
Date:TodayDate ,
}
}).then(function(result) {
var opening_balance = 0
//chenking the data is present in the selected date
if(result.docs.length>0){
result.docs.forEach((element, ) => {
saletotal = (isNaN(element.TotalAmount) ? 0: parseInt(element.TotalAmount));
Totalinvest = (isNaN(element.TotalInvestment) ? 0: parseInt(element.TotalInvestment));
Totalexp = (isNaN(element.TotalExpenses) ? 0: parseInt(element.TotalExpenses));
opening_balance+=saletotal+Totalinvest-Totalexp
console.log(opening_balance)
});
}
// if there is no data on particular date subtracting one day from date from date and calling the
//get day function with yesterday value as parameter to check if the data is present in previos date
else{
let d = new Date(TodayDate);
d.setDate(d.getDate() - 1);
var yesterdaydate = d.toISOString().split('T')[0];
getday(yesterdaydate)
}
});
}
//selecting the date
<input type="date" name="daybook" id="daybook-firstdate" onchange="getday()" >
i want to execute the find until there is data in previous date from selected date
any one help me with code
day book object: //find is based on Date object

Compare Dates in Array of MongoDB Object with different formats

I am using a datepicker in my frontend which sends a date using an AJAX Request.
Now this date is in the format - 'YYYY-MM-DD'. I have an array of dates which I have to compare this date with and pick out which are equal. This array is from a Mongo DB collection where I have TimeZone included in the field as well, for example - "2020-06-03T00:00:00.000Z"
My Slot Json Object
{
date: [
2020-06-03T00:00:00.000Z,
2020-06-05T00:00:00.000Z,
2020-06-07T00:00:00.000Z
],
__v: 0
}
I have to loop over each date in the date array and compare it with the date I get from frontend.
Let's say user input date is
let userInput = '2020-06-03';
And now I have to compare it with the date array
How do I ensure that the following comparision leads me to get a true value for
'2020-06-03' and '2020-06-03T00:00:00.000Z'
I am looking at a solution which is appropriate when looping over all these array elements.
assuming, that your date array always is at 00:00:00 o'clock, i'd suggest something like this:
const dates = [
'2020-06-03T00:00:00.000Z',
'2020-06-05T00:00:00.000Z',
'2020-06-07T00:00:00.000Z'
];
let userInput = '2020-06-03';
function findDates(date) {
const searchDate = Date.parse(date);
const foundDates = [];
dates.forEach(date => {
const tempDate = Date.parse(date);
if (tempDate === searchDate) {
foundDates.push(date);
}
});
return foundDates;
}
console.log(findDates(userInput));
if there's more to consider, please say so
Have the format of date strings(input) and the array of dates be properly specified for parsing. Then you could straight away use the isSame() to check if the dates are equal
let input = "2020-06-03";
let arr = [
"2020-06-03T00:00:00.000Z",
"2020-06-05T00:00:00.000Z",
"2020-06-07T00:00:00.000Z"
];
let result = arr.map(item =>
moment(item, "YYYY-MM-DD hh:mm:ss zz").isSame(input, "YYYY-MM-DD")
);
console.log(result);
<script src="https://momentjs.com/downloads/moment.js"></script>

BSON.Timestamp conversion issue in Nodejs Mongodb Stitch?

I am using stitch function to get the last 7 days from event collection.below is my code.this is execute in stitch function.
exports = async function() {
var events = context.services.get("mongodb-atlas").db("utilo").collection("events");
var today = new Date();
var lastWeek = BSON.Timestamp(today.setDate(today.getDate()-7),0);
console.log(lastWeek);
var document = null;
do{
document = await cursor.next();
if(document){
var result = {};
result.community = document._id;
result.data.newUsersCount = await events.count({community:document._id,type:"join",status:"completed",ts:{$gt:lastWeek}});
}}
}
In the above code, I tried to get last 7 days records from event collection.Here (today.setDate(today.getDate()-7),0) getting the correct Timestamp value but after adding BSON.Timestamp, the timestamp will change to lower year or higher year like either 2004 or 2024. without changing the Timestamp value, can we convert to the Timestamp?
How can i store value in last week like TImestamp(1520801145,0)?
or How to write the code for get the last 7 days record from events collection (ts stored in timestamp)

Extract specific digits from a .txt file

I have been asked to count the number of tweets per hour by day (0 - 23) in a huge text file of random tweets. The date is not interesting, only the tweet per hour. I want to return them in a new array of objects. Each object should have properties hour and count like this:
{hour: x, count: y},
I've made a function where I'm declaring an empty array, in which I will put my data:
function(tweets) {
let result = [];
and I think I need to push them like this:
result.push({hour: x, count: y});
But I don't know how to extract the specific hour from my object (key and value).
in the huge, raw data file, each tweet is logged with a date like this:
created_at: "30-06-2015 14:27",
Any suggestions or experience? I'm currently learning about regex and for loops. Should I use them in this code or is there a smarter way?
Edit: as you asked for more details:
The raw data are object in an array with the following structure:
{
time: Date-object,
created_at: "30-06-2015 14:27",
fromUsername: "victor",
text: "asyl og integration",
lang: "da",
source: "Twitter for Android",
}
About extracting text I see good answer here. Instead of console.log add parsing and saving to your array.
About regexp - I think it should be something like
var re = /created_at: \"([^\"]*)\",/g;
What I would do is work from a different angle:
create an object with a dateTimeHour for the start of each hour that you care about. It should presumably be a limited timespan like for all tweets that happened before now:
So generate something that looks like this dynamically:
{
'2019-03-01T17:22:30Z': 0, // or simply '1552667443928'
'2019-03-01T18:22:30Z': 0,
'2019-03-01T19:22:30Z': 0,
'2019-03-01T20:22:30Z': 0,
...etc
}
Which you can do using current Date and then a loop to create additional previous date times:
const now = new Date()
// you can use a generator here or simply a while loop:
const dateTimes = {}
while(now > REQUIRED_DATE)
dateTimes[new Date(now.setHours(now.getHours() - 1))] = 0
Now you have an exhausted list of all the hours.
Then, check if the given tweet is within that hour:
check if item.created_at < currentHourBeingLooked because you should loop through the Object.keys(dateTimes).
Then, loop through each item in your list and check if it fits that dateTime if so increment dateTimes[currentHour]++.
So, the hardest part will be converting created_at to a normal looking date time string:
const [datePortion, timePortion] = "30-06-2015 14:27".split(' ')
const [day, month, year] = datePortion.split('-')
const [hour, minute] = timePortion.split(':')
now with all those date, month, year, hour, and minute you can build a time object in javascript:
It follows the formula:
From MDN:
new Date(year, monthIndex [, day [, hours [, minutes [, seconds [, milliseconds]]]]]);
AKA:
new Date(year, monthIndex, day, hours, minutes, seconds);
So for December 17, 2019 # 3:24am it'll be:
const = new Date(2019, 11, 17, 3, 24, 0);
I'll assume that you already know to use regex from the post pointed by Ralkov to get all of your created_at dates, and my answer will go from that.
You said the date is not important so once you have the string
'created_at: "30-06-2015 14:27"'
we need to get rid of everything except for the hour, i did it by extracting substrings, feel free to try other approaches, this is just to get you started.
var date = obj.substr(obj.indexOf(' ') + 1);
var time = date.substr(date.indexOf(' ') + 1);
var hour = time.substr(0, time.indexOf(':'));
will get yo the hour
"14"
Note that this only works for one day, you need to do some additional changes if you'd like to store tweet hour count for different days in the same data structure
When you write your for-loop use the following function each time you find a tweet and already extracted the hour, it stores a combination of value-pairs into a map variable defined outside the function, creating a new pair if necessary or just updates it with the new tweet count.
function newTweet(hour, tweetsPerHour) {
var tweetsThisHour = tweetsPerHour.get(hour);
tweetsThisHour = tweetsThisHour === undefined ? 0 : tweetsThisHour;
tweetsPerHour.set(hour, ++tweetsThisHour);
console.log(tweetsThisHour)
}
complete code:
var obj = 'created_at: "30-06-2015 14:27"';
var date = obj.substr(obj.indexOf(' ')+1);
var time = date.substr(date.indexOf(' ')+1);
var hour = time.substr(0, time.indexOf(':'));
var tweetsPerHour = new Map();
newTweet(hour, tweetsPerHour); //this is the extracted hour
newTweet("16", tweetsPerHour); //you can try different hours as well
newTweet("17", tweetsPerHour);
function newTweet(hour, tweetsPerHour) {
var tweetsThisHour = tweetsPerHour.get(hour);
tweetsThisHour = tweetsThisHour === undefined ? 0 : tweetsThisHour;
tweetsPerHour.set(hour, ++tweetsThisHour);
console.log(hour + " tweet count: " + tweetsThisHour)
}
what the code is doing is storing the hour and count of tweets in pairs:
[{"14":1} ,{"16":1}, {17:1}]
for example if you add "14" again it would update to
[{"14":2}, {"16":1}, {17:1}]
dig into JavaScript Map Objects as well.
Your code flow is something like the following:
Read .txt file
loop through dates -> get hour from date -> newTweet(hour,
tweetsPerHour).

Categories