We Keep Coding

Extract specific digits from a .txt file

I have been asked to count the number of tweets per hour by day (0 - 23) in a huge text file of random tweets. The date is not interesting, only the tweet per hour. I want to return them in a new array of objects. Each object should have properties hour and count like this:
{hour: x, count: y},
I've made a function where I'm declaring an empty array, in which I will put my data:
function(tweets) {
let result = [];
and I think I need to push them like this:
result.push({hour: x, count: y});
But I don't know how to extract the specific hour from my object (key and value).
in the huge, raw data file, each tweet is logged with a date like this:
created_at: "30-06-2015 14:27",
Any suggestions or experience? I'm currently learning about regex and for loops. Should I use them in this code or is there a smarter way?
Edit: as you asked for more details:
The raw data are object in an array with the following structure:
{
time: Date-object,
created_at: "30-06-2015 14:27",
fromUsername: "victor",
text: "asyl og integration",
lang: "da",
source: "Twitter for Android",
}

About extracting text I see good answer here. Instead of console.log add parsing and saving to your array.
About regexp - I think it should be something like
var re = /created_at: \"([^\"]*)\",/g;

What I would do is work from a different angle:
create an object with a dateTimeHour for the start of each hour that you care about. It should presumably be a limited timespan like for all tweets that happened before now:
So generate something that looks like this dynamically:
{
'2019-03-01T17:22:30Z': 0, // or simply '1552667443928'
'2019-03-01T18:22:30Z': 0,
'2019-03-01T19:22:30Z': 0,
'2019-03-01T20:22:30Z': 0,
...etc
}
Which you can do using current Date and then a loop to create additional previous date times:
const now = new Date()
// you can use a generator here or simply a while loop:
const dateTimes = {}
while(now > REQUIRED_DATE)
dateTimes[new Date(now.setHours(now.getHours() - 1))] = 0
Now you have an exhausted list of all the hours.
Then, check if the given tweet is within that hour:
check if item.created_at < currentHourBeingLooked because you should loop through the Object.keys(dateTimes).
Then, loop through each item in your list and check if it fits that dateTime if so increment dateTimes[currentHour]++.
So, the hardest part will be converting created_at to a normal looking date time string:
const [datePortion, timePortion] = "30-06-2015 14:27".split(' ')
const [day, month, year] = datePortion.split('-')
const [hour, minute] = timePortion.split(':')
now with all those date, month, year, hour, and minute you can build a time object in javascript:
It follows the formula:
From MDN:
new Date(year, monthIndex [, day [, hours [, minutes [, seconds [, milliseconds]]]]]);
AKA:
new Date(year, monthIndex, day, hours, minutes, seconds);
So for December 17, 2019 # 3:24am it'll be:
const = new Date(2019, 11, 17, 3, 24, 0);

I'll assume that you already know to use regex from the post pointed by Ralkov to get all of your created_at dates, and my answer will go from that.
You said the date is not important so once you have the string
'created_at: "30-06-2015 14:27"'
we need to get rid of everything except for the hour, i did it by extracting substrings, feel free to try other approaches, this is just to get you started.
var date = obj.substr(obj.indexOf(' ') + 1);
var time = date.substr(date.indexOf(' ') + 1);
var hour = time.substr(0, time.indexOf(':'));
will get yo the hour
"14"
Note that this only works for one day, you need to do some additional changes if you'd like to store tweet hour count for different days in the same data structure
When you write your for-loop use the following function each time you find a tweet and already extracted the hour, it stores a combination of value-pairs into a map variable defined outside the function, creating a new pair if necessary or just updates it with the new tweet count.
function newTweet(hour, tweetsPerHour) {
var tweetsThisHour = tweetsPerHour.get(hour);
tweetsThisHour = tweetsThisHour === undefined ? 0 : tweetsThisHour;
tweetsPerHour.set(hour, ++tweetsThisHour);
console.log(tweetsThisHour)
}
complete code:
var obj = 'created_at: "30-06-2015 14:27"';
var date = obj.substr(obj.indexOf(' ')+1);
var time = date.substr(date.indexOf(' ')+1);
var hour = time.substr(0, time.indexOf(':'));
var tweetsPerHour = new Map();
newTweet(hour, tweetsPerHour); //this is the extracted hour
newTweet("16", tweetsPerHour); //you can try different hours as well
newTweet("17", tweetsPerHour);
function newTweet(hour, tweetsPerHour) {
var tweetsThisHour = tweetsPerHour.get(hour);
tweetsThisHour = tweetsThisHour === undefined ? 0 : tweetsThisHour;
tweetsPerHour.set(hour, ++tweetsThisHour);
console.log(hour + " tweet count: " + tweetsThisHour)
}
what the code is doing is storing the hour and count of tweets in pairs:
[{"14":1} ,{"16":1}, {17:1}]
for example if you add "14" again it would update to
[{"14":2}, {"16":1}, {17:1}]
dig into JavaScript Map Objects as well.
Your code flow is something like the following:
Read .txt file
loop through dates -> get hour from date -> newTweet(hour,
tweetsPerHour).

Javascript moment js split time based on hours from current time to previous midnight

What am looking forward to get is a split of time from current time to yesterdays midninght split by one hour
so that is
eg: if now its 03:34
so i would like to get this
var durations = [
{from:03:00, to:03:34},
{from:02:00, to:03:00},
{from:01:00, to:02:00},
{from:00:00, to:01:00}
]
SO from the above example the value of 03:34 is the current time
SO far i have been able to get the first segment via
{from:moment().startOf('hour'), to:moment()}
Now am stuck on how to split the other durations with a difference of 1 hour each
So its something like
let hours_from_midnight = moment.duration(end.diff(moment().startOf('day'))).asHours();
But now how do i proceed to use the number of hours from midnight to split the next durations and achieve the durations desired

What you need to do is keep subtracting an hour and cloning the date, as all the methods mutate the existing object:
function periods(initialTime) {
// our lower bound
const midnight = initialTime.clone().startOf("day");
// our current period start
const periodStart = initialTime.clone().startOf("hour");
const periods = [];
if (!(periodStart.isSame(initialTime))) {
// only add the last period if it isn't empty
periods.push({ start: periodStart.clone(), end: initialTime });
}
while (periodStart.isAfter(midnight)) {
// the last start is our new end
const to = periodStart.clone();
// our new start is one hour earlier
const from = periodStart.subtract(1, "hour").clone();
periods.push({
from,
to
});
}
return periods;
}
console.log(periods(moment("03:34", "HH:mm")));
console.log(periods(moment("18:00", "HH:mm")));

Find next instance of a given weekday (ie. Monday) with moment.js

I want to get the date of the next Monday or Thursday (or today if it is Mon or Thurs). As Moment.js works within the bounds of a Sunday - Saturday, I'm having to work out the current day and calculate the next Monday or Thursday based on that:
if (moment().format("dddd")=="Sunday") { var nextDay = moment().day(1); }
if (moment().format("dddd")=="Monday") { var nextDay = moment().day(1); }
if (moment().format("dddd")=="Tuesday") { var nextDay = moment().day(4); }
if (moment().format("dddd")=="Wednesday") { var nextDay = moment().day(4); }
if (moment().format("dddd")=="Thursday") { var nextDay = moment().day(4); }
if (moment().format("dddd")=="Friday") { var nextDay = moment(.day(8); }
if (moment().format("dddd")=="Saturday") { var nextDay = moment().day(8); }
This works, but surely there's a better way!

The trick here isn't in using Moment to go to a particular day from today. It's generalizing it, so you can use it with any day, regardless of where you are in the week.
First you need to know where you are in the week: moment().day(), or the slightly more predictable (in spite of locale) moment().isoWeekday(). Critically, these methods return an integer, which makes it easy to use comparison operators to determine where you are in the week, relative to your targets.
Use that to know if today's day is smaller or bigger than the day you want. If it's smaller/equal, you can simply use this week's instance of Monday or Thursday...
const dayINeed = 4; // for Thursday
const today = moment().isoWeekday();
if (today <= dayINeed) {
return moment().isoWeekday(dayINeed);
}
But, if today is bigger than the day we want, you want to use the same day of next week: "the monday of next week", regardless of where you are in the current week. In a nutshell, you want to first go into next week, using moment().add(1, 'weeks'). Once you're in next week, you can select the day you want, using moment().day(1).
Together:
const dayINeed = 4; // for Thursday
const today = moment().isoWeekday();
// if we haven't yet passed the day of the week that I need:
if (today <= dayINeed) {
// then just give me this week's instance of that day
return moment().isoWeekday(dayINeed);
} else {
// otherwise, give me *next week's* instance of that same day
return moment().add(1, 'weeks').isoWeekday(dayINeed);
}
See also https://stackoverflow.com/a/27305748/800457
EDIT: other commenters have pointed out that the OP wanted something more specific than this: the next of an array of values ("the next Monday or Thursday"), not merely the next instance of some arbitrary day. OK, cool.
The general solution is the beginning of the total solution. Instead of comparing for a single day, we're comparing to an array of days: [1,4]:
const daysINeed = [1,4]; // Monday, Thursday
// we will assume the days are in order for this demo, but inputs should be sanitized and sorted
function isThisInFuture(targetDayNum) {
// param: positive integer for weekday
// returns: matching moment or false
const todayNum = moment().isoWeekday();
if (todayNum <= targetDayNum) {
return moment().isoWeekday(targetDayNum);
}
return false;
}
function findNextInstanceInDaysArray(daysArray) {
// iterate the array of days and find all possible matches
const tests = daysINeed.map(isThisInFuture);
// select the first matching day of this week, ignoring subsequent ones, by finding the first moment object
const thisWeek = tests.find((sample) => {return sample instanceof moment});
// but if there are none, we'll return the first valid day of next week (again, assuming the days are sorted)
return thisWeek || moment().add(1, 'weeks').isoWeekday(daysINeed[0]);;
}
findNextInstanceInDaysArray(daysINeed);
I'll note that some later posters provided a very lean solution that hard-codes an array of valid numeric values. If you always expect to search the same days, and don't need to generalize for other searches, that'll be the more computationally efficient solution, although not the easiest to read, and impossible to extend.

get the next monday using moment
moment().startOf('isoWeek').add(1, 'week');

moment().day() will give you a number referring to the day_of_week.
What's even better: moment().day(1 + 7) and moment().day(4 + 7) will give you next Monday, next Thursday respectively.
See more: http://momentjs.com/docs/#/get-set/day/

The following can be used to get any next weekday date from now (or any date)
var weekDayToFind = moment().day('Monday').weekday(); //change to searched day name
var searchDate = moment(); //now or change to any date
while (searchDate.weekday() !== weekDayToFind){
searchDate.add(1, 'day');
}

Most of these answers do not address the OP's question. Andrejs Kuzmins' is the best, but I would improve on it a little more so the algorithm accounts for locale.
var nextMoOrTh = moment().isoWeekday([1,4,4,4,8,8,8][moment().isoWeekday()-1]);

Here's a solution to find the next Monday, or today if it is Monday:
const dayOfWeek = moment().day('monday').hour(0).minute(0).second(0);
const endOfToday = moment().hour(23).minute(59).second(59);
if(dayOfWeek.isBefore(endOfToday)) {
dayOfWeek.add(1, 'weeks');
}

Next Monday or any other day
moment().startOf('isoWeek').add(1, 'week').day("monday");

IMHO more elegant way:
var setDays = [ 1, 1, 4, 4, 4, 8, 8 ],
nextDay = moment().day( setDays[moment().day()] );

Here's e.g. next Monday:
var chosenWeekday = 1 // Monday
var nextChosenWeekday = chosenWeekday < moment().weekday() ? moment().weekday(chosenWeekday + 7) : moment().weekday(chosenWeekday)

The idea is similar to the one of XML, but avoids the if / else statement by simply adding the missing days to the current day.
const desiredWeekday = 4; // Thursday
const currentWeekday = moment().isoWeekday();
const missingDays = ((desiredWeekday - currentWeekday) + 7) % 7;
const nextThursday = moment().add(missingDays, "days");
We only go "to the future" by ensuring that the days added are between 0 and 6.

d3.max return wrong date

I have a kind a weird problem. I'm loading data for a overview page. I need to show max date, so users know which time period they're dealing with. Now, I know which date this should be (today or yesterdays date, this is updated once a night) but I would like to do a max function anyway, in case something goes wrong during the update.
The problem I'm having is that both d3.max and a custom max-date function returns the wrong date. One month in the future, so today should show 7/9 2015 but instead it displays 7/10 2015. When I filter the data on 7/10 i get an empty array. In fiddle this works alright so there is something fishy going on. I do run an d3.tsv(tsv is right, even if the extension says .csv the file is in fact tab-separated), might be something there that's causing trouble? Any ideas where i might go wrong? The parsefunction alone returns the right results, the dates when read have the following format: dd.mm.yyyy
//Dateformatting
function parseDate (dateStr) {
var s1 = dateStr.split(" ");
var s1dat = s1[0].split(".");
return new Date(s1dat[2], s1dat[1], s1dat[0])
};
var dateArr = [];
//Occupation
d3.tsv("data.csv", function(error, data, tsv) {
datasetIn = data;
datasetIn.forEach(function(d) {
d.datum = parseDate(d.datum);
d.Patients = +d.Patients.replace(",", ".");
d.Beds= +d.Antal_vardplats.replace(",", ".");
});
for (index = 0; index < datasetIn.length; ++index) {
dateArr.push(datasetIn[index].datum);
}
var maxYear = d3.max(dateArr).getFullYear();
var maxDate = d3.max(dateArr);
})

In JavaScript month values are zero-based beginning with 0 for January to 11 for December. Passing in a human-readable 9 representing September to new Date() is supposed to create a Date object for October instead.
new Date("2015", "9", "7").toDateString(); // "Wed Oct 07 2015"
Adjusting for this should give you the correct result when parsing human-readable date values.

How to get the total month by given months

I have an array like this
var array = [
{ date:2014-11-11,
title:test },
{ date:2014-11-12,
title:test },
{ date:2014-11-13,
title:test },
…more
…more
{ date:2015-01-01
title:test},
{ date:2015-01-02
title:test},
…more
…more
{ date:2015-03-01
title:test}
]
My questions is how to get the total month of each year.
For example, I need to have 2 months (nov to dec) in 2014 and 3 months (Jan to March) in 2015.
var firstYear = parseInt($filter('date')(array[0].date, 'yyyy'));
var lastYear = parseInt($filter('date')(array[array.length-1].date, 'yyyy'));
I am not sure what I can do next to get the total month of the year
Can anyone help me about it? Thanks!

Your array syntax is not valid javascript. Do you really have date strings like:
date: '2014-11-11',
or is the value a Date object representing that date? Is the date local or UTC? Anyway, I'll assume you have strings and whether they are UTC or local doesn't matter.
My questions is how to get the total month of each year. For example, I need to have 2 months (nov to dec) in 2014 and 3 months (Jan to March) in 2015.
I'm not exactly sure what you want, you should provide an example of your expected output. The following returns the month ranges in particular years, there is no conversion of Strings to Dates:
// Sample data
var array = [
{ date:'2014-11-11',
title:'test'},
{ date:'2014-11-12',
title:'test'},
{ date:'2014-11-13',
title:'test'},
{ date:'2015-01-01',
title:'test'},
{ date:'2015-01-02',
title:'test'},
{ date:'2015-03-01',
title:'test'}
];
And the function:
function getMonthCount2(arr) {
var yearsMonths = arr.map(function(v){return v.date.substr(0,7).split(/\D/)}).sort();
var monthRanges = {}
yearsMonths.forEach(function(v,i) {
if (monthRanges[v[0]]) {
monthRanges[v[0]] += v[1] - yearsMonths[--i][1];
} else {
monthRanges[v[0]] = 1;
}
});
return monthRanges;
}
console.log(getMonthCount2(array)); // {'2014': 2, '2015': 3}
The above assumes valid input, you may want to put in a validation step to ensure the data is clean before passing it to the function.

If you're dealing with dates and times you should probably think about using a library as there are many nuances that go along with working dates and times.
I just did something similar to this and I solved it using moment js and the date range extension.
Looking at the docs for moment-range it seems that you can do something like this:
var start = new Date(2012, 2, 1);
var end = new Date(2012, 7, 5);
var range1 = moment().range(start, end);
range1.by('months', function(moment) {
// Do something with `moment`
});