We Keep Coding

Extracting monthly values from data with irregular dates

I have bunch of electricity meter readings which have irregular dates. See below :
ReadingDate Meter
19/01/2021 5270
06/03/2021 5915
11/05/2021 6792
08/07/2021 7367
9/9/2021 8095
8/11/2021 8849
02/12/2021 9065
17/01/2022 9950
Now I'd like to transform this into monthly readings, using just this data, to end up with a table like this
Month Usage
2021-01 452
2021-02 393
2021-03 416
2021-04 399
2021-05 341
2021-06 297
2021-07 347
2021-08 358
2021-09 369
2021-10 389
2021-11 295
2021-12 586
2022-01 308
Now, I have a working solution, but I'm sure there's a more beautiful concise way of doing it.
What I do is to create an intermediate array that has one line for each date between first and last meter readings.
Each item in the array has 3 values :
the date
the average value for that date (calculated by counting the days between meter readings and dividing that by change in the meter.
the corresponding month
The last step then is to loop over this intermediate array and sum the values for each different month.
Here's the working code (its taken from Google Apps Script so please ignore the spreadsheet specific stuff:
var DailyAveragesArray = [['Date','Usage','Month']];
var monthlyObject = {};
var monthlyArray = [['Month','Usage']];
function calculateAverageDailyFigures() {
// give indices for the useful columns, 0 numbered
var ReadingDateColumn = 0;
var MeterReading = 1;
// Read into an array
var MeterReadingData = ss.getDataRange().getValues() // Get array of values
const sortedReadings = MeterReadingData.slice(1).sort((a, b) => a[0] - b[0]);
// from https://flaviocopes.com/how-to-sort-array-by-date-javascript/
// First calculate the number of days and average daily figure for each row
// Note we don't do this for the last row
for(i=0; i < sortedReadings.length - 1 ; i++){
var NumberOfDays = (sortedReadings[i+1][0] - sortedReadings[i][0])/(1000*3600*24);
sortedReadings[i].push(NumberOfDays);
var MeterDifference = sortedReadings[i+1][1] - sortedReadings[i][1];
var AverageDailyFigure = MeterDifference/NumberOfDays;
sortedReadings[i].push(AverageDailyFigure);
}
BuildDailyArray(sortedReadings);
}
function BuildDailyArray(sortedReadings){
// For each row in sorted , loop from the date to the next date-1 and create columns date and Usage
for(i=0; i<sortedReadings.length -1 ;i++){
for (var d = sortedReadings[i][0]; d < sortedReadings[i+1][0]; d.setDate(d.getDate() + 1)) {
var newDate = new Date(d);
var month = newDate.getFullYear() + '-' + ('0' + (newDate.getMonth() + 1)).slice(-2);
DailyAveragesArray.push([newDate,sortedReadings[i][3],month]);
// Check if the month is in the object and add value, otherwise create object an add value
if(month in monthlyObject){
monthlyObject[month] = monthlyObject[month] + sortedReadings[i][3];
} else {
Logger.log('Didnt find month so create it');
monthlyObject[month] = sortedReadings[i][3];
}
}
}
Logger.log(DailyAveragesArray.length);
Logger.log(monthlyObject);
var DailyUsageData = ss.getRange('D1:F'+DailyAveragesArray.length);
DailyUsageData.setValues(DailyAveragesArray);
BuildMonthlyArray();
}
function BuildMonthlyArray(){
const keys = Object.keys(monthlyObject);
Logger.log(keys);
keys.forEach((key, index) => {
monthlyArray.push([key,Math.round(monthlyObject[key])]);
});
var MonthlyUsageData = ss.getRange('H1:I'+monthlyArray.length);
MonthlyUsageData.setValues(monthlyArray);
}
So, my question is, how would I do this nicer, more beautifully, not so verbose ?
I'm not sure what the correct term is for what I want to do. I don't think it's resampling .
I'd appreciate any comments.
Thanks / Colm

Here is my shot on this.
The way i'm doing it:
Initializing all days and its value
Grouping by month
Calculating the average per month
Explanation a bit more precise
initDateFromString
The method initDateFromString takes a dates with the format DD/MM/YYYY and return the associated js date object
initAllDates
The method initAllDates will split the data into day and add the average value of the difference for each day
for example, for the first two readings, it will result to an array of dates looking like :
date
value
19/01/2021
14.02
20/01/2021
14.02
....
....
05/03/2021
14.02
06/03/2021
14.02
The value 14.02 comme from the following calcul :
(newReadingMeter - oldReadingMeter)/nbDaysBetweenDates
Which in this example is (5915 - 5270)/46 = 14.02
joinToMonth
The joinToMonth method will then group the days into month with all the days value summed !
const data = [{
ReadingDate: '19/01/2021',
Meter: 5270
},
{
ReadingDate: '06/03/2021',
Meter: 5915
},
{
ReadingDate: '11/05/2021',
Meter: 6792
},
{
ReadingDate: '08/07/2021',
Meter: 7367
},
{
ReadingDate: '9/9/2021',
Meter: 8095
},
{
ReadingDate: '8/11/2021',
Meter: 8849
},
{
ReadingDate: '02/12/2021',
Meter: 9065
},
{
ReadingDate: '17/01/2022',
Meter: 9950
}
]
function initDateFromString(dateString){
let dateParts = dateString.split("/");
return new Date(+dateParts[2], dateParts[1] - 1, +dateParts[0]);
}
function initAllDates(data){
let dates = []
let currentValue = data.shift()
const currentDate = initDateFromString(currentValue.ReadingDate)
data.forEach(metric => {
const date = initDateFromString(metric.ReadingDate)
const newDates = []
while(currentDate < date){
newDates.push({date: new Date(currentDate)})
currentDate.setDate(currentDate.getDate() + 1)
}
dates = dates.concat(newDates.map(x => {
return {Usage: (metric.Meter - currentValue.Meter) / newDates.length, date: x.date}}
))
currentDate.setDate(date.getDate())
currentValue = metric
})
return dates
}
function joinToMonth(dates){
return dates.reduce((months, day) => {
const month = day.date.getMonth()
const year = day.date.getFullYear()
const existingObject = months.find(x => x.month === month && x.year === year)
if (existingObject) {
existingObject.total += day.Usage
} else {
months.push({
month: day.date.getMonth(),
year: day.date.getFullYear(),
total: day.Usage,
})
}
return months;
}, []);
}
const dates = initAllDates(data)
const joinedData = joinToMonth(dates)
console.log(joinedData)

Adding years to array of dates

if by end of the period person is greater than 18 i want childEndDate to be the date of person's date when he/she will become 18 years old.
in my else if statement i am using date-fns library to add 18 years to dates i have in my this.childBirthDate array. but returned output is wrong: ["1988-01-01T00:00:02.012Z", "1988-01-01T00:00:02.010Z", "2031-11-08T13:24:43.704Z"]
output i want returned is: ['2030-02-16T20:00:00.000Z', '2028-05-19T20:00:00.000Z', 2031-11-08T13:24:43.704Z]
here is my stackblitz
this.childBirthDate = [
'2012-02-16T20:00:00.000Z',
'2010-05-19T20:00:00.000Z',
'2016-05-19T20:00:00.000Z',
];
//enddate
const endYear = date.getFullYear() + 10;
date.setFullYear(endYear);
this.endDate = date.toISOString();
this.childBirthDate.forEach((element) => {
const birthYear = element.substring(0, 4);
this.childbirthYear.push(+birthYear);
});
const periodEndYear = +this.endDate.substring(0, 4);
// calculate child endDate
this.childbirthYear.forEach(element => {
if (periodEndYear - element < 18) {
this.childEndDate = this.endDate;
} else if (periodEndYear - element >= 18) {
this.childEndDate = addYears(new Date(element), 18).toISOString();
}
this.final.push(this.childEndDate);
});
console.log(this.final)
this.personalInfo = {
personalInfoId: 0,
underageChildInfo: this.data.underageChildInfo?.map((i, index) => ({
firstName: i.name,
endDate: this.final[index],
})),
};

I think your logic to get the end date is reproducing what you want. However, I think your code may be hard to follow due to all the extra variables.
If you remove the foreach loops in favor of some map operators, your code will log the dates you are looking to get.
e.g.
// in ngOnInit
this.finalDates = this.childBirthDate.map(date => {
return this.calculateDate(date);
});
// outputs ["2030-02-16T20:00:00.000Z", "2028-05-19T20:00:00.000Z", "2031-11-08T14:28:01.761Z"]
console.log(this.finalDates);
// custom method on the class
calculateDate(date: string): string {
const periodEndYear = +this.endDate.substring(0, 4);
const year = parseInt(date.substring(0, 4), 10); // get a number from string
if (periodEndYear - year < 18) {
return this.endDate;
} else if (periodEndYear - year >= 18) {
return addYears(new Date(date), 18).toISOString();
}
return date;
}
here is a fork of your blitz

How to check if the date is included in holidays?

i want to exclude the holiday dates from two selected dates ,
i have write the code , by checking if the range between start date and end date is included in holiday array then make the total days -1 . but is not working , so how i can do it
this is my code
const workday_count = (start: moment.Moment, end: moment.Moment, publicHolidays: Date[]) => {
const first = start.clone().endOf('week');
const last = end.clone().startOf('week');
const days = (last.diff(first, 'days') * 5) / 7;
let wfirst = first.day() - start.day();
const sta = start.toDate();
const en = end.toDate();
const range = enumerateDaysBetweenDates(sta, en);
if (start.day() === 6) --wfirst;
let wlast = end.day() - last.day();
if (end.day() === 5) --wlast;
const withOutWeekend = wfirst + Math.floor(days) + wlast;
const count = 0;
range.forEach((date) => {
if (publicHolidays.includes(date)) {
return count + 1;
}
});
return withOutWeekend;
};

it seems publicHolidays.includes(date) will always return false, so you can try publicHolidays.find() to check if date in this range
when the start day is Sunday(start.day() === 0), your wfirst would be 5 instead of 6
when the end day is Saturday(end.day() === 6), your wlast would be 5 instead of 6
Here is the code:
const workday_count = (start, end, publicHolidays) => {
const first = start.clone().endOf("week");
const last = end.clone().startOf("week");
const days = (last.diff(first, "days") * 5) / 7;
let wfirst = first.day() - start.day();
// when start is Sunday
if (start.day() === 0) --wfirst;
let wlast = end.day() - last.day();
// when end is Saturday
if (end.day() === 6) --wlast;
const sta = start.toDate();
const en = end.toDate();
const range = enumerateDaysBetweenDates(sta, en);
let withOutWeekend = wfirst + Math.floor(days) + wlast;
let count = 0;
range.forEach((date) => {
// check if the date is in publicHolidays
if (
publicHolidays.find((holiday) => holiday.getTime() === date.getTime())
) {
count++;
}
});
return withOutWeekend - count;
};
BTW, I think you can describe more detailed in your question about how is your code not working, so that we can have a better understanding of the problem. :)

How to sum values over a given date range

I'm using LocalStorage to save an array of Dates and Costs.
When I'm writing localStorage.getItem("todos"); into the console, the format will be like this:
"[{"due":"28/10/2017","task":"80"},{"due":"06/10/2017","task":"15"}]"
Where due is the Date, and TASK is the AMOUNT.
I managed to get the TOTAL of AMOUNTS by:
total: {
type: String,
value: () => {
var values = localStorage.getItem("todos");
if (values === undefined || values === null) {
return "0";
}
var data = JSON.parse(values);
var sum = 0;
data.forEach(function(ele){ sum+=Number(ele.task)}); return sum;
}
}
Now I'm trying to get the TOTAL of last 6 MONTHS.
I have no idea on how to approach this.
How should I be able to do this?

During your iteration you need to add a check to make sure the sum is only including values where the due date is within your range. If you can use a library like moment, this would greatly simplify your logic.
const data = [
{ due: '28/10/2017', task: 80 },
{ due: '06/10/2017', task: 15 },
{ due: '10/05/2000', task: 3000 }
];
const sixMonthsAgo = moment().subtract(6, 'months');
const total = data.reduce((acc, item) => {
const dueDate = moment(item.due, 'DD/MM/YYYY');
return acc + (dueDate.isAfter(sixMonthsAgo) ? item.task : 0);
}, 0);
console.log('total should equal 95: ', total);
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.19.1/moment.min.js"></script>

Here is a solution for your issue :
make a test in the forEach loop :
I've put 4 dates : 2 under 6 months and 2 older
The result is 80+15 = 95
// After JSON.parse
var todos=[{"due":"28/10/2017","task":"80"},{"due":"06/10/2017","task":"15"},{"due":"06/04/2017","task":"15"},{"due":"06/02/2017","task":"15"}];
var sum = 0;
var minDate = new Date();
var month = minDate.getMonth()+1-6; // get month minus 6 months
var year = minDate.getFullYear(); // get year
if(month < 1){ // if month is under January then change year
month+=6;
year-= 1;
}
minDate.setMonth(month); // Replace our min date with our - 6 m
minDate.setYear(year); // set year in case we have changed
todos.forEach(function(ele){
var arr = ele.due.split("/"); // split french string date into d,m,y
if(arr.length==3){
var dueDate = new Date(arr[2],arr[1],arr[0]); // get the task date
if(dueDate>minDate){ // if task is not to old then
sum+=parseInt(ele.task); // sum it
}
}
});
console.log(sum);

Calculating weekdays issue, javascript

i have an array of user selected days, presented as such:
days_selected[1] = true;
days_selected[2] = false;
days_selected[3] = false;
days_selected[4] = true;
days_selected[5] = true;
days_selected[6] = true;
days_selected[7] = true;
Key presents weekday, and true/false presents if the user checked the day
now, i have the current weekday,
date_now.getDay()
Lets say the current weekday is monday, 1 i need to find the amount of days between monday and the first day which is marked as true;
I know i could simply loop the days, find the current day, and keep looping until i stumble on another true day, and then subtract the values, but how do i count in day 6 being true, while day 7 and day 1 are false, in that case, it would also be 2 days in between, i am horrible at math :-)
passed = false;
day = 0;
$.each( days_selected, function( key, value ) {
if (passed == true && value == true) {
day = key;
return false;
}
if (key == date_now.getDay()) {
passed = true;
}
});

What about this? (assuming indexes starting at 0)
var today = date_now.getDay()
, i = today + 1;
while (i % 7 != today && !days_selected[i % 7]) {
i += 1;
}
var interval = i - today - 1;
Check http://jsbin.com/jozotoke/1/edit

How about something like
function nextTrue(arr, selected, toCheck) {
var first = arr.indexOf(toCheck, selected+1),
res = first > selected ? first : arr.indexOf(toCheck);
return Math.abs(res - selected);
}
to be used as
var diff = nextTrue(days_selected, date_now.getDay(), true);
FIDDLE