I'm using an elasticsearch date histogram to group responses by count over time. The date histogram facet works great for this but if an interval doesn't have any responses that fall within in it it doesn't show up in the json. I figured the best way to combat this is to use javascript to fill in the gaps in a charting library. (ideally in highcharts but d3 or something else is possible). Months seem pretty easy to do but it get more complicated when I need to do it by week and day as well. Basically my problem is:
{ date: April: 5, count: 5 }, { date: June, count: 10 }
needs to be more like
{ date: April: 5, count: 5 }, {date: May, count: null }, { date: June, count: 10 }
min_doc_count=0 only creates intervals in between nonempty buckets. If you want to plot empty intervals outside your buckets (a few months ahead or behind of the start of your data), then add extended_bounds (docs).
In elasticsearch_dsl, to allow empty buckets out to two years ago, this looks like
A(
"date_histogram",
field="publishedAt",
calendar_interval="month",
format="MMM yyyy",
min_doc_count=0,
extended_bounds={"min": f"{date:%b %Y}||-2y"},
),
I had the same issue for a while after searching and reading the documentation I found out extended_bounds will fix my problem:
{
"aggs": {
"total": {
"date_histogram": {
"extended_bounds": {
"max": "2022-11-01",
"min": "2015-09-04"
},
"field": "eventDate",
"calendar_interval": "1d",
"min_doc_count": 0
}
}
}
}
Related
I have a challenge with displaying data in a barchart. I choose to use Chartjs as it seemed the best free fit for my needs.
Though the principle of datasets is clear, I can't get my head around the following:
I need to display 2 assignment ratings (y-axis) of 10 people grouped per course (x-axis).
"id": "15686a84-a0cb-4944-b268-f25f2b3b89e1",
"user_id": 5,
"assignment": {
"course_id": 100002,
"difficulty": 4,
"fun": 1
}
},
{
"id": "8e567e3d-3618-445d-8148-e1977016e238",
"user_id": 6,
"assignment": {
"course_id": 100012,
"difficulty": 1,
"fun": 3
}
},
Here are two assignment records shown for two different users (2, 6) for two different courses (10002, 100012) with both two ratings (fun, difficulty) so you can see how the data is constructed.
if i use:
data: assignments
.filter((assignment) => assignment.user_id === s.id)
.map((a) => [a.assignment.difficulty, a.assignment.fun]),
Chartjs will interpret the two values of difficulty and fun as a min and max. But I want them to be treated as two seperate values.
I tried everything but I didn't found the holy grail yet.
Can any one help me digging into this?
I had completed all 14 user stories, except 2, userstory #6 and #12 they are related to each other (about YScale domain and range, and converting it to minutes)
It's been two days I stuck on this, I will share my code
Can anyone tell me what is the bug in my code.
##https://codepen.io/codebrakerk/pen/ZEWOPpz
You need to access "Time" from your dataset.
{
"Time": "36:50",
"Place": 1,
"Seconds": 2210,
"Name": "Marco Pantani",
"Year": 1995,
"Nationality": "ITA",
"Doping": "Alleged drug use during 1995 due to high hematocrit levels",
"URL": "https://en.wikipedia.org/wiki/Marco_Pantani#Alleged_drug_use"
}
I guess the "Seconds" attribute implies the time it took for the person to complete his run. Your time code should run as follows:
(item) => {
let t = item["Time"].split(':');
return new Date(minutes=t[0], seconds=t[1]);
}
I would like to sort and modify the output of an object array by browsing week by week from the earliest date, to sort by week the sum of qt.
Datas: [
{qt: 3, date: 03-03-2020},
{qt: 2, date: 02-14-2020},
{qt: 4, date: 13-03-2020},
{qt: 3, date: 04-02-2020},
]
Desired result:
[
week1: 3,
week2: 6,
week3: 0,
week4: 0,
week5: 3
]
I’ve been trying to turn it all up since yesterday, but I'm starting to have a lot of trouble. I'm working on React with Akita. If you have possible algorithm orientations, I am interested.
You can use lodash library.
var _ = require('lodash');
const ordered = _.orderBy(Datas, 'date', 'asc');
it sort your Datas according to date by ascending.I realize that you want erroneous form as desired result you cannot make it [week1: 3] it should be [{week1:3}] but it doesn't matter. You should make a list [3,6,0,0,3] and assume that their indexes represent their week number.So following code gives you the list:
const desiredResult = ordered.map(obj => {
return obj.qt;
});
Also you may want visit this website to learn how to install lodash
The difficulty is above all to travel PER WEEK in a date range (the oldest to that of today). It is a question of isolating even in the first emps to obtain:
[{
week1: {qt: ..., date: ...}, //week1 = date range between Monday and Sunday
week2: {qt: ..., date: ...},
...]
I’m putting together an web-app that plots time series of weather data for the current year and the past 10 years on one dygraph. I want the time series for the current year (2015 at this time) to be highlighted relative to the past years, so I’m trying to increase the strokewidth for the current year from a default of 2 to 5. My problem is that I’m having trouble getting this to work programatically. The daily data for each year is in native array format, so the time series are identified by the contents of the array Yr0Labels:
["Date", "2005", "2006", "2007", "2008", "2009", "2010", "2011", "2012", "2013", "2014", "2015"]
Next January 1 the Yr0Labels will programmatically change to:
["Date", "2006", "2007", "2008", "2009", "2010", "2011", "2012", "2013", "2014", "2015",”2016”]
In the dygraph options parameter I set default strokeWidth to 2 and the strokeWidth for 2015 to 5. Because I want the current calendar year data to be highlighted properly after Jan. 1 , I’ve tried identifying the series to be highlighted by the contents of yr0Labels[11],
{
……
strokeWidth: 2,
labels: yr0Labels,
series: { yr0Labels[11] : { strokeWidth: 5} },
……..
}
This produced a syntax error - SyntaxError: missing : after property id
Suspecting that dygraph didn’t want to see array syntax in the series identifier, I tried to identify the series via a string variable “cyear”,
cyear = yr0Labels[11];
{…strokeWidth: 2,
labels: yr0Labels,
series: { cyear : { strokeWidth: 5} },
…….}
This didn’t produce an error, but also didn’t highlight the series.
The only way I’ve been able to make this work is to directly enter the current year value as the series identifier,
{…strokeWidth: 2,
labels: yr0Labels,
series : { ‘2015’ : { strokeWidth: 5} },
…….}
This worked, but I’d have to edit the dygraph option parameter every Jan. 1 to make data for the current calendar year plot properly.
How I can make this highlighting work programmatically?
The issue is with how you're using keys in JavaScript object literals.
This:
{ foo: 'bar' }
is the same as:
{ 'foo': 'bar' }
even if there's a variable in scope called foo. To achieve the result you want, you need to do fill out your objects using something like:
var foo = 'blah';
var o = {};
o[foo] = 'bar';
I have the below array of object from my MongoDB query to get some statistics based on time:
[
{
_id: { year: 2014, dayOfYear: 128, hour: 9 },
count: 2,
avg: 0.12455,
min: 0.1245,
max: 0.1246,
gv: 7.98954654895666,
bv: 0.9950000000000001
},
{
_id: { year: 2014, dayOfYear: 134, hour: 14 },
count: 8,
avg: 0.12217854,
min: 0.1212,
max: 0.12345678,
gv: 25.869999999999997,
bv: 3.1652450614477337
},
{
_id: { year: 2014, dayOfYear:126, hour: 19 },
count: 3,
avg: 0.11234099999999998,
min: 0.112341,
max: 0.112341,
gv: 29.849999999999998,
bv: 3.3533788500000004
}
]
I want to basically convert the _id object into a unix timestamp with moment.js I just cant to find the right underscore function to iterate through the objects, get the ID data and make an array out of it again.
In the process i could just do moment().year('2014').format('Z')
result wanted:
{
unixtime: 1400599394,
count: 3,
avg: 0.11234099999999998,
min: 0.112341,
max: 0.112341,
gv: 29.849999999999998,
bv: 3.3533788500000004
}
I know that you did ask for how to do this with promises, but I'm just taking a stab here based on the naming of the fields under _id that this is actually output from the aggregation framework and therefore there is another way to do this.
The date operators that you appear to have used are great for their purposes, but as you seem to want a unix timestamp as the output then the logical thing to do is keep that field in unix timestamp format, just with the grouping boundaries you need.
This seems to be grouping "per hour" within each day, so let's apply the date math in order to alter that timestamp value. So instead of using the date operators to break up the date in your $group pipeline key, do this instead:
{ "$group": {
"_id": {
"$subtract": [
{ "$subtract": [ "$created", new Date("1970-01-01") ] },
{ "$mod": [
{ "$subtract": [ "$created", new Date("1970-01-01") ] },
1000*60*60
]}
]
}
}}
Of course include all the other things you have in your aggregation pipeline ( which are not supplied in this question ), but the result is the actual epoch timestamp value being truncated to the hour of the day from your original date field which is named "created".
Just to break down the parts of this consider this date object:
var date = new Date()
date.valueOf()
1400639001169
So there you see the epoch timestamp representation extracted from the date object. In terms of the math, retrieving that value via the .valueOf() method is exactly the same as:
date - new Date("1970-01-01")
1400639001169
Which provides you with the seconds ( or milliseconds in this case ) elapsed since "1970-01-01", which is exactly what epoch time is. To further this we can get the milliseconds elapsed in 1 hour as a portion of this date by obtaining the modulus:
date % ( 1000*60*60 )
1401169
So with 1000 milliseconds in a second, 60 seconds in a minute and 60 minutes in an hour we obtain this result. All that is left now is to subtract that value from the original date value:
date - date % ( 1000*60*60 )
1400637600000
And that value is now the boundary of the hour in the day and suitable for grouping as well as being the desired result you want in your output. Just for reference, here are the string formatted values of the initial and converted timestamp:
Initial: Wed, 21 May 2014 02:23:21 GMT
Converted: Wed, 21 May 2014 02:00:00 GMT
So basically you are doing the same thing for grouping that you are using the date operators for, but actually getting the output format you want without requiring additional conversion.
Just a cool trick that if I'm right about how you got here should be useful to you and if not, then at least should be useful to others who might have arrived here in exactly that way.