We Keep Coding

Why is my reducer behaving differently between the first filter and subsequent filters applied in dc.js?

I'm working on a data visualization that has an odd little bug:
It's a little tricky to see, but essentially, when I click on a point in the line chart, that point corresponds to a specific issue of a magazine. The choropleth updates to reflect geodata for that issue, but, critically, the geodata is for a sampled period that corresponds to the issue. Essentially, the choropleth will look the same for any issue between January-June or July-December of a given year.
As you can see, I have a key called Sampled Issue Date (for Geodata), and the value should be the date of the issue for which the geodata is based on (basically, they would get geographical distribution for one specific issue and call it representative of ALL data in a six month period.) Yet, when I initially click on an issue, I'm always getting the last sampled date in my data. All of the geodata is correct, and, annoyingly, all subsequent clicks display the correct information. So it's only that first click (after refreshing the page OR clearing an issue) that I have a problem.
Honestly, my code is a nightmare right now because I'm focused on debugging, but you can see my reducer for the remove function on GitHub which is also copy/pasted below:
// Reducer function for raw geodata
function geoReducerAdd(p, v) {
// console.log(p.sampled_issue_date, v.sampled_issue_date, state.periodEnding, state.periodStart)
++p.count
p.sampled_mail_subscriptions += v.sampled_mail_subscriptions
p.sampled_single_copy_sales += v.sampled_single_copy_sales
p.sampled_total_sales += v.sampled_total_sales
p.state_population = v.state_population // only valid for population viz
p.sampled_issue_date = v.sampled_issue_date
return p
}
function geoReducerRemove(p, v) {
const currDate = new Date(v.sampled_issue_date)
// if(currDate.getFullYear() === 1921) {
// console.log(currDate)
// }
currDate <= state.periodEnding && currDate >= state.periodStart ? console.log(v.sampled_issue_date, p.sampled_issue_date) : null
const dateToRender = currDate <= state.periodEnding && currDate >= state.periodStart ? v.sampled_issue_date : p.sampled_issue_date
--p.count
p.sampled_mail_subscriptions -= v.sampled_mail_subscriptions
p.sampled_single_copy_sales -= v.sampled_single_copy_sales
p.sampled_total_sales -= v.sampled_total_sales
p.state_population = v.state_population // only valid for population viz
p.sampled_issue_date = dateToRender
return p
}
// generic georeducer
function geoReducerDefault() {
return {
count: 0,
sampled_mail_subscriptions: 0,
sampled_single_copy_sales: 0,
sampled_total_sales: 0,
state_population: 0,
sampled_issue_date: ""
}
}
The problem could be somewhere else, but I don't think it's a crossfilter issue (I'm not running into the "two groups from the same dimension" problem for sure) and adding additional logic to the add reducer makes things even less predictable (understandably - I don't ever really need to render the sample date for all values anyway.) The point of this is that I'm completely lost about where the flaw in my logic is, and I'd love some help!
EDIT: Note that the reducers are for the reduce method on a dc.js dimension, not the native javascript reducer! :D

Two crossfilters! Always fun to see that... but it can be tricky because nothing in dc.js directly supports that, except for the chart registry. You're on your own for filtering between different chart groups, and it can be tricky to map between data sets with different time resolutions and so on.
The problem
As I understand your app, when a date is selected in the line chart, the choropleth and accompanying text should have exactly one row from the geodata dataset selected per state.
The essential problem is that Crossfilter is not great at telling you which rows are in any given bin. So even though there's just one row selected, you don't know what it is!
This is the same problem that makes minimum, maximum, and median reductions surprisingly complicated. You often end up building new data structures to capture what crossfilter throws away in the name of efficiency.
A general solution
I'll go with a general solution that's more that you need, but can be helpful in similar situations. The only alternative that I know is to go completely outside crossfilter and look in the original dataset. That's fine too, and maybe more efficient. But it can be buggy and it's nice to work within the system.
So let's keep track of which dates we've seen per bin. When we start out, every bin will have all the dates. Once a date is selected, there will be only one date (but not exactly the one that was selected, because of your two-crossfilter setup).
Instead of the sampled_issue_date stuff, we'll keep track of an object called date_counts now:
// Reducer function for raw geodata
function geoReducerAdd(p, v) {
// ...
const canonDate = new Date(v.sampled_issue_date).getTime()
p.date_counts[canonDate] = (p.date_counts[canonDate] || 0) + 1
return p
}
function geoReducerRemove(p, v) {
// ...
const canonDate = new Date(v.sampled_issue_date).getTime()
if(!--p.date_counts[canonDate])
delete p.date_counts[canonDate]
return p
}
// generic georeducer
function geoReducerDefault() {
return {
// ...
date_counts: {}
}
}
What does it do?
Line by line
const canonDate = new Date(v.sampled_issue_date).getTime()
Maybe this is paranoid, but this canonicalizes the input dates by converting them to the number of milliseconds since 1970. I'm sure you'd be safe using the string dates directly, but who knows there could be a space or a zero or something.
You can't index an object with a date object, you have to convert it to an integer.
p.date_counts[canonDate] = (p.date_counts[canonDate] || 0) + 1
When we add a row, we'll check if we currently have a count for the row's date. If so, we'll use the count we have. Otherwise we'll default to zero. Then we'll add one.
if(!--p.date_counts[canonDate])
delete p.date_counts[canonDate]
When we remove a row, we know that we have a count for the date for that row (because crossfilter won't tell us it's removing the row unless it was added earlier). So we can go ahead and decrement the count. Then if it hits zero we can remove the entry.
Like I said, it's overkill. In your case, the count will only go to 1 and then drop to 0. But it's not much more expensive to this rather than just keep
Rendering the side panel
When we render the side panel, there should only be one date left in date_counts for that selected item.
console.assert(Object.keys(date_counts).length === 1) // only one entry
console.assert(Object.entries(date_counts)[0][1] === 1) // with count 1
document.getElementById('geo-issue-date').textContent = new Date(+Object.keys(date_counts)[0]).format('mmm dd, yyyy')
Usability notes
From a usability perspective, I would recommend not to filter(null) on mouseleave, or if you really want to, then put it on a timeout which gets cancelled when you see a mouseenter. One should be able to "scrub" over the line chart and see the changes over time in the choropleth without accidentally switching back to the unfiltered colors.
I also noticed (and filed) an issue because I noticed that dots to the right of the mouse pointer are shown, making them difficult to click. The reason is that the dots are overlapping, so only a little sliver of a crescent is hoverable. At least with my trackpad, the click causes the pointer to travel leftward. (I can see the date go back a week in the tooltip and then return.) It's not as much of a problem when you're zoomed in.

Get max and min values of a facet in Algolia

I'm trying to use the algolia javascript api to create a pretty simple page with some basic filters.
I'm also trying to implement a range slider similar to this one here made by algolia - https://community.algolia.com/instantsearch.js/
I need the min and max values because 'facets_stats' returns the min and max for the current query (but I need the absolute minimum and absolute maximum so I can keep them constant at the edges of the slider)
After digging through the code for that instantsearch widget, I see they are using the algoliasearch helper to get the values (I think) but after playing around with the helper for a bit, I wasn't able to get the values I needed.
I've almost completed my project so it would be a huge waste of time to rewrite it using those components but I can't seem to figure out how to get the min and max values for a facet.
Is there any simple way to get the maximum/minimum value for a numeric facet using a single api call?

What you can do is performing an initial empty (blank) query to retrieve the facets_stats and therefore the underlying min/max values for all the records.
index.search('' /* match all */, {facets: 'myfacet', hitsPerPage: 0}).then(function(content) {
console.log(content.facets_stats.myfacet.min, content.facets_stats.myfacet.max);
});
You could then save those values and setup your slider accordingly.

Adding dynamic mean to time series chart

I will try to explain my problem as much accurate as possible. I am looking for a javascript chart library filling the two following conditions:
From an ajax request retrieving time series,
display dynamically data when changing the time window.
such as it is perfectly done on highstocks: http://www.highcharts.com/stock/demo/basic-line
And
plot an horizontal line corresponding to the mean,
changing when the user update the time window on the chart.
Actually it is possible to display an horizontal line. But they are fixed on the whole data and do not change accordingly when the time window is modified:
http://www.highcharts.com/stock/demo/yaxis-plotlines
I am quite new to the topic and would like to know if it'sp ossible to modify Highstock classes to have such a result. Or maybe some other js libraries exists?

Using a combination of the answer here:
Highchart, get total of visible series data after setExtremes
And an example of dynamic average using all visible series that I made for a previous question, here:
http://jsfiddle.net/jlbriggs/gweuLegq/
I put together this example, using the afterSetExtremes event, like this:
xAxis : {
events:{
afterSetExtremes:function() {
var ext = this.getExtremes();
getAverage(this.chart, ext.min, ext.max, show, width, avgColor, dashStyle);
}
}
},
Working example here:
http://jsfiddle.net/jlbriggs/c93543yL/
The idea is:
1) capture the afterSetExtremes event
2) get the resulting axis min and max
3) loop through the series data
4) if a point is between the min and max, increment the count, and add the
point's y value to the sum
5) calculate the average accordingly, check for existence of average series, if exists, update, if not, add
It could as easily use a plot line that you add/remove as needed instead of a series, but I like having it as a series so that it has a legend entry.

multi stage range validation

So I want to have essentially have two range validators, so there is two limits.
Ex. I have a text box that has a range validator of min. 1 and max 10 and if out of that range the value is not accepted but inside that range there is another range validator which is min. 3 and max 6 and if outside that range a messagebox pops up saying your value is out of operating range, do you still want to submit the value? It also needs to be at runtime.
Im not quite sure how to go about this, if anyone can help that would be awesome.

You might take a look at this post. You can use a CustomValidator to do what you need.
https://stackoverflow.com/questions/251842/asp-net-extended-range-validation

Selecting a new value in a dropdown list while remembering the old value in Jquery

I'm working on a webshop where customers can choose an amount of items to purchase via a drop down list. Each item have a point value and I calculate the total of their purchase via jQuery (onChange event):
function showPoints(itemPointValue, ddlAmount){
var customerPoints = parseInt($('p#points').html());
currentPoints = customerPoints - (ddlAmount * itemPointValue);
$('p#points').html(currentPoints);
}//showPoints
The only problem here is if they change their amount from say 5 to 4, another 4 * the point value gets taken off their "total points". Their total points ends up becoming totally inaccurate and they can even go below 0. I thought of using jquery.data to set an "oldValue" variable but IE does not support this. Any suggestions?

You could store the previous value on a custom attribute as stated above, and you could use the focus event to set the old value, something in the lines of this should work:
$('p#points').on('focus', function () {
$(this).attr('old-value', $(this).val();
});

Why not just recalculate the total point cost any time any dropdown is changed, and re-calculate currentPoints by subtracting the new total from customerPoints? This, to me, seems like a much cleaner solution than adding and subtracting values

How about, creating custom attribute on the element:
$('p#points').attr('old-value',5);
and retrieve it like this:
$('p#points').attr('old-value');

I solved this issue by using two events on my drop down lists. onClick to save the old data using jquery.data and onChange to check the old value against the new value. If the new value is smaller then the old value I adjust the point total accordingly.
HTML:
<select name="dropdown onClick="saveOld(this.value, this.name)" onchange="showPoints(points, this.value,this.name)">
JS/Jquery
function showPoints(pointCategory, ddlAmount, name){
old = jQuery.data(document.body,name);
var Points = parseInt($('p#points').html());
if(old > ddlAmount){
diff = old - ddlAmount;
currentPoints = Points + (diff * pointCategory);
}else{
currentPoints = Points - (ddlAmount * pointCategory);
}
$('p#points').html(currentPoints);
}//showPoints
function saveOld(oldAmount, name){
$(document.body).data(name,oldAmount);
}//saveOld
Thank you all for your answers!