I'm working on a crossfilter between a line chart and choropleth. I recently got a much better understanding of the reduce method in dc.js, so I want to pass through some more metadata about each data point to my line chart and my choropleth. This is working really well for the line chart, and I now have a tooltip showing lots of information about each point.
For my choropleth, however, the transition to using reduce instead of reduceSum has really messed up my data. For example:
The value getting passed to the tooltip isn't the data I would expect, and I have no idea where the calculation is coming from (it almost seems like it's from the SVG path, or even the color accessor?)
As I toggle between different renderings of the choropleth, my choropleth changes, but the value on the tooltip stays exactly the same
The initial render of the choropleth is showing a fully blue map, so it seems like the initial value might be incorrect, anyway.
I'm trying to understand how I can identify the data point that's coming from the group, use that to render the choropleth based on a specific value (total_sampled_sales) and then pass that data to the tooltip so that each state's value and metadata can be displayed.
Also, any tips for debugging this type of issue would be greatly appreciated. As you may be able to see from my console.logs, I'm having a hard time tracing the data through to the tooltip. This is presumably the problem block:
us1Chart.customUpdate = () => {
us1Chart.colorDomain(generateScale(returnGroup()))
us1Chart.group(returnGroup())
}
us1Chart.width(us1Width)
.height(us1Height)
.dimension(stateRegion)
.group(returnGroup())
.colors(d3.scaleQuantize().range(colorScales.blue))
.colorDomain(generateScale(returnGroup()))
.colorAccessor(d => {
// console.log('colorAccessor:', d)
return d ? d : 0
})
.overlayGeoJson(statesJson.features, "state", d => {
// console.log(`geojson`, d, d.properties.name)
return d.properties.name
})
.projection(d3.geoAlbersUsa()
.scale(Math.min(getWidth('us1-chart') * 2.5, getHeight('us1-chart') * 1.7))
.translate([getWidth('us1-chart') / 2.5, getHeight('us1-chart') / 2.5])
)
.valueAccessor(kv => {
// console.log(kv.value)
if (kv.value !== undefined) return kv.value
})
.renderTitle(false)
.on('pretransition', (chart) => {
chart.selectAll('path')
.call(mapTip)
.on('mouseover.mapTip', mapTip.show)
.on('mouseout.mapTip', mapTip.hide);
})
https://jsfiddle.net/ayyrickay/f1vLwhmq/19/
Note that the data's a little wonky because I removed half of the records just for size constraints
Because of the data binding with choropleths, I'm now using the data that's passed through (specifically, the US State that was selected) and then identifying the data point in the crossfilter group:
const selectedState = returnGroup().all().filter(item => item.key === d.properties.name)[0]
So I have a returnGroup method that selects the correct choropleth group (based on some app state), gets the list, and then I filter to see which item matches the name property passed to the tooltip. Because filter returns an array, I'm just being optimistic that it'll filter to one item and then use that item. Then I have access to the full data point, and can render it in the tooltip accordingly. Not elegant, but it works.
Related
I'm working on a data visualization that has an odd little bug:
It's a little tricky to see, but essentially, when I click on a point in the line chart, that point corresponds to a specific issue of a magazine. The choropleth updates to reflect geodata for that issue, but, critically, the geodata is for a sampled period that corresponds to the issue. Essentially, the choropleth will look the same for any issue between January-June or July-December of a given year.
As you can see, I have a key called Sampled Issue Date (for Geodata), and the value should be the date of the issue for which the geodata is based on (basically, they would get geographical distribution for one specific issue and call it representative of ALL data in a six month period.) Yet, when I initially click on an issue, I'm always getting the last sampled date in my data. All of the geodata is correct, and, annoyingly, all subsequent clicks display the correct information. So it's only that first click (after refreshing the page OR clearing an issue) that I have a problem.
Honestly, my code is a nightmare right now because I'm focused on debugging, but you can see my reducer for the remove function on GitHub which is also copy/pasted below:
// Reducer function for raw geodata
function geoReducerAdd(p, v) {
// console.log(p.sampled_issue_date, v.sampled_issue_date, state.periodEnding, state.periodStart)
++p.count
p.sampled_mail_subscriptions += v.sampled_mail_subscriptions
p.sampled_single_copy_sales += v.sampled_single_copy_sales
p.sampled_total_sales += v.sampled_total_sales
p.state_population = v.state_population // only valid for population viz
p.sampled_issue_date = v.sampled_issue_date
return p
}
function geoReducerRemove(p, v) {
const currDate = new Date(v.sampled_issue_date)
// if(currDate.getFullYear() === 1921) {
// console.log(currDate)
// }
currDate <= state.periodEnding && currDate >= state.periodStart ? console.log(v.sampled_issue_date, p.sampled_issue_date) : null
const dateToRender = currDate <= state.periodEnding && currDate >= state.periodStart ? v.sampled_issue_date : p.sampled_issue_date
--p.count
p.sampled_mail_subscriptions -= v.sampled_mail_subscriptions
p.sampled_single_copy_sales -= v.sampled_single_copy_sales
p.sampled_total_sales -= v.sampled_total_sales
p.state_population = v.state_population // only valid for population viz
p.sampled_issue_date = dateToRender
return p
}
// generic georeducer
function geoReducerDefault() {
return {
count: 0,
sampled_mail_subscriptions: 0,
sampled_single_copy_sales: 0,
sampled_total_sales: 0,
state_population: 0,
sampled_issue_date: ""
}
}
The problem could be somewhere else, but I don't think it's a crossfilter issue (I'm not running into the "two groups from the same dimension" problem for sure) and adding additional logic to the add reducer makes things even less predictable (understandably - I don't ever really need to render the sample date for all values anyway.) The point of this is that I'm completely lost about where the flaw in my logic is, and I'd love some help!
EDIT: Note that the reducers are for the reduce method on a dc.js dimension, not the native javascript reducer! :D
Two crossfilters! Always fun to see that... but it can be tricky because nothing in dc.js directly supports that, except for the chart registry. You're on your own for filtering between different chart groups, and it can be tricky to map between data sets with different time resolutions and so on.
The problem
As I understand your app, when a date is selected in the line chart, the choropleth and accompanying text should have exactly one row from the geodata dataset selected per state.
The essential problem is that Crossfilter is not great at telling you which rows are in any given bin. So even though there's just one row selected, you don't know what it is!
This is the same problem that makes minimum, maximum, and median reductions surprisingly complicated. You often end up building new data structures to capture what crossfilter throws away in the name of efficiency.
A general solution
I'll go with a general solution that's more that you need, but can be helpful in similar situations. The only alternative that I know is to go completely outside crossfilter and look in the original dataset. That's fine too, and maybe more efficient. But it can be buggy and it's nice to work within the system.
So let's keep track of which dates we've seen per bin. When we start out, every bin will have all the dates. Once a date is selected, there will be only one date (but not exactly the one that was selected, because of your two-crossfilter setup).
Instead of the sampled_issue_date stuff, we'll keep track of an object called date_counts now:
// Reducer function for raw geodata
function geoReducerAdd(p, v) {
// ...
const canonDate = new Date(v.sampled_issue_date).getTime()
p.date_counts[canonDate] = (p.date_counts[canonDate] || 0) + 1
return p
}
function geoReducerRemove(p, v) {
// ...
const canonDate = new Date(v.sampled_issue_date).getTime()
if(!--p.date_counts[canonDate])
delete p.date_counts[canonDate]
return p
}
// generic georeducer
function geoReducerDefault() {
return {
// ...
date_counts: {}
}
}
What does it do?
Line by line
const canonDate = new Date(v.sampled_issue_date).getTime()
Maybe this is paranoid, but this canonicalizes the input dates by converting them to the number of milliseconds since 1970. I'm sure you'd be safe using the string dates directly, but who knows there could be a space or a zero or something.
You can't index an object with a date object, you have to convert it to an integer.
p.date_counts[canonDate] = (p.date_counts[canonDate] || 0) + 1
When we add a row, we'll check if we currently have a count for the row's date. If so, we'll use the count we have. Otherwise we'll default to zero. Then we'll add one.
if(!--p.date_counts[canonDate])
delete p.date_counts[canonDate]
When we remove a row, we know that we have a count for the date for that row (because crossfilter won't tell us it's removing the row unless it was added earlier). So we can go ahead and decrement the count. Then if it hits zero we can remove the entry.
Like I said, it's overkill. In your case, the count will only go to 1 and then drop to 0. But it's not much more expensive to this rather than just keep
Rendering the side panel
When we render the side panel, there should only be one date left in date_counts for that selected item.
console.assert(Object.keys(date_counts).length === 1) // only one entry
console.assert(Object.entries(date_counts)[0][1] === 1) // with count 1
document.getElementById('geo-issue-date').textContent = new Date(+Object.keys(date_counts)[0]).format('mmm dd, yyyy')
Usability notes
From a usability perspective, I would recommend not to filter(null) on mouseleave, or if you really want to, then put it on a timeout which gets cancelled when you see a mouseenter. One should be able to "scrub" over the line chart and see the changes over time in the choropleth without accidentally switching back to the unfiltered colors.
I also noticed (and filed) an issue because I noticed that dots to the right of the mouse pointer are shown, making them difficult to click. The reason is that the dots are overlapping, so only a little sliver of a crescent is hoverable. At least with my trackpad, the click causes the pointer to travel leftward. (I can see the date go back a week in the tooltip and then return.) It's not as much of a problem when you're zoomed in.
I have data which updates every 10 seconds and I would like to check that all the data is valid before progressing with updates. I am currently getting false data intermittently which occurs as a negative number in one of the values. If one of the objects has a negative value then I don't trust the whole set and don't want to update any elements.
Ideally I don't want to update some items and then bail once the incorrect value occurs, but rather, determine if the whole set is good before updating anything
I'm not sure how d3 can manage this but I've tried with this and it seems to work. But it doesn't seem particularly in keeping with the elegance of D3 so I think there's probably a correct and better way to do it. But maybe not?!
var dataValid = true;
abcItems.each(function (d, i) {
if (0 > dd.Number1 - dd.Number2) dataValid = false;
});
if (dataValid) {
abcItems.each(function (d, i) {
// updating elements here
});
} else {
console.log("negative value occurred");
}
Is there a better way to manage this through D3?
A little bit more context:
The data (JSON provided via a RESTful API) and visualisation (a bar chart) are updating every 10 seconds. The glitch in the API results in incorrect data once every hour or so at the most (sometimes it doesn't happen all day). The effect of the glitch is that the bars all change dramatically whereas the data should only change by ones or twos each iteration. In the next fetch of data 10 seconds later the data is fine and the visualisation comes right.
The data itself is always "well-formed" it's just that the values provided are incorrect. Therefore even during the glitch it is safe to bind the data to elements.
What I want to do, is skip the entire iteration and update phase if the data contains one of these negative values.
Perhaps also worth noting is that the items in the data are always the same, that is to say the only "enter" phase that occurs is on page load and there are no items that exit (though I do include these operations to capture any unexpected fluctuations in the data). The values for these items do change though.
Looking at your code it seams you already have bound the dataset to your DOM elements abcItems.each(...).
Why not bail out of the update function when the data is not valid.
d3.json("bar-tooltip.json", function(dataset) {
if (!dataset.every(d => d.Number2 <= d.Number1)) return;
// do the update of the graph
});
The example assumes you call d3.json() froma function that is called every update interval, but you can use a different update method.
JSFiddle here: https://jsfiddle.net/ayyrickay/k1crg7xu/47/
My code is a bit of a mess right now, but essentially, I have two choropleths, and I want to render a multiline chart based on the choropleth data - I just have no idea how to wrangle the data to make it work.
The line chart is be a composite line chart. One line would be New Yorker circulation data, the other would be Saturday Evening Post circulation data. The y axis is issue_circulation, the x axis is actual_issue_date
In the current implementation I’ve set up two crossfilters (one for each data set) and I’m creating a dimension for the choropleth and one for the line chart. The choropleths render properly, but I’ve yet to get the line charts to render. I can’t tell if its because of the format of my data ({key: date, value: y-axis-value}) or if my implementation of crossfilter is just too janky. I'm trying to understand based on other StackOverflow questions, but nothing I've tried seems to work (this includes prefiltering the data like I'm doing now, creating two different crossfilters and separate dimensions, trying to be meticulous apart parsing dates, etc.)
When you're using a time scale for the X axis, the keys of your group should be Date objects. So it won't work to format the dates as strings when creating the dimensions & groups; instead just use raw Date objects.
Since Dates are slow, I suggest doing this as a data preprocessing step:
data.forEach(function(d) {
d.actual_issue_date = new Date(d.actual_issue_date);
})
Then your dimension key functions just extract the date object:
const dimension1 = title1Circulation.dimension(d => d.actual_issue_date)
const lineChartYear1 = title1Circulation.dimension(d => d.actual_issue_date)
const lineChartYear2 = title2Circulation.dimension(d => d.actual_issue_date)
This ends up looking kind of messy, because the Saturday Evening Post data fluctuates a lot by week:
Zoomed in:
Assuming this isn't a data cleaning problem (kind of looks like it?), one way to improve the display would be to aggregate by month:
const circulationGroup1 = lineChartYear1.group(d => d3.timeMonth(d)).reduceSum(d => d.issue_circulation)
const circulationGroup2 = lineChartYear2.group(d => d3.timeMonth(d)).reduceSum(d => d.issue_circulation)
composite
.xUnits(d3.timeMonths)
This rounds the group key down to the beginning of each month, adding together all the values for each month.
Still kind of messy, but better:
Welp, you still have some work to do, but anyway, that's why the data was not displaying!
Fork of your fiddle.
In JavaScript, I am unable to figure out how to use the data object in the function below to get the position of the clicked-on data point (e.g. third data point in the series).
Using chartsNew.js, a popular fork of charts.js, this code shows the value of the datapoint at the mouse click:
function fnMouseDownLeft(event,ctx,config,data,other){
if(other != null){
window.alert("["+ data.labels[other.v12]+", "+data.datasets[other.v11].data[other.v12] +"]");
}else{
window.alert("You click Left but not on a data");
}
}
How do I display the clicked element's position in the data series?
jsFiddle Example
This seems the most promising but I don't understand the relationship between data.datasets, data.datasets[other] and data.datasets[other].data[other]
window.alert("Position: " + data.datasets[other.v11].data[other.v3] );
Here is documentation:
https://github.com/FVANCOP/ChartNew.js/wiki/100_095_Mouse_Actions
https://github.com/FVANCOP/ChartNew.js/wiki/120_Template_variables#inGraphDataTmpl-annotateLabel
My confusion: v12 (for a line chart) should display the position of data in the series (which is what I want) but instead it displays the x-axis value for that datapoint.
other.v12 seems to do the trick
alert(other.v12);
http://jsfiddle.net/wesn0xm5/1/
Not sure why it's not giving you the series, it does for me.
I'm having a bit of a problem.
I'm using D3 to make a pie chart for an application I'm building. I basically it have it working, but I'm annoyed by one aspect of the chart. I've adapted the chart from here: http://jsfiddle.net/vfkSs/1/ to work with my application.
The data is passed in here:
data = data ? data : { "slice1": Math.floor((Math.random()*10)+1),
"slice2": Math.floor((Math.random()*10)+1),
"slice3": Math.floor((Math.random()*10)+1),
"slice4": Math.floor((Math.random()*10)+1) };
But somewhere in this file these slices are being ordered by value, which is not what I want.
The issue with this chart is that when it updates it adjusts all pieces of the chart to keep them in ascending order. For example the largest portion of the pie is always on the right, and smallest on the left. I would like these to remain in the order they are when the data is passed in.
This is buried a bit in the documentation.
pie.sort([comparator])
If comparator is specified, sets the sort order of data for the layout
using the specified comparator function. Pass null to disable sorting.
(bolding mine)
So, modify your .pie call to:
var cv_pie = d3.layout.pie().sort(null).value(function (d) { return d.value });
Updated fiddle.