We Keep Coding

How do I fix this botched transition resulting from missing data in my D3.js line chart?

I have a D3.js line chart that updates when the user chooses a new country. Overall I'm satisfied with the way my chart updates and it looks very clean when two conditions are met:
There is no missing data for either country
Country one has no missing data but country two has missing data.
Here is the problem scenario: country one has missing data but country two has no missing data.
If country one has missing data on the left, when the chart updates, the new segments of the line get appended on the right side (usually on top of the existing segments of the line) and then gradually slide over to the left. It's a very ugly transition and words might not suffice so here are a few pictures.
Country One: Missing data on left
Transition to Country Two (which has no missing data)
Country Two: No missing data
I have a hunch about the source of the problem and how I might be able to fix it.
A potential solution?
This line chart by Mike Bostock is interesting. It has missing data but it uses linear interpolation to fill in the gaps. Would I be able to solve my problem by filling in the gaps like this? I tried implementing this solution but I just can't get the second line for the missing data to appear.
https://observablehq.com/#d3/line-with-missing-data
The Code for my Line Chart
I have two separate functions for my line chart. The render function initializes the chart and then I have to use the update function to transition to new data. I know that's a bit weird but I'm relatively inexperienced with D3.js and I can't get it to work with one function. For the That is another potential problem in and of itself, but I don't think it's causing the botched transition, is it?
For the sake of brevity, I only included the parts where the line gets appended and where it gets updated.
function render(url){
d3.json(url).then(function (data) {
data.forEach(function (d) {
d.Year = +d.Year;
d.Value = +d.Value / 1000;
});
...
...
omitted code
...
...
svg.append("path")
.datum(data)
.attr('id','mainline')
.attr("fill", "none")
.attr("stroke", "steelblue")
.attr("stroke-width", 3)
.attr("d", d3.line()
.defined(d => !isNaN(d.Value))
.x(function (d) {
return x(d.Year)
})
.y(function (d) {
return y(d.Value)
})
);
}
function update(url) {
//Reading in new data and updating values
d3.json(url).then(function (data) {
data.forEach(function (d) {
d.Year = +d.Year;
d.Value = +d.Value / 1000;
});
...
...
omitted code
...
...
var lines = svg.selectAll("#mainline").datum(data).attr("class","line");
lines.transition().ease(d3.easeLinear).duration(1500)
.attr("d", d3.line()
.defined(d => !isNaN(d.Value))
.x(function (d) {
return x(d.Year)
})
.y(function (d) {
return y(d.Value)
})
);
Any help would be greatly appreciated. Thanks!

getting and using data from CSV file d3 v5

I am trying to create a map visualization using d3.
I have gotten the map to work and am trying to add points on the map.
Depending on some other data in the CSV file (inspection results) I want certain points to have a different color.
I can add the points using this code, but I cannot get the colors to come out correctly (there should be red ones, but all of them are green). I think it is a problem with how I'm trying to get the data out, or just a problem with my JavaScript in general. Any help would be appreciated. Thank you!
function colorr(d) {
if (d == 0) {
return "red";
} else {
return "green";
}
}
var dataset = []
// load data about food
// https://stackoverflow.com/questions/47821332/plot-points-in-map-d3-javascript
// https://groups.google.com/forum/#!topic/d3-js/AVEa7nPCFAk
// https://stackoverflow.com/questions/10805184/show-data-on-mouseover-of-circle
d3.csv('data6.csv').then( function(data) {
// don't know if this actually does anything...
dataset=data.map(function(d) { return [+d["InspectionScore"],+d["Longitude"],+d["Latitude"]];});
g.selectAll('circle')
.data(data)
.enter()
.append('circle')
.attr("cx",function(d) { return projection([d.Longitude,d.Latitude])[0]; }).merge(g)
.attr("cy",function(d) { return projection([d.Longitude,d.Latitude])[1]; }).merge(g)
.attr("r", .4)
.attr("fill", d3.color(colorr( function(d) { return d.InspectionScore } ) ));
});

This can be resolved by changing the last line to:
.attr("fill", d => d3.color(colorr(d.InspectionScore))));
The reason this works is that d3's attr allows you to take the attribute value from a function. In this case you want to transform the data element to either red or blue. This is what the arrow function does in the above code. It is equivalent to :
.attr("fill", function(d) {
return d3.color(colorr(d.InspectionScore));
})
To get a deeper understanding of how D3 works, you can check the tutorials.

d3.js - Multiple line chart doesn't update circles

I'm working on a d3 chart (Multiple line chart).
I'm trying to represent a stock prediction, so basically the chart contains two lines: stock values line and an other one for my prediction.
The prediction is monthly, all days of month are represented in the chart.
In order to choose the month I have added a dropdown menu.
I appended a circle on each daily data, and works well for the first time. When user tries to change the month, the old circles are not updated, but the new ones are added.
Follow the code about circles:
topicEnter.append("g").selectAll("circle")
.data(function(d){return d.values})
.enter()
.append("circle")
.attr("r", 5)
.attr("cx", function(dd){return x(dd.date)})
.attr("cy", function(dd){return y(dd.probability)})
.attr("fill", "none")
.attr("stroke", "black");
I have done a fiddle to understand better the situation and in order to show code.
What am I missing here? Why don't the circles update themself with the lines?

To solve the issue about circles not updating you can do the following:
function update(topics) {
// Calculate min and max values with arrow functions
const minValue = d3.min(topics, t => d3.min(t.values, v => v.probability));
const maxValue = d3.max(topics, t => d3.max(t.values, v => v.probability));
y.domain([minValue, maxValue]);
x2.domain(x.domain());
y2.domain(y.domain());
// update axes
d3.transition(svg).select('.y.axis').call(yAxis);
d3.transition(svg).select('.x.axis').call(xAxis);
// Update context
var contextUpdate = context.selectAll(".topic").data(topics);
contextUpdate.exit().remove();
contextUpdate.select('path')
.transition().duration(600)
.call(drawCtxPath);
contextUpdate.enter().append('g') // append new topics
.attr('class', 'topic')
.append('path').call(drawCtxPath);
// New data join
var focusUpdate = focus.selectAll('.topic').data(topics);
// Remove extra topics not found in data
focusUpdate.exit().remove(); //remove topics
// Update paths
focusUpdate.select('path')
.transition().duration(600)
.call(drawPath)
// Update circles
var circlesUpdate = focusUpdate
.selectAll('.topic-circle')
.data(d => d.values);
circlesUpdate.exit().remove();
circlesUpdate.transition().duration(600).call(drawCircle);
circlesUpdate.enter().append('circle').call(drawCircle);
// Add new topics
var newTopics = focusUpdate.enter().append('g') // append new topics
.attr('class', 'topic');
// Add new paths
newTopics.append('path').call(drawPath)
// Add new circles
newTopics.selectAll('.topic-circle')
.data(d => d.values)
.enter()
.append('circle')
.call(drawCircle);
}
With these helper functions to reduce code duplication:
function drawCtxPath(path) {
path.attr("d", d => line2(d.values))
.style("stroke", d => color(d.name));
}
function drawPath(path) {
path.attr("d", d => line(d.values))
.attr('clip-path', 'url(#clip)')
.style("stroke", d => color(d.name));
}
function drawCircle(circle) {
circle.attr('class', 'topic-circle')
.attr('clip-path', 'url(#clip)')
.attr("r", d => 5)
.attr("cx", d => x(d.date))
.attr("cy", d => y(d.probability))
.attr("fill", "none")
.attr("stroke", "black");
}
I think there are some additional issues in your code, when you select the same month twice you get an error, we can fix that by doing the following:
d3.select('#month_chart').on("change", function() {
// Get selected value of the select
var month = this.options[this.selectedIndex].value;
// Since you have hardcoded data we need to return a new array
// This is why if you select the same month twice your code breaks
// since parseDate will fail since the data will be already parsed
// the second time
var monthData = get_monthly_data(month).map(d => {
return {
date: parseDate(d.date),
predicted_bool: d.predicted_bool,
target: d.target
};
});
// Lets use arrow functions!
var keys = d3.keys(monthData[0]).filter(k => k !== 'date');
color.domain(keys);
// More arrow functions!
var topics = keys.map(key => {
return {
name: key,
values: monthData.map(d => {
return {
date: d.date,
probability: +d[key]
};
})
};
});
x.domain(d3.extent(monthData, d => d.date));
update(topics);
});
// A good ol' switch
function get_monthly_data(month) {
switch (month) {
case 'gennaio':
return data_1;
case 'febbraio':
return data_2;
case 'marzo':
return data_3;
default:
return data_1;
}
}
Working jsfiddle:
https://jsfiddle.net/g699scgt/37/

The problem is your update cycle, but there are a good number of examples of the enter, update, exit process in d3.
But essentially:
You append a new g element for each batch of circles, which means you have an empty selection (no circles are in that g yet) each time and each data point is appended (and none are removed). You don't need this extra append. Take a look at the DOM structure on each append in your existing code.
Your enter() selection returns new elements - not modified elements. So if your total number of elements remains the same you will have an empty enter() selection. You'll want to update existing elements separately (alternatively, remove them all and append them all every time).
You'll want something closer to this:
// set the data
circles = topic.selectAll("circle")
.data(function(d){return d.values});
// update existing circles
circles.attr("cx", function(dd){return x(dd.date)})
.attr("cy", function(dd){return y(dd.probability)});
// add new circles
circles.enter()
.append("circle")
.attr("r", 5)
.attr("cx", function(dd){return x(dd.date)})
.attr("cy", function(dd){return y(dd.probability)})
.attr("fill", "none")
.attr("stroke", "black");
// remove excess circles
circles.exit().remove();
You'll likely also want to revise the lines that append the lines to reflect the enter, update, exit cycle in d3.

enter() and exit() with D3 nested selections

I have a follow-up question to this question that I closed too hastily.
How/where would I put a .exit() or .transition() for the answers? In my original code, I'm saving the main chart in a variable (questions_chart), so I can say something like questions_chart.exit().remove(). However, if I put .exit().remove() after the .text() (last line in the code in Peter's answer), I get an error saying that object array has no method 'exit'

I have not tested this but will give it a shot...you would need to preserve the variable binding the data...like so:
var divs = d3.select("body").selectAll("div")
.data(data.questions);
divs
.enter().append("div") // this creates the question divs
.text(function(d) { return d.value; })
.selectAll("div")
.data(function(d) { return d.answers; })
.enter().append("div") // this creates the nested answer divs
.text(function(d) { return d.value; });
divs.exit().remove();
I am assuming you don't need to remove just the divs that are answers. Hope this helps.
UPDATE: giving it a shot at dealing with questions and answers...
var divs = d3.select("body").selectAll("div")
.data(data.questions);
var questionDivs = divs
.enter().append("div") // this creates the question divs
.text(function(d) { return d.value; });
divs.exit().remove();
var answerDivs = questionDivs
.selectAll("div")
.data(function(d) { return d.answers; });
answerDivs
.enter().append("div") // this creates the nested answer divs
.text(function(d) { return d.value; });
answerDivs.exit().remove();

D3js sorting issue on transition applied

I have a dataset already binded to svg:g via a d.id
var categorized = g1.selectAll("g.node")
.data(dataset, function(d){return d.id})
.classed('filtered', false);
categorized.enter()
.append("g")
.attr("class", "node")
...
I use a function to order it from a data value like this:
var sorted = dataset
.filter(function(d) { return d.notation[3].value >=50 } )
.sort(function(a, b) { return d3.descending(a.notation[3].value,
b.notation[3].value) });
It returns the correct order when I console.log it
var filtered = g1.selectAll("g.node")
.data(sorted, function(d) {return d.id})
.classed('filtered', true);
Still in the right order if I console.log it,
but if I apply a delay it reverses the result order
scored.transition()
.delay(500).duration(1000)
.attr("id", function(d) {
console.log(d.id);
});
but keeps it well sorted if I remove the delay.
My question : am I doing something in a bad way?

I think you're observing that d3.js generally uses the "optimized" for loop that iterates in reverse (see Are loops really faster in reverse? among other references).
Would it work to simply reverse your selection? I'm not sure what you're transitioning such that you need the tween steps to be applied in a certain order.