We Keep Coding

D3js v3 to v5 upgrade: unable to convert strings to integers

I had a D3js code which produces bar graphs and works fine with version 3.x. I wanted to upgrade the code to version 5 in the interest of being updated. Upon doing so I was able to correct a number of syntax updates such as scaleLinear, scaleBand, etc. The data is imported via tsv. The code is able to show the graph on the page with the correct x axis widths for the bars. However, the yAxis bars go out of bounds and the scale on the y-axis is very short. For example, the data shows the maximum value of the data to be 30000, but the yaxis is only from 0-90. Upon further investigation the d.RFU values from which the y data is generated seems to be not converted from string to integers. In the v3 code, I had a function at the end which converted the type of d.RFU to integer using the unary operator
d.RFU = +d.RFU
However, it seems to be not working in v5. Could this be due to the promises implementation in replacement of the asynchronous code?
Any solutions on how to fix this in version 5?
Please let me know if you need any more information and forgive me if I have missed out anything as I am new to programming and this website. Any help is appreciated.
Here is parts of the code which I have right now:
//set dimensions of SVG and margins
var margin = { top: 30, right: 100, bottom: 50, left: 100, },
width = divWidth - margin.left - margin.right,
height = 250 - margin.top - margin.bottom,
x = d3.scaleBand()
.range([0, width - 20], 0.1),
y = d3.scaleLinear()
.range([height,0]);
//setup the axis
var xAxis = d3.axisBottom(x);
var yAxis = d3.axisLeft(y);
var svg = d3.select("#bargraphID")
.append("svg")
.attr("width", width + margin.left + margin.right - 100)
.attr("height", height + margin.top + margin.bottom - 10)
.append("g")
.attr("transform", "translate (" + margin.left + "," + margin.top + ")");
d3.tsv(filename).then(function(data) {
// get x values from the document id
x.domain(data.map(function(d) {
return d.ID;
}));
yMax = d3.max(data, function(d) {
return d.RFU;
});
// get the y values from the document RFU tab
y.domain([0, yMax]);
//create the x-axis
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate (0, " + height + ")")
.call(xAxis)
.selectAll("text")
.style("text-anchor", "middle")
.attr("dx", "0em")
.attr("dy", "-0.55em")
.attr("y", 30)
.attr("class", "x-axisticks");
//create the y-axis
svg.append("g")
.attr("class", "y axis")
.call(yAxis);
//add the data as bars
var bar = svg.selectAll("bar")
.data(data)
.enter()
.append("rect")
.style("fill", barColor)
.attr("fill-opacity", "0.3")
.attr("x", function(d) {
return x(d.ID);
})
.attr("width", x.bandwidth())
//set initial coords for bars for animation.
.attr("y", height)
.attr("height", 0)
//animate bars to final heights
.transition()
.duration(700)
.attr("y", function(d) {
return y(d.RFU);
})
.attr("height", function(d) {
return height - y(d.RFU);
})
.attr("fill-opacity", "1.0")
.attr("class", "y-data");
});
//convert RFU to integers
function type(d) {
d.RFU = +d.RFU;
return d;
}

Just like with the old v3 and v4 versions, you have to pass the row conversion function to d3.tsv in D3 v5:
d3.tsv(filename, type)
Then, use the promise with then. Have in mind that d3.tsv always return strings (be it D3 v3, v4 or v5), because:
If a row conversion function is not specified, field values are strings.
Here is the demo with fake data:
var tsv = URL.createObjectURL(new Blob([
`name RFU
foo 12
bar 42
baz 17`
]));
d3.tsv(tsv, type).then(function(data) {
console.log(data)
})
function type(d) {
d.RFU = +d.RFU;
return d;
}
<script src="https://d3js.org/d3.v5.min.js"></script>
PS: Since SO snippet may have a problem loading that blob in some browsers, here is the same code in JSFiddle, check the console: https://jsfiddle.net/kv0ea0x2/

CSV File opened using JS not showing the updated version of the file when replaced by another one with the same name. d3.csv

Before everything is said, I've not much knowledge about using JS and especially not d3.csv. These are all code taken from a template from the internet.
Altho, I am pretty sure the problem lies within the file being saved in the Cache.
But for specificity's sake, I'm going to post the code.
On our page is a button that opens a table that takes it's values from the data in a CSV file.
This is the code in the HTML, the function which calls the script to view the table.
<script>
$(".bar-tab").click(function()
{
$.getScript("js/d3-bar.js");
})
</script>
This is the "d3-bar.js" script itself.
!(function (d3) {
$("barcontent").empty();
var margin = {top: 20, right: 30, bottom: 30, left: 40},
width = 960 - margin.left - margin.right,
height = 500 - margin.top - margin.bottom;
var x = d3.scale.ordinal()
.rangeRoundBands([0, width], .1);
var y = d3.scale.linear()
.range([height, 0]);
var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom");
var yAxis = d3.svg.axis()
.scale(y)
.orient("left");
var chart = d3.select(".chart")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
d3.csv("data/bar-data.csv", type, function(error, data) {
x.domain(data.map(function(d) { return d.name; }));
y.domain([0, d3.max(data, function(d) { return d.value; })]);
chart.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);
chart.append("g")
.attr("class", "y axis")
.call(yAxis);
chart.selectAll(".bar")
.data(data)
.enter().append("rect")
.attr("class", "bar")
.attr("x", function(d) { return x(d.name); })
.attr("y", function(d) { return y(d.value); })
.attr("height", function(d) { return height - y(d.value); })
.attr("width", x.rangeBand());
});
function type(d) {
d.value = +d.value; // coerce to number
return d;
}
})(d3);
The problem with this is that when we replace the "bar-data.csv" file with another one, for we need to change it from time to time, it doesn't change.
It changes when we manually clear the cache. Which brings us to what we think is the reason that causing the problem. Is there a way to bypass this? For the browser to not check the cache for the file we need?

It's caching working like it's supposed to.
A quick way to do this is to just "bust the cache". This is normally done by adding useless query parameter to the url with a value that's always different on each request. Timestamps are good values for this. The browser will think the url is new, and it will request a fresh copy of it instead of the one it cached.
d3.csv(`data/bar-data.csv?t=${Date.now()}`, type, function(error, data) {
Alternatively, if you have access to server-side configuration, you can probably tell the server to tell the browser to not cache certain paths served. This would depend on the server software you're using though, so read their respective manuals.

This is a common problem. You can work around this by adding a random GET request after the file type like "data/bar-data.csv?version=123"

Single Stacked Bar Graph Data Setup?

I'm trying to make a singular column for a bar graph in d3.js, the purpose of which is to bar-graph the coefficients of the other line graph in my program. I'm familiar with how they are made when the data is in .csv format, but in this case right now I'm trying to make it from three variables. The three variables are:
var xtwo;
var xone;
var xzero;
which have values put into them in a later part. I've built a skeleton based on what I know and have seen, which is right here:
//Bar Graph
var barmargin = {
top: 20,
right: 20,
bottom: 30,
left: 60
},
barwidth = 500 - barmargin.left - barmargin.right,
barheight = 350 - barmargin.top - barmargin.bottom;
//X scale
var barx = d3.scale.ordinal()
.rangeRoundBands([0, barwidth], .1);
//Y scale
var bary = d3.scale.linear()
.rangeRound([barheight, 0]);
//bar graph colors
var color = d3.scale.ordinal()
.range(["#FF5C33", "#F48C00", "#FFFF5C"]);
// Use X scale to set a bottom axis
var barxAxis = d3.svg.axis()
.scale(barx)
.orient("bottom");
// Same for y
var baryAxis = d3.svg.axis()
.scale(bary)
.orient("left")
.tickFormat(d3.format(".2s"));
// Addchart to the #chart div
var svg = d3.select("#chart").append("svg")
.attr("width", barwidth + barmargin.left + barmargin.right)
.attr("height", barheight + barmargin.top + barmargin.bottom)
.append("g")
.attr("transform", "translate(" + barmargin.left + "," + barmargin.top + ")");
//Where data sorting happens normally
var bardata.data([xzero, xone, xtwo]);
//Y domain is from zero to 5
y.domain([0, 5]);
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + barheight + ")")
.call(barxAxis);
svg.append("g")
.attr("class", "y axis")
.call(baryAxis);
bardata.selectAll("rect")
.data(function(d) {
return d.types;
})
.enter().append("rect")
.attr("width", barx.rangeBand())
.attr("y", function(d) {
return bary(d.y1);
})
.attr("height", function(d) {
return bary(d.y0) - bary(d.y1);
})
.style("fill", function(d) {
return color(d.name);
});
but I can't really figure out how to make it work correctly. I thought that I could manually make the .data but it seems to not be working that way.
Full code if necessary: https://jsfiddle.net/tqj5maza/1/

Broadly speaking: you want to create three bars, sat on top of each other, from three different values. The three values will be enough to scale the bars, but they in themselves won't be enough to position the bars- each bar needs to be offset by the size of the bars that have gone before.
d3 can only read the values that are already in the data you send it- you can't really access the previous values as you go, as each datum is bound to a separate element. Thus, what you need to do is to create some new data, which has all the numbers required to display it.
Here's one way that you might do that:
var canvas = d3.select("#canvas").append("svg").attr({width: 400, height: 400})
var values = [50, 90, 30]
var colours = ['#FA0', '#0AF', '#AF0']
var data = []
var yOffset = 0
//Process the data
for(var i = 0; i < values.length; i++) {
var datum = {
value : values[i],
colour : colours[i],
x: 0,
y: yOffset
}
yOffset += values[i]
data.push(datum)
}
var bars = canvas.selectAll('rect').data(data)
bars
.enter()
.append('rect')
.attr({
width : 30,
height : function(d) {
return d.value
},
y : function(d) {
return d.y //
}
})
.style({
fill : function(d) {
return d.colour
}
})
http://jsfiddle.net/r3sazt7m/
d3's layout functions all do more or less this- you pass them a set of data, and they return new data containing the values that the SVG drawing instructions require.

Smoothing arcs/plot points in D3.js/GeoJSON/TopoJSON/Shapefile (somewhere along the way)

I've been looking around a while for an answer to this, and I haven't been able to figure it out.
I'm ultimately creating a TopoJSON file from grid based data (GRIB files).
I can pretty easily interpolate the data down to a finer resolution grid so the plot points appear smoother when zoomed out, but when zoomed in, it's inevitable to see the blocky grid points.
I've also looked into simplification, which does help a bit but its not quite smoothing.
I'm using D3 to render the data.
Is this something that can be done on the front end or should/can it be done in the raw TopoJSON data?
I essentially don't want you to be able to tell that it's a grid, even if you zoom in 10,000%.
Here's an example of what I'm after:

Is this something that can be done on the front end or should/can it be done in the raw TopoJSON data?
This is something that should be done on the front end. If you were to smooth the data before you wrote it to the JSON file, the file would be needlessly big.

If you're using D3.js, and you're working with lines, the built-in interpolate() function is the way to go.
Here is a working example of D3's line.interpolate() using "cardinal" smoothing:
http://codepen.io/gracefulcode/pen/doPmOK
Here's the code:
var margin = {
top: 30,
right: 20,
bottom: 30,
left: 50
},
width = 600 - margin.left - margin.right,
height = 270 - margin.top - margin.bottom;
// Parse the date / time
var parseDate = d3.time.format("%d-%b-%y").parse;
// Set the ranges
var x = d3.time.scale().range([0, width]);
var y = d3.scale.linear().range([height, 0]);
// Define the axes
var xAxis = d3.svg.axis().scale(x).orient("bottom").ticks(5);
var yAxis = d3.svg.axis().scale(y).orient("left").ticks(5);
// Define the line
var valueline = d3.svg.line()
.interpolate("cardinal")
.x(function(d) {
return x(d.date);
})
.y(function(d) {
return y(d.close);
});
// Adds the svg canvas
var svg = d3.select("body").append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform",
"translate(" + margin.left + "," + margin.top + ")");
// Get the data
d3.json('https://api.myjson.com/bins/175jl', function(error, data) {
data.forEach(function(d) {
d.date = parseDate(d.date);
d.close = +d.close;
});
// Scale the range of the data
// Starting with a basic graph 14
x.domain(d3.extent(data, function(d) {
return d.date;
}));
y.domain([0, d3.max(data, function(d) {
return d.close;
})]);
// Add the valueline path.
svg.append("path")
.attr("class", "line")
.attr("d", valueline(data));
// Add the X Axis
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);
// Add the Y Axis
svg.append("g")
.attr("class", "y axis")
.call(yAxis);
});

Maybe d3 line.interpolate() is what you're looking for?
More info:
http://www.d3noob.org/2013/01/smoothing-out-lines-in-d3js.html

d3.js scatter plot - zoom/drag boundaries, zoom buttons, reset zoom, calculate median

I've built a d3.js scatter plot with zoom/pan functionality. You can see the full thing here (click 'Open in a new window' to see the whole thing):
http://bl.ocks.org/129f64bfa2b0d48d27c9
There are a couple of features that I've been unable to figure out, that I'd love a hand with it if someone can point me in the right direction:
I want to apply X/Y zoom/pan boundaries to the area, so that you can't drag it below a certain point (e.g. zero).
I've also made a stab at creating Google Maps style +/- zoom buttons, without any success. Any ideas?
Much less importantly, there are also a couple of areas where I've figured out a solution but it's very rough, so if you have a better solution then please do let me know:
I've added a 'reset zoom' button but it merely deletes the graph and generates a new one in its place, rather than actually zooming the objects. Ideally it should actually reset the zoom.
I've written my own function to calculate the median of the X and Y data. However I'm sure that there must be a better way to do this with d3.median but I can't figure out how to make it work.
var xMed = median(_.map(data,function(d){ return d.TotalEmployed2011;}));
var yMed = median(_.map(data,function(d){ return d.MedianSalary2011;}));
function median(values) {
values.sort( function(a,b) {return a - b;} );
var half = Math.floor(values.length/2);
if(values.length % 2)
return values[half];
else
return (parseFloat(values[half-1]) + parseFloat(values[half])) / 2.0;
};
A very simplified (i.e. old) version of the JS is below. You can find the full script at https://gist.github.com/richardwestenra/129f64bfa2b0d48d27c9#file-main-js
d3.csv("js/AllOccupations.csv", function(data) {
var margin = {top: 30, right: 10, bottom: 50, left: 60},
width = 960 - margin.left - margin.right,
height = 500 - margin.top - margin.bottom;
var xMax = d3.max(data, function(d) { return +d.TotalEmployed2011; }),
xMin = 0,
yMax = d3.max(data, function(d) { return +d.MedianSalary2011; }),
yMin = 0;
//Define scales
var x = d3.scale.linear()
.domain([xMin, xMax])
.range([0, width]);
var y = d3.scale.linear()
.domain([yMin, yMax])
.range([height, 0]);
var colourScale = function(val){
var colours = ['#9d3d38','#c5653a','#f9b743','#9bd6d7'];
if (val > 30) {
return colours[0];
} else if (val > 10) {
return colours[1];
} else if (val > 0) {
return colours[2];
} else {
return colours[3];
}
};
//Define X axis
var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom")
.tickSize(-height)
.tickFormat(d3.format("s"));
//Define Y axis
var yAxis = d3.svg.axis()
.scale(y)
.orient("left")
.ticks(5)
.tickSize(-width)
.tickFormat(d3.format("s"));
var svg = d3.select("#chart").append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")")
.call(d3.behavior.zoom().x(x).y(y).scaleExtent([1, 8]).on("zoom", zoom));
svg.append("rect")
.attr("width", width)
.attr("height", height);
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);
svg.append("g")
.attr("class", "y axis")
.call(yAxis);
// Create points
svg.selectAll("polygon")
.data(data)
.enter()
.append("polygon")
.attr("transform", function(d, i) {
return "translate("+x(d.TotalEmployed2011)+","+y(d.MedianSalary2011)+")";
})
.attr('points','4.569,2.637 0,5.276 -4.569,2.637 -4.569,-2.637 0,-5.276 4.569,-2.637')
.attr("opacity","0.8")
.attr("fill",function(d) {
return colourScale(d.ProjectedGrowth2020);
});
// Create X Axis label
svg.append("text")
.attr("class", "x label")
.attr("text-anchor", "end")
.attr("x", width)
.attr("y", height + margin.bottom - 10)
.text("Total Employment in 2011");
// Create Y Axis label
svg.append("text")
.attr("class", "y label")
.attr("text-anchor", "end")
.attr("y", -margin.left)
.attr("x", 0)
.attr("dy", ".75em")
.attr("transform", "rotate(-90)")
.text("Median Annual Salary in 2011 ($)");
function zoom() {
svg.select(".x.axis").call(xAxis);
svg.select(".y.axis").call(yAxis);
svg.selectAll("polygon")
.attr("transform", function(d) {
return "translate("+x(d.TotalEmployed2011)+","+y(d.MedianSalary2011)+")";
});
};
}
});
Any help would be massively appreciated. Thanks!
Edit: Here is a summary of the fixes I used, based on Superboggly's suggestions below:
// Zoom in/out buttons:
d3.select('#zoomIn').on('click',function(){
d3.event.preventDefault();
if (zm.scale()< maxScale) {
zm.translate([trans(0,-10),trans(1,-350)]);
zm.scale(zm.scale()*2);
zoom();
}
});
d3.select('#zoomOut').on('click',function(){
d3.event.preventDefault();
if (zm.scale()> minScale) {
zm.scale(zm.scale()*0.5);
zm.translate([trans(0,10),trans(1,350)]);
zoom();
}
});
// Reset zoom button:
d3.select('#zoomReset').on('click',function(){
d3.event.preventDefault();
zm.scale(1);
zm.translate([0,0]);
zoom();
});
function zoom() {
// To restrict translation to 0 value
if(y.domain()[0] < 0 && x.domain()[0] < 0) {
zm.translate([0, height * (1 - zm.scale())]);
} else if(y.domain()[0] < 0) {
zm.translate([d3.event.translate[0], height * (1 - zm.scale())]);
} else if(x.domain()[0] < 0) {
zm.translate([0, d3.event.translate[1]]);
}
...
};
The zoom translation that I used is very ad hoc and basically uses abitrary constants to keep the positioning more or less in the right place. It's not ideal, and I'd be willing to entertain suggestions for a more universally sound technique. However, it works well enough in this case.

To start with the median function just takes an array and an optional accessor. So you can use it the same way you use max:
var med = d3.median(data, function(d) { return +d.TotalEmployed2011; });
As for the others if you pull out your zoom behaviour you can control it a bit better. So for example instead of
var svg = d3.select()...call(d3.behavior.zoom()...)
try:
var zm = d3.behavior.zoom().x(x).y(y).scaleExtent([1, 8]).on("zoom", zoom);
var svg = d3.select()...call(zm);
Then you can set the zoom level and translation directly:
function zoomIn() {
zm.scale(zm.scale()*2);
// probably need to compute a new translation also
}
function reset() {
zm.scale(1);
zm.translate([0,0]);
}
Restricting the panning range is a bit trickier. You can simply not update when the translate or scale is not to your liking inside you zoom function (or set the zoom's "translate" to what you need it to be). Something like (I think in your case):
function zoom() {
if(y.domain()[0] < 0) {
// To restrict translation to 0 value
zm.translate([d3.event.translate[0], height * (1 - zm.scale())]);
}
....
}
Keep in mind that if you want zooming in to allow a negative on the axis, but panning not to you will find you get into some tricky scenarios.
This might be dated, but check out Limiting domain when zooming or panning in D3.js
Note also that the zoom behaviour did have functionality for limiting panning and zooming at one point. But the code was taken out in a later update.

I don't like to reinvent the wheel. I was searching for scatter plots which allow zooming. Highcharts is one of them, but there's plotly, which is based on D3 and not only allows zooming, but you can also have line datasets too on the scatter plot, which I desire with some of my datasets, and that's hard to find with other plot libraries. I'd give it a try:
https://plot.ly/javascript/line-and-scatter/
https://github.com/plotly/plotly.js
Using such nice library can save you a lot of time and pain.