d3: Contour or Surface plot from Irregular/Scattered Data - javascript

I can take a set of triplets [X,Y,Z] and immediately generate a (smooth) contour plot using Python and matplotlib with a single call to tricontour(). One can also generate contours 'easily' using plot.ly, but I find it to be unacceptably slow. (Also, I'm not interested in the MATLAB solution, which is similar to the Python)
I'm looking for similar functionality using d3.js. I would settle for a "surface plot" instead of contours, or a "heat map" without contour lines.
I can see how to generate a colored Delaunay triangulation and/or a colored Voronoi Tesselation, but the question of how to generate a contour plot in d3 from irregular data points seems to still be an open one (even though the question on this was prematurely closed!).
So far, all I've seen are approaches "by hand", using Radial basis functions (gaussian blur) or grid interpolation using Barycentric interpolation.
I'd even be willing to 'live with' Gouraud-shading or Coon-gradients on a Delaunay triangulation, but apparently "advanced shading methods" like Gourand or Coon gradients are not in "regular" SVG but are proposed for SVG2...not sure where that leaves me with d3 & (regular) SVG. It seems like doing this SVG gradient-shading by hand would be a major pain.
Is there a "better" package-y way to do this, i.e. something that doesn't require so much 'custom' code? (Maybe via some multidimensional Bezier routine I haven't found yet?)
I'll post a Fiddle with my starting point: a colored Voronoi tesselation: https://jsfiddle.net/k2v2jy7s/1/. Can you help me take this from "blocky" to "smooth" (and maybe even show contour lines)?
<svg width="960" height="500"></svg>
<script src="https://d3js.org/d3.v4.min.js"></script>
<script>
var svg = d3.select("svg"),
width = +svg.attr("width"),
height = +svg.attr("height");
var npoints = 1000;
var sites = d3.range(npoints)
.map(function(d) { return [Math.random() * width, Math.random() * height]; });
// values at data points / colors being mapped = "zvals"
var kx = 3.14159/(width*0.5);
var ky = 3.14159/(height*0.5);
var zvals = d3.range(npoints)
for (i = 0; i < npoints; i++) {
zvals[i] = (1.0 + Math.cos(kx*sites[i][0]) * Math.cos(ky*sites[i][1]))/2.0;
zvals[i] *= zvals[i];
}
var g = svg.append("g")
.attr("transform", "translate(" + 0+ "," + 0 + ")");
var voronoi = d3.voronoi()
.extent([[-1, -1], [width + 1, height + 1]]);
var polygon = svg.append("g")
.attr("class", "polygons")
.selectAll("path")
.data(voronoi.polygons(sites))
.enter().append("path")
.style('fill', function(d,i){ return d3.hsl( zvals[i]*310, 1, .5); })
.call(redrawPolygon);
function redrawPolygon(polygon) {
polygon
.attr("d", function(d) { return d ? "M" + d.join("L") + "Z" : null; });
}
</script>
Update: Also found this blocks.org post on "Gradient Heatmaps", which as I mentioned is the sort of result I'd be willing to live with, but again that's a large quantity of custom code. Would really prefer a compact 'stock' solution, a la tricontour().

5 ½ years, and no answers to this question!
Well, I've also been looking into how to generate contours from a series of [X,Y,Z] points in Javascript, but have not yet found the best or most complete solution. A lot of solutions I came across via Googling (such as d3-contour) are designed for an evenly spaced grid of values, not an irregular series of points as you might obtain from a land survey.
d3-tricontour
The d3-tricontour library looks perhaps the most promising, though, so I might have a play around with it.
Here's an example of what it can generate:
(The labels are optional.)
Apparently it uses the delaunay and meandering triangles algorithms to convert arbitrary points into triangles and then contour geometry. The algorithm works in O(n) where n is the number of edges, meaning it's very fast and scales perfectly well.
To learn more you can visit their:
Github repository
Many examples on Observable
Alternatives
Otherwise, there might be other ways to do this. If working with one of the grid-based libraries, I think the general process would be to:
Convert arbitary [X,Y,Z] points into a grid — the Delaunay algorithm is probably a great place to start (see d3-delaunay or other delaunay libraries)
Find the Z value for each point in the grid using some kind of interpolation (the maths for that, I'm not sure about)
Then feed that result into one of the grid-based contouring libraries
Constraining Contours
Also take note that creating contours from real world terrain also requires "constraining" some edges so that contours don't crossover ridgelines where they shouldn't.
CDT-JS is a library web app (with no separate library available as yet) that calculates constrained Delaunay triangulation, which might be useful for this case.
Otherwise, in theory, you might be able to create this kind of functionality by injecting additional [X,Y,Z] points along your lines of contraint prior to rendering. But I haven't tested this approach.

Related

How to get D3 to plot all elements of a GeoJson file? [duplicate]

I am trying to visualize russians regions. I got data from here, validate here and all was well - picture.
But when I try to draw it, I receive only one big black rectangle.
var width = 700, height = 400;
var svg = d3.select(".graph").append("svg")
.attr("viewBox", "0 0 " + (width) + " " + (height))
.style("max-width", "700px")
.style("margin", "10px auto");
d3.json("83.json", function (error, mapData) {
var features = mapData.features;
var path = d3.geoPath().projection(d3.geoMercator());
svg.append("g")
.attr("class", "region")
.selectAll("path")
.data(features)
.enter()
.append("path")
.attr("d", path)
});
Example - http://ustnv.ru/d3/index.html
Geojson file - http://ustnv.ru/d3/83.json
The issue is the winding order of the coordinates (see this block). Most tools/utilities/libraries/validators don't really care about winding order because they treat geoJSON as containing Cartesian coordinates. Not so with D3 - D3 uses ellipsoidal math - benefits of this is include being able to cross the antimeridian easily and being able to select an inverted polygon.
The consequence of using ellipsoidal coordinates is the wrong winding order will create a feature of everything on the planet that is not your target (inverted polygon). Your polygons actually contain a combination of both winding orders. You can see this by inspecting the svg paths:
Here one path appears to be accurately drawn, while another path on top of it covers the entire planet - except for the portion it is supposed to (the space it is supposed to occupy covered by other paths that cover the whole world).
This can be simple to fix - you just need to reorder the coordinates - but as you have features that contain both windings in the same collection, it'll be easier to use a library such as turf.js to create a new array of properly wound features:
var fixed = features.map(function(feature) {
return turf.rewind(feature,{reverse:true});
})
Note the reverse winding order - through an odd quirk, D3, which is probably the most widespread platform where winding order matters actually doesn't follow the geoJSON spec (RFC 7946) on winding order, it uses the opposite winding order, see this comment by Mike Bostock:
I’m disappointed that RFC 7946 standardizes the opposite winding order
to D3, Shapefiles and PostGIS. And I don’t see an easy way for D3 to
change its behavior, since it would break all existing (spherical)
GeoJSON used by D3. (source)
By rewinding each polygon we get a slightly more useful map:
An improvement, but the features are a bit small with these projection settings.
By adding a fitSize method to scale and translate we get a much better looking map (see block here):
Here's a quick fix to your problem, projection needs a little tuning, also path has fill:#000 by default and stroke: #FFF could make it more legible.
var width = 700, height = 400;
var svg = d3.select(".graph").append("svg")
.attr("viewBox", "0 0 " + (width) + " " + (height))
.style("max-width", "700px")
.style("margin", "10px auto");
d3.json("mercator_files/83.json", function (error, mapData) {
var features = mapData.features;
var center = d3.geoCentroid(mapData);
//arbitrary
var scale = 7000;
var offset = [width/2, height/2];
var projection = d3.geoMercator().scale(scale).center(center)
.translate(offset);
var path = d3.geoPath().projection(projection);
svg.append("g")
.attr("class", "region")
.selectAll("path")
.data(features)
.enter()
.append("path")
.attr("d", path)
});

D3 Arc With Chevron Shaped End

I've drawn an arc using D3.js which by default has square shaped ends.
var arc = d3.arc()
.innerRadius(0)
.outerRadius(100)
.startAngle(0)
.endAngle(Math.PI);
d3.selectAll('svg')
.append('path')
.attr('d', function() {
return arc();
});
How can I draw an arc with a chevron shape on one end of it.
I think I understand what you are looking for, so I'll give it a go:
As you probably guess from the d3.js documentation, d3.arc() does not have the methods needed to make a point at one end. Padding and rounded corners are applied on both ends, and I can't see how they would work to form a point at both ends let alone one.
Two solutions come to mind (and there are probably many that I can't even conceive of)
Lop off the end of each arc, based on its end angle, and append a triangle or other similar shape (alternatively, apply some sort of mask to trim the end into a point)
Attempt to rework d3.arc() to your needs, taking up the invitation to develop/refine d3 in a modular fashion.
Personally, I think option one is probably much less clean and probably harder to design. Option two should be doable, and with this encouragement to dive in and make modules:
Small files are nice, but modularity is also about making D3 more fun. Microlibraries are easier to understand, develop and test. They make it easier for new people to get involved and contribute. They reduce the distinction between a “core module” and a “plugin”, and increase the pace of development in D3 features. (https://github.com/d3/d3/blob/master/CHANGES.md)
I thought I'd give this a go.
I've put together an attempt that might be a start for a chevron tipped arc module based on the d3.arc() function.
The rounded corners portion of the d3.arc() function in the d3-shape.js module is likely the best place to look as it shows modifications to the arc ends. The portions of the module that modify the arc, in the event of rounded corners, look like:
context.arc(t0.cx, t0.cy, rc1, Math.atan2(t0.y01, t0.x01), Math.atan2(t0.y11, t0.x11), !cw);
context.arc(0, 0, r1, Math.atan2(t0.cy + t0.y11, t0.cx + t0.x11), Math.atan2(t1.cy + t1.y11, t1.cx + t1.x11), !cw);
context.arc(t1.cx, t1.cy, rc1, Math.atan2(t1.y11, t1.x11), Math.atan2(t1.y01, t1.x01), !cw);
The outer edge is handled first (and shown above). The first line is the rounding on the rear outside corner, the third line is the rounding on the forward outside corner. Simply removing the third line allows for a pointed arc (if you remove it from the inside edge too). Then the remaining challenge is making the other end of the arc flat, which I did by using the start angle and the inner & outer radii to find the corners of the arc to create a flat end.
The end result was something like:
// get tail coordinate (outer)
var tailOuter = {};
tailOuter.x = Math.cos(a0) * r1; // a0 = starting angle
tailOuter.y = Math.sin(a0) * r1; // r1 = outer radius
context.moveTo(tailOuter.x, tailOuter.y);
context.arc(0, 0, r1, Math.atan2(t0.cy + t0.y11, t0.cx + t0.x11), Math.atan2(t1.cy + t1.y11, t1.cx + t1.x11), !cw);
I've put together a quick and dirty module that takes the d3.arc() function and creates a d3.cheveronArc() function instead. It's a gutted modification to d3.arc() and has only four methods (inner/outerRadius(),start/endAngle()). It has no means to check for parameters that will likely cause misbehavior (eg: chevron is longer than the arc). It is merely a proof of concept, though I am happy with how it looks for a rather quick attempt:
As you might notice, the inner most circle has an odd shape near its tail, small inner radii seem to cause some problems like that.
The code can be viewed at:
http://bl.ocks.org/andrew-reid/3375e602cc6c00c4e3ea4799d171ee27
Looking at it, I feel like I want to add the option to add the inverse of the chevron to the rear end of the arcs for a better visual effect, but that's a different problem.
I would just use d3's path generator with an SVG marker. Add any shape to any path. Edit the "ends" by editing the marker definition. Use D3 path generator to define your arc (or any path).
It's worth noting that if you take this approach you have to use the d3 path generator rather than the d3 arc generator because the arc generator implicitly closes the path (putting your "end" marker back at the beginning of the path).
In my example, I added the chevron to the start as well just to show that it's as trivial as adding .attr("marker-start","url(#chevron)") and .attr("marker-end","url(#chevron)")
D3 Path Generator | https://github.com/d3/d3-path/blob/master/README.md#path
SVG Markers | https://developer.mozilla.org/en-US/docs/Web/SVG/Element/marker
edit: and now that I think of it, you can probably use d3.symbols to generate your markers/ends for you instead of manually defining the shape path. The chevron would have to be custom, but you could probably use the triangle symbol.
D3 Symbols | https://github.com/d3/d3-shape#symbols
console.clear()
var path = d3.path()
path.arc(225,80,70,1,-.5)
var path2 = d3.path()
path2.moveTo(20,20)
path2.bezierCurveTo(150,300,200,0,450,100)
d3.select("svg").append("path")
.attr("d", path2.toString())
.attr("stroke","steelblue")
.attr("fill","none")
.attr("stroke-width","20")
.attr("marker-start","url(#chevron)")
.attr("marker-end","url(#chevron)")
d3.select("svg").append("path")
.attr("d", path.toString())
.attr("stroke","#43A2CA")
.attr("fill","none")
.attr("stroke-width","20")
.attr("marker-start","url(#chevron)")
.attr("marker-end","url(#chevron)")
<script src="https://unpkg.com/d3#4.4.0"></script>
<?xml version="1.0"?>
<svg width="500" height="200" viewBox="0 0 500 200">
<defs>
<marker id="chevron"
viewBox="0 0 20 20" refX="10" refY="10"
markerUnits="userSpaceOnUse"
markerWidth="20" markerHeight="20"
orient="auto" fill="black">
<path d="M0 0 10 0 20 10 10 20 0 20 10 10Z" />
</marker>
</defs>
</svg>

Compute Voronoï diagram on a large dataset of very close points

I have a large dataset of geographical points (around 22 000 points, but I could be more in the future) and I need to compute their Voronoï diagram. I first project my points from (lat,lng) to (x,y) (using latLngToLayerPoint() from Leaflet) and then compute the diagram based on a Javascript implementation of Fortune's algorithm . I recover each cells of the diagrams or more precisely va and vb, being respectively :
"A Voronoi.Vertex object with an x and a y property defining the start
point (relative to the Voronoi site on the left) of this Voronoi.Edge
object."
and
"A Voronoi.Vertex object with an x and a y property defining the end
point (relative to Voronoi site on the left) of this Voronoi.Edge
object."
(cf. Documentation)
Finally, I project back these points to display the diagram using leaflet. I know that, in order to compute the diagram each point needs to be unique, so I get rid of duplicates before computing the diagram. But the thing is, I end up with a pretty bad result (non-noded intersections, complex polygons):
Close-up
I have holes in the diagram and I'm not sure why. The points are house Address so some of them, even if they are not equals, are really (really) close. And I wonder if the issue doesn't come from the projection (if (lat1,lng1) and (lat2,lng2) are almost equals, will (x1,y1) and (x2,y2) be equals ?). I strongly suspect that is where the issue come from, but I don't know how to workaround (establish a threshold ?)
Edit : I precise that I delete the duplicates after the projection, so it's not about the precision of the projection but more about what happen if two points are one-pixel apart ?
So I found the solution to my problem, I post it in case of anyone need to compute a Voronoï diagram on a map using Leaflet and Turf and is having troubles implementing the Fortune's algorithm (until turf-voronoi works).
Other sources of how to compute a Voronoï diagram on map can be found (but using d3) (I think d3 also use this Javascript implementation of Fortune's algorithm)
The problem was not caused by the size of the dataset or the proximity of the points, but by how I recovered the cells.
So you first need to project your point from (lat,lng) to (x,y)(using latLngToLayerPoint()), compute the diagram : voronoi.compute(sites,bbox), where the sites are your points looking like this [ {x: 200, y: 200}, {x: 50, y: 250}, {x: 400, y: 100} /* , ... */ ] (note that your sites needs to be unique) and if you want the frame of the screen for your current zoom to be your bbox juste use :
var xl = 0,
xr = $(document).width(),
yt = 0,
yb = $(document).height();
Once you computed the diagram, just recover the cells (be carfull, if you want the right polygons you need the edges to be counterclockwise ordered (or clockwise ordered, but you them to be ordered), thankfully the algorithm provides the half edges of a given Voronoï.Vertex counterclockwise ordered). To recover the vertex of each cell you can use either getStartpoint() or getEndpoint() without forgetting to project them back from (x,y) to (lat,lng) (using layerPointToLatLng())
diagram.cells.forEach(function (c) {
var edges=[];
var size = c.halfedges.length;
for (var i = 0; i < size; i++) {
var pt = c.halfedges[i].getEndpoint();
edges.push(map.layerPointToLatLng(L.point(pt.x,pt.y)));
};
voronoi_cells.push(L.polygon(edges));
});
Finally, you have to use a FeatureCollection to display the diagram :
I highly recomment you don't implement a Voronoi tesselation algorithm by yourself, and use https://github.com/Turfjs/turf-voronoi instead.

D3 geo: getting projection.clipAngle to work on all specified elements

I'm a newcomer to D3 and I'm trying to make a world globe with some points ("pins") on it. Demo here: http://bl.ocks.org/nltesown/66eee134d6fd3babb716
Quite commonly, the projection is defined as:
var proj = d3.geo.orthographic()
.center([0, 0])
.rotate([50, -20, 0])
.scale(250)
.clipAngle(90)
.translate([(width / 2), (height / 2)]);
the clipAngle works well for the svg paths, but not the pins (which are svg circles). As you can see on the demo, the pin that sits between Iceland and Greenland should be hidden (it's Taiwan).
So I suppose the problem comes from these lines, but I can't understand why:
.attr("transform", function(d) {
return "translate(" + proj([ d.lng, d.lat ]) + ")";
});
It is not sufficient to just set the clipping radius via clipAngle() to get the desired behavior. The projection alone will not do the clipping, but just calculate the projected coordinates without taking into account any clipping. That is the reason, why Taiwan gets rendered, although you expected it to be hidden.
But, thanks to D3, salvation is near. You just need to re-think the way you are inserting your circles representing places. D3 has the mighty concept of geo path generators which will take care of the majority of the work needed. When fed a projection having a clipping angle set, the path generator will take this into account when calculating which features to actually render. In fact, you have already set up a proper path generator as your variable path. You are even correctly applying it for the globe, the land and the arcs.
The path generator will operate on GeoJSON data, so all you need to do is convert your places to valid GeoJSON features of type Point. This could be done with a little helper function similar to that used for the arcs:
function geoPlaces(places) {
return places.map(function(d) {
return {
type: "Point",
coordinates: [d.lng, d.lat]
};
});
}
With only minor changes you are then able to bind these GeoJSON data objects to make them available for the path generator which in turn takes care of the clipping:
svg.selectAll(".pin") // Places
.data(geoPlaces(places))
.enter().append("path")
.attr("class", "pin")
.attr("d", path);
Have a look at my fork of your example for a working demo.

D3 + Leaflet: d3.geo.path() resampling

We've adapted Mike Bostock's original D3 + Leaflet example:
http://bost.ocks.org/mike/leaflet/
so that it does not redraw all paths on each zoom in Leaflet.
Our code is here: https://github.com/madeincluj/Leaflet.D3/blob/master/js/leaflet.d3.js
Specifically, the projection from geographical coordinates to pixels happens here:
https://github.com/madeincluj/Leaflet.D3/blob/master/js/leaflet.d3.js#L30-L35
We draw the SVG paths on the first load, then simply scale/translate the SVG to match the map.
This works very well, except for one issue: D3's path resampling, which looks great at the first zoom level, but looks progressively more broken once you start zooming in.
Is there a way to disable the resampling?
As to why we're doing this: We want to draw a lot of shapes (thousands) and redrawing them all on each zoom is impractical.
Edit
After some digging, seems that resampling happens here:
function d3_geo_pathProjectStream(project) {
var resample = d3_geo_resample(function(x, y) {
return project([ x * d3_degrees, y * d3_degrees ]);
});
return function(stream) {
return d3_geo_projectionRadians(resample(stream));
};
}
Is there a way to skip the resampling step?
Edit 2
What a red herring! We had switched back and forth between sending a raw function to d3.geo.path().projection and a d3.geo.transform object, to no avail.
But in fact the problem is with leaflet's latLngToLayerPoint, which (obviously!) rounds point.x & point.y to integers. Which means that the more zoomed out you are when you initialize the SVG rendering, the more precision you will lose.
The solution is to use a custom function like this:
function latLngToPoint(latlng) {
return map.project(latlng)._subtract(map.getPixelOrigin());
};
var t = d3.geo.transform({
point: function(x, y) {
var point = latLngToPoint(new L.LatLng(y, x));
return this.stream.point(point.x, point.y);
}
});
this.path = d3.geo.path().projection(t);
It's similar to leaflet's own latLngToLayerPoint, but without the rounding. (Note that map.getPixelOrigin() is rounded as well, so probably you'll need to rewrite it)
You learn something every day, don't you.
Coincidentally, I updated the tutorial recently to use the new d3.geo.transform feature, which makes it easy to implement a custom geometric transform. In this case the transform uses Leaflet’s built-in projection without any of D3’s advanced cartographic features, thus disabling adaptive resampling.
The new implementation looks like this:
var transform = d3.geo.transform({point: projectPoint}),
path = d3.geo.path().projection(transform);
function projectPoint(x, y) {
var point = map.latLngToLayerPoint(new L.LatLng(y, x));
this.stream.point(point.x, point.y);
}
As before, you can continue to pass a raw projection function to d3.geo.path, but you’ll get adaptive resampling and antimeridian cutting automatically. So to disable those features, you need to define a custom projection, and d3.geo.transform is an easy way to do this for simple point-based transformations.

Categories