I am creating a map with D3.js.
I began by downloading the country (Canada) shapefile here:
https://www.arcgis.com/home/item.html?id=dcbcdf86939548af81efbd2d732336db
..and converted it into a geojson here (link to file below):
http://mapshaper.org/
So far all I see is a coloured block, without any errors on the console. My question is, how can I tell if my json file or my code is incorrect?
Here is my code and on bottom is a link to json file.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>D3: Setting path fills</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.6/d3.min.js"></script>
<!-- <script src="https://d3js.org/topojson.v1.min.js"></script> -->
<style type="text/css">
/* styles */
</style>
</head>
<body>
<script type="text/javascript">
var canvas = d3.select("body").append("svg")
.attr("width", 760)
.attr("height", 700)
d3.json("canada.geo.json", function(data) {
var group = canvas.selectAll("g")
.data(data.features)
.enter()
.append("g")
var projection = d3.geo.mercator();
var path = d3.geo.path().projection(projection);
var areas = group.append("path")
.attr("d",path)
.attr("class","area")
})
</script>
</body>
</html>
Link to json file:
https://github.com/returnOfTheYeti/CanadaJSON/blob/master/canada.geo.json
A d3 geoProjection uses unprojected coordinates - coordinates on a three dimensional globe. The geoProjection takes those coordinates and projects them onto a two dimensional plane. The units of unprojected coordinates are generally degrees longitude and latitude, and a d3 geoProjection expects this. The problem is that your data is already projected.
How can I tell if the data is projected?
There are two quick methods to determine if your data is projected:
look at the meta data of the data
look at the geographic coordinates themselves
Look at the Geographic Metadata
The projection your data uses is defined in the .prj file that forms part of the collection of files that makes up a shapefile:
PROJCS["Canada_Albers_Equal_Area_Conic",
GEOGCS["GCS_North_American_1983",
DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],
PRIMEM["Greenwich",0.0],
UNIT["Degree",0.0174532925199433]],
PROJECTION["Albers"],
PARAMETER["False_Easting",0.0],
PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-96.0],
PARAMETER["Standard_Parallel_1",50.0],
PARAMETER["Standard_Parallel_2",70.0],
PARAMETER["Latitude_Of_Origin",40.0],
UNIT["Meter",1.0]]
Your data is already projected with an Albers projection, and the unit of measurement is the meter. Projecting this data as though it consists of lat/long pairs will not work.
If you only have a geojson file and no reference shapefile, some geojson files will specify an EPSG number in a projection propery, if this number is something other than 4326 you probably have projected data.
Look at the Coordinates
You can tell your data doesn't have unprojected data because the values of each coordinate are outside the bounds of longitude and latitude (+/-180 degrees east/west, +/- 90 degrees north south):
"coordinates":[[[[899144.944639163,2633537.
Your coordinates translate around the globe several times: this is why your projection results in an svg is filled entirely with features.
Ok, Now What?
There are two primary solutions available for you:
Convert the projection so that the geojson consists of latitude and longitude points
Use d3.geoTransform or d3.geoIdentity to transform your projected data.
Convert the Projection
To do this you want to "unproject" your data, or alternatively, project it so that it consists of longitude, latitude points.
Most GIS software offers the ability to reproject data. It's much easier with a shapefile than a geojson, as shapefiles are much more common in GIS software. GDAL, QGIS, ArcMap offer relatively easy conversion.
There are also online converters, mapshaper.org is probably the easiest for this, and has added benefits when dealing with d3 - simplification (many shapefiles contain way too much detail for the purposes of web mapping). Drag all the files of the shapefile into the mapshaper window, open the console and type: proj wgs84. Export as geojson (after simplification), and you've got a geojson ready for d3.
After reprojecting, you may notice that your data is awkward looking. Don't worry, it's unprojected (well, kind of unprojected, it's shown as 2d, but with a very simple projection that assumes Cartesian input data).
With your unprojected data, you are now ready to project your data in d3.
Here's an example with your data (d3-v4. data is simplified and reprojected on mapshaper (no affiliation to me))
Using d3.geoIdentity or d3.geoTransform
For this I would recommend using d3v4 (I see your code is v3). While geo.transform is available in v3, it is much more cumbersome without the new methods available in v4, namely: d3.geoIdentity and projection.fitSize. I will address the v4 method of using projected data here
With your data you can define a different sort of projection:
var projection = d3.geoIdentity();
However, this type of "projection" will give you trouble if you aren't careful. It basically spits out the x,y values it is given. However, geographic projected coordinate spaces typically have [0,0] somewhere in the bottom left, and svg coordinates space has [0,0] in the top left. In svg coordinate space, y values increase as you go down the coordinate plane, in the projected coordinate space of your data, y values increase as you go up. Using an identity will therefore project your data upside down.
Luckily we can use:
var projection = d3.geoIdentity()
.reflectY(true);
One last problem remains: the coordinates in the geojson are not scaled or translated so that the features are properly centered. For this there is the fitSize method:
var projection = d3.geoIdentity()
.reflectY(true)
.fitSize([width,height],geojsonObject)
Here width and height are the width and height of the SVG (or parent container we want to display the feature in), and the geojsonObject is a geojson feature. Note it won't take an array of features, if you have an array of features, place them in a feature collection.
Here's your data shown taking this approach (I still simplified the geojson).
You can also use a geoTransform, this is a bit more complex, but allows you to specify your own transform equation. For most situations it is probably overkill, stick with geoIdentity.
Pros and Cons of Each Option:
Unprojecting the data:
Beyond the initial leg work to unproject the data, by unprojecting the data so that it consists of longitude latitude pairs you have to do some extra processing each time you show the data.
But, you also have a high degree of flexibility in how you show that data by accessing any d3 geoProjection. Using unprojected data also allows you to more easily align different layers: you don't have to worry about rescaling and transforming multiple layers individually.
Keeping the Projected Data
By keeping the projection the data comes in, you save on computing time by not having to do spherical math. The downsides are the upsides listed above, it's difficult to match data that doesn't share this projection (which is fine if you export everything using this projection), and your stuck with the representation - a d3.geoTransform doesn't offer much in the way of converting your projection from say a Mercator to an Albers.
Note that that the fit.size() method I used for option two above is available for all geoProjections (v4).
In the two examples, I used your code where possible. A couple caveats though, I changed to d3v4 (d3.geo.path -> d3.geoPath, d3.geo.mercator -> d3.geoMercator, for example). I also changed the name of your variable canvas to svg, since it is a selection of an svg and not a canvas. Lastly, in the first example I didn't modify your projection, and a mercator's center defaults to [0,0] (in long/lat), which explains the odd positioning
If by correct you mean it is valid JSON - then you can simply parse it via javascript and check for any errors.
In this example any errors will be logged to the console. Otherwise the parsed object will be logged if valid.
// where data is your JSON data
try {
console.log(JSON.parse(data));
} catch (e) {
console.error(e);
}
However as you are already using D3 you can simply use the d3.json method to test for errors. Add an extra parameter to the d3 call.
d3.json("canada.geo.json", function(error, canada) {
if (error) return console.error(error);
console.log(canada);
});
See: https://github.com/d3/d3-3.x-api-reference/blob/master/Requests.md
Related
I come back for a problem. I'm still learning d3js for 5 days and I have a problem about the border rendering.
Why I have that ?
The problem occurs when I add region borders.
svgMap.append("path")
.attr("class", "reg_contour")
.datum(topojson.mesh(fr[0], fr[0].objects.reg_GEN_WGS84_UTF8, function(a, b) { return a !== b; }))
.attr("d", path);
Here is my code : https://plnkr.co/edit/mD1PzxtedWGrZd5ave28?p=preview
Just for the record, the json file combines two layers (departments and regions) created with the same shp and compiled by QGIS for geojson output. After I convert this to topojson by mapshapper.
The error lies in your topojson - the two different feature types, departments and regions do not share the same coordinates along their common boundaries.
First, in this sort of situation it is desirable to check to make sure there is no layering issue (often other features are drawn ontop of others, hiding them or portions of them), we can do this by just showing the regional boundaries:
(plunkr)
So the problem can't be layering, if we look at a particular feature in the topojson, say the department of Creuse:
{"arcs":[[-29,-28,-27,-26,202,-297,-296,205,-295,-410,419]],"type":"Polygon","properties":{"codgeo":"23","pop":120581,"libgeo":"Creuse","libgeo_m":"CREUSE","codreg":"75","libreg":"Nouvelle Aquitaine"}}
We see that the department is drawn using 11 arcs representing each portion of the boundary based on shared boundaries between multiple features so that shared boundaries are only represented once in the data.
If we zoom in on Creuse we can see those 11 arc segments shared between either other departments, regions, or with nothing at all:
The thick portions of the boundary correspond to the thick white boundaries in the image in the question, I've only changed the styling and zoom from the original plunkr
This looks problematic, the department should only have 6 arcs:
Why are there additional arcs? Because the boundaries of the departments are not aligned properly - the shared boundaries between departments do not always share the same arcs in your topojson. Chances are the departments might use a different scale than the regions, a different precision or projection, or were made somehow differently. This has resulted in minute nearly imperceptible differences that have resulted in boundaries that share coordinates in reality not sharing the same coordinates in the data, and thus the shared arcs are going unrecognized.
Since you are generating the mesh like this:
topojson.mesh(fr[0], fr[0].objects.reg_GEN_WGS84_UTF8, function(a, b) { return a !== b; })
Only shared boundaries are drawn, which explains the gaps.
We can rectify this a few ways, the easiest way would be to remove the regions altogether. The departments record which region they are in, we can check to see if a boundary should be drawn if the departments on each side of that boundary are in different regions:
.datum(topojson.mesh(fr[0], fr[0].objects.dep_GEN_WGS84_UTF8, function(a, b) { return a.properties.libreg !== b.properties.libreg; }))
Which gives us:
(plunkr)
Alternatively, we can re-habilitate the regional boundaries by importing them into a GIS platform such as ArcGIS and repairing the geometry. We could also import the departments and make a new layer based on region properties in a dissolve. Using the repair geometry tool in Arc, I get a nice boundary (when shown with the same code as the first image here):
There are other methods, such as including a tolerance in aligning arcs, but these might be more difficult than the above.
I'm trying to load my topojson file of the map of ontario using d3, but all I get is a bunch of random lines, just like Random lines when loading a TopoJSON file in D3.
My file is using WGS84 so that's not the issue either. What am I doing wrong? Js code below.
var width = 960, height = 700;
var svg = d3.select('#map').append('svg')
.attr('width', width)
.attr('height', height)
d3.json('CensusSubDiv.json', function(error, CensusSubDiv) {
if (error) return console.error(error);
svg.append('path')
.datum(topojson.feature(CensusSubDiv, CensusSubDiv.objects.CensusSubDivision))
.attr('d', d3.geo.path().projection(d3.geo.mercator()))
.attr('id', 'ont')
;
});
So, as I mentioned in the comments, your data may use WGS84 as a datum, but D3 requires WGS84 to be used as a "projection" (d3 requires latitude and longitude, which represent points on a three dimensional globe, and thus are actually unprojected, hence my quotes). Projected data, consequently, is labelled with WGS84 sometimes as well as other identifiers.
The good news is you can display projected data fairly easily - the better news is that this is faster. Instead of doing three dimensional math, we just transform and scale the data before plotting it. The bad news is that this works easiest with d3 v4 (which my example uses, it only changes a few points: eg: d3.geo.path -> d3.geoPath, d3.geo.projection -> d3.geoProjection, etc).
As your data is already projected, we can use d3.geoIdentity which allows us to use the projection.fitSize() (which modifies scale and translate) method but otherwise doesn't project the data. FitSize takes an array for the width/height of the display area and a geojson object:
projection.fitSize([width,height],geojsonObject)
fitSize is not part of d3v3, but we're on d3v5, so updating isn't a bad thing, projection.fitExtent allows for margins
As in SVG coordinate space, Y values start at 0 at the top of the screen and increase as one moves down, and in most projected coordinate spaces, Y values start at the bottom of the map and increase as one moves north, we also need to flip the identity on the y axis:
var projection = d3.geoIdentity()
.reflectY(true)
.fitSize([width,height],geojsonObject)
This allows us to display the data fairly easily. See this block for your data with the above noted changes.
But, here's the caveat, you have data that is already projected, if you don't know how that data was projected, you can't align other geographic data that is unprojected or otherwise projected: you can't project a point with given longitude latitude because you don't know how to project it in the same manner as your already projected data.
This might not be a concern, for example in a choropleth perhaps you only need the outline of the census subdivisions. But if you wanted to place cities on top of the map, you'd face some difficulties if the coordinates for those cities weren't already projected in the same manner as your census data.
If you wanted to overlay other geographic data (unprojected data) over Ontario, then you will need to unproject your data. In mapshaper, when you initially import the data into the window (ensuring you drag the .prj file too - otherwise mapshaper won't know what projection to reproject the data from), you can open the console and type proj WGS84, this should give you coordinates that are in degrees (though the topojson will still have encoded integer coordinates, the coordinates are stored in plain text if exporting to geojson). Of course, you'll need to use more typical projection, such as a d3.geoMercator() again if you chose to unproject your data.
I am working on a project using D3.JS and Flask trying to display a map of the U.S. The problem is that the map is not displaying. I know that that the SVG element is being attached, that the json data is coming through but the map itself is not coming up. I then tried to create a simple index.html document and placed the json file within that project and using a Python simple server was able to see the map. So now for the code.
To start here is the JS code:
<script type="text/javascript">
//Define default path generator
let path = d3.geo.path();
//Creating the SVG element and attaching it to the page.
let svg = d3.select("#us-map")
.append("svg")
.attr("width", 900)
.attr("height", 700);
//This function will get the data for the map of the U.S
d3.json("/json", function(json){
console.log(json.features) //This shows an array of 52 objects
//Bind data and create one path per GeoJSON feature
svg.selectAll("path")
.data(json.features)
.enter()
.append("path")
.attr("d", path);
});
</script>
Python code:
#app.route("/json")
def json():
data = Data()
data_file = data.convert_json_for_d3()
#return data_file
return data_file.to_json()
The method in the class that I am calling from the above code:
def convert_json_for_d3(self):
self.__data = pd.read_json('us-states.json')
df = self.__data
return df
The HTML:
<section id="us-map">
<h1>UFO Sitings in the U.S.</h1>
</section>
I have been working on this for a day now. At first, getting the json data working with the D3 was the hard part. I figured I had it at that point. However, as I stated, the map is not showing up. It does work if I place it into a simple index.html file along with the json. So I am wondering if the issue is with Flask? One final point, I am following along code from the book Interactive data visualization for the Web. The github repo that I was pulling code from is here: https://github.com/alignedleft/d3-book/blob/master/chapter_12/01_paths.html
Working with a simple HTML file that data appears as this:
object {type: "FeatureCollection", features: Array(52)}
features:Array(52)
0:Object
geometry: Object
coordinates: Array(1)
0: Array (33)
0
1
:
1:Object
:
Data in my Flask Project looks like this:
Object {features: Object, type: Object}
features: Object
0: Object
geometry: Object
coordinates: Array(1)
0: Array (33)
0:Array(2)
1: Object
:
So both the array of objects are almost identical-except for the very first line.
Thank you for any help.
Two issues (based on comments below and updated question)
One is the geojson in your Flask project is not valid. For example features should be an array - not an object, type should still be featureCollection (This is discussed in the comments below). Somewhere along the line your geojson structure is changed to something that isn't accepted by d3.geo.path() as valid geojson.
Two is that you are not using a projection to convert your latitude and longitude pairs (in the geometry of each feature) to svg coordinate space.
For the second issue:
When using d3.geo.path(); you need to specify a projection (eg: d3.geo.path().projection(d3.geo.mercator());). If you do not provide a projection, a null projection is used. This interprets the provided geojson without applying any transformation - essentially your values are translated into pixel values on the svg.
The United States is in the western hemisphere, all the longitudes are therefore negative (and as north south geographic values increase as one goes north and svg y values increase as one moves down, the image will be upside down). A null projection (with no other transform on the svg) will draw a path that is to the left of your svg's boundaries. This is why these three things can happen:
the "SVG element is being attached"
"that the json data is coming through", but
"the map itself is not [visually] coming up"
The geojson referenced in the chapter you reference is unprojected data - it contains a latitude and longitude pair for each vertex: [-133.05341,56.125739]. This is ideal, as a d3.geo.projection takes this type of coordinate, points on a three dimensional ellipsoid, and projects them onto a two dimensional Cartesian plane. If you had already planar data you would need to use a geo transform.
So, to visualize your data you will need to select a projection to convert latitude and longitude pairs into appropriate values. USA albers might be easiest as it is a composite projection that scales down Alaska and moves both Alaska and Hawaii closer to the continental US. The Albers USA also sets all the projection parameters to show the US without modification to projection parameters (center, rotation, parallels, scale etc), while other projections will require setting proper parameters.
I got your data to display with an albersUsa projection using the following code:
var projection = d3.geo.albersUsa().translate([width/2,height/2]);
var path = d3.geo.path().projection(projection);
d3.json("us.json",function(data) {
svg.append("path")
.datum(data)
.attr('d',path)
.attr('fill','steelblue')
.attr('stroke','black');
});
Here is a working demo and a screen shot:
If your map is too large for your svg (or vice versa), you can set the scale with projection.scale(n), with smaller numbers zooming out, larger numbers zooming in.
I would like to create a map which has been tilted back on the Z-axis, as if you were looking at it laid flat on a table. No problem: use Mike Bostock's custom projection approach, define a simple transformation that scales back as it goes towards the top, call it a day.
Except in the process I learned that d3 was defaulting to the Albers USA projection, and that using my custom projection put Alaska and Hawaii back to their correct placements (which I don't want.)
Also, I hadn't realized Puerto Rico wasn't on the Albers USA projection, so I'd actually like to switch to Albers USA + PR.
I would prefer not to remake the projection if I can help it, because someday I'll find an Albers that also has the various other US territories.
Are projections composable somehow?
Without knowing the exact details of your implemenation I have forked Mike Bostock's Block AlbersUSA + PR to show a way this could be done.
At its core this uses the Albers USA projection which includes Puerto Rico as requested. This is a normal D3 projection:
var albersUsaPrProj = albersUsaPr()
.scale(1070)
.translate([width / 2, height / 2]);
Next, I have implemented a rather naïve projection to the table which might need some refinement, but should be enough to get you started. This one uses d3.geo.transform() to create a stream wrapper for the calculations needed to project the map on the desktop. The wrapped stream listener needs to implement just the point method that will be called with the x, y screen coordinates which are the results from the geo projection.
// The scale is used for convenient calculations only.
var yScale = d3.scale.linear()
.domain([0,height])
.range([0.25,1]);
var desktopProj = d3.geo.transform({
point: function(x,y) {
this.stream.point((x - 0.5*width) * yScale(y) + (0.5*width), y);
}
});
Both projections are easily combined into one new stream wrapper by creating an object implementing the .stream() method.
var combinedProj = {
stream: function(s) {
return albersUsaPrProj.stream(desktopProj.stream(s));
}
};
According to the documentation on projection.stream(listener):
Returns a projecting stream wrapper for the specified listener. Any geometry streamed to the wrapper is projected before being streamed to the wrapped listener.
This will first let albersUsaPrProj take care of the map's Albers USA projection and, afterwards, stream the resulting screen coordinates to desktopProj.
This combined projection may then be passed to path.projection([projection]):
For more control over the stream transformation, the projection may be specified as an object that implements the stream method. (See example.) The stream method takes an output stream as input, and returns a wrapped stream that projects the input geometry; in other words, it implements projection.stream.
This give us the final call as
var path = d3.geo.path()
.projection(combinedProj);
I'm using the ESRI JavaScript API v3.8. (I know 3.11 is out - can't upgrade yet.)
What I'm trying to do is to create a geometric buffer of a size provided by the user from an arbitrary line (or point) selected by the user. Some of the relevant code is shown below:
var params = new esri.tasks.BufferParameters();
params.distances = [values.distance]; //the input distance
params.geometries = [gr.geometry]; //the input geometry
params.unit = esri.tasks.GeometryService.UNIT_FOOT;
params.outSpatialReference = mapView.map.spatialReference; //always 3857
params.bufferSpatialReference = gr.geometry.spatialReference; //always 3857
esri.config.defaults.io.corsEnabledServers.push('mydomain.com');
esri.config.defaults.io.proxyUrl = 'https://serverdomain.com/proxy';
var gsvc = new esri.tasks.GeometryService('https://tasks.arcgisonline.com/ArcGIS/rest/services/Geometry/GeometryServer');
gsvc.simplify(params.geometries, function(geometries){
params.geometries = geometries;
gsvc.buffer(params, function(geometries){
//add output geometry to the map and perform spatial query with it
}, function(err){
//handle error
});
}, function(err){
//handle error
});
The problem is that, if I use an input distance of 500 (feet), then measure the distance from the center line of the input geometry on self._queryGeometry, using ESRI's measurement tool, the actual width of the polygon created is something like 370 feet on either side of the center line.
I've managed to get this to work more accurately using the Illinois State Plane spatial reference, as my test objects are in Illinois, but the logic needs to work everywhere.
When I try various incarnations of doing a geodesic buffer, the input distance unit seems to get ignored and, using an input distance value of 500, I get a buffer that spans the entire world! Either that or the results are exactly the same, depending on how things are set up.
I believe I need to do a geodesic buffer, but I have absolutely no idea how to go about that in such a way that the geometry service will actually pay attention to the units I'm sending in.
Any ideas would be greatly appreciated. Let me know if I've left anything out.
Sounds like there might be a spatial reference issue somewhere. You can try re-projecting geometry into 3857 if that's what the map is and I would inspect the geometry being returned from the buffer and simplify to make sure it looks like what your expecting. I have had issues with the geometry service area's and length's returning slightly incorrect geometries and in my case it ended up being an issue with an incorrect spatial reference. Also, I know you said you can't upgrade, but 3.13 is out and can do geometry options locally without the need for a proxy or any network requests, if possible, it would be worth trying out.