We Keep Coding

D3.js projection shows only random lines [duplicate]

I am trying to make this map of the us scale smaller. Either to my SVG, or even manually.
This is my code in its simplest from:
function initializeMapDifferent(){
var svg = d3.select("#map").append("svg")
.attr("width", 1000)
.attr("height", 500);
d3.json("https://d3js.org/us-10m.v1.json", function (error, us){
svg.append("g")
.attr("class", "states")
.selectAll("path")
.data(topojson.feature(us, us.objects.states).features)
.enter().append("path")
.attr("fill", "gray")
.attr("d", d3.geoPath());
});
}
I have tried something like:
var path = d3.geoPath()
.projection(d3.geoConicConformal()
.parallels([33, 45])
.rotate([96, -39])
.fitSize([width, height], conus));
but every time I add anything to my path variable I get NAN errors from the internal parts of D3. Thanks for any help!

Why the data doesn't project properly
The key issue is that your data is already projected. D3 geoProjections use data that is unprojected, or in lat long pairs. Data in the WGS84 datum. Essentially a d3 geoProjection takes spherical coordinates and translates them into planar cartesian x,y coordinates.
Your data does not conform to this - it is already planar. You can see most evidently because Alaska is not where it should be (unless someone changed the lat long pairs of Alaska, which is unlikely). Other signs and symptoms of already projected data may be a feature that covers the entire planet, and NaN errors.
That this is a composite projection makes it hard to unproject, but you can display already projected data in d3.js.
"Projecting" already projected data
Null Projection:
Most simply, you can define your projection as null:
var path = d3.geoPath(null);
This will take the x,y data from the geojson geometries and display it as x,y data. However, if your x,y coordinates exceed the width and height of your svg, the map will not be contained within your svg (as you found in your example with .attr("d", d3.geoPath());).
The particular file in this question is pre-projected to fit a 960x600 map, so this is ideal for a null projection - it was designed with the dimensions in mind. Its units are pixels and all coordinates fall within the desired dimensions. However, most projected geometries use coordinate systems with units such as meters, so that the bounding box of the feature's coordinates may be millions of units across. In these cases the null projection won't work - it'll convert a map unit value to a pixel value with no scaling.
With d3, A null projection is commonly used with geojson/topojson that is preprojected to fit a specified viewport using a d3 projection. See command line cartography for an example (the example uses unprojected source files - the same issues that arise from using a d3 projection on projected data apply in both browser and command line). The primary advantage of preprojecting a file for use with a null projection is performance.
geoIdentity
If all you need is to scale and center the features, you can use a geoIdentity. This is implements a geoTransform but with standard projection methods such as scale, translate, and most importantly - fitSize/fitExtent. So, we can set the projection to a geoIdentity:
var projection = d3.geoIdentity();
This currently does the same as the null projection used above, it takes x,y data from the geojson geometries and displays it as x,y data with no transform - treating each coordinate in the geojson as a pixel coordinate. But, we can apply fitSize to this (or fitExtent) which will automatically scale and translate the data into the specified bounding box:
var projection = d3.geoIdentity()
.fitSize([width,height],geojsonObject);
or
var projection = d3.geoIdentity()
.fitExtent([[left,top],[right,bottom]], geojsonObject);
Keep in mind, most projected data uses geographic conventions, y=0 is at the bottom, with y values increasing as one moves north. In svg/canvas coordinate space, y=0 is at the top, with y values increasing as one moves down. So, we will often need to flip the y axis:
var projection = d3.geoIdentity()
.fitExtent([width,height],geojsonObject)
.reflectY(true);
This particular dataset: https://d3js.org/us-10m.v1.json was projected with a d3 projection, so its y axis has already been flipped as d3 projections project to a svg or canvas coordinate space.
geoIdentity Demo
var width = 600;
var height = 300;
var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height);
d3.json("https://d3js.org/us-10m.v1.json", function (error, us){
var featureCollection = topojson.feature(us, us.objects.states);
var projection = d3.geoIdentity()
.fitExtent([[50,50],[600-50,300-50]], featureCollection)
var path = d3.geoPath().projection(projection)
svg.append("g")
.attr("class", "states")
.selectAll("path")
.data(featureCollection.features)
.enter().append("path")
.attr("fill", "gray")
.attr("d", path);
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/4.6.0/d3.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/topojson/2.2.0/topojson.js"></script>
geoTransform
If you want a little more control over how that data is displayed you can use a geoTransform.
From Mike Bostock:
But what if your geometry is already planar? That is, what if you just
want to take projected geometry, but still translate or scale it to
fit the viewport?
You can implement a custom geometry transform to gain complete control
over the projection process.
To use a geoTransform is relatively straightforward assuming that you do not want to change the type of projection. For example, if you want to scale the data you could implement a short function for scaling with geoTransform:
function scale (scaleFactor) {
return d3.geoTransform({
point: function(x, y) {
this.stream.point(x * scaleFactor, y * scaleFactor);
}
});
}
var path = d3.geoPath().projection(scale(0.2));
Though, this will scale everything into the top left corner as you zoom out. To keep things centered, you could add some code to center the projection:
function scale (scaleFactor,width,height) {
return d3.geoTransform({
point: function(x, y) {
this.stream.point( (x - width/2) * scaleFactor + width/2 , (y - height/2) * scaleFactor + height/2);
}
});
}
var path = d3.geoPath().projection(scale(0.2,width,height))
geoTransform Demo:
Here is an example using your file and a geoTransform:
var width = 600;
var height = 300;
var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height);
function scale (scaleFactor,width,height) {
return d3.geoTransform({
point: function(x, y) {
this.stream.point( (x - width/2) * scaleFactor + width/2 , (y - height/2) * scaleFactor + height/2);
}
});
}
d3.json("https://d3js.org/us-10m.v1.json", function (error, us){
var path = d3.geoPath().projection(scale(0.2,width,height))
svg.append("g")
.attr("class", "states")
.selectAll("path")
.data(topojson.feature(us, us.objects.states).features)
.enter().append("path")
.attr("fill", "gray")
.attr("d", path);
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/4.6.0/d3.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/topojson/2.2.0/topojson.js"></script>
Unproject the data
This method is useful under certain circumstances. But it requires you to know the projection that was used to create your data. Using QGIS/ArcGIS or even mapshaper you can change the data's projection so that it is "projected" as WGS84 (aka EPSG 4326). Once converted you have unprojected data.
In Mapshaper this is pretty easy with shapefiles, drag in the .dbf, .shp, and .prj files of a shapefile into the window. Open the console in mapshaper and type proj wgs84.
If you don't know the projection used to create the data, you can't unproject it - you don't know what transformation was applied and with what parameters.
Once unprojected, you can use regular d3 projections as normal as you have coordinates in the correct coordinate space: longitude latitude pairs.
Unprojecting is useful if you also have unprojected data and want to mix both in the same map. Alternatively you could project the unprojected data so that both use the same coordinate system. Combining unmatched coordinate systems in a map with d3 is not easy and d3 is likely not the correct vehicle for this. If you really want to replicate a specific projection with d3 to match features that are already projected with unprojected features, then this question may be useful.
How can you tell if your data is projected already?
You could do check to see that the geometry of your features respect the limits of latitude and longitude. For example, if you were to log:
d3.json("https://d3js.org/us-10m.v1.json", function (error, us){
console.log(topojson.feature(us, us.objects.states).features);
});
You will quickly see that values are in excess of +/- 90 degrees N/S and +/- 180 degrees E/W. Unlikely to be lat long pairs.
Alternatively, you could import your data to an online service such as mapshaper.org and compare against another topojson/geojson that you know is unprojected (or 'projected' using WGS84).
If dealing with geojson, you may be lucky enough to see a property that defines the projection, such as: "name": "urn:ogc:def:crs:OGC:1.3:CRS84" (CRS stands for coordinate reference system) or an EPSG number: EPSG:4326 (EPSG stands for European Petroleum Survey Group).
Also, if your data projects with a null projection but not a standard projection (scaled/zoomed out to ensure you aren't looking in the wrong area), you might be dealing with projected data. Likewise if your viewport is entirely covered by one feature (and you aren't zoomed in). NaN coordinates are also a potential indicator. However, these last indicators of projected data can also mean other problems.
Lastly, the data source may also indicate data is already projected either in meta data or how it is used: Looking at this block, we can see that no projection was used when the geoPath is defined.

Random lines when loading topojson file?

I'm trying to load my topojson file of the map of ontario using d3, but all I get is a bunch of random lines, just like Random lines when loading a TopoJSON file in D3.
My file is using WGS84 so that's not the issue either. What am I doing wrong? Js code below.
var width = 960, height = 700;
var svg = d3.select('#map').append('svg')
.attr('width', width)
.attr('height', height)
d3.json('CensusSubDiv.json', function(error, CensusSubDiv) {
if (error) return console.error(error);
svg.append('path')
.datum(topojson.feature(CensusSubDiv, CensusSubDiv.objects.CensusSubDivision))
.attr('d', d3.geo.path().projection(d3.geo.mercator()))
.attr('id', 'ont')
;
});

So, as I mentioned in the comments, your data may use WGS84 as a datum, but D3 requires WGS84 to be used as a "projection" (d3 requires latitude and longitude, which represent points on a three dimensional globe, and thus are actually unprojected, hence my quotes). Projected data, consequently, is labelled with WGS84 sometimes as well as other identifiers.
The good news is you can display projected data fairly easily - the better news is that this is faster. Instead of doing three dimensional math, we just transform and scale the data before plotting it. The bad news is that this works easiest with d3 v4 (which my example uses, it only changes a few points: eg: d3.geo.path -> d3.geoPath, d3.geo.projection -> d3.geoProjection, etc).
As your data is already projected, we can use d3.geoIdentity which allows us to use the projection.fitSize() (which modifies scale and translate) method but otherwise doesn't project the data. FitSize takes an array for the width/height of the display area and a geojson object:
projection.fitSize([width,height],geojsonObject)
fitSize is not part of d3v3, but we're on d3v5, so updating isn't a bad thing, projection.fitExtent allows for margins
As in SVG coordinate space, Y values start at 0 at the top of the screen and increase as one moves down, and in most projected coordinate spaces, Y values start at the bottom of the map and increase as one moves north, we also need to flip the identity on the y axis:
var projection = d3.geoIdentity()
.reflectY(true)
.fitSize([width,height],geojsonObject)
This allows us to display the data fairly easily. See this block for your data with the above noted changes.
But, here's the caveat, you have data that is already projected, if you don't know how that data was projected, you can't align other geographic data that is unprojected or otherwise projected: you can't project a point with given longitude latitude because you don't know how to project it in the same manner as your already projected data.
This might not be a concern, for example in a choropleth perhaps you only need the outline of the census subdivisions. But if you wanted to place cities on top of the map, you'd face some difficulties if the coordinates for those cities weren't already projected in the same manner as your census data.
If you wanted to overlay other geographic data (unprojected data) over Ontario, then you will need to unproject your data. In mapshaper, when you initially import the data into the window (ensuring you drag the .prj file too - otherwise mapshaper won't know what projection to reproject the data from), you can open the console and type proj WGS84, this should give you coordinates that are in degrees (though the topojson will still have encoded integer coordinates, the coordinates are stored in plain text if exporting to geojson). Of course, you'll need to use more typical projection, such as a d3.geoMercator() again if you chose to unproject your data.

Issue with D3.JS and Flask-trying to get map of U.S

I am working on a project using D3.JS and Flask trying to display a map of the U.S. The problem is that the map is not displaying. I know that that the SVG element is being attached, that the json data is coming through but the map itself is not coming up. I then tried to create a simple index.html document and placed the json file within that project and using a Python simple server was able to see the map. So now for the code.
To start here is the JS code:
<script type="text/javascript">
//Define default path generator
let path = d3.geo.path();
//Creating the SVG element and attaching it to the page.
let svg = d3.select("#us-map")
.append("svg")
.attr("width", 900)
.attr("height", 700);
//This function will get the data for the map of the U.S
d3.json("/json", function(json){
console.log(json.features) //This shows an array of 52 objects
//Bind data and create one path per GeoJSON feature
svg.selectAll("path")
.data(json.features)
.enter()
.append("path")
.attr("d", path);
});
</script>
Python code:
#app.route("/json")
def json():
data = Data()
data_file = data.convert_json_for_d3()
#return data_file
return data_file.to_json()
The method in the class that I am calling from the above code:
def convert_json_for_d3(self):
self.__data = pd.read_json('us-states.json')
df = self.__data
return df
The HTML:
<section id="us-map">
<h1>UFO Sitings in the U.S.</h1>
</section>
I have been working on this for a day now. At first, getting the json data working with the D3 was the hard part. I figured I had it at that point. However, as I stated, the map is not showing up. It does work if I place it into a simple index.html file along with the json. So I am wondering if the issue is with Flask? One final point, I am following along code from the book Interactive data visualization for the Web. The github repo that I was pulling code from is here: https://github.com/alignedleft/d3-book/blob/master/chapter_12/01_paths.html
Working with a simple HTML file that data appears as this:
object {type: "FeatureCollection", features: Array(52)}
features:Array(52)
0:Object
geometry: Object
coordinates: Array(1)
0: Array (33)
0
1
:
1:Object
:
Data in my Flask Project looks like this:
Object {features: Object, type: Object}
features: Object
0: Object
geometry: Object
coordinates: Array(1)
0: Array (33)
0:Array(2)
1: Object
:
So both the array of objects are almost identical-except for the very first line.
Thank you for any help.

Two issues (based on comments below and updated question)
One is the geojson in your Flask project is not valid. For example features should be an array - not an object, type should still be featureCollection (This is discussed in the comments below). Somewhere along the line your geojson structure is changed to something that isn't accepted by d3.geo.path() as valid geojson.
Two is that you are not using a projection to convert your latitude and longitude pairs (in the geometry of each feature) to svg coordinate space.
For the second issue:
When using d3.geo.path(); you need to specify a projection (eg: d3.geo.path().projection(d3.geo.mercator());). If you do not provide a projection, a null projection is used. This interprets the provided geojson without applying any transformation - essentially your values are translated into pixel values on the svg.
The United States is in the western hemisphere, all the longitudes are therefore negative (and as north south geographic values increase as one goes north and svg y values increase as one moves down, the image will be upside down). A null projection (with no other transform on the svg) will draw a path that is to the left of your svg's boundaries. This is why these three things can happen:
the "SVG element is being attached"
"that the json data is coming through", but
"the map itself is not [visually] coming up"
The geojson referenced in the chapter you reference is unprojected data - it contains a latitude and longitude pair for each vertex: [-133.05341,56.125739]. This is ideal, as a d3.geo.projection takes this type of coordinate, points on a three dimensional ellipsoid, and projects them onto a two dimensional Cartesian plane. If you had already planar data you would need to use a geo transform.
So, to visualize your data you will need to select a projection to convert latitude and longitude pairs into appropriate values. USA albers might be easiest as it is a composite projection that scales down Alaska and moves both Alaska and Hawaii closer to the continental US. The Albers USA also sets all the projection parameters to show the US without modification to projection parameters (center, rotation, parallels, scale etc), while other projections will require setting proper parameters.
I got your data to display with an albersUsa projection using the following code:
var projection = d3.geo.albersUsa().translate([width/2,height/2]);
var path = d3.geo.path().projection(projection);
d3.json("us.json",function(data) {
svg.append("path")
.datum(data)
.attr('d',path)
.attr('fill','steelblue')
.attr('stroke','black');
});
Here is a working demo and a screen shot:
If your map is too large for your svg (or vice versa), you can set the scale with projection.scale(n), with smaller numbers zooming out, larger numbers zooming in.

Importing JSON for D3 map manipulation

tl;dr
I'm creating arcs between two points on a map as shown here, but I want to save that huge array of coordinates in a json/csv file. How should I save that file and what should I change in the script so it correctly parses the json/csv file.
Long version
I'm trying to draw arcs between two points on a map as shown here.
Here's what I did.
First I defined my coordinates (hard-coded). Notice that they are lon/lat.
var trainRoutes = [
{sourceLocation: [94.91542,27.485983],targetLocation: [77.549934,8.079252]}
];
Then I defined my arcs.
var arcs = svg.append("g").attr("class","arcs"); // adding a class for CSS stuff
And finally the code for drawing them.
arcs.selectAll("path")
.data(trainRoutes)
.enter()
.append("path")
.attr('d', function(d) {
return makeArc(d, 'sourceLocation', 'targetLocation', 1);
});
makeArc is just a function that returns a string for the path to be drawn, again, as shown here.
As you can see, I'm just creating one arc with two sets of coordinates (say city A and city B). I would like to draw more arcs but not clutter my index.html. I want to put the coordinates in a JSON file and create arcs from there rather than declaring coordinates within index.html.
I did try putting the coordinates in a JSON file and used d3.json to do the same.
d3.json("trainRoutes.json", function(json){
arcs.selectAll("path")
.data(json)
.enter()
.append("path")
.attr('d', function(d) {
return makeArc(d, d.sourceLocation, d.targetLocation, 1); });
});
But this didn't work and the console says arcs.selectAll("path") is not a function. How do I solve this issue? I'm open to using both d3.csv/d3.json, just want to move the coordinates to another file.
Here's what my JSON file (trainRoutes.json) looked like in case my declaration of JSON was wrong.
[
{sourceLocation: [94.91542,27.485983],targetLocation: [77.549934,8.079252]}
]

D3 geo: getting projection.clipAngle to work on all specified elements

I'm a newcomer to D3 and I'm trying to make a world globe with some points ("pins") on it. Demo here: http://bl.ocks.org/nltesown/66eee134d6fd3babb716
Quite commonly, the projection is defined as:
var proj = d3.geo.orthographic()
.center([0, 0])
.rotate([50, -20, 0])
.scale(250)
.clipAngle(90)
.translate([(width / 2), (height / 2)]);
the clipAngle works well for the svg paths, but not the pins (which are svg circles). As you can see on the demo, the pin that sits between Iceland and Greenland should be hidden (it's Taiwan).
So I suppose the problem comes from these lines, but I can't understand why:
.attr("transform", function(d) {
return "translate(" + proj([ d.lng, d.lat ]) + ")";
});

It is not sufficient to just set the clipping radius via clipAngle() to get the desired behavior. The projection alone will not do the clipping, but just calculate the projected coordinates without taking into account any clipping. That is the reason, why Taiwan gets rendered, although you expected it to be hidden.
But, thanks to D3, salvation is near. You just need to re-think the way you are inserting your circles representing places. D3 has the mighty concept of geo path generators which will take care of the majority of the work needed. When fed a projection having a clipping angle set, the path generator will take this into account when calculating which features to actually render. In fact, you have already set up a proper path generator as your variable path. You are even correctly applying it for the globe, the land and the arcs.
The path generator will operate on GeoJSON data, so all you need to do is convert your places to valid GeoJSON features of type Point. This could be done with a little helper function similar to that used for the arcs:
function geoPlaces(places) {
return places.map(function(d) {
return {
type: "Point",
coordinates: [d.lng, d.lat]
};
});
}
With only minor changes you are then able to bind these GeoJSON data objects to make them available for the path generator which in turn takes care of the clipping:
svg.selectAll(".pin") // Places
.data(geoPlaces(places))
.enter().append("path")
.attr("class", "pin")
.attr("d", path);
Have a look at my fork of your example for a working demo.