I am using the natural earth 10m files to reduce, simplify, and quantize them into TopoJSON. However, for my use case, I'm attempting to store the data on the server and request a country or region I am currently interested in to display with d3. Because of this I need to keep my file size fairly small.
I was able to write some hacky code to strip out the region geometry data and arcs from the full TopoJSON file and serve that at a reasonably small size. Unfortunately, I've come to realize that doing a mesh on this does not give me any results. If, however, I do not reindex all of the arcs and instead just inject empty arrays into the arc positions I don't need, mesh works fine.
Basically, my question is, is there a way for me to strip out a specific country or region from a large TopoJSON file? I don't mean just the geometry object since that's easy enough to do, but also reduce the main arcs array to only include what is needed. I'm struggling to find a clean way to do this.
Thanks!
Related
I am working with ~150,000 polylines that span North America. Across all the lines I'd like to draw/animate from the start to the end of each line. I started down the road of adding vertices to an array and updating a Mapbox source like this example. This was not feasible given the size of the entire dataset. In geojson form it's about 800mb and manipulating this in the browser seemed unreasonable.
As an alternate approach, I've broken each polyline into equally sized segments of 10km in length. I added an attribute on each segment which represents its percent distance from the start. I created an interval below that triggers a setFilter event, filtering segments based in the percent from start.
currentLinPos=0;
lineInterval=setInterval(function(){
currentLinPos+=.01;
if(currentLinPos>0.9999){
currentLinPos=0
}
map.setFilter('lines',['<','linePrc',currentLinPos])
}, 250);
Even when working with a small subset of the data, performance choppy and lags. I expected a better result from Mapbox GL but perhaps I need to rethink my data structure or approach entirely. Or would using something like DECK.GL be far superior here?
I am working on a system to procedurally build meshes for "mines", right now I don't want to achieve visual perfection I am more focused on the basic.
I got the point in which I am able to generate the shape of the mines and from that generating the 2 meshes, one for the ground and one for the "walls" of the mine.
Now I am working on getting the UV mapping right but my problem is that the ground is really hard to map to UV coordinates properly and I am currently not able to get it right.
For the tessellation I am using a constrained version of the delaunay triangulation to which I added a sub-tessellation what simply splits the triangles at least once and keeps splitting them if the area of the triangle is greater than X.
Here a 2D rendering of the tessellation that highlight the contours, the triangles and the edges
Here the result of the 3D rendering (using three.js and webgl) with my current UV mapping applied (a displacement map as well, please ignore it for now).
I am taking a naive approach to the UV mapping, each vertex of a triangle in the grid is translated to values between 0 and 1 and that's it.
I think that, in theory should be right, but the issue is with the order of the vertexes that is creating a problem but if that would be the case the texture should be shown rotated or oddly not just oddly AND stretched like that.
Once I will get the UV mapping right, the next step would be to correctly implement the
I am currently writing this in javascript but any hint or solution in any language would be alright, I don't mind converting and/or re-engineering it to make it work.
My goal is to be able to procedurally build the mesh, send it to multiple clients and achieve the same visual rendering. I need to add quite a few bits and pieces after this other step is implemented so I can't rely on shaders on the client side because otherwise being able to place tracks, carts or something else on the ground would just be impossible for the server.
Once I will get these things sorted out, I will switch to Unity 3D for the rendering on the client side, webgl and three.js are currently being used just to have a quick and easy way to view what's being produced without the need of a client/server whole infrastructure.
Any suggestion?
Thanks!
I sorted out the issue in my code, it was pretty stupid though: by mistake I was adding 3 UV mappings per triangle and not 1 per point causing an huge visual mess. Sorted out that, I was able to achieve what I needed!
https://www.youtube.com/watch?v=DHF4YWYG7FM
Still a lot of work to do but starts to look decent!
I have longitude and latitude coordinates which represent different types of geometry shapes (point, multi-point, polygon, multi-polygon and line string etc) that are stored in MongoDB. I'd like plot them (say each shape as a layer) using JavaScript. Map is not needed as a background although these coordinates represent building etc in the real world. I understand there are calculations as such are involved to convert coordinates to x and y on graph/canvas but not sure where to begin tbh. I'm very new to GIS and JavaScript so go easy on me please. I know this question sounds a bit vague but bottom line is I have coordinates that represent real world elements and want to plot them without a map background. I'd appreciate advises and suggestions. Many thanks.
The easiest way to do it is to use a Javascript mapping library such as OpenLayers or Leaflet. You could also do it with D3, although should you want to make your map more full-featured later, it will be much easier with a library that was designed for that type of thing.
If you are storing your data in Mongo as GeoJSON (GeoJSON website), you can add it to OpenLayers as a GeoJSON layer.
Disclaimer- My opinions might be slightly biased because I used to work for Boundless
I am using D3 to render a vector map, based off of a GeoJSON dataset.
I have an existing path and I need to transform the path using a tween. To do this I need to use D3 to lookup the current projection object for this path. However, I can't seem to figure out the syntax to achieve this. Is it possible to look up the projection of an existing D3 path?
I can look up the path with:
const mapPath = d3.select('path.states');
But, I'm not sure where to go from there to get the projection object from this path. Thanks for any help or direction you might be able to provide.
Since (based on the discussion in the comments) it sounds like the code that used a projection to render the map lives in the same app as the code that needs that projection elsewhere in the app, then this is really a question of app architecture — not projection detection. There are certainly ways to architect it such that the projection is "handed down" to the subroutine that need it, but that really depends on the frameworks and your implementation so far etc.
So without getting into architecture, another way to do this, is following Mark's suggestion (and in fact indicated by the code snippet in your original question) using d3's data binding to hang the projection off of the DOM node that the projection helped produce. IMO, compared with an app architecture solution, using data binding for this purpose is the lesser option, because it could make your code harder to follow bug prone. But it will work and, in a simple app at least, it's reasonable to do it this way. So...
Presumably, somewhere you have code that created and rendered the paths, and it might look something like this:
var projection = d3.geo.mercator();
var mapG = svg.append('g').attr('class', 'map');
var mapG.selectAll('path').data(myMapData).enter()
.append('path')
.attr("d", d3.geo.path().projection(projection));
I'm sure yours looks different, but hopefully it's close to the above, because I need to point out that the are N paths, corresponding to N entries in myMapData, and that each path already has a datum associated with it (the datum of the geo feature it represents). So a good place to hang your projection is NOT on the paths (because there are multiples, and they have have data bound to them already) but rather on the parent of the paths — mapG in this case. The way you do it is using .datum() to bind the projection to the DOM node:
mapG.datum(projection);
Now, from that other place in your code, where you need to look up the projection, you'd use datum() to extract it:
var mapG = d3.select('svg g.map')
var projection = mapG.datum()
I'm looking at plotting a scatterplot with a large number of points (500,000 and upwards).
Currently, we're doing this in Python with Matplotlib. It plots the points, and it provides controls to pan and zoom. I don't believe it provides any clustering or points, it just plots them all - doesn't make much sense at the zoomed out view, I suppose, but you can zoom in and they're all there.
I was looking at doing the chart in JavaScript, to make it a bit easier to distribute. I was looking at D3.js, to see if something similar is feasible there. I did find this example of a basic scatterplot:
http://bl.ocks.org/mbostock/3887118
Firstly, would you be able to plot that number of points? (500,000 and upwards) I was under the impression you couldn't due to the overhead of all the DOM objects? Are there ways around this?
Secondly, is there any kind of clustering available, either a library or even just an example of this being done in D3.js?
Thirdly, if anybody knows any good examples of pan/zoom functionality and clustering, or even just a packaged JS library that handles it, that would be awesome.
Fourth, it would be also nice to have click handlers for each point - and to display some text either in a overlay, or even just in a separate window. Any thoughts on this?
Can you draw half a million points with D3? Sure, but not with SVG. You'll have to use canvas (here's a simple example with 10,000 points that includes brush-based selection: http://bl.ocks.org/emeeks/306e64e0d687a4374bcd) and that means that you no longer have individual elements to assign click handlers to. You will not be able to render half a million points with SVG, because all those DOM elements will choke your interface, as you mentioned.
D3 does include quadtree support that can be leveraged for clustering. It's in use in the above example to speed up search but you could use it to nest elements in proximity at certain scales.
Ultimately, your choices are:
1) Some other library/custom implementation that renders in canvas and polls the mouse position to give you the data element rendered at that point.
2) A sophisticated custom D3 approach that nests elements in proximity and only renders SVG elements appropriate at the zoom level and canvas position (pan) you're at.
Yes, D3.js can be made to work with million scale data with two things:
pre-rendering on the server side. For more see here: https://mango-is.com/blog/engineering/pre-render-d3-js-charts-at-server-side/
By aggregating (or clustering) part of the data so that user can interact and expand the graph if need be. For this use collapsible nodes if you can (http://bl.ocks.org/mbostock/1062288).
Also avoid using force layout. It takes time to settle and converge to a stable positioning.
For clustering libraries, I would pick one up off the shelf. I would choose the scikits library from python, there are many in JavaScript but they are not very robust as they mostly cover k-means or hierarchical clustering. I would precalculate the coordinates using scikits by clustering and then render it using D3.
D3 handles Pan and zoom. Again click handlers and text display are available in D3. (http://bl.ocks.org/robschmuecker/7880033)