I'm relatively new to Mapbox and its GL JS Library, but so far have been really impressed at its capabilities.
I'm currently working with a dataset of approximately 100,000 points and am trying to devise a way of quickly generating/visualising a continuous interpolated surface within the convex hull of the dataset I have (essentially trying to generate something that's as fast/responsive as the heatmap function, but looking to interpolate from the point data values rather than spatial density).
The documentation for the heatmap functionality discusses this exact scenario:
Among maps you'll find on the web, there are two common categories of
heatmaps: those that encourage the user to explore dense point data,
and those that interpolate discrete values over a continuous surface,
creating a smooth gradient between those points. The latter is less
common and most often used in scientific publications or when a
phenomenon is distributed over an area in a predictable way. For
example, your town may only have a few weather stations, but your
favorite weather app displays a smooth gradient of temperatures across
the entire area of your town. For your local weather service, it is
reasonable to assume that, if two adjacent stations report different
temperatures, the temperature between them will transition gradually
from one to the next.
But then proceeds to explain this is less common and there's no documentation/example provided for this type of application.
At this stage I've tried converting the points to voronoi cell polygons and colour coding by data value (a nearest neighbour approach to visualising), but the render seems to struggle with 100,000 points at lower zoom levels (0 through 8). Does anyone know if it's possible to create a fast-rendering surface interpolation from point values? Any examples would be fantastic.
The Mapbox Delaunator library is a very fast JavaScript library for Delaunay triangulation of 2D points. d3-delaunay and d3-geo-voronoi are both based off of this library. You could use these libraries and then display the results on your map via a custom style layer.
Looks like what you are trying to do is some sort of Spatial Interpolation.
Here is a summary of popular methods typically used for this purpose.
https://michaelminn.net/tutorials/r-spatial-interpolation/
Related
Creating a map similar to the one found here:
https://www.plantmaps.com/interactive-california-2012-usda-plant-zone-hardiness-map.php
Have all the data needed.
Create polygons around each one without distance inbetween. No overlapping or similar.
Drawing it all by hand on geojson.io, seems impossible.
I want to create polygons around each [point] and make sure there is no distance between them.
What you're describing here is a tessellation. Depending on your data, you might opt for a regular tessellation (i.e. creating a grid of squares or hexagons, paying a modicum of attention to the units of your coordinate system) or the well-known-among-GIS-people Voronoi tessellation. Note that a Voronoi tessellation created over a regular grid of points will result in a regular grid of polygons.
There are plenty of tools for Voronoi tessellations. For javascript and GeoJSON, my tool of choice would be turf.js's voronoi module.
Here are some things to consider:
creating polygon from a point is simple enough. For example you use the point as the center of a regular polygon and devide 2PI by the number of sides and step through the points to create the polygon. But, what is the radius? That depends on the projection you are using. I happen to use OpenLayers and really like this map control. It's default projection it EPSG:3857. So, the coordinates are already in meters - so easy. But if your points are in long/lat then you have to do some math. it may be easier to transform to a different projection temporarily. An opensource library that is really nice for gis calculations if you need one is Turf.
You mentioned also about having non-overlapping polygons? Well, in this case you will have a lot of gaps if you use regular polygons. To have non-overlapping polygons with, as you put it no distance between them is an interesting constraint. Now you are dealing with different shaped polygons. And an algorithm for handling that is pretty intense. I know that MapInfo GIS has a feature for adjusting polygons to be non-overlapping. But, in a JavaScript environment with GeoJSON, you are probably talking about server side logic for this.
That map you are looking at looks like it is using Leaflet with svg overlays.
I designed an ad hoc library to write SVG images. I use it for the graphical visualisation of large sparse matrices : homology matrices of the comparison of two genomes.
In my case, a usual homology matrix is 20 000 x 20 000 with around 30 000 scattered dots (representing homologies), for instance this homology matrix.
My ad hoc library is not mature enough for new developpements and I am interested by the 3D.js library. I also found the graphical library Cairo and its binding pyCairo interesting.
My needs are :
the possibility to output SVG, PNG or PDF images of the matrix.
when the mouse get over a dot of the homology matrix I would like to show specific informations.
a plus would be the ability to zoom in an area of the matrix, and show new details above a zoom threshold.
I know that 3D.js has dynamic features (interesting for navigating/zooming through the sparse matrix) but I wonder if it will display fast enough my data. It might be a better idea to use Cairo in an application and recompute the image for different zoom levels.
In brief I feel that I have 2 solutions :
write a HTML file, with a 3D.js script that adds SVG elements of the homology matrix and interactivity. 3D.js would be great for sharing matrices from a server.
write a dedicated app (e.g. with GTK) and draw the matrix inside the app with Cairo. Interactions would then be added by asking Cairo to re-draw each time the user interacts with the app.
Do I miss something ? Which solution would you use ?
I'm looking at plotting a scatterplot with a large number of points (500,000 and upwards).
Currently, we're doing this in Python with Matplotlib. It plots the points, and it provides controls to pan and zoom. I don't believe it provides any clustering or points, it just plots them all - doesn't make much sense at the zoomed out view, I suppose, but you can zoom in and they're all there.
I was looking at doing the chart in JavaScript, to make it a bit easier to distribute. I was looking at D3.js, to see if something similar is feasible there. I did find this example of a basic scatterplot:
http://bl.ocks.org/mbostock/3887118
Firstly, would you be able to plot that number of points? (500,000 and upwards) I was under the impression you couldn't due to the overhead of all the DOM objects? Are there ways around this?
Secondly, is there any kind of clustering available, either a library or even just an example of this being done in D3.js?
Thirdly, if anybody knows any good examples of pan/zoom functionality and clustering, or even just a packaged JS library that handles it, that would be awesome.
Fourth, it would be also nice to have click handlers for each point - and to display some text either in a overlay, or even just in a separate window. Any thoughts on this?
Can you draw half a million points with D3? Sure, but not with SVG. You'll have to use canvas (here's a simple example with 10,000 points that includes brush-based selection: http://bl.ocks.org/emeeks/306e64e0d687a4374bcd) and that means that you no longer have individual elements to assign click handlers to. You will not be able to render half a million points with SVG, because all those DOM elements will choke your interface, as you mentioned.
D3 does include quadtree support that can be leveraged for clustering. It's in use in the above example to speed up search but you could use it to nest elements in proximity at certain scales.
Ultimately, your choices are:
1) Some other library/custom implementation that renders in canvas and polls the mouse position to give you the data element rendered at that point.
2) A sophisticated custom D3 approach that nests elements in proximity and only renders SVG elements appropriate at the zoom level and canvas position (pan) you're at.
Yes, D3.js can be made to work with million scale data with two things:
pre-rendering on the server side. For more see here: https://mango-is.com/blog/engineering/pre-render-d3-js-charts-at-server-side/
By aggregating (or clustering) part of the data so that user can interact and expand the graph if need be. For this use collapsible nodes if you can (http://bl.ocks.org/mbostock/1062288).
Also avoid using force layout. It takes time to settle and converge to a stable positioning.
For clustering libraries, I would pick one up off the shelf. I would choose the scikits library from python, there are many in JavaScript but they are not very robust as they mostly cover k-means or hierarchical clustering. I would precalculate the coordinates using scikits by clustering and then render it using D3.
D3 handles Pan and zoom. Again click handlers and text display are available in D3. (http://bl.ocks.org/robschmuecker/7880033)
My mobile web app utilizes navigator.geolocation.watchPosition to track polylines and waypoints for historical walking tours. (A customer requirement no-native app)
My first testing was done in a vehicle due to bad weather and seemed to produce reasonable results when plotting the polyline on an embedded Google Map in real-time. However, when walking it produced a more zig zag line.
Some research has revealed that I need to filter the retrieved location instead of just calculating the distance of two points.
The suggested filter, Kalman filter, seems to be something over my head, and I'm wondering whether there is an open source implementation for JavaScript available, which can be of help for my use case.
An open source C/C++ implementation of the Kalman filter that could be compiled to JavaScript using emscripten might be of help too.
I posted a simple Kalman filter that can be used for geolocation data in an answer to this question: Smooth GPS data . It was designed for Android, where the GPS location readings are accompanied by a single error-bar number called accuracy which is measured in metres. That means that for each GPS location reading, with 67% probability the true location is within the circle centered on the specified location, and the radius of the circle is given by the accuracy number. If your situation is different then a bit of work will be required, and if so I would suggest that you edit the question to spec out in more detail what inputs you have for the Kalman filter.
I wanted to ask you about the way to make spatial interpolation in google maps API. This means- I have values for some points inside an area and want to recalculate values for every point in this area (and create an raster overlay).
In "offline" GIS softwares, there are more methods to do this. For example Kriging (http://en.wikipedia.org/wiki/Kriging).
Thanks for answer
There is no real practical way to do such a thing. Spatial Interpolation is something you really need a full-blown GIS to accomplish. It requires storing, analyzing, and presenting data in way that ArcGis or QGis is best suited for. There is some new feature in ArcGis Online that works with Google Maps API but I don't know much about it and am unsure if it would be any help for you but a link is here.
I know, its not something geostatiscically correct, but I have made this:
- I have created a net of rectangle polygons.
- I have connected all my points to polygon they lie and have made mean value for polygons with at least one point
- I have checked every polygon without points and at least 2 neighbourly polygons with points, and have made mean from this neighbourly values
- I have checked again all my polygons with values and at least 2 neighbourly polygons with values and smoothed origin values according to the neighbours´ values.