I'm trying to compare an array of lat/lng coordinates for a map to see if any "cluster" or group together. I want to remove the ones that are too close together so if there are 4-5 stacking on top of each other on a map, it wil only show 1 until you zoom in a bit more, and then it will recalculate all of them again.
I've tried comparing the array to itself, but it doesn't seem to give consistant results. Has anyone attempted something like this before?
JSON Example:
[
{
Latitude = "44.033843";
Longitude = "-79.48865499999999";
},
{
Latitude = "44.033843";
Longitude = "-79.48865499999999";
}]
Iterate the nodes and for a zoom level only display those that are beyond a set distance from each other. The haversine formula is simple enough to implement: example in JS here.
http://www.movable-type.co.uk/scripts/latlong.html
For the efficiency aspect, you probably don't want to calculate the entire list against the temporary list on every iteration so as a first level declutter a simple rounding works (every degree being ~60 miles from each other) ... start with rounding to the nearest 5 degrees, then 1, then 10ths, 100ths, etc as you zoom in. Ordering these lists and pulling unique array values first - then calculating distances from the resultant list.
There are certainly many other algorithms to do it - but at some point you have to calculate distances.
EDIT: this assumes you're happy to fudge things a little bit, and instead of worrying about the actual distance between points on the globe, you look at the "Manhattan" distance of their lat/long coordinates. It depends on how precise you need to be, and whether you have points near the Earth's poles. But for most practical purposes this assumption should be fine.
Suppose your desired precision is one decimal place. Then I would just iterate through the array, building up a has where the keys are the rounded coordinates and the values are arrays of lat/long pairs which round to the corresponding key.
hash = Hash.new
latLongArray.each { |point|
key = [point.lat.round(1), point.long.round(1)]
hash[key] = Array(hash[key]) + [point.lat, point.long]
}
This way you have them clustered, and you can in fact just put markers at the coordinates given by the keys themselves.
Related
I'm having a problem
I would like to ask what the most efficient way is to check if latitude and longitude coordinates are inside a range (for example 100 meters) from a list of latitudes and longitude points.
For example I have this list of coordinates:
[[48.34483,51.16.24517],[48.484,16.2585],[48.361,51.87739419],[6.38477205,51.87745015],[48.3645,51.16.73167],[6.38391099,51.87755068],[48.3575,16.725],[6.38380232,51.87720004],[6.38376297,51.87708017],[6.38375183,51.87704018],[6.38373055,51.8769829]]
I would like somehow that all points that are in a specific range (100m for example),
to be somehow grouped.
Is there any way how I can indicate that for example from the above list:
[48.484,16.2585],[48.361,51.87739419] and [48.3575,16.725]
are in a radius of 100m ( distance between these points is less then 100m) and they should be groped
Sounds like a great question for a GIS professional; you could perhaps post on gis.stackexchange.com. Are you using a mapping technology where you already have access to an API? The functionality that you're looking for are referred to as geometric operations. I'd start by looking into geometry functions available in an API which calculate the distance between points. You could find the geometric center of all of the points, then request the geometry API to create a buffer around that point. Next, query if each point falls within that buffer.
Found a post which might help with finding the center of the points here:
How do I find the center of a number of geographic points?
Also found a post on stackexchange which sounds very similar to yours, only the post is in reference to ArcGIS and the Point Distance (Analysis) tool:
https://gis.stackexchange.com/q/91571/81346
Ideally you'd use a geospatial db for this, to avoid performance issues when dealing with increasing numbers of points. MySQL, Postgres etc all support geospatial functions.
But as you've tagged your question with javascript, I'll post a JS solution. There's an npm package called haversine - with it, you should be able to loop through each point and return the other points that are within 100m. Something like:
// bring in haversine from npm
var haversine = require("haversine");
// define the full list of points
var data = [
[48.34483,51.1624517],
[48.484,16.2585],
[48.361,51.87739419],
[6.38477205,51.87745015],
[48.3645,51.1673167],
[6.38391099,51.87755068],
[48.3575,16.725],
[6.38380232,51.87720004],
[6.38376297,51.87708017],
[6.38375183,51.87704018],
[6.38373055,51.8769829]
];
var points = data.map(point => new Object({latitude: point[0], longitude: point[1]}));
// var to store results in
var results = [];
// loop through the points
points.forEach((pair) => {
var nearby = points;
// filter the full list to those within 100m of pair
nearby.filter(point => haversine(pair, point, {unit: 'mile'}) <= 100);
results.push({
'point': pair,
'nearby': nearby
});
});
console.log(results);
Note: I corrected some of the points in your list, which had double decimals so weren't valid
Conceptual Question
I am building a flight simulator in Three.js. I intend to rip CSV data for Latitude, Longitude, and Elevation from Google Earth and transfer it into arcGIS to create a Digital Elevation Model (DEM). I then want to create the terrain based on the DEM. I already have a splat map texture shader I wrote and things are looking good.
However, I will need to add models and more specifically text and landing zones for the towns. This will require accurate XYZ coordinates.
I figure this is an interesting problem. I have seen one question before on stackoverflow similar to this but it was not quite to the same depth I'm looking for.
1) How to create coordinate system that maps actual XYZ, Latitude, Longitude, Elevation data to a PlaneBufferGeometry?
My assumption is that if I take a hypothetical 100,000 x 100,000 map sample then I will need to create a Plane that has matching vert count and then maps 1:1.
new THREE.PlaneBufferGeometry( 100000, 100000, 100000, 100000 );
Then the trickier part of mapping lat long coordinates to this. Perhaps just a multiplier like * 100 or so per lat, long degrees?
2) How to create the most efficient data structure for this. It will contain a lot of data.
I am thinking the most efficient data structure would be an array with Z integers.
let vertArray = new Array(10000000000);
for (i = 0; i < 9999999999; i++) {
vertArray[i] = planeBufferGeometry.vertices[i].z;
}
Each 100,000 in the array would represent a Y coordinate, while each i value in said sections would be an X coordinate. The value of the Z coordinate would be stored in the array itself.
So hypothetically if I wanted to get X: 3, Y: 4, Z: ? it would be...
const xCoord = 3,
yCoord = 4,
index = (yCoord * 100000) + xCoord,
zCoord = vertArray[index];
This is the smallest overhead approach I can think of... defining Array length ahead of time, keeping the array one dimensional, filling with only integers. Any better ideas? Perhaps creating an array would be unneeded and I could create an equation that pulls vert data directly from the rendered mesh?
3) Are there ways to decrease the impact of large data stored in browser memory?
I have yet to implement this but the idea of a 10 Million Length array in the browser is quite a lot in my mind. I would prefer being able to load the entire thing rather than doing some sort of AJAX call when the helicopter gets near the edge of a sub-plane. Think "zones" in MMORPG's.
How can I perform a query on the database as per the user's location value? The application was developed with HTML5, CSS, Javascript, PHP has a database with columns as in the below table.
On the html webpage the users geo coordinates are collected and are to be compared with the values in the database to find the nearest place to the user with the places in the database.
Please let me know how to achieve this. Any examples / samples will be appreciated.
There is a question that compares the capabilities of various spatial databases, GIS: PostGIS/PostgreSQL vs. MySql vs. SQL Server?, where Postgis comes out a pretty clear winner over MySQL.
Whether you use MySQL or Postgis, you would be much better off, if you can, storing your latitude and longitude values as a geometry/geography (Point), as the functions that can be used to find things nearby, ST_Distance, ST_Distance_Sphere and the more obscure <-> operator, see Find n Nearest Neighbors for given Point using PostGIS? (for example usage) work directly on geometry/geography columns. Even more importantly, you can add a spatial index, which these functions need to work properly, which will outperform searches on separately indexed latitude and longitude columns by a large margin (this will depend on table size, but will grow as table size grows),
In Postgis, you can convert lat and lon to a geometry with:
alter table mytable add column geom (Geometry, 4326);
update mytable set geom = ST_SetSRID(ST_MakePoint(lon, lat), 4326)
create index ix_spatial_mytable_geom on mytable using gist(geom);
At this point, you will be able to very efficient queries to find points near other points, using any of the examples in the above links.
You can do similar things in MySQL, although, it does not support a spatial reference system, ie, the 4326 above, which means lat/lon, and it lacks a ST_MakePoint function, so you would need to use STGeomFromText and concatenate the lat/lon together to make a POINT. It also does everything in planar coordinates, as Claudio and others have stated, which is not an issue with Postgis.
I apologize for a long and somewhat tangential answer, but having done various migrations between databases on large amounts of data (MySQL, SQL Server and Postgres/GIS) and made lots of mistakes on the way, I hope I can set you off in the right direction (and add a bit of future proofing, if you want to start using some other spatial functionality, which Postigs has in spades).
For a rough measure I would try something like the following (only Euclidean geometry, it doesn't take into account the Earth curvature or problems like this).
First you could compute the difference between the user's coordinates and the coordinates of places in the database. Like this:
distLat = abs(userLat - placeLat)
distLong = abs(userLong - placeLong)
Then I would compute the distance between the two points using Pythagora's theorem. So:
distance = squareRoot(distLat * distLat + distLong * distLong)
You can compare the distances of all places in the database and take the minimum, which teoreticaly is the place nearest to the user's position.
If you use MySQL I think that a query like this should work:
SELECT * FROM places ORDER BY MIN(SQRT((p.latitude - userLatitude) * (p.latitude - userLatitude) + (p.longitude - userLongitude) * (p.longitude - userLongitude))) LIMIT 1
Beware that this query could be very slow depending on how many places you have, because it needs to read all the rows in the table and compute the distance for each one. Indexes have no effects.
Anyway, for this kind of problems you should better use GIS or databases with good geospatial extensions. MySQL geospatial extension is not very powerful, even in MySQL 5.6. It has a ST_DISTANCE function but still uses Euclidean geometry which is not very accurate for calculation on a spherical surface like the Earth. Anyway, if you use MySQL 5.6, I think that it should be better to use the ST_DISTANCE function, which is for sure much more optimized than doing calculations "manually" in the query.
See also this article for a deep explanation and more examples: http://www.plumislandmedia.net/mysql/haversine-mysql-nearest-loc
EDIT
As requested by the OP, I add more details on how to deal with negative coordinates.
Negative coordinates are generally not a big issue. I will try to explain this with some examples.
Take for example the coordinates of the Brooklyn bridge: 40.704391, -73.994675.
With the above formula, the distance of the Brooklyn bridge from the Statue of Liberty is:
sqrt((40.704391 - 40.689167)^2 + (-73.994675 - -74.044444)^2) = 0.052045399
Now consider the distance between the Statue of Liberty and the Brooklyn Bowl (coordinates 40.7179666, -73.9670125), that is:
sqrt((40.7179666 - 40.689167)^2 + (-73.9670125 - -74.044444)^2) = 0.082613886
As you can see the distance of the Brooklyn Bowl from the Statue of Liberty is bigger than the distance of the Brooklyn Bridge. It is correct since the Brooklyn Bowl is 4 miles away from the Statue of Liberty, while the Brooklyn Bridge is only 1 mile away from it.
In this example both the two points has a negative longitude. But the formula works even if one of the two has positive coordinates. For example, the distance between the Statue of Liberty and the Tour Eiffel (Paris, coordinates 48.858360, 2.294460), is:
sqrt((48.858360 - 40.689167)^2 + (2.294460 - -74.044444)^2) = 76.77476134
Then calculate also the distance between the Statue of Liberty and the Colosseum (Rome, coordinates 41.890238, 12.492242):
sqrt((41.890238 - 40.689167)^2 + (12.492242 - -74.044444)^2) = 86.54502063
As you can see it works, the distance of the Colosseum is bigger since it is about 8000km away from the Statue of Liberty, while the Tour Eiffel is about 800 km closer.
The only issue I see is when you have to calculate the distance between two places that are in the far east and in the far west respectively. The above formula will give you a very high distance, but actually the they could be very close. Take for example Anchorage, a city in Alaska (coordinates 61.252240, -149.896769), and Beringovskij, a city in the very east of Russia (coordinates 63.049797, 179.310011). They have a distance of only about 1500 km but with the above formula you get:
sqrt((61.252240 - 63.049797)^2 + (-149.896769 - 179.310011)^2) = 329.2116875
Definitely a too high value for only 1500 km: I would expect something less than 50.
The problem is that the formula calculates the distance taking the central meridian as a reference point, that is the meridian with a 0 degrees latitude. This is good until the distance is "no more that half the Earth".
I think that a solution could be to calculate two distances.
The first with a reference point of 0 degrees: it is what the above formula does.
The second with a reference point of 180 degrees. It's like calculating the distance on a world map shifted by 180 degrees, like this: http://www.bouwman.com/world/Formilab-180.html.
And then take the minimum of these two distances.
Thus the formula becomes a little more complex:
distance = squareRoot(min((userLat - placeLat)^2, (userLat - placeLat - 360)^2) + (userLong - placeLong)^2)
Note that we subtract 360 because it is the distance between the degree -180 and the degree 180.
With this new formula we get correct results for places that are more then 180 degrees away from each other, and we get also the same result given by the previous formula when comparing places that are less then 180 degrees away from each other. The calculate distance Anchorage - Beringovskij is now: 30.84564166.
Of course, as I have already said, this is not an exact method for calculating distances. You can take a look at this article for more "scientific" techniques: https://en.wikipedia.org/wiki/Geographical_distance :D
I am having a problem that I have traced to unexpected behavior in the d3.geo.bounds function.
var bounds = d3.geo.bounds(data);
For a geoJSON FeatureCollection containing only point features, a proper bounding box isn't returned: specifically, the x-values of the first and last points in the collection appear to used instead of the min and max x-values for the group of points. This can be seen in action at http://jsbin.com/icosof/4/edit.
The bounding-box coordinates returned thus includes the min and max y-values, but not the largest and smallest x-values. Instead, these seemed to be grabbed from the first and last features in the set:
[[6113.30166221, -6663.98951731], [5153.32052977, -77.3529517971]]
Generating a bounds object by manipulating the coordinates directly
xvalues = [];
$.each(data.features, function(i,el){
xvalues.push(el.geometry.coordinates[0]);
});
yvalues = [];
$.each(data.features, function(i,el){
yvalues.push(el.geometry.coordinates[1]);
});
console.log([[Math.min.apply(null,xvalues),Math.min.apply(null,yvalues)],
[Math.max.apply(null,xvalues),Math.max.apply(null,yvalues)]]);
produces the desired result:
[[3397.33954824, -6663.98951731], [6504.18296202, -77.3529517971]]
Does d3.geo.boundsexpect a FeatureCollection sorted by x-coordinates (and if so how would I work around this), or is this a bug?
I think the confusion here is that d3.geo.bounds expects features to have latitude/longitude values, not x/y values. So your input is taken as % 360, and you won't get the max/min you expect.
You may need to convert your coordinates to lat/lon values to get this to work - left as an exercise for the reader, as I don't know what your coordinate system is.
I'm currently trying to build a kind of pie chart / voronoi diagram hybrid (in canvas/javascript) .I don't know if it's even possible. I'm very new to this, and I haven't tried any approaches yet.
Assume I have a circle, and a set of numbers 2, 3, 5, 7, 11.
I want to subdivide the circle into sections equivalent to the numbers (much like a pie chart) but forming a lattice / honeycomb like shape.
Is this even possible? Is it ridiculously difficult, especially for someone who's only done some basic pie chart rendering?
This is my view on this after a quick look.
A general solution, assuming there are to be n polygons with k vertices/edges, will depend on the solution to n equations, where each equation has no more than 2nk, (but exactly 2k non-zero) variables. The variables in each polygon's equation are the same x_1, x_2, x_3... x_nk and y_1, y_2, y_3... y_nk variables. Exactly four of x_1, x_2, x_3... x_nk have non-zero coefficients and exactly four of y_1, y_2, y_3... y_nk have non-zero coefficients for each polygon's equation. x_i and y_i are bounded differently depending on the parent shape.. For the sake of simplicity, we'll assume the shape is a circle. The boundary condition is: (x_i)^2 + (y_i)^2 <= r^2
Note: I say no more than 2nk, because I am unsure of the lowerbound, but know that it can not be more than 2nk. This is a result of polygons, as a requirement, sharing vertices.
The equations are the collection of definite, but variable-bounded, integrals representing the area of each polygon, with the area equal for the ith polygon:
A_i = pi*r^2/S_i
where r is the radius of the parent circle and S_i is the number assigned to the polygon, as in your diagram.
The four separate pairs of (x_j,y_j), both with non-zero coefficients in a polygon's equation will yield the vertices for the polygon.
This may prove to be considerably difficult.
Is the boundary fixed from the beginning, or can you deform it a bit?
If I had to solve this, I would sort the areas from large to small. Then, starting with the largest area, I would first generate a random convex polygon (vertices along a circle) with the required size. The next area would share an edge with the first area, but would be otherwise also random and convex. Each polygon after that would choose an existing edge from already-present polygons, and would also share any 'convex' edges that start from there (where 'convex edge' is one that, if used for the new polygon, would result in the new polygon still being convex).
By evaluating different prospective polygon positions for 'total boundary approaches desired boundary', you can probably generate a cheap approximation to your initial goal. This is quite similar to what word-clouds do: place things incrementally from largest to smallest while trying to fill in a more-or-less enclosed space.
Given a set of voronio centres (i.e. a list of the coordinates of the centre for each one), we can calculate the area closest to each centre:
area[i] = areaClosestTo(i,positions)
Assume these are a bit wrong, because we haven't got the centres in the right place. So we can calculate the error in our current set by comparing the areas to the ideal areas:
var areaIndexSq = 0;
var desiredAreasMagSq = 0;
for(var i = 0; i < areas.length; ++i) {
var contrib = (areas[i] - desiredAreas[i]);
areaIndexSq += contrib*contrib;
desiredAreasMagSq += desiredAreas[i]*desiredAreas[i];
}
var areaIndex = Math.sqrt(areaIndexSq/desiredAreasMagSq);
This is the vector norm of the difference vector between the areas and the desiredAreas. Think of it like a measure of how good a least squares fit line is.
We also want some kind of honeycomb pattern, so we can call that honeycombness(positions), and get an overall measure of the quality of the thing (this is just a starter, the weighting or form of this can be whatever floats your boat):
var overallMeasure = areaIndex + honeycombnessIndex;
Then we have a mechanism to know how bad a guess is, and we can combine this with a mechanism for modifying the positions; the simplest is just to add a random amount to the x and y coords of each centre. Alternatively you can try moving each point towards neighbour areas which have an area too high, and away from those with an area too low.
This is not a straight solve, but it requires minimal maths apart from calculating the area closest to each point, and it's approachable. The difficult part may be recognising local minima and dealing with them.
Incidentally, it should be fairly easy to get the start points for the process; the centroids of the pie slices shouldn't be too far from the truth.
A definite plus is that you could use the intermediate calculations to animate a transition from pie to voronoi.