Double parse JSON data (or use another method to split) - javascript

Wondering what the best way is to break a large data object coming from AJAX apart. If I send just one portion (says paths) I use JSON.parse(data) What I'd really like to do is split the object apart first into it's individual blocks, then be able to do something like JSON.parse(data['paths']).
Here's a clipped sample of the JSON data
{
"paths": {
"type": "FeatureCollection",
"features": [{
"type": "Feature",
"geometry": {
"type": "MultiLineString",
"coordinates": [
[
[-122.32074805731085, 47.634990818586026],
[-122.32074412999432, 47.63497931696752],
[-122.32107629703529, 47.63465666282262]
]
]
},
"properties": {
"path_name": "Woodland path"
},
"id": 2
}]
},
"beds": {
"type": "FeatureCollection",
"features": [{
"type": "Feature",
"geometry": {
"type": "MultiPolygon",
"coordinates": [
[
[
[-122.32073753862116, 47.6347629704532],
[-122.32071585642394, 47.63470617810399],
[-122.32073753862116, 47.6347629704532]
]
]
]
},
"properties": {
"bed_name": "Azalea Triangle"
},
"id": 1
}]
}
}
Here's what I have in javascript
$.ajax
dataType: 'text'
url: 'map.json'
success: (data) ->
Here's the Rails code that generates the call
data = { buildings: #geopaths, lawns: #geobeds }
respond_to do |format|
format.json { render json: data }
format.html
end
UPDATE:
I had sort of avoided explaining what I was wanting to do because I thought it would confuse the issue. In a nut shell - I am collecting data from a database, and sending it to Javascript to be displayed as layers on a map. Each layer has a name (paths, beds, etc), and gets encoded as GeoJSON in Rails before being sent to Javascript with an AJAX command. If I only send one layer I have no trouble parsing the data and getting it onto the map. A typical line of code would look like pathMarkers = L.geoJSON(JSON.parse(data)).
I now need to pass multiple layers to the map. My understanding is AJAX can only handle one object so I combine both paths and beds into one object. When I get to the Javascript side I don't know what to do. In other words I need to get only that portion of the object that has path data for the pathMarkers, and only that portion of the object that has bed data for the bedMarkers.
Graphically this is what I'm trying to do:
paths = a bunch of GeoJSON data
beds = a bunch of GeoJSON data
Use AJAX to send paths and beds to javascript
build pathMarkers with JSON.parse for the paths data
build bedsMarkers with JSON.parse for the beds data
I could build a sample and post it to bitbucket if it would help.

Assuming that I understood correctly and your concern is bringing in distinct data layers into a geo library like Leaflet.js, a single AJAX request is fine unless the JSON payload is so large that it crashes the browser.
As you don't provide much of your code, the following is a general example of how you would do it.
First you create the map object. Obviously :)
const map = L.map(id).setView([1.2345, -1.2345], 10);
Start the AJAX request to fetch the geoJSON file.
$.ajax({
dataType: "json",
url: '/json/lives/here.json',
data: {} /* any props you'd like to pass as query string to your server */,
success: success
});
And the crux of the issue: "How do I access each feature collection?"
The sucess or done callback is where you can be sure you received the data, and can add it to the map.
jQuery's AJAX method, when called with dataType: 'json', automatically runs JSON.parse() for you (and a couple of other things). Once the JSON is parsed, it can be accessed as any other object in JS.
At this point the success callback receives the JSON-turned-into-object, which you can access with traditional JS methods. Like so:
function success (data) {
// data is the parsed JSON. It is now just a JS object.
// Below: for every "key" in the data object, pass its data to L.geoJSON() and add it to the map.
for (var geojsonFeatureCollection in data) {
if (data.hasOwnProperty(geojsonFeatureCollection)) {
L.geoJSON(geojsonFeatureCollection, {/* options */}).addTo(map);
}
}
}
To answer your point about AJAX and a single object: AJAX is a just like any other browser request.
Yes you do send one request at a time. And likewise you receive one response from the server. But what is contained in the response can be absolutely any data. So what you are doing server side is totally OK!
In your case the data consists of a JSON text file, which is later parsed and turned into a JS object for you to actually do something with. The object contains a bunch of "keys" (beds, paths) and all you need to do is iterate over each of those keys and pass each one to Leaflet's geoJSON method for rendering.

I've run into this problem before with incredibly large payloads on mobile devices (iOS with Phonegap, to be precise). You may want to look into a library named OboeJS, at http://www.juancaicedo.com/oboe.js-website/.
Essentially, that library streams the JSON request so that you can process it in chunks. You should be able to use this to suit your needs.

Related

I have some raw JSON stored in a variable and just want to post it to an API

I have been traversing through Stackoverflow and everywhere else on the web to try and find a solution to my issue..
I am working in Javascript and attempting to POST a small section of JSON to an endpoint in the API i know is working (I have completes the GET and POST manually in Postman)
Here is my issue..
I want dont really want to do the "GET" in my programme I just want to either reference the file or even just store it in a little variable.
So for example I have in my code:
var OauthUpload = {
"objects": [
{
"name": "api",
"serviceID": 16,
"properties": {}
}
],
"version": "integration",
"environment": "redshift"
}
Then I am trying to reference this in the JS function:
function ApiPostOauth (port) {
$.post(OauthUpload, "http://docker.dc.test.com:" + getActualPort(port) + "/rest/v1/oauth/import", runner);
}
But I am having no joy! I have seen a few different silutions but none seem to fit for me.
Basically I want a way to just:
Reference my own JSON as a variable and then insert tht so my function "ApiPostOauth" has that inserted before it runs?
Thanks guys
Steve
I have put together an example for your use. When executing this code, the server will return the same object it is sent. So the 'OauthUpload` object is sent as the request body and the server returns the exact same object. Note: if you don't see output in the output panel when running the sample I will need to restart the server (leave a comment). This is here to demonstrate:
[EDIT] - after re-reading your question, it appears you would like to pass the 'OauthUpload` object into the function. I've updated the example.
You have a mistake in your call to jQuery post() method. As shown in the comments, the first two arguments are reversed in the call to post().
Since you didn't pick up on that, I decided to provide an example using your code. Since I don't have your server, I stood up a server for this example. So the URL and port will be different, but the AJAX call will be the same.
Please pay close attention to the OauthUpload object. Notice the property names are no longer surrounded by ". I removed these quotes because they seemed to be causing you confusion about JavaScript objects and JSON strings (there is no such thing as a JSON Object regardless of what you read on the web - JSON is a string format).
Next, look at the differences between the call made to $.post() in my example and your code. You will see the URL comes first in that call.
let url = "//SimpleCORSEnabledServer--randycasburn.repl.co/rest/v1/oauth/import";
let OauthUpload = {
objects: [{
name: "api",
serviceID: 16,
properties: {}
}],
version: "integration",
environment: "redshift"
}
ApiPostOauth(OauthUpload);
function ApiPostOauth(data) {
$.post(url, data, runner)
}
function runner(data) {
document.querySelector('pre').textContent = JSON.stringify(data, null, 2);
}
<pre></pre>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>

JSON parsing and stringify

I am currently working on a Javascript project where I have to parse tons of data. The project requires that I parse some JSON data and bring specific data into another array. Right now, I am using the JSON.stringify method and I can console.log all of the data that I need. The data looks something like this:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-85.3865810000125,
33.90171899971196
],
[
-85.38659500025622,
33.9017919996593
],
yes, this is not all that data-it is about 1200 pages long! I only pasted in the top segment of it. All I really need is how to get to the coordinates aspect of it and into an array. So what I am currently doing is this:
var work = JSON.stringify(response, null, 4)
console.log(work);
Which gives me the above response. However, I am not that familiar with JSON.stringify so if I do:
console.log(work.type);
Attempting to see what the value is for type, I get a response of undefined. Now if I try doing this:
var work = JSON.parse(response)
console.log(work);
I get a response of: VM296:1 Uncaught SyntaxError: Unexpected token o in JSON at position 1
Thus, I cannot get the data parsed which I want to get to which will be the coordinates data. Any help with using stringify would great help me. I have read a lot about how it turns the data into a string but have not really seen a lot about how to parse it. Do I have to use parse? Thank you for your help!
You dont' have to use stringify or parse for this. The response object you are receiving is already a javascript object so you can reference its properties normally
response.type; // => "FeatureCollection"
If you try and stringify the response it will lose its type property, for instance:
typeof "some string".someProperty; // => undefined

Required JSON file not returning all properties every time

I have an API in a Node.js which requires a JSON file, and on some requests maps in content from it. Simple enough, and works perfectly for the first request.
On subsequent requests one of the properties is left off. This is obviously a stripped down version of the code, but is all I'm doing to get the content.
Essentially my JSON looks like:
{"sections": {
"city": {
"title": "Seattle",
"info": "some info",
"tips": [ "tip 1", "tip 2" ]
}
}
}
and I require it:
var Content = require("content");
// some code
return req.status(200).json({ data: data, content: Content.sections.city });
The first request returns the whole content object. Every request after that returns only title and info, but not tips.
Edit -- I'm a dummy. In the //some code section, I work with the tips and eventually call delete on this version of the object. Apparently passing by reference still hasn't sunken in.

JSON Response from web service - best practices question

I'm writing a simple web service that return a JSON response. It'll be heavily used, so I want to try and make the JSON response as small as possible for performance reasons. I'm on the fence over a design decision; penny for your thoughts!
My JSON response from the server looks like this:
{
"customers":
[
{
"id": "337",
"key": "APIfe45904c"
},
{
"id": "338",
"key": "somethingDifferent"
},
{
"id": "339",
"key": "APIfe45904c"
},
{
"id": "340",
"key": "APIfe45904c"
}
]
}
The APIfe45904c here is used in about 60-70% of the records, so I could also modify the the JSON response to remove the repeated information and add a default_key i.e. if there's no key specified, the client should assume the default_key like this:
{
"default_key": "APIfe45904c",
"customers":
[
{
"id": "337"
},
{
"id": "338",
"key": "somethingDifferent"
},
{
"id": "339"
},
{
"id": "340"
}
]
}
No client is using the web service yet, so this wouldn't break anything. Is this good practice? It works, and makes for a small JSON response, but I'm conflicted. I like the KISS principle for developers using the service, but I also want as small a JSON response as possible.
I was tempted to replace customers with c, id with i and key with k to aid reducing the file size, but I figured this will be a problem if I want to get other clients to start using it. Should I drop the idea of default_key for the same reason?
Each JSON response will likely be no more 200 lines of id/key pairs, so I don't need to incorporate pagination, etc.
I would keep it simple as you say, and then use gzip to compress it. It should compress very well as it is repetitive, and remains convenient for programmers.
See here for pointers in outputting gzip headers for AJAX: Is gzip encoding compatible with JSON?
Unless you have very special performance needs, I would always choose clarity over brevity. Especially for an API that is going to be used by many developers.
You should use the consistent format where each record has an id and a key field. What you lose in bandwidth you gain from not having to pre-process the JSON on the client-side.
I tend to analyze my JSON data structure like you but in the end it isn't worth the tiny bit of space you save. Your JSON data structure looks good... have you seen Twitter's JSON data structure? Now that is ugly.
I would go with the default key idea, but I wouldn't go as far as shortening the attribute names since that can be confusing. Perhaps you can take an argument from the web service call (from query string) that specifies whether or not the client desires to have shortened attribute names.

Parse complex JSON in Sencha Touch

I've asked over on the Sencha Touch forums but haven't had a great response (well, none in fact). I wonder if anyone can help. I want to create a List of items from a JSON response received from a PHP script. Here's an example of the JSON received back from the PHP:
{
"friends":[
{
"friend":{
"id":"4",
"forename":"Chris",
"surname":"Major",
"verboseName":"Chris Major",
"phoneNumber":"07931655247",
"longitude":"-0.410909",
"latitude":"52.999245",
"email":"major#lincsmps.co.uk",
"lastRefresh":null,
"joinDate":"1294839423"
},
"networks":null,
"approved":"1"
},
{
"friend":{
"id":"2",
"forename":"Marta",
"surname":"Urbanowicz",
"verboseName":"Marta Urbanowicz",
"phoneNumber":"07716021468",
"longitude":"-0.0338518",
"latitude":"52.9773876",
"email":"urb.marta#googlemail.com",
"lastRefresh":null,
"joinDate":"1294836801"
},
"networks":null,
"approved":"1"
}
]
}
I would like to know how I can parse this into a DataStore and then output a grouped List in Sencha Touch. Although there are examples of using standard JSON responses for a ListView, there doesn't seem to be a nested example such as this...
Any help gratefully received.
Ben,
What you may want to do is declare your store of Friends with the fields with the fields you desire (maybe id, surname, email, network, approved)
Then when make a remote call with the Ajax remote call, which will just pass back the raw data.
At that point you'll have the "friends" object. You can then loop through each "friend" and instead of adding the "friend" in total create new objects:
var friend = {id:'2', surname:'whatever', email:'whatever#blah.com'}
Make an array of those new friend structures, and pass those into the store. This will flatten out or simplify your complex json structure.
Hope that helps, and I can provide more details if you need them.

Categories