I've been reading all the stuff I can find about this, but none of the solutions seems to provide the answer I need.
Specifically, I read this Access / process (nested) objects, arrays or JSON carefully, as well as dozens of other posts.
Here's what I'm trying to accomplish:
I have a large object, called allData -- that I got as a json object from MongoDB, and it contains an array of all the data from each reading.
{pitch: -7.97, roll: -4.3, temp: 98, yaw: -129.83, time: "01/22/2016 17:28:47", …}
{pitch: -8.04, roll: -4.41, temp: 97, yaw: -130.81, time: "01/22/2016 17:28:58", …}
...
What I'd LIKE to do is be able to extract all the pitch readings with something along the lines of allData.pitch but obviously that doesn't work, since each data reading is in an element of the allData array. So I could go through in a loop and do allData[x].pitch but I was hoping for cleaner, faster way to do do this -- since I'll probably want to extract each type of data.
Unfortunately at this point I don't have the ability to simply request the pitch data from the db, so I get this whole set back.
One last wrinkle is that ONE of the elements of the array above is already a data object.
You can utilise Array.prototype.map() for this
var pitches = allData.map(function(d) {
return {
"pitch": d.pitch,
"time": d.time
};
});
If you can't control the data returned from the server (i.e., to only retrieve the pitch values you want), you are going to have to loop through to get them all.
Related
I am getting stuck on a project where I am trying to pull out relevant data from a super massive Google Maps Timeline json, and running into the problem of the structure not being as orderly as I thought it was. Essentially, I am trying to pull out an address, time, date and mileage out of this json for every trip in my car. to use this data, I pasted it into a normal javascript file and named it so I can use it as an object. I then take this data and create a string that will format that info like a CSV file.
From going over the structure of the json by looking at only a few trips, I was able to determine the following general structure:
const google = {
timelineObjects: [
0: {activitySegment: {A NUMBER OF OBJECTS}},
1: {placeVisit : {A NUMBER OF OBJECTS}},
2: {activitySegment: {A NUMBER OF OBJECTS}},
3: {placeVisit : {A NUMBER OF OBJECTS}}
]
}
activitySegment has all the travelling info, like distance, travel times, etc. placeVisit has info about the destination. In my small sample, I was able to just loop through each using an if statement with i%2=0, and just change what I wanted to pull out from each, which worked well.
When I tried adding a larger sample, I was finding that Google occasionally did not create a activitySegment object and only had a placeVisit, which was throwing "Uncaught TypeError: Cannot read property 'distance' of undefined".
I am sure that the even/odd sorting will not work out any more. Is there a way to use a conditional to show if google[timelineObjects][i] is either a {activitySegment} or {placeVisit}? Or, is there a better way to figuring out what object is next in the array?
You can test to see if the Object at the particular array index has a given property using Object.prototype.hasOwnProperty():
const google = {
timelineObjects: [
{activitySegment: {}},
{placeVisit : {}},
{activitySegment: {}},
{placeVisit : {}}
]
};
console.log(google.timelineObjects[0].hasOwnProperty("activitySegment")); // true
console.log(google.timelineObjects[1].hasOwnProperty("activitySegment")); // false
If your objective to see what type of object you get. You can iterate over each object, see what the key of the object is and process the data depending on the key value. Something like this.
Object.entries(google).forEach(([key, value]) => {
if(key==='activitySegment') {
//process activeSegment here
}else {
//process placeVisit here
}
})
I need to use a Zapier webhook to take some incoming JSON data, which contains an array of items, loop that array and do an action for each element.
Here's a sample of incoming JSON data:
{
"first_name": "Bryan",
"last_name": "Helmig",
"age": 27,
"data": [
{
"title": "Two Down, One to Go",
"type": "Left"
},
{
"title": "Talk the Talk",
"type": "Right"
},
{
"title": "Know the Ropes",
"type": "Top"
}
]
}
The size of the array will be dynamic.
The problem is that when I import this data in the hook, it gives me
data
title: Two Down, One to Go
type: Left
title: Talk the Talk
type: Right
title: Know the Ropes
type: Top
So, basically it says that data is just a big string of all this stuff together.
Can anyone help me figure out if it's possible to have a Zap loop over this and do something, e.g., insert data into a sheet, for ever item in the array? I'm aware of the "Code" actions, I've chosen JavaScript, which I could parse out the string, but that doesn't seem efficient. Plus, in reality, there will be a lot of data in the objects inside the JSON array.
EDIT: SOLVED! ANSWER BELOW
So, the first part is to Catch Raw Hook for a trigger. It's the normal "Webhooks", but you have to click to show the less common variations. With the Catch Raw Hook, your data will not be turned automatically turned into variables via the Zapier app, you'll have the raw JSON data.
Once you have the raw JSON, in my case, you'll have an action, and this will be the "Code" action. I'm using JavaScript. In my template, I'm grabbing the entire JSON string (your whole imported JSON is a string right now, not an object, so we can't use "." (dot) notation to access parts of it).
You'll need to JSON.parse() the string in the code. But first, let me explain that Zapier has a pre-defined variable called inputData that you'll use in your code. Then in the top of the "Edit Template" section of your "Code" Action, you'll see you can name the variable of that JSON string you imported.
Now the fun part! In the code, you'll type:
// of course, you can change the variables to what you want
// but 'inputData' is unique, can't change that
const myData = JSON.parse(inputData.rawJsonData);
So, my raw data is a string, it's not JSON yet, so this line of code makes it a JSON object. And now, as an object we can loop over it or .map or access 'this.that' or whatever you want.
The next important thing to mention about "Code" in Zapier, is that to get your stuff out, you return. So, in the next few lines, I'm going to return a .map function that returns each item in an array. And it's tough to grasp how Zapier treats this, but it actually runs the next "Action" you create (e.g. adding a row to a sheet) for each time you loop in that .map. So, let's take a look below:
return myData.data.map(item => {
return item;
});
If you remember, I had an array called "data" in my raw JSON I listed in the original question. It will loop over that array and since I'm returning, then it will perform an "Add Row to Sheet" (in my case) for each loop, thus, inserting all of my data as multiple rows in my spreadsheet.
So the finished code:
const myData = JSON.parse(inputData.rawJsonData);
return myData.data.map(item => {
return item;
});
I'm trying to come up with an efficient method of storing static data in JSON so that it can be used in queries client-side.
Currently, this data consists of about 60 CSV files which each have approx. 2000-2200 entries each. I parse this data server-side and have a webservice that handles queries coming from the client. As mentioned, I'd like to be able to move this to the client side so that the web application could potentially work offline using the application cache.
A small sample of the data is below:
Battle Axe,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1
150,19EK,21EK,23EK,25EK,33ES,33ES,36ES,36ES,34ES,36ES,40ES,40ES,34ES,34ES,39ES,42ES,38ES,41ES,44ES,46ES
149,19ES,21ES,23ES,25ES,33ES,33ES,36ES,36ES,34ES,36ES,40ES,40ES,34ES,34ES,39ES,42ES,38ES,41ES,44ES,46ES
148,19EK,21EK,23EK,25EK,33ES,33ES,36ES,36ES,34ES,36ES,39ES,40ES,34ES,34ES,39ES,42ES,37ES,40ES,44ES,45ES
147,19ES,21ES,23ES,25ES,33ES,32ES,35ES,35ES,33ES,35ES,39ES,39ES,33ES,33ES,38ES,41ES,37ES,40ES,43ES,45ES
My original attempt at converting to JSON was based on the following:
Each file was one JSON Object (lets call this object 'weapon')
Each row in the file was another JSON object stored in an array under the corresponding weapon object
Each entry for a row was stored in a fixed length array under the row object
All of the 'weapon' objects were stored in an array.
This meant I had one array that consisted of approx. 60 objects, which in turn had on average 100 objects stored within them. Each of these 100 objects had an array of 20 objects for each entry which consisted of the actual value and some additional meta data. As you can see, I am already at 120K objects... the resulting minified json string was 3mb. Small sample below:
var weapon =
{
Name: 'Broadsword',
HitEntries: [
{
High: 150,
Low: 150,
Unmodified: false,
Hits: [ { Hits: '12', Critical: 'CK', Fail: false},...,{ Hits: '1', Critical: '', Fail: false}]
},
...
{
High: 50,
Low: 47,
Unmodified: false,
Hits: [ { Hits: '3', Critical: '', Fail: false}]
}
]
}
An example of a query that will be run is below. It will be based on the sample data csv shown above:
Battle Axe weapon is selected
A value of 147 is selected for the roll (row)
A value for 9 is selected for the armour type (column heading)
The result of the above should be 39ES (cross reference between row and heading)
At this point I decided it was probably a good idea to get some advice before heading down this path. Any input is appreciated =)
You can make a few optimizations here:
Use WebSockets to stream data if possible
Convert the data to TypedArrays (blobs) - you'll end up dealing with something like a 10K file.
Use the index DB to query if needed
I am getting the data in the following format
[Sl.No, Amount, Transaction Type, Account Number]
[01, $10000, Deposit, 473882829]
[02, $10202, Deposit, 348844844]
[02, $10202, Withdrawal, 348844844]
What is the best way to store this data in Javascript for faster retrieval
var data = ["02", "$10202", "Withdrawal", 348844844]
//empty object
var list = {};
//Constructing the index by concatenating the last two elements of the data.
//Probably this will give the primary key to the data.
var index = data[2] + data[3].toString();
//Store the data using the index
list[index] = data;
You can retrieve the data using the index constructed above.
Determine how the data needs to be accessed. I am guessing it needs to be accessed linearly, so as to not accidentally overdraw (withdraw before deposit), etc -- in this case an Array is generally a suitable data structure. A simple for-loop should be able to find most simple "queries".
Define how the object is represented in Javascript -- is each "transaction item" an array of [x, amount, type, account] or is it an object with the signature {x: ..., amount: ..., type: ..., account: ...} (I often opt for the latter as it adds some self-documentation). It could also be an object created with new TransactionItem (or whatnot) that includes methods.
Determine how to convert the raw data into the chosen object representation. For a well-defined format a simple regular expression or 'split' may work.
Use data. Remember that even a shoddy O(n^2) algorithm is generally "fast enough" for small n. Best to get it working first.
Happy coding.
What would be the syntax path to thumbnails.data?
also, could such an output be simplified, to just {}'s, rather than []'s and {}'s?
{
"returnValue":true,
"results":[
{
"_id":"++HUS_WBo9OoOpWA",
"_kind":"com.palm.media.audio.file:1",
"_rev":3357,
"album":"Elements of Love: Ballads",
"albumArtist":"Earth, Wind & Fire",
"artist":"Earth, Wind & Fire",
"bookmark":0,
"createdTime":0,
"disc":{
"position":1,
"total":1
},
"duration":0,
"genre":"Rhythm & Blues",
"isRingtone":false,
"modifiedTime":1300682209,
"path":"/media/internal/Track 03 - Devotion.mp3",
"searchKey":"Earth, Wind & Fire Elements of Love: Ballads Devotion",
"size":6976284,
"sortKey":{
"trackAndDisc":100003
},
"thumbnails":[
{
"_id":"d1e",
"data":"/media/internal/Track 03 - Devotion.mp3:216:5998",
"type":"embedded"
}
],
"title":"Devotion",
"track":{
"position":3,
"total":0
}
}
]
}
Thanks
If myData holds the data structure in question, you would use
myData.results[0].thumbnails[0].data
As for simplification of your output, yes, it certainly could be simplified, but we'd have to see the code which generates that to tell you how to accomplish it.
How do I get to the thumbnails data?
Assuming your JSON object is stored in the variable myData:
myData.results[0].thumbnails[0].data
Note that this is for the specific example you posted and will always return the first thumbnails data for the first result. In actual code, you will probably loop over both the arrays (results and thumbnails) to extract all the thumbnails data for all the results objects.
Can this JSON object be simplified?
It most certainly could be - it depends on what the purpose is and how it's being generated. If it's being returned by a webservice that you have no control over, then no, you can't change it, obviously. If you're generating it, then sure, you define the object and it's meaning. For example, you could restrict the number of thumbnails to just 1 always, and so, instead of having an array of thumbnails, you'd have just a thumbnail object.
However, as I see it right now, it makes a lot sense - your results could include 1 or more items, hence an array; there could more than 1 thumbnail images and so an array is used there as well...
Are you planning to have more than one thumbnail per result? If not, you can just have:
"thumbnails":{
"_id":"d1e",
"data":"/media/internal/Track 03 - Devotion.mp3:216:5998",
"type":"embedded"
},
and access it as: results[i].thumbnails.data