Short question:
Given table based data (such as SQL results), what is the most accepted method of converting this into a specific json string / js objectstring with C#?
Scenario:
I have a web page using d3.js to general graphical reports. For the data, I call an API which executes a stored procedure and returns the result set. I need a way of converting that data into a suitable json object which can easily be processed within my d3 report.
Example:
With the following table being returned by my stored procedure
What would be the most suitable kind of object to convert this data into for use with javascript / d3, and how would one go about achieving that?
What I've tried:
Since my API is written in C#, I've tried using the JsonConvert.SerializeObject method, as below.
DataTable dt = new DataTable();
using (SqlDataAdapter sql = new SqlDataAdapter(getData))
{
sql.Fill(dt);
}
returnStr = JsonConvert.SerializeObject(dt);
Which produces a simple array, one item per row of data, for example:
[
{
"Group": "Group01", "TrueFalse": 1, "SubGroup": "SubGroup01", "Owner": "ME"
},
{
"Group": "Group01", "TrueFalse": 1, "SubGroup": "SubGroup02", "Owner": "ME"
},
{
"Group": "Group01", "TrueFalse": 0, "SubGroup": "SubGroup02", "Owner": "You"
},
{
"Group": "Group01", "TrueFalse": 1, "SubGroup": "SubGroup03, "Owner": "You"
},
{
"Group": "Group02", "TrueFalse": 0, "SubGroup": "SubGroup01", "Owner": "Someone"
},
// etc...
]
While its quite possible to work with this, I don't know if it's the best way of going about doing so, or whether we can create an object that's more suitable. If you can give any pointers on best practises, and possible examples, that would be great.
This is more of preference issue and perhaps consistency in your data model. What you have there is perhaps the preferred method as it is easy to iterate through while knowing the object structure in the array. Another way of doing it, and is harder to understand if you are an incoming developer, is to have an array for each column in the table and iterating by the index of the array to get the values. This would require more code and is less readable. So rather than indexing each field for each iteration through the array, you simply have objects that have a consistent structure, even if a field in the object is empty, which leads to another issue with the array-per-column approach: you would have to be absolutely sure that a field cannot be empty. Otherwise, your indexing counts will not be consistent.
Long story short, what you have there is semantically the most appropriate. Table columns equate to fields in your object, and the JSON object equates to a row in your table.I hope this makes sense to you and helps out.
Related
I'm building an app where I need to delete items stored in the database. Here's a (shortened) example of user data I have in my DynamoDB table called 'registeredUsers':
{
"userId": "f3a0f858-57b4-4420-81fa-1f0acdec979d"
"aboutMe": "My name is Mary, and I just love jigsaw puzzles! My favourite jigsaw category is Architecture, but I also like ones with plants in them.",
"age": 27,
"email": "mary_smith#gmail.com",
"favourites": {
"imageLibrary": [
{
"id": "71ff8060-fcf2-4523-98e5-f48127d7d88b",
"name": "bird.jpg",
"rating": 5,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/bird.jpg"
},
{
"id": "fea4fd2a-851b-411f-8dc2-1ae0e144188a",
"name": "porsche.jpg",
"rating": 3,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/porsche.jpg"
},
{
"id": "328b913f-b364-47df-929d-925676156e97",
"name": "rose.jpg",
"rating": 0,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/rose.jpg"
}
]
}
}
I want to be able to delete the item 'rose.jpg' in the user.favourites.imageLibrary array. In order to select the correct user, I can provide the userId as the primary key. Then, in order to select the correct image in the array, I can pass the AWS.DocumentClient the 'id' of the item in order to delete it. However, I'm having trouble understanding the AWS API Reference docs. The examples given in the developer guide do not describe how to delete an item by looking at one of it's attributes. I know I have to provide an UpdateExpression and an ExpressionAttributeValues object. When I wanted to change a user setting, I found it pretty easy to do:
const params = {
TableName: REGISTERED_USERS_TABLE,
Key: { userId },
UpdateExpression: "set userPreferences.difficulty.showGridOverlay = :d",
ExpressionAttributeValues: {
":d": !showGridOverlay
},
ReturnValues: "UPDATED_NEW"
};
To conclude, I need a suitable Key, UpdateExpression and ExpressionAttributeValues object to access the rose.jpg item in the favourites array.
Unfortunately, the UpdateExpression syntax is not as powerful as you would have liked. It supports entire nested documents inside the item, but not sophisticated expressions to search in them or to modify them. The only ability it gives you inside a list is to access or modify its Nth element. For example:
REMOVE #favorites.#imagelibrary[3]
Will remove the 3rd element of imagelibrary (note that the "#imagelibrary" will need to be defined in ExpressionAttributeNames), and you can also have a condition on #favorites.#imagelibrary[3].#id, for example, in ConditionExpression. But unfortunately, there is no way to specify more complex combinations of conditions and updates, such as "find me the i where #favorites.#imagelibrary[i].#id is equal something, and then REMOVE this specific element".
Your remaining option is to read the full value of the item (or with ProjectionExpression just the #favorties.#imagelibrary array), and then in your own code find which of the elements you want to remove (e.g., discover that it is the 3rd element), and then in a separate update, remove the 3rd element.
Note that if there's a possibility that some other parallel operation also changes the item, you must use a conditional update (both UpdateExpression and ConditionExpression) for the element removal, to ensure the element that you are removing still has the id you expected. If the condition fails, you need to repeat the whole operation again - read the modified item again, find the element again, and try to remove it again. This is an example of the so-called "optimistic locking" technique which is often used with DynamoDB.
This is an odd question, but I hope someone can help me out.
I have a JS object (it's actually JSON string which is then converted back and object.).
{
"legs": [{
"pods": [{
"11": {
"id": 6,
"name": "tesla",
"text": "tesla model2",
"battery": "60"
},
"21": {
"id": 9,
"name": "porsche",
"text": "porsche electric",
"battery": "100"
}
}]
}]
}
So in my javascript I'll call a specific item... say, legs.pods[11].name
I pass that into a function like editThisField(legs.pods[11].name);
On that function editThisField(whichDataElement){.....} I'll show a modal box where the user can provide new text for the name value... what I would like is to have the whichDataElement updated to the new text such that which ever object variable I pass into that field, is updated in the legs object... I'm not sure if I'm explaining this very well, but I can't think where to start looking at a good approach to this.
Essentially I'm making a text area where I paste in the JSON, and the system allows me to edit the JSON object via modal popups. So the modal appears and dynamically shows a form based on the data in the specific sub object. When you fill in the form, the original JSON is updated... thoughts?
Not really working as I need... Maybe this helps clarify:
var mything = {
subthing: {
thisthing: "here"
}
}
var edit = mything.subthing.thisthing;
edit = "Changed";
console.log(mything); // then the listthing value should be changed.
It sounds like you're on the right track. If you pass in an object to your editThisField function you can change it's contents, but in your example you are passing in a primitive (which won't work as it's passed by value).
Another option... since you're loading a modal I assume you know what attribute you want to change so you can just specify the object keys or array indices dynamically. See example here https://jsfiddle.net/jroden1f/. Note, I changed your data model slightly as I wasn't sure why you had legs and pods as arrays and it made the example easier if I just made them objects.
I need to use a Zapier webhook to take some incoming JSON data, which contains an array of items, loop that array and do an action for each element.
Here's a sample of incoming JSON data:
{
"first_name": "Bryan",
"last_name": "Helmig",
"age": 27,
"data": [
{
"title": "Two Down, One to Go",
"type": "Left"
},
{
"title": "Talk the Talk",
"type": "Right"
},
{
"title": "Know the Ropes",
"type": "Top"
}
]
}
The size of the array will be dynamic.
The problem is that when I import this data in the hook, it gives me
data
title: Two Down, One to Go
type: Left
title: Talk the Talk
type: Right
title: Know the Ropes
type: Top
So, basically it says that data is just a big string of all this stuff together.
Can anyone help me figure out if it's possible to have a Zap loop over this and do something, e.g., insert data into a sheet, for ever item in the array? I'm aware of the "Code" actions, I've chosen JavaScript, which I could parse out the string, but that doesn't seem efficient. Plus, in reality, there will be a lot of data in the objects inside the JSON array.
EDIT: SOLVED! ANSWER BELOW
So, the first part is to Catch Raw Hook for a trigger. It's the normal "Webhooks", but you have to click to show the less common variations. With the Catch Raw Hook, your data will not be turned automatically turned into variables via the Zapier app, you'll have the raw JSON data.
Once you have the raw JSON, in my case, you'll have an action, and this will be the "Code" action. I'm using JavaScript. In my template, I'm grabbing the entire JSON string (your whole imported JSON is a string right now, not an object, so we can't use "." (dot) notation to access parts of it).
You'll need to JSON.parse() the string in the code. But first, let me explain that Zapier has a pre-defined variable called inputData that you'll use in your code. Then in the top of the "Edit Template" section of your "Code" Action, you'll see you can name the variable of that JSON string you imported.
Now the fun part! In the code, you'll type:
// of course, you can change the variables to what you want
// but 'inputData' is unique, can't change that
const myData = JSON.parse(inputData.rawJsonData);
So, my raw data is a string, it's not JSON yet, so this line of code makes it a JSON object. And now, as an object we can loop over it or .map or access 'this.that' or whatever you want.
The next important thing to mention about "Code" in Zapier, is that to get your stuff out, you return. So, in the next few lines, I'm going to return a .map function that returns each item in an array. And it's tough to grasp how Zapier treats this, but it actually runs the next "Action" you create (e.g. adding a row to a sheet) for each time you loop in that .map. So, let's take a look below:
return myData.data.map(item => {
return item;
});
If you remember, I had an array called "data" in my raw JSON I listed in the original question. It will loop over that array and since I'm returning, then it will perform an "Add Row to Sheet" (in my case) for each loop, thus, inserting all of my data as multiple rows in my spreadsheet.
So the finished code:
const myData = JSON.parse(inputData.rawJsonData);
return myData.data.map(item => {
return item;
});
I am a completely new to D3.
I'd like to visualize easy results of tests in bubbles, e.g. Name: mathematics, completed-tests: 340, name: Latin, completed tests: 550.
When I look at the example on https://bl.ocks.org/mbostock/4063269, I can see that only a CSV file is loaded.
According to the documentation it is also possible to load a JSON file, but I didn't find the correct "semantic" for the JSON file.
What should the JSON file look like and how should it be called in the JavaScript section of the index.html?
Note: I want to use version 4.0. Does it matter?
Your question here is not about the correct structure of the JSON file, since there is no correct structure: it depends on the code.
Your question here is, given that specific code you linked, how can you change the CSV for a JSON.
To answer that, you have to understand how d3.csv creates an array of objects. In the Bostock's code you linked, each object in that array has an id and a value key/value pair.
Therefore, your JSON need to have a structure like this:
[{
"id": "flare.analytics.cluster.AgglomerativeCluster",
"value": 1938
}, {
"id": "flare.analytics.cluster.CommunityStructure",
"value": 3812
}, {
"id": "flare.analytics.cluster.HierarchicalCluster",
"value": 2714
}, {
"id": "flare.analytics.cluster.MergeEdge",
"value": 1743
}]
Since you are creating your own JSON you don't need to mind the row function, which removes the objects without value and coerces their values to a number.
Here is the updated bl.ocks using a JSON file (with just some objects): https://bl.ocks.org/anonymous/4ca57ea4393a37bc92091325eba295dd
Have in mind that, if you use the data structure you described in your question...
[{"Name": "mathematics", "completed-tests": 340}, etc...]
... you will have to make several changes in Bostock's code, both in d3.hierarchy and in the node selection.
For a web site I'm creating, I have to create a quote based on data provided as a JSON string from the server. I've been looking through this site (and various others) but still am unsure on the best way to query/search the data.
For example, I need to get the Area Name from the Area ID. I need to get the maximum age for an area and also the price for a given minimum/maximum age.
I also want to get an array of prices.
Is it best to create a Javascript object from the string using the eval method? Or should I be using jQuery.
Thanks for your help.
({"SkiPass":[{"Id":1,"Area":"Chamonix","Rates":[{"Id":1,"AgeMin":0,"AgeMax":2,"Price":2.5},{"Id":2,"AgeMin":3,"AgeMax":17,"Price":5.0},{"Id":3,"AgeMin":18,"AgeMax":30,"Price":6.2},{"Id":4,"AgeMin":31,"AgeMax":59,"Price":7.4}]},
{"Id":2,"Area":"Megeve","Rates":[{"Id":1,"AgeMin":0,"AgeMax":2,"Price":1},{"Id":2,"AgeMin":3,"AgeMax":17,"Price":2.0},{"Id":3,"AgeMin":18,"AgeMax":30,"Price":2.2},{"Id":4,"AgeMin":31,"AgeMax":59,"Price":4.4}]},
{"Id":3,"Area":"Verbier","Rates":[{"Id":1,"AgeMin":0,"AgeMax":2,"Price":1.5},{"Id":2,"AgeMin":3,"AgeMax":17,"Price":3.0},{"Id":3,"AgeMin":18,"AgeMax":30,"Price":4.2},{"Id":4,"AgeMin":31,"AgeMax":59,"Price":5.4}]}]})
Create a JavaScript object from the string, most definitely, but do it with legitimate JSON parsing facilities and not "eval()". You could use jQuery, but there are other solutions, such as the JSON tools available from json.org, which are small and simple.
Once it's a JavaScript object, well then your needs should guide you as to whether some query solution is necessary, or instead that it's just a simple matter of programming.
I think the best method is jLinq: http://hugoware.net/Projects/jLinq it's like doing a SQL query on JSON.
It doesn't needs jQuery.
I use it, and it's great.
Create the object from the JSON string using JSON.parse() or jQuery.parseJSON() if you are already using jQuery -- or just pass it as from the server side as JSON.
You can then iterate through the object to find the record you want. Or, you can use build your objects so that you can naturally grab data from them.
FloatLeft - as Dan points out, your task would be much easier if you could use XPath but there is no need to re-write your data in XML format. With DefiantJS (http://defiantjs.com) you can now query JSON structure with XPath expressions.
DefiantJS extends the global object JSON with the method "search", which enables XPath queries and returns an array with the matches (empty array if no matches were found). The returned array is equipped with aggregate functions as well; one of these "sortDesc".
Check out this working fiddle;
http://jsfiddle.net/hbi99/H3PR3/
var data = {
"SkiPass": [
...
{
"Id": 3,
"Area": "Verbier",
"Rates": [
{ "Id": 1, "AgeMin": 0, "AgeMax": 2, "Price": 1.5 },
{ "Id": 2, "AgeMin": 3, "AgeMax": 17, "Price": 3 },
{ "Id": 3, "AgeMin": 18, "AgeMax": 30, "Price": 4.2 },
{ "Id": 4, "AgeMin": 31, "AgeMax": 59, "Price": 5.4 }
]
}
]
},
res1 = JSON.search( data, '//SkiPass[Id=3]/Area' ),
res2 = JSON.search( data, '//*[Area and Id=3]/Rates' )
.sortDesc('AgeMax'); // <-- sorting descending by the value of "AgeMax"
document.getElementById('name').innerHTML = res1;
document.getElementById('max_age').innerHTML = res2[0].AgeMax;
document.getElementById('price').innerHTML = res2[0].Price;