We Keep Coding

How to extract specific values from a complex and dynamic JavaScript style dictionary using Regex? [duplicate]

This question already has answers here:
Python - Parsing JSON formatted text file with regex
(4 answers)
Closed 2 years ago.
I have an extensive JS dictionary retrieved from an HTML webpage and I want to extract data from it without parsing the Javascript. Currently I am trying to accomplish this using Regular Expression.
The problem is that the dictionary is quite complex and dynamic, meaning that, on occasion, I could find some new keys inserted, yet I expect my target keys to stay the same.
This is highly trimmed data with some values ommited, but it maintains the complexity.
{"compactVideoRenderer":{"videoId":"abcDE123-_","thumbnail":{"thumbnails":[{"url":"OMMITED_URL","width":168,"height":94},{"url":"OMMITED_URL_TWO","width":336,"height":188}]},"title":{"accessibility":{"accessibilityData":{"label":"OMMITED_TITLE"}},"simpleText":"OMMITED_TITLE_SIMPLE"}}}
From the above, I need to extract the values of the following:
compactVideoRenderer -> videoId ("abcDE123-_")
compactVideoRenderer -> accessibility -> simpleText ("OMMITED_TITLE_SIMPLE")
The solution must be flexible enough that if I insert another key value pair at any location (as long as it does not change the 'address' of the target keys), the regex should still be able to find the target values.
As Regex is universal in terms of programming languages, code in any language will help, however, code or suggestions in Python are extra helpful!

Use https://pypi.org/project/jsonfinder/ to extract the JSON object from the HTML string. Then you can work with a normal Python dict. No regex needed.

Why use regex when you can access the elements the natural way?
If you must, there are dupes: Python - Parsing JSON formatted text file with regex
In Python3 you can do
import json
from types import SimpleNamespace
# Parse JSON into an object with attributes corresponding to dict keys.
x = json.loads(data, object_hook=lambda d: SimpleNamespace(**d))
print(data.compactVideoRenderer.videoId)
In JS:
const data = JSON.parse(`{
"compactVideoRenderer": {
"videoId": "abcDE123-_",
"thumbnail": {
"thumbnails": [{
"url": "OMMITED_URL",
"width": 168,
"height": 94
}, {
"url": "OMMITED_URL_TWO",
"width": 336,
"height": 188
}]
},
"title": {
"accessibility": {
"accessibilityData": {
"label": "OMMITED_TITLE"
}
},
"simpleText": "OMMITED_TITLE_SIMPLE"
}
}
}`)
console.log(data.compactVideoRenderer.videoId)
console.log(data.compactVideoRenderer.title.simpleText)

Converting data table to suitable javascript object (C#)

Short question:
Given table based data (such as SQL results), what is the most accepted method of converting this into a specific json string / js objectstring with C#?
Scenario:
I have a web page using d3.js to general graphical reports. For the data, I call an API which executes a stored procedure and returns the result set. I need a way of converting that data into a suitable json object which can easily be processed within my d3 report.
Example:
With the following table being returned by my stored procedure
What would be the most suitable kind of object to convert this data into for use with javascript / d3, and how would one go about achieving that?
What I've tried:
Since my API is written in C#, I've tried using the JsonConvert.SerializeObject method, as below.
DataTable dt = new DataTable();
using (SqlDataAdapter sql = new SqlDataAdapter(getData))
{
sql.Fill(dt);
}
returnStr = JsonConvert.SerializeObject(dt);
Which produces a simple array, one item per row of data, for example:
[
{
"Group": "Group01", "TrueFalse": 1, "SubGroup": "SubGroup01", "Owner": "ME"
},
{
"Group": "Group01", "TrueFalse": 1, "SubGroup": "SubGroup02", "Owner": "ME"
},
{
"Group": "Group01", "TrueFalse": 0, "SubGroup": "SubGroup02", "Owner": "You"
},
{
"Group": "Group01", "TrueFalse": 1, "SubGroup": "SubGroup03, "Owner": "You"
},
{
"Group": "Group02", "TrueFalse": 0, "SubGroup": "SubGroup01", "Owner": "Someone"
},
// etc...
]
While its quite possible to work with this, I don't know if it's the best way of going about doing so, or whether we can create an object that's more suitable. If you can give any pointers on best practises, and possible examples, that would be great.

This is more of preference issue and perhaps consistency in your data model. What you have there is perhaps the preferred method as it is easy to iterate through while knowing the object structure in the array. Another way of doing it, and is harder to understand if you are an incoming developer, is to have an array for each column in the table and iterating by the index of the array to get the values. This would require more code and is less readable. So rather than indexing each field for each iteration through the array, you simply have objects that have a consistent structure, even if a field in the object is empty, which leads to another issue with the array-per-column approach: you would have to be absolutely sure that a field cannot be empty. Otherwise, your indexing counts will not be consistent.
Long story short, what you have there is semantically the most appropriate. Table columns equate to fields in your object, and the JSON object equates to a row in your table.I hope this makes sense to you and helps out.

why do only some JSON subcategories use "[" and some other "{"

In JSON, subcategories are sometimes defined using "{" and at other times using "[".
in this example: games -> box -> template , why "[" after games only?
How should the following XML be defined in JSON. How and when should I use "[" and "{"?
<games>
<game id="21934">
<name>Star Wars: The Old Republic</name>
<popularity>30</popularity>
</game>
</games>
Can you give me a good comparison with XML ?
"games": [
{
"name": "Star Wars: The Old Republic",
"popularity": 30,
"id": 21934,
"giantbomb_id": 24205,
"box": {
"template": "http://static-cdn.jtvnw.net/ttv-boxart/Star%20Wars%3A%20The%20Old%20Republic-{width}x{height}.jpg",
"small": "http://static-cdn.jtvnw.net/ttv-boxart/Star%20Wars%3A%20The%20Old%20Republic-52x72.jpg",
"medium": "http://static-cdn.jtvnw.net/ttv-boxart/Star%20Wars%3A%20The%20Old%20Republic-136x190.jpg",
"large": "http://static-cdn.jtvnw.net/ttv-boxart/Star%20Wars%3A%20The%20Old%20Republic-272x380.jpg"
},

You can best answer this question by reading the documentation at json.org.
[ ] are used to define arrays, whereas { } are used to declare objects. Objects are really a form of associative array (mapping name indices to values instead of number
indices to values). In JSON arrays however, the number indices are implicit.
The main advantages of JSON are that it is a subset of Javascript and that it is a compact data interchange format when compared to XML, which is more verbose. JSON data only needs minimal validation whereas XML requires complex parsing. JSON also sacrifices the so called readabilty element of XML, although personally speaking I find it easier to scan JSON to find mistakes than I do wading through XML elements and attributes.
To take your games example, in XML a list of games would be something like this:
<games>
<game id="21934">
<name>Star Wars: The Old Republic</name>
<type>MMORG</type>
</game>
<!-- more game blocks here -->
<game id="12345">...</game>
</games>
In the above example I have skipped niceties such as declaring the fact it is an XML document, linking the above file to a Data Type Definition (DTD) etc.
In JSON the file would probably just be something like this:
{
"games": [
{ "id": 21934, "name" : "Star Wars: The Old Republic", "type": "MMORG" },
{ "id": 12345, .... }
]
}
You could read the above object directly into a Javascript variable and it would be accepted as valid javascript without further processing. It's much faster and easier to get along with. One thing to note is that despite the fact that "games" is an array of objects, it has been encapsulated in {} to be read as a single object.
So in summary, XML is a formal way of exchanging information, whereas JSON sacrifices the formality for ease and speed of use. Be warned however that JSON does have rules and very minor infractions can cause failure to read some or all of the data, depending on browser implementation.

The [] syntax is for arrays where you locate members by number.
The {} syntax is for objects where you locate members by name.

How to access object properties in a JSON file using literals obtained from the same file

I'm trying to create a stacked bar chart with d3 from data in a JSON file. Each bar would have ties, losses, and wins for a particular month.
JSON file:
{
"name": "FHS",
"points": 3000,
"ties": 3,
"losses": 19,
"wins": 50,
"games": {
"2010-09": { "ties": 1, "losses": 2, "wins": 16 },
"2010-10": { "ties": 2, "losses": 5, "wins": 13 },
"2010-11": { "ties": 0, "losses": 12, "wins": 21 }
}
}
I've already used some of the data just fine using the d3.json functionality. For example:
d3.json("data.json",function(error,data)
{
alert(data.name);
alert(data.points);
// etc.
});
But here's where it gets interesting (fyi, I'm using the underscore library to grab each property of "games"...perhaps this is where I'm going wrong?). If I try the following:
_.keys(data.games).forEach(function(d)
{
alert(d);
});
I get 2010-09, 2010-10, 2010-11 in three different alert boxes...perfect! BUT, if I try the following:
_.keys(data.games).forEach(function(d)
{
alert(d.ties);
});
I get undefined :/ In fact, another problem that I see arising is that
alert(data.games.2010-09.ties);
should work (if the property names didn't begin with integers or have hyphens...), but won't because of the format of the literal ("2010-09")...so, my main question is how to dynamically access nested object properties in such a manner that would make generating a chart simple. I've tried to include enough context, but please let me know if you would like any more information. Thanks in advance!

You get undefined in the alert box because d, inside this function, is a string – "2010-09", "2010-10", "2010-11", etc – since you're iterating over the array of strings returned by _.keys(data.games).
So, alert(d.ties) is like calling alert("2010-09".ties), which is expectedly undefined.
But, building on what Lars explained,
_.keys(data.games).forEach(function(d)
{
alert(data.games[d].ties);
});
is how you'd get at the data.
P.S. console.log(data.games[d].ties) is the more robust way (and hence usually the preferred way) to debug javascript.

You can enclose keys in [""] to access the value, i.e. alert(data.games["2010-09"].ties); will work. You might also be interested in the d3 functions to work with key-value structures; see the documentation.

Best way to JSON parsing using JavaScript [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Length of a JavaScript object (that is, associative array)
I have JSON in the following format
[{"student":{"name" : "ABCD",
"age":8,
}
"user":{ "firstName":"ABCD",
"age": 9,
}
},
{"student":{"name" : "XCYS",
"age":10,
}
"user":{ "firstName":"GGG",
"age": 11,
}
},]
I tried using (data.student[i].length), which did not work (just to see what the length of the object is), and I also tried (data.user[i]) to no avail.
I am basically very confused on how I can get the length of one of the objects in the JSON and how I can effectively display it. How can I do this?

The content in your questions doesn't parse as valid JSON as all keys need to be wrapped in quotes (so you should have "age": 11 instead of age: 11). All key/value pairs in an object should be separated by a comma (so you should have "firstName": "GGG", "age": 11 instead of "firstName": "GGG" "age": 11. You also have a trailing comma in the outer array.
So long as you have valid JSON, you should be able to use JSON.parse(data) in all browsers newer than IE7 to convert the string into an actual object. Once parsed into an object, you can use:
var data = "your JSON string";
var object = JSON.parse(data);
console.log(object.length); // the number of objects in the array
console.log(object[0].student.name; // the name of the first student
If you are supporting browsers IE7 and older, check out Douglas Crockford's JSON polyfill: https://github.com/douglascrockford/JSON-js

Please validate your json string in some websites
http://jsonformatter.curiousconcept.com/
Since javascript is a script language, it will interpret the script and terminate the interpretation at the moment when it has some syntax errors. Try to use some javascript console in your browser
IE - Developer Tools
Firefox - firebug plugin

Let's assume that the actual object you have is valid (no missing commas etc), and you've parsed the string correctly, you're trying to access objects with array indexes and vice versa. Let's rebuild the object from ground up:
First, you have an array.
[]
You access the first element in the array using data[0]. The first element is an object.
[
{}
]
The object has two keys, which both hold objects.
[
{
"student": {},
"user": {}
}
]
You access the student key with data[0].student. It contains and object with two keys.
[
{
"student": {
"name": "ABCD",
"age": 8
},
"user": {}
}
]
You get the number of values in that object with:
Object.keys( data[0].student ).length;
For better browser support see Length of a JavaScript object

first, the json is not structured correctly - some element names are missing quotes, commsa in the wrong places, missing, etc. Compare yours to this:
var json = [{"student":{"name" : "ABCD","age":8}, "user":{ "firstName":"ABCD", "age": 9}},
{"student":{"name" : "XCYS","age":10}, "user":{ "firstName":"GGG", "age": 11}}
];
json is easier to proof read that way than spread out vertically. Then, with a simple loop:
for(var ix in json)
{
alert(json[ix].student.name +"="+json[ix].student.age);
}
hopefully you can see how to go from there.

I figured out how to parse it , seems very simple
I used data[i].student.name and data[i].user.name
this seems to alert the correct value
Thank you all for the responses !!