Complex JSON Structure - javascript

I am tackling frontend development (AngularJS) and rather than pull data from the backend (which isn't complete but renders everything to JSON), I am looking to just use hardcoded JSON.
However, I am new to this and can't seem to find anything about complex JSON structure. In a basic sense, my web app has users and the content they create. So, in my mind, there will be two databases, but I'm not sure if I'm approaching it correctly.
Users - username, location, created content, comments, etc.
"user": [
{
"userID": "12",
"userUserName": "My Username",
"userRealName": "My Real Name",
"mainInterests": [
{
"interest": "Guitar"
},
{
"interest": "Web Design"
},
{
"interest": "Hiking"
}
],
"userAbout": "All about me...",
"userComments": [
{
"comment": "this is a comment", "contentID" : "12"
},
{
"comment": "this is another comment", "contentID" : "123"
}
],
}
]
Created Content - title, description, username, comments, etc.
"mainItem": [
{
"mainID": "1",
"mainTitle": "Guitar Lessons",
"mainCreatorUserName": "My Username",
"mainCreatorRealName": "My Real Name",
"mainTags": [
{
"tag": "Intermediate"
},
{
"tag": "Acoustic"
},
{
"tag": "Guitar"
}
],
"mainAbout": "Learn guitar!",
"mainSeries": [
{
"videoFile": "file.mov",
"thumbnail": "url.jpg",
"time": "9:37",
"seriesNumber": "1",
"title": "Learn Scales"
},
{
"videoFile": "file.mov",
"thumbnail": "url.jpg",
"time": "8:12",
"seriesNumber": "2",
"title": "Learn Chords"
}
],
"userComments": [
{
"comment": "this is a comment", "userID" : "12"
},
{
"comment": "this is another comment", "userID" : "123"
}
]
}
]
And there is more complexity than that, but I just would like to understand if I'm approaching this right. Maybe I'm even approaching this entirely incorrectly (for instance, CRUD vs. REST? Does it matter here? As I understand it, REST implies that each of the objects above are resources with their own unique URI? So would JSON rendered be impacted?). I really am not sure. But ultimately, I need to use the JSON structure properly pull data into my frontend. Assumably, whatever said structure is will be mirrored and rendered in the backend.
Edit* thank you guys for the replies. I think part of my question, where I clarify "complex", is missing. So I'd like to explain. I guess more than the JSON itself, I really mean the structure of the data. For instance, in my example, I am structuring my data to all be beneath two unique objects (user and content). Is this correct? Or should I think about my data more diverse? For instance, technically I could have a comments database (where each comment is the main object). Or is that still implied in my dataset? Perhaps my question isn't even about JSON as much as it is the data structure which will happen to get rendered in JSON. Hopefully this clarifies what I mean by complex?
Any and all help is appreciated.

I'm not sure why you're making what seems to be objects into single-item arrays (as evidenced by the opening square brackets). Other than that, it looks fine to me. Generally speaking single items (like "User") are structured as an object and multiples are arrays of objects.
As for the Angular stuff, if you want to pull direct from a JSON file as a test, take a look here:
var services = angular.module('app.services', [])
services.factory('User', function($http) {
var User = function(data) {
return data;
}
User.findOne = function(id) {
return $http.get('/test_user.json').then(function(response) {
return new User(response.data);
});
};
return User;
});
I also recomment looking into Deployed for doing development without access to live data services.

Related

Chrome Extension: when being directed to a new page, replace assorted URL parameters with a different value before the URL request gets sent off

When I click I a link (or am getting redirected by a site), I want to replace the values of assorted tracking parameters in the requested URL with a different, universal value. For this example, let's use my username, so a link that directs to
www.abc.com/?utm_source=originalsource&utm_campaign=originalcampaign&cid=originalcid&gclid=originalgclid
would become www.abc.com/?utm_source=WTTDOTM&utm_campaign=WTTDOTMcampaign&cid=WTTDOTM&gclid=WTTDOTM
It’s important that these parameters get edited before the request is sent off, as the idea is to avoid the UTM/cid/gclid/etc tracking and leave a mark in the traffic data of whoever is looking at those tracking parameters. I know I have to use webRequest, and the logic seems pretty simple, but I don’t know how to find-and-replace in the URL on the fly with js. This answer looks like its the closest to what I’m asking, but I don’t know how it would work with just modifying smaller slices of a URL as I'm pretty inexperienced with javascript and most of my coding knowledge is just slapping stuff together until it works.
To start with a simple sub-problem my question is: how can I use the webRequest API to replace utm_campaign=*** in a URL with utm_campaign=WTTDOTM before the page is requested?
----EDIT:-----
#wOxxOm thank you for your helpful answer! I think I have the general structure down now, but I am still struggling with the formatting of regex rules, which I'm pretty unfamiliar with. The way I understand it, the rule ^(.*?utm_source=)(.+?(?=\\&)|$)(.*) (and its "medium" variant) will capture all values before 'utm_source=' into group 1 and all values (or none in the case of it being the last parameter) after '&' into group 3 and then I should be able to make the regexSubstition group1+WTTDOTM+group3, right? When I currently try to upload the unpacked extension below, I get hit with an error that says "rules.json: Rule with id 1 specifies an incorrect value for the "regexFilter" key." I think I am just formatting it wrong, but I do not know what I need to fix. Can you help? Here are my rules.json and manifest.json files.
rules.json
[
{
"id": 1,
"priority": 1,
"action": {
"type": "redirect",
"redirect": {
"regexSubstitution": "\\1WTTDOTM\\3"
}
},
"condition": {
"regexFilter": "^(.*?utm_source=)(.+?(?=\\&)|$)(.*)",
"resourceTypes": [
"main_frame"
]
}
},
{
"id": 2,
"priority": 2,
"action": {
"type": "redirect",
"redirect": {
"regexSubstitution": "\\1WTTDOTM\\3"
}
},
"condition": {
"regexFilter": "^(.*?utm_medium=)(.+?(?=\\&)|$)(.*)",
"resourceTypes": [
"main_frame"
]
}
}
]
manifest.json
{
"manifest_version": 2,
"name": "WTTDOTM is a UTM",
"version": "1.3",
"permissions": [
"declarativeNetRequest",
"declarativeNetRequestFeedback",
"<all_urls>"
],
"description": "Replaces all UTM values with 'WTTDOTM'",
"declarative_net_request" : {
"rule_resources" : [{
"id": "1",
"enabled": true,
"path": "rules.json"
},
{
"id": "2",
"enabled": true,
"path": "rules.json"
}],
"icons": {
"128": "icon128.png" }
}
}
It's probably not impossible to do this with regular expressions but it's going to be frustrating. The text that you're trying to capture could contain almost anything, so building a regex that captures what's after each query parameter and only what's after each query parameter is going to be difficult. Instead I would suggest using Chrome's built in URLSearchParams API.
Combined with Javascript's URL object, what you're trying to do would probably look like this:
const replaceTrackerParams = (urlString, replacementString) => {
const url = new URL(urlString);
const params = new URLSearchParams(url.search);
// If campagin parameters exist, replace them
params.has('utm_source') && params.set('utm_source', replacementString)
params.has('utm_campaign') && params.set('utm_campaign', replacementString)
// Return modified URL
return url.hostname + '?' + params.toString()
}
Replace shorthand syntax with more complicated logic as needed.
Another answer suggested using declarativeNetRequest because it is the newer, more efficient API. This is true, but I don't think it supports callback functions yet.

Why flatten API responses in Node application

I've been asked to look at flatting API responses on a Node.js project I'm working on but I'm not entirely sure why it needs to be flattened. No responses are nested beyond 2 levels so to me it's not too bad. Can anybody tell me why an API response would be flattened? I'd be keen to know the pros and cons aswell an hear any suggestions on how to do it? I'm currently looking at npm package flat.
Here's an example response:
{
"users": [
{
"id": 1,
"name": "John Doe",
"email": "john#doe.com",
"suppliers": [
{
"id": 1,
"name": "Supplier1",
}
]
},
{
"id": 2,
"name": "Jane Doe",
"email": "jane#doe.com",
"suppliers": [
{
"id": 1,
"name": "Supplier1",
},{
"id": 2,
"name": "Supplier2",
}
]
}
]
}
Can anybody tell me why an API response would be flattened? I'd be
keen to know the pros and cons aswell an hear any suggestions on how
to do it?
Pros:
An API needs to be flattened if the consumer of that API requires it to be flattened. For example, if the purpose of the API is to provide data that gets loaded into a table view of some kind, it would be much more convenient to the consumer if the API returned the data flattened to match the shape of that table.
Cons:
Flattening hierarchical data often times requires either limiting the number of child elements that can be returned or creating more rows that makes sense.
Consider your data flattened in these two approaches:
Flattened approach #1
{
"users": [
{
"id": 1,
"name": "John Doe",
"email": "john#doe.com",
"suppliers_1_id": 1,
"suppliers_1_name": "Supplier1",
"suppliers_2_id": null,
"suppliers_2_name": null
}, {
"id": 2,
"name": "Jane Doe",
"email": "jane#doe.com",
"suppliers_1_id": 1,
"suppliers_1_name": "Supplier1",
"suppliers_2_id": 2,
"suppliers_2_name": "Supplier2"
}
]
}
In this example, it has to be decided ahead of time what the maximum number of suppliers that can be returned. I really dislike this kind of design for storing data, but often it is the requested way for data to be displayed in a table. If for example there are 3 suppliers for a user, you would have no way to return that data without adding more columns. It very quickly becomes unmanageable unless you have a very small and finite maximum number of child rows.
Flattened approach #2
{
"users": [
{
"id": 1,
"name": "John Doe",
"email": "john#doe.com",
"supplier_id": 1,
"supplier_name": "Supplier1"
}, {
"id": 2,
"name": "Jane Doe",
"email": "jane#doe.com",
"supplier_id": 1,
"supplier_name": "Supplier1"
}, {
"id": 2,
"name": "Jane Doe",
"email": "jane#doe.com",
"supplier_id": 2,
"supplier_name": "Supplier2"
}
]
}
This approach simply repeats each user for the number of suppliers. This format is convenient for making it possible to convert the flattened data back to hierarchical data. It is a common approach when retrieving hierarchical data from a relational database in a single rowset. A client application can easily reassemble the original hierarchy by grouping by user id. The negative to this approach is it returns more than one row per top-level object, which can cause larger data payloads. If you have a single, internal consumer of the API, this approach may work. If it is for a public API, you'll have to spend more time documenting and supporting the API, because it may not make sense to your API consumers.
I'm not entirely sure why it needs to be flattened.
Whoever is asking you to flatten it should specify that actual shape they are looking for, and that may provide clarity. I've described two possible ways to flatten your data, but there are plenty more possibilities.

How can I get [nested] data from my JSON file in a database (mlab) using Javascript?

I know this question seems like it would obviously be answered somewhere, but I can't find it despite searching even the api.
My database looks like this:
{
"_id": {
"$oid": "1234567890"
},
"data": [
{
"Name": "Cow",
"Type": "animal",
"Type2": "",
},
{
"Name": "Apricot, pack of 3",
"Type": "animal",
"Type2": "treetype",
}
]
}
I want to do something like the user inputting "Apricot" and then it brings up "Apricot, pack of 3" and returns everything (name, type, type2)
So like (in awesome pseudocode)
var x= askuser
data = jsonarray(data)
arrayhandle = search(data(x))
console.print(arrayhandle.Name)
console.print(arrayhandle.Type)
console.print(arrayhandle.Type2)
Also I know it may be too much to ask (even breaching on fuzzy search), but I would really like it if my users could ask something like "Turkey" and the program respond will all the information for something named "Turkey, count 2"
Or if there were two entries that started with turkey, return all of them. So like the user inputs "turkey" and the name→desc is returned for "Turkey, count of 2" and "Turkey, count of 1/2" and "Turkey, pack of 20".
That would be perfect. But right now I just need to get the data. Also, it's sitting in mlab right now so I need to find a way to grab it from there and move it into my javascript file in glitch.
Thank you!

how to structure data in Firebase?

i come from a MySQL workflow and now i am trying to do something in Firebase, and i have some dilemmas with structuring my data, because i don't know how would i query it.
fallowing the example below where i have some users and some comments, what is a good way to find
how many post likes a user had
how many post comments a user had
what were the posts that a user liked
...
i was thinking on adding that information to a user, like:
"user:2": {
"name": "Doe",
"email": "doe#bob.com",
"post_likes": {
"post:1": 1,
"post:2": 1
},
"post_comments": {
"post:1": 15,
"post:2": 5
}
}
but this seems redundant and duplicates data..
i need to find a way to search in posts everything that has to do with user:1, maybe i need to create another object that connects the users and posts ??
any ideas?
{
"array": {
"users": {
"user:1": {
"name": "John",
"email": "john#bob.com"
},
"user:2": {
"name": "Doe",
"email": "doe#bob.com"
}
},
"posts": {
"post:1": {
"name": "First Post",
"content": "some post content",
"comments": {}
},
"post:2": {
"name": "Second Post",
"content": "some other post content",
"likes": 2,
"comments": {
"comment:1": {
"uid": "user:1",
"email": "some comment"
},
"comment:2": {
"uid": "user:2",
"email": "some other comment"
}
}
}
}
}
}
How to structure data is not a simple question to answer and is highly dependent on how the data will be read. The general rule is to work hard on writes to make reads easy. Firebase offers two guides on understanding data and on structuring data.
They won't fit nicely into this text box, so you'll find better answers there. Here's a birds-eye view.
It's JSON data.
Flatten your data. Don't nest just because you can. Treat each logical data component like you would in SQL and keep it in its own path.
Avoid arrays in distributed systems. Sequential numeric ids are error prone.
Utilize indices to create sub-lists of master data rather than trying to nest lists in lists in lists.

Formatting JSON using the RESTSerializer in the latest Ember Data version

I'm struggling to 'munge' my JSON into the correct format.
To illustrate i've made a quick, JSfiddle.
http://jsfiddle.net/chrismasters/NQKvy/638/
The format the server returns the data has a couple of differences to the preferred format recommended by Ember Data now.
Here is the raw JSON output
{
"video": {
"uuid": "8a660002-03c6-4b8e-bd8b-4ce28fa0dacd",
"state": "pending",
"theme": "basic",
"resolution": "nHD",
"title": "Test title",
"track": {
"uuid": "376fc3bb-d703-49e7-9d92-bce7f6bf8b56",
"state": "complete",
"source": "upload"
}
}
}
The first is that rather than use IDs it uses a UUID that is a string.
I seem to have managed to fix that using the normalizeHash, for video at least - but i'm not sure whether the same approach will fix the track model too - especially if I use embedding as I need to.
This is where the big problems start to appear, if I comment out the belongsTo relationship from the video model then it works OK, so I think... it is clearly a problem with the JSON formatting for the embedded track data.
Here are the model definitions and the serialization
App.Video = DS.Model.extend({
title: DS.attr('string'),
//track: DS.belongsTo('track', { embedded: true })
});
App.VideoSerializer = DS.RESTSerializer.extend({
normalizeHash: {
video: function(hash) {
hash.id = hash.uuid;
delete hash.uuid;
return hash;
}
}
});
I'd really appreciate some advice on how to format this response into a format that Ember Data recognises.
Also - does anyone know of a tool or good way of debugging these serialization transformations because at the moment the error message from Ember is not very helpful in terms of debugging or seeing what the serialization output is.
Many thanks for any help you can suggest.
Chris
In case anyone else has the same confusion over serializations I thought i'd include an explanation how to solve this problem.
Here is the working jsbin:
http://jsbin.com/fuzu/4
The main points are:
Primary Keys
primaryKey: 'uuid'
Is useful to convert the id into the correct naming & needs to be applied explicitly to any serializers (using globally on a ApplicationSerializer didn't seem to work).
Model Relationships
track: DS.belongsTo('track', {embedded: true} )
Ensure the definition of the relationship includes embedding & only on one side.
Extract Single
extractSingle: function(store, type, payload, id, requestType) {
var tracks = [];
var track = payload.video.track;
var video = payload.video;
tracks.push(track);
video.track = payload.video.track.uuid;
payload = { video: video, track: tracks };
return this._super(store, type, payload, id, requestType);
}
Pluralization is really important for Ember Data to understand the relationships, even though the model relationship is a belongsTo.
You can see this clearly in the desired (working) JSON
{
"video": {
"id": "8a660002-03c6-4b8e-bd8b-4ce28fa0dacd",
"state": "pending",
"theme": "basic",
"resolution": "nHD",
"title": "Test title",
"track": "2"
},
"track": [{
"id": "2",
"state": "complete",
"source": "upload"
}]
}
The track value in video isn't wrapped in an array, yet the root track value is an array.
For this reason I found it very useful first define the desired JSON and test it working first, then try to munge the real JSON into that format.
I think a tool to help with this process (visualising real-time JSON output from seraliziation) could be a great addition to Ember Data & something I'm going to look into creating.

Categories