We Keep Coding

Parse JSON data, and then pass all its values to another function

I'm building this website: http://collections.design
The way it works is by reading all tools data from a JSON, using jQuery (I don't know much javascript). Then, you can click on an item and a side panels opens with further information. But there's a lot of repeated code, so I'm trying to optimise it a bit.
First I parse the JSON:
// The data source
var data_source = "../data/tools/tools.json";
// Parsing the JSON
$.getJSON(data_source, function(data) {
$.each(data, function(key,val) {
// And I'm storing all of its values in variables, to make them easier to read:
var name = val.availability.name;
var linux = val.os.linux;
// Then I'm using all that to render each item on screen
…
});
});
Each of the items has a button that calls another function to create and open the side panel. The side panel reuses that item's data from the JSON. This function to create the side panel is using the name variable as parameter, but then inside is parsing the JSON again to get the rest of the values it needs.
My question is:
How can I "encapsulate" all variables when I do the JSON parsing, then pass it as a parameter to the other function; and finally, individually read each of those values in the other function?
I tried working with arrays. But didn't manage it to work, also keeping in mind that I'm trying to simplify things, not repeat myself, and keep short names…
Maybe I'm asking too much, but any pointers or links to doc will be appreciated.

I see two ways of doing this.
1) Save the JSON data outside the scope so you can reuse it and pass the index of the data you want.
Something like this
// The data source
var data_source = "../data/tools/tools.json";
var all_data;
// Parsing the JSON
$.getJSON(data_source, function(data) {
all_data = data;
$.each(data, function(key,val) {
$('.button').on('click', function() { callToOtherFunction(key) })
});
});
function callToOtherFunction(key) {
console.log(all_data[key]);
}
2) As Sam Axe said, pass the data directly to the function
// The data source
var data_source = "../data/tools/tools.json";
// Parsing the JSON
$.getJSON(data_source, function(data) {
$.each(data, function(key,val) {
$('.button').on('click', function() { callToOtherFunction(key) })
});
});
function callToOtherFunction(val) {
console.log(val);
}
Here's a working fiddle.

The data is already "encapsulated" in the data object. Pass that object to the function that you want to use the data in.
You could always construct a new object - but what's the point - it's already in the data object.

JSON.stringify is very slow for large objects

I have a very big object in javascript (about 10MB).
And when I stringify it, it takes a long time, so I send it to backend and parse it to an object( actually nested objects with arrays), and that takes long time too but it's not our problem in this question.
The problem:
How can I make JSON.stringify faster, any ideas or alternatives, I need a javaScript solution, libraries I can use or ideas here.
What I've tried
I googled a lot and looks there is no better performance than JSON.stringify or my googling skills got rusty!
Result
I accept any suggestion that may solve me the long saving (sending to backend) in the request (I know its big request).
Code Sample of problem (details about problem)
Request URL:http://localhost:8081/systemName/controllerA/update.html;jsessionid=FB3848B6C0F4AD9873EA12DBE61E6008
Request Method:POST
Status Code:200 OK
Am sending a POST to backend and then in JAVA
request.getParameter("BigPostParameter")
and I read it to convert to object using
public boolean fromJSON(String string) {
if (string != null && !string.isEmpty()) {
ObjectMapper json = new ObjectMapper();
DateFormat dateFormat = new SimpleDateFormat(YYYY_MM_DD_T_HH_MM_SS_SSS_Z);
dateFormat.setTimeZone(TimeZone.getDefault());
json.setDateFormat(dateFormat);
json.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
WebObject object;
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "Start");
try {
object = json.readValue(string, this.getClass());
} catch (IOException ex) {
Logger.getLogger(JSON_ERROR).log(Level.SEVERE, "JSON Error: {0}", ex.getMessage());
return false;
}
// Logger.getLogger("JSON Tracker").log(Level.SEVERE, "END");
return this.setThis(object);
}
return false;
}
Like This
BigObject someObj = new BigObject();
someObj.fromJSON(request.getParameter("BigPostParameter"))
P.S : FYI this line object = json.readValue(string, this.getClass());
is also very very very slow.
Again to summarize
Problem in posting time (stringify) JavaScript bottle nick.
Another problem parsing that stringified into an object (using jackson), and mainly I have svg tags content in that stringified object as a style column, and other columns are strings, int mainly

As commenters said - there is no way to make parsing faster.
If the concern is that the app is blocked while it's stringifying/parsing then try to split data into separate objects, stringily them and assemble back into one object before saving on the server.
If loading time of the app is not a problem you could try to ad-hoc incremental change on top of the existing app.
... App loading
Load map data
Make full copy of the data
... End loading
... App working without changes
... When saving changes
diff copy with changed data to get JSON diff
send changes (much smaller then full data)
... On server
apply JSON diff changes on the server to the full data stored on server
save changed data
I used json-diff https://github.com/andreyvit/json-diff to calc changes, and there are few analogs.

Parsing is a slow process. If what you want is to POST a 10MB object, turn it into a file, a blob, or a buffer. Send that file/blob/buffer using formdata instead of application/json and application/x-www-form-urlencoded.
Reference
An example using express/multer

Solution
Well just as most big "repeatable" problems go, you could use async!
But wait, isn't JS still single-threaded even when it does async... yes... but you can use Service-Workers to get true async and serialize an object way faster by parallelizing the process.
General Approach
mainPage.js
//= Functions / Classes =============================================================|
// To tell JSON stringify that this is already processed, don't touch
class SerializedChunk {
constructor(data){this.data = data}
toJSON() {return this.data}
}
// Attach all events and props we need on workers to handle this use case
const mapCommonBindings = w => {
w.addEventListener('message', e => w._res(e.data), false)
w.addEventListener('error', e => w._rej(e.data), false)
w.solve = obj => {
w._state && await w._state.catch(_=>_) // Wait for any older tasks to complete if there is another queued
w._state = new Promise((_res, _rej) => {
// Give this object promise bindings that can be handled by the event bindings
// (just make sure not to fire 2 errors or 2 messages at the same time)
Object.assign(w, {_res, _rej})
})
w.postMessage(obj)
return await w._state // Return the final output, when we get the `message` event
}
}
//= Initialization ===================================================================|
// Let's make our 10 workers
const workers = Array(10).fill(0).map(_ => new Worker('worker.js'))
workers.forEach(mapCommonBindings)
// A helper function that schedules workers in a round-robin
workers.schedule = async task => {
workers._c = ((workers._c || -1) + 1) % workers.length
const worker = workers[workers._c]
return await worker.solve(task)
}
// A helper used below that takes an object key, value pair and uses a worker to solve it
const _asyncHandleValuePair = async ([key, value]) => [key, new SerializedChunk(
await workers.schedule(value)
)]
//= Final Function ===================================================================|
// The new function (You could improve the runtime by changing how this function schedules tasks)
// Note! This is async now, obviously
const jsonStringifyThreaded = async o => {
const f_pairs = await Promise.all(Object.entries(o).map(_asyncHandleValuePair))
// Take all final processed pairs, create a new object, JSON stringify top level
final = f_pairs.reduce((o, ([key, chunk]) => (
o[key] = chunk, // Add current key / chunk to object
o // Return the object to next reduce
), {}) // Seed empty object that will contain all the data
return JSON.stringify(final)
}
/* lot of other code, till the function that actually uses this code */
async function submitter() {
// other stuff
const payload = await jsonStringifyThreaded(input.value)
await server.send(payload)
console.log('Done!')
}
worker.js
self.addEventListener('message', function(e) {
const obj = e.data
self.postMessage(JSON.stringify(obj))
}, false)
Notes:
This works the following way:
Creates a list of 10 workers, and adds a few methods and props to them
We care about async .solve(Object): String which solves our tasks using promises while masking away callback hell
Use a new method: async jsonStringifyThreaded(Object): String which does the JSON.stringify asynchronously
We break the object into entries and solve each one parallelly (this can be optimized to be recursive to a certain depth, use best judgement :))
Processed chunks are cast into SerializedChunk which the JSON.stringify will use as is, and not try to process (since it has .toJSON())
Internally if the number of keys exceeds the workers, we round-robin back to the first worker and overschedule them (remember, they can handle queued tasks)
Optimizations
You may want to consider a few more things to improve performance:
Use of Transferable Objects which will decrease the overhead of passing objects to service workers significantly
Redesign jsonStringifyThreaded() to schedule more objects at deeper levels.

You can explore libraries like fast-json-stringify which use a template schema and use it while converting the json object, to boost the performance. Check the below article.
https://developpaper.com/how-to-improve-the-performance-of-json-stringify/

How to access and use multiple data from json object? Do I need to make an array?

I am a beginner and using $.get to retrieve data from a rest API such as:
[{"id":"1","url":"http:\/\/123.456.78.910\/workforce\/images\/item1.jpg","price":"99","description":"Mobile Phone"},
{"id":"2","url":"http:\/\/123.456.78.910\/workforce\/images\/item2.jpg","price":"98","description":"Laptop"}
{"id":"3","url":"http:\/\/123.456.78.910\/workforce\/images\/item3.jpg","price":"92","description":"Console"}] }
$.get('http://xxxxxxxxxxx,
function (data) {
var obj = $.parseJSON(data);
So from what I understand I have retrieved the data from the REST API and parsed it so it is stored in a variable called obj.
My question is, how do I access and use each unique record in the obj variable?
Each record has it's own picture (item1.jpg, item2.jpg etc).
Whem my app loads I want it to show the item1.jpg image, and I want to be able to navigate to the other item pictures using buttons (previous / next).
I also want the description and price to be displayed underneath in some text input fields.
What I have figured so far is that I should:
Iterate through the obj variable, and store each record into an array.
Upon app initialisation I can set the default value for the image placeholder to array[index0].url, and set the description and price fields.
I can then set the previous and next buttons to array[currentIndex-1] or array[currentIndex+1].
Would this be the best way to do it?
Or can I just do this without using an array and manipulate the obj.data directly?
Thanks!!!

I may not be understanding what exactly what you want to do but I think I have the gist. If you just want to show the picture then the array of just images probably wouldn't be a bad idea. However, it looks like the Jason you're getting is already in an array. You can just use array index notation to get to what you want.
ie)
var arr = //your json response ;
var current = 0; //sets currently displayed object to the first in the array
var setCurrent = function () {
var image = arr[current]["url"];
}
You can then modify current however you want (on click on arrow iterate up/down, etc) then call the setCurrent function to set your image the the one you want. Hope that helps!

You can use the response you have from $.get() directly.
It is an array of objects.
You can use it like this:
console.log(data[2].description);
// outputs: "Console"
I've made a CodePen demo where it has a 4th object with a real image url to show you how to use the url info...
EDIT
Just in case you wouldn't know this:
You can use the response inside the scope of the $.get() callback...
You can not use it straith after the $.get() outside the callback since $.get() is asynchronous.
You can use it in some other handler wich will happen after the response is received.
var getResponse;
$.get('http://xxxxxxxxxxx', function (data) {
getResponse = data;
console.log(data[2].description);
// outputs: "Console"
console.log(getResponse[2].description);
// outputs: "Console"
});
console.log(getResponse[2].description);
// outputs: "Undefined"
// But since this handler will be triggered long after the response is obtained:
$("#somebutton").click(function(){
console.log(getResponse[2].description);
// outputs: "console"
});

In order for your page javascript to be able to access the data retrieved from your ajax request, you'll need to assign it to some variable which exists outside the callback function.
You will need to wait until the ajax request has been processed before you can read the array. So you might want to set the actual default image to be something that doesn't rely on the ajax request (a local image).
Here's a simple approach
// fake testing ajax func
function fakeget (url, callback) {
setTimeout(callback(JSON.stringify([
{"id":"1","url":"http:\/\/123.456.78.910\/workforce\/images\/item1.jpg","price":"99","description":"Mobile Phone"}, {"id":"2","url":"http:\/\/123.456.78.910\/workforce\/images\/item2.jpg","price":"98","description":"Laptop"},
{"id":"3","url":"http:\/\/123.456.78.910\/workforce\/images\/item3.jpg","price":"92","description":"Console"}
])), 1000);
}
// real code starts here
// global variables for ajax callback and setImg func to update
var imageData, currentImg;
// change this back to $.get for real
fakeget('http://xxxxxxxxxxx',
function (data) {
imageData = $.parseJSON(data);
setImg(0);
}
);
function setImg(index) {
// turns negative indices into expected "wraparound" index
currentImg = (index % imageData.length + imageData.length) % imageData.length;
var r = imageData[currentImg];
$("#theImg").attr('src', r.url);
$('#theDescription').text(r.price + " " + r.description);
}
$("#prev").click(function () {
setImg(currentImg - 1);
});
$("#next").click(function () {
setImg(currentImg + 1);
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<div>
<img id='theImg' src='somedefault.jpg'>
<div id='theDescription'></div>
</div>
<button id='prev'>Prev</button>
<button id='next'>Next</button>

Few observations :
Your JSON Object is not a valid JSON.
No need to parse it again your data is already a JSON Object.
Working fiddle
var data = [{"id":"1","url":"http:\/\/123.456.78.910\/workforce\/images\/item1.jpg","price":"99","description":"Mobile Phone"},{"id":"2","url":"http:\/\/123.456.78.910\/workforce\/images\/item2.jpg","price":"98","description":"Laptop"}, {"id":"3","url":"http:\/\/123.456.78.910\/workforce\/images\/item3.jpg","price":"92","description":"Console"}];
for (var i in data) {
var imgUrl = data[i].url;
console.log(imgUrl);
}

Add data / populate data to the user object

I'm new to FeathersJS and want to add additional data to the user object after authorization. (I want to access the data with app.get('user').newData)
First I tried to set a value in hook.result.newData, but it doesn't work. I'm only able to change an existing value like email.
module.exports = function(options) {
return function(hook) {
hook.result.households = hook.app.service('households').find({'people.userId': hook.id}).then(household => {
console.log('households: ', household.data);
return household.data;
});
hook.result.email = '!' + hook.result.email + '?'; // this works
hook.result.newData = 'test'; // this doesn't work
};
};
Another approach I tried was modifying the code snippet found here: https://github.com/feathersjs/feathers-hooks/issues/56 but also with no success.
But I'm also not sure how to populate in my case. I want to add objects of another service, which have an array of user ids. And I only want those objects, which contain the own user id among others in the array.

Object Compare in AngularJS

I have a form which when loaded sends GET request to server and recives data which will stored in 'master' and i copy that data to 'local' as below.
$scope.dirty = false;
init(data);
function init(data) {
$scope.master = angular.copy(data.data);
$scope.local = angular.copy($scope.master);
}
Now, I use local object as model for my form and I have to button submit and reset. I watch the local object as below.
$scope.$watchCollection('local', function (newLocal, oldLocal) {
$scope.dirty = !angular.equals(newLocal, $scope.master);
});
So, If dirty is true then i can know that data has been modified but since I am using Objects AngularJS adds $$hasKey to $scope.local and because of that $scope.dirty always sets to true.
So, is there any way to handle this problem? I am new to AngularJS so may be this can be funny question but I'm stuck.

You could convert your object to a JSON string before comparing:
function init(data) {
// store json data into $scope.master for later comparison
$scope.master = angular.toJson(data.data);
$scope.local = angular.copy(data.data);
}
$scope.$watchCollection('local', function (newLocal, oldLocal) {
var json = angular.toJson(newLocal); // new local without $$ key
$scope.status.dirty = !angular.equals(json, $scope.master);
// $scope.local is still a javascript object
});

I was sending data form PHP and PHP treats Number and string as seperate datatype.
So I converted those Number data to string and now It works as desired and also I leart that whenever I use <form name='newForm> angularJS creates a new scope named newForm so that I can you many properties of that scope like $dirty, $pristinem $submitted and many more. so Now I dont have to write this logic by myself