I'm adding some APIs to a JavaScript project to replace what used to be multi-field manual data entry with lookups. (E.g., we want to stop asking you 20 questions about a car, and instead just ask your VIN and autopopulate the other 19 answers from a VIN decoder.)
I'm spending more time than I'd like mapping data from the response schema into the existing internal schema of my app. Lots of the work requires a human touch to suss out synonyms, like:
internal.postal_code = api.zipCode;
Some times I find myself writing a really gnarly if to avoid a sometimes-null or missing object half way down a deep tree, like
if(api.a && api.a.b && api.a.b.c){
internal.z = api.a.b.c.d;
}
Is there a good library that would let me write a simple map and do all this work for me? A map might look like:
map = {
'zipCode' : 'postal_code',
'a.b.c.d' : 'z'
};
mapperTool( api, internal, map );
(Note the internal object is stitched together from several APIs and pre-existing tools, so adding or overwriting properties on internal is better than outputting a new object.)
Try this:
function mapperTool(source, desc, map) {
Object.keys(map).forEach(function(key) {
var value = key.split('.').reduce(function(obj, name) {
if (obj && obj[name]) {
return obj[name];
}
}, source);
desc[map[key]] = value;
});
}
var map = {
'zipCode' : 'postal_code',
'a.b.c.d' : 'z'
};
var internal = {};
mapperTool({a:{b:{c:{d: 10}}},zipCode:20}, internal, map);
document.body.innerHTML = '<pre>' + JSON.stringify(internal, true, 4) + '</pre>';
Related
I need to serialize and deserialize JavaScript objects to store them in a DB.
Note that these objects contain functions, so I can't store them as JSON, so I can't use json2.js.
What's the state of the art in [de]serialization of JavaScript objects (in JavaScript of course).
In general, there's no way (in a browser) to serialize objects with functions attached to them: Every function has a reference to its outer scope, that scope won't exist when you deserialize it, so serialized references to that scope will be invalid.
What I would do is use the built-in (or json2.js) JSON.stringify and JSON.parse functions with the replacer and reviver parameters. Here's a partial example of how it would work:
JSON.stringify(yourObject, function(name, value) {
if (value instanceof LatLng) { // Could also check the name if you want
return 'LatLng(' + value.lat() + ',' + value.lng() + ')';
}
else if (...) {
// Some other type that needs custom serialization
}
else {
return value;
}
});
JSON.parse(jsonString, function(name, value) {
if (/^LatLng\(/.test(value)) { // Checking the name would be safer
var match = /LatLng\(([^,]+),([^,]+)\)/.exec(value);
return new LatLng(match[1], match[2]);
}
else if (...) {
...
}
else {
return value;
}
});
You can use any serialization format you want in your custom types. The "LatLng(latitude,longitude)" format is just one way of doing it. You could even return a JavaScript object that can be serialized to JSON natively.
You don't want to serialize logic such as functions.
If you have to update your logic / js functions in the future, you don't (always) want the older logic to be loaded back with the data neccessarily. Beware.
use gserializer:
http://www.onegeek.com.au/articles/programming/javascript-serialization.php
the code in google :
http://code.google.com/p/gserializer/
GSerializer is a javascript library to
serialize/deserialize javascript
objects to and from strings, for
persistance in say, a Cookie. Unlike
many other implementations,
GSerializer can also serialize
functions and non-JSON notation.
On Node.js, there is also the JASON package.
Here is the example:
var JASON = require("JASON");
str = JASON.stringify(obj);
obj = JASON.parse(str);
Install the package by: npm install JASON.
If you're using ES6 versions of Node, you can check out a small package I wrote called JSOFF. It's the JavaScript Object-Function Format; a drop-in replacement for JSON that handles functions.
It's super tiny and simple, so Babeljs or Browserify may be your friends.
Install via: npm install jsoff or yarn add jsoff.
Here is the example how to create an object with functions:
const JSOFF = require('jsoff');
var obj = {
abc: 123,
def: function (a,b) { return a * 2 + b * 3; },
ghi: a => { return a * 2 },
jkl: (a,b) => { return ((d,e) => { return a*d + b*e })(2,4) }
};
var str = JSOFF.stringify(obj);
// str is now:
// '{"abc":123,"def":"function (a,b) { return a * 2 + b * 3; }","ghi":"a => { return a * 2 }","jkl":"(a,b) => { return ((d,e) => { return a*d + b*e })(2,4) }"}');
});
var clone = JSOFF.parse(str);
clone.def(10,5) // 35
clone.ghi(5) // 10
clone.jkl(10,20) // 100
I wouldn't serialize JS functions because of security reasons. Through a public API all kinds of nasty things could be sent to the database. As for deserialisation I've got a different approach. I'm mixing model objects defined on client side with the data coming from JSON. I have a small tool to do that, take a look at it on GitHub at khayll/jsonmix.
JsonMix provides a kind of deserialisation from JSON into JavaScript Objects complete with functions.
It would look like something:
//model definition (just an example)
var LatLng = function() {}
LatLng.prototype.getMapTypeId = function() {
return this.mapTypeId;
}
//deserializing done like this
var result = JSMix(jsonString).withObject(LatLng.prototype, "latLngs").build();
//all items in the latLngs collection have the functions coming from the model
console.log(result.latLngs[5].getMapTypeId());
As title, when I try to do:
myString = JSON.stringify($('#calendar').fullcalendar('clientEvents'));
it fails. I tried to alert myString but I see a series of [Object object], ... . But if i try to alert myArray[0].title for example, it returns correctly.
Where I'm doing wrong?
P.S. The goal is to obtain a string to save on a file via AJAX.
Your results tell you that the objects in the array that the fullCalendar clientEvents method gives you can't be directly converted to JSON. I get slightly different results on the http://fullcalendar.io page (I get an error about trying to convert a circular structure); I assume that's down to differences either in the FullCalendar version you're using vs. they're using, or differences in how your browser and mine deal with circular structures. Either way, the objects apparently can't be used as-is.
The goal is to obtain a string to save on a file via AJAX.
You can do that by using map on the array to get objects that can be converted to JSON successfully, whitelisting the properties you want (or blacklisting the ones you don't want).
Here's an example whitelisting the start, end, and title properties:
var json = JSON.stringify($("#calendar").fullCalendar("clientEvents").map(function(e) {
return {
start: e.start,
end: e.end,
title: e.title
};
}));
Heres one blacklisting source and any property starting with _:
var json = JSON.stringify($("#calendar").fullCalendar("clientEvents").map(function(e) {
var rv = {};
Object.keys(e)
.filter(function(k) {
return k != "source" && !k.startsWith("_");
})
.forEach(function(k) {
rv[k] = e[k];
});
return rv;
}));
...which worked for me on their site.
Here are ES2015 versions of both of those:
Whitelisting:
let json = JSON.stringify($("#calendar").fullCalendar("clientEvents").map(e => ({
start: e.start,
end: e.end,
title: e.title
})));
Blacklisting:
let json = JSON.stringify($("#calendar").fullCalendar("clientEvents").map(e => {
let rv = {};
Object.keys(e)
.filter(k => k != "source" && !k.startsWith("_"))
.forEach(k => {
rv[k] = e[k];
});
return rv;
}));
I have a http server built in node.js and 'connect'. The web service currently parses JSON requests to an Object, does "stuff", and returns a synchronous response. The JSON is from an ecommerce cart and I end up with an Object that looks like (abridged version):
var myObj = {"request":{"industry":"social","transactionId":"gdfs23s","billing": {"addressLine1":"911 Fallen Street","addressLine2":"1 2"},"shipping":{"addressLine1":"1523 Willow Tree LAne","addressLine2":"unit 15"}}}
I want to clean up the data, performing tasks such as removing extra white space, normalizing postal abbreviation, ie street to st, etc.
I've written a series of regular expression that successfully achieve the cleansing/normalization.
However what I am unsure of is how to do this efficiently AND elegantly apply these cleanse processes to an Object in JS /Node.js. I will have scenarios where I want to cleanse request.billing.addressLine1, request.shipping.addressLine1 with the same regex pattern
I could of course do something like:
var addressCleaner= new RegExp("( str| street| strt)","g");
myObj.request.billing.addressLine1.replace(addressCleaner, "st");
myObj.request.shipping.addressLine1.replace(addressCleaner, "st");
But I dont feel this is very DRY and furthermore its not being done very "node"ishly.
Any suggestions or example approaches? I would like to avoid using a package like Mongoose etc. to do this as the type of normalizing i'm doing does not just consist of making sure a string is a string ,etc.
Thanks in advance.
So, I would suggest to have a hash with all normalizers, and seperately to have a list of properties,
which needs to be normalized. To have the idea here some code:
var Normalizers = {
// -> function
trim: function(str) { return str.trim(); },
// -> array [RegExp, StringToReplace]
street: [ /(str|street)/g, 'st']
//...
};
var properties = {
'request.billing.addressLine1': ['trim', 'street'],
// ..
};
obj_normalize(myObj, properties, Normalizers);
The code for obj_normalize/obj_getProprety/obj_setProperty I moved to the gist.
If your regex is applicable to every string found within the object you can simply recurse through the object and apply the regex to every string.
A general purpose object traversal function is very useful for something like this:
function object_traverse (name,obj,fn) {
obj = fn(name,obj);
if (obj instanceof Array) {
for (var n=0;n<obj.length;n++) {
obj[n] = object_traverse(n,obj[n],fn);
}
}
else if (typeof obj != "string" && typeof obj != "number") {
for (var n in obj) {
obj[n] = object_traverse(n,obj[n],fn);
}
}
return obj;
}
Now you can do this:
myObj = object_traverse('',myObj,function(name,obj){
if (typeof obj == "string" && name.match(/address/i)) {
obj = obj.replace(addressCleaner, "st");
}
return obj;
});
I'd have a model built from JSON files and serialize it as I see fit. This would avoid matching or searching for properties which couldn't possibly exist in the source. Some example:
function makeAddress(rawAddress) {
return { street: rawAddress["str"] ||
rawAddress["street"] ||
rawAddress["strt"],
. . . };
Being equipped with this function, say, then you have an array of "address" object, then converting them would be a matter of:
addresses.map(makeAddress);
I've seen very similar questions to this, but I can't quite decide if they was answered clearly - maybe I'm being a bit dense, sorry.
I want to have the convenience (and clarity) of my own object, call it a CardboardBox(). It won't contain code, just data. I want to write this to a database and read it back later, but obviously, it is a type Object() when it's read back. All I can think of to find out what it used to be is:
Have a member variable type that I set to CARDBOARD_BOX
Instantiate a new CarbardBox() and use a function (in the box) to copy the properties of Object() to the new CardboardBox() object
Is there a better way of doing this? I'm pretty sure I can change the actual type.
function CardboardBox() {
this.type = "CARDBOARD_BOX"
this.name = "No set";
this.populate = new function(obj) {
// populate this object with obj properties
}
var box = new CarboardBox(); // CarboardBox
box.name = "My Box";
send = JSON.stringyfy(box);
.
.
.
obj = JSON.parse(send); // Object
if (obj.type == "CARDBOARD_BOX") {
savedBox = new CardboardBox();
savedBox.populate(obj);
}
Thanks in advance...
Steve
[edit] My test code.
function CardboardBox(n) {
this.name = n;
}
var box = new CardboardBox("My Box");
send = JSON.stringify(box); // JSON CarboardBox()
obj = JSON.parse(send, function fn(obj) { // Object() returned
log("OB: "+obj.type);
return obj.type === 'CardboardBox' ? new CardboardBox(obj) : CardboardBox;
});
console.log(obj);
Output is:
OB: undefined utils.js:40
OB: undefined utils.js:40
function CardboardBox(n) {
this.name = n;
}
One possible solution is the following:
function CardboardBox(n) {
if(typeof(n) == 'string') {
//build from name string
this.name = n;
} else {
//build from object
this.name = n.name;
}
//add in this object's "type" in a place
//that is unlikely to exist in other JSON strings
this.__type = 'CardboardBox';
}
var box = new CardboardBox("My Box");
send = JSON.stringify(box), // JSON CarboardBox()
obj = JSON.parse(send, function(key, val) {
//if this is an object, and is CardboardBox
if(typeof(val) === 'object' && val.__type === 'CardboardBox')
return new CardboardBox(val);
return val;
//or if your object is in a context (like window), and there are many of
//them that could be in there, you can do:
//
//if(typeof(val) === 'object' && context[val.__type])
// return new context[val.__type](val);
});
console.log(obj);
Basically store the object type in a place you know to look for later on when parsing the json. if you have multiple objects you can instantiate in a single scope the second parse method may be more appropriate. This also will account for objects in the JSON that are not CarboardBoxs.
Edit Here is a jsFiddle of this method in action.
Overall, you're correct: Javascript doesn't have any built-in way to serialize anything beyond plain objects, so going to and from JSON will not produce a particular class when you deserialize it. So you need to either work out serialization/deserialization yourself, or use a library that provides some support.
I personally like Backbone.js for this problem, as it handles serializing and deserializing quite well. You define a model class, which include a method to save its data to a server in a serialized form, and a method to deserialize it back to the model. The key design issue here is that deserializing is performed knowing the model you're deserializing to:
you either call myModel.fetch() to get data from the server based on the model id, or
you pass a bunch of new data to the model constructor: new Model(serializedData), or
you pass an array of data for multiple models to a collection that knows the model type: new ModelCollection(arrayOfSerializedData).
What Backbone doesn't do is deal with type-casting data of an unknown type. When I've dealt with this, I've usually done something similar to #Chad's response, but using an intermediary; you could see this as a proxy model, or as a factory:
var classes = {
CardboardBox: ...,
AluminumBox: ...
}
function Deserializer(json) {
// parse if you're actually dealing with a string
var data = JSON.parse(json),
// now look for some custom type flag - you'll need to set this yourself
type = data.type,
// class lookup, perhaps with a default
Cls = classes[type] || DefaultType;
return new Cls(data);
}
var obj = new Deserializer(send);
obj instanceof CardboardBox; // should work
This still relies on a custom flag to switch types, though - I'm not sure there's any way around this.
I need to serialize and deserialize JavaScript objects to store them in a DB.
Note that these objects contain functions, so I can't store them as JSON, so I can't use json2.js.
What's the state of the art in [de]serialization of JavaScript objects (in JavaScript of course).
In general, there's no way (in a browser) to serialize objects with functions attached to them: Every function has a reference to its outer scope, that scope won't exist when you deserialize it, so serialized references to that scope will be invalid.
What I would do is use the built-in (or json2.js) JSON.stringify and JSON.parse functions with the replacer and reviver parameters. Here's a partial example of how it would work:
JSON.stringify(yourObject, function(name, value) {
if (value instanceof LatLng) { // Could also check the name if you want
return 'LatLng(' + value.lat() + ',' + value.lng() + ')';
}
else if (...) {
// Some other type that needs custom serialization
}
else {
return value;
}
});
JSON.parse(jsonString, function(name, value) {
if (/^LatLng\(/.test(value)) { // Checking the name would be safer
var match = /LatLng\(([^,]+),([^,]+)\)/.exec(value);
return new LatLng(match[1], match[2]);
}
else if (...) {
...
}
else {
return value;
}
});
You can use any serialization format you want in your custom types. The "LatLng(latitude,longitude)" format is just one way of doing it. You could even return a JavaScript object that can be serialized to JSON natively.
You don't want to serialize logic such as functions.
If you have to update your logic / js functions in the future, you don't (always) want the older logic to be loaded back with the data neccessarily. Beware.
use gserializer:
http://www.onegeek.com.au/articles/programming/javascript-serialization.php
the code in google :
http://code.google.com/p/gserializer/
GSerializer is a javascript library to
serialize/deserialize javascript
objects to and from strings, for
persistance in say, a Cookie. Unlike
many other implementations,
GSerializer can also serialize
functions and non-JSON notation.
On Node.js, there is also the JASON package.
Here is the example:
var JASON = require("JASON");
str = JASON.stringify(obj);
obj = JASON.parse(str);
Install the package by: npm install JASON.
If you're using ES6 versions of Node, you can check out a small package I wrote called JSOFF. It's the JavaScript Object-Function Format; a drop-in replacement for JSON that handles functions.
It's super tiny and simple, so Babeljs or Browserify may be your friends.
Install via: npm install jsoff or yarn add jsoff.
Here is the example how to create an object with functions:
const JSOFF = require('jsoff');
var obj = {
abc: 123,
def: function (a,b) { return a * 2 + b * 3; },
ghi: a => { return a * 2 },
jkl: (a,b) => { return ((d,e) => { return a*d + b*e })(2,4) }
};
var str = JSOFF.stringify(obj);
// str is now:
// '{"abc":123,"def":"function (a,b) { return a * 2 + b * 3; }","ghi":"a => { return a * 2 }","jkl":"(a,b) => { return ((d,e) => { return a*d + b*e })(2,4) }"}');
});
var clone = JSOFF.parse(str);
clone.def(10,5) // 35
clone.ghi(5) // 10
clone.jkl(10,20) // 100
I wouldn't serialize JS functions because of security reasons. Through a public API all kinds of nasty things could be sent to the database. As for deserialisation I've got a different approach. I'm mixing model objects defined on client side with the data coming from JSON. I have a small tool to do that, take a look at it on GitHub at khayll/jsonmix.
JsonMix provides a kind of deserialisation from JSON into JavaScript Objects complete with functions.
It would look like something:
//model definition (just an example)
var LatLng = function() {}
LatLng.prototype.getMapTypeId = function() {
return this.mapTypeId;
}
//deserializing done like this
var result = JSMix(jsonString).withObject(LatLng.prototype, "latLngs").build();
//all items in the latLngs collection have the functions coming from the model
console.log(result.latLngs[5].getMapTypeId());