I need to somehow deep clone the entire set of bindings of my ScriptEngine object.
What I have tried
I have tried so far the Cloner library to clone the entire Bindings structure. This would be great if it worked because it would have ensured a precise copy, including private variables. But this leads to jvm heap corruption (the jvm just crashes with exit code -1073740940). Sometimes it doesn't crash but weird things happen, like the System.out.println() stops working as it should...
I have also looked into cloning the objects using js code inside the ScriptEngine, so that I can get those as NativeObjects and manage them in some java maps. But all cloning methods which I found have flaws. I want a precise snapshot of the objects. For instance if each of two objects a and b contain fields (say a.fa and b.fb) which reference the same object c, when cloned using jQuery.extend() (for instance) the fields a.fa and b.fb of the cloned a and b will reference different clones of c, instead of referencing one same clone. And many other edge issues.
I also tried to clone the entire ScriptEngine using Cloner (not only the bindings), and I also tried using Rhino's js engine and clone the entire scope (instead of the bundeled ScriptEngine wrapper). But the heap corruption issue persists.
Why I need to do it
I need this because I must be able to restore the values of the entire ScriptEngine bindings to some previous point. I need to make precise snapshots of the bindings.
The application is part of my doctoral research project which consists of running state machines with nodes (implemented in java) which have js code attached. The js code is typed in by the end user and it is being evaled at runtime. When final state can't be reached through a path, the algorithm makes steps backwards, trying to find alternative paths. On each step backward it must undo any changes that might have occurred in the js engine bindings.
All the global variables names are known before js eval-ing, and are objects (the user types in code for the nodes and this is then organized (within java) into js objects with certain name patterns). But their content can be anything because that is controlled by the user js code.
So I guess my only sollution now is to clone js object using js code.
Aside from "edge cases", jQuery.extend can be used in the way you mention. a b and their clones will all reference the same object c.
var c = { f:'see' };
var a = { fa: c };
var b = { fb: c };
var cloneA = $.extend({}, a);
var cloneB = $.extend({}, b);
console.log(a.fa === b.fb, cloneA.fa === cloneB.fb, a.fa === cloneB.fb);
// true true true
But it seems like you want to clone all the objects (including c) while keeping track of the objects' relationships. For this, it may be best to use object relation tables.
I see this a lot with nested javascript objects and JSON because people tend to forget that JSON is purely a text format. There are no actual javascript objects in a JSON file except for a single string of text instanceof String. There are no beans or pickles or any preservative-heavy foods in javascript.
In an object relation table, each 'table' is just an array of "flat" objects with only primitive valued properties and pointers (not references) to other objects in the table (or in another table). The pointer can just be the index of the target object.
So a JSON version of the above object relationship might look something like
{
"table-1":[
{ "a": { "fa":["table-2",0] } },
{ "b": { "fb":["table-2",0] } }
],
"table-2":[
{ "c": { "name":"see" } },
{ "d": { "name":"dee" } },
{ "e": { "name":"eh.."} }
]
}
And a parser might look like
var tables = JSON.parse(jsonString);
for(var key in tables){
var table = tables[key];
for(var i = 0; i < table.length; i++){
var name = Object.keys(table[i])
var obj = table[i][name];
for(var key2 in obj){
if(obj[key2] instanceof Array && obj[key2][0] in tables){
var refTable = obj[key2][0];
var refID = obj[key2][1];
var refObj = tables[refTable][refID];
var refKey = Object.keys(refObj)[0];
obj[key2] = refObj[refKey];
}
}
this[name] = obj;
}
}
console.log(a.fa === b.fb, b.fb === c);
// true true
I realize object relationship mapping has it's downfalls, but taking snapshots of the scripting engine does sound a bit crazy. Especially since your intention is to be able to recall each previous step, because then you need a new snapshot for every step... that will very quickly take up shit ton of disk space.. unless you're just keeping track of the snapshot diffs between each step, like a git repo. That sounds like an awful lot of work to implement a seemingly simple "undo" method.
Damn.. come to think of it, why not just store each step in a history file? Then if you need to step back, just truncate the history file at the previous step, and run each step over again in a fresh environment.
Not sure how practical that would be (performance wise) using java. Nodejs (as it exists today) is faster than any java scripting engine will ever be. In fact, I'm just gonna call it ECMAscript from now on. Sorry bout the rant, its just that
Java is slow, but you already know.
Because it readily shows, that's as fast as it goes.
May I suggest a different approach.
Don't try to clone the ScriptEngine. It doesn't implement Serializable/Externalizable and the API doesn't support cloning. Attempts to force a clone will be workarounds that might break in some future Java version.
Serialize Bindings to JSON using cycle.js. It will encode object references in the form {$ref: PATH}. When you deserialize it will restore the references.
As far as I can tell cycle.js doesn't serialize functions but its possible to add function serialization yourself using Function.toString() (See below and Example)
Alternatively if using a library is not an option its fairly straightforward to implement your own serialization to suit your needs:
var jsonString = JSON.stringify(obj, function(key, val) {
if (typeof(value) === 'function')
return val.toString();
// or do something else with value like detect a reference
return val
})
Related
Don't do it
This question has some comments with a low opinion of the very notion of reconstructing the objects. The commenters either couldn't or wouldn't explain why they thought it was a bad idea, but since asking I have come the to same conclusion. Here's why.
If you think about MVVM, the purpose of having a model and a view-model is to separate behaviour from data. This is kind of funny, because the point of object-orientation is to combine them. But in a distributed world, the data has to be shipped around. If your code and data are all munged together then you have to either invent MVVM or keep de- and re-constructing objects.
The code to de- and re-construct objects is a testing and maintenance time-sink you don't need, and introduces two failure modes. Don't do it. Have a method-less class to hold the state and a stateless class that operates on the method-less class. This is the essence of MVVM, and really nothing more than application of Memento pattern.
Memento (283)
Without violating encapsulation, capture and externalize an object's internal state so that the object can be restored to this state later.
Design Patterns, Gamma et al, 1995
Original question
The data of my view models is passed back and forth between client JS and server Web APIs as JSON.
It is well understood that JSON.stringify(object) serialises only members that have a non-null value that is not a Function. Thus, JSON.parse(JSON.stringify(someObject)) will remove all the methods from the object.
My current implementation has each graph node implemented as a Typescript class with Serialise and Deserialise methods. JQuery.ajax calls a Web API and implicitly parses the resultant JSON into a DAG of object definitions, each of which has a Type property indicating which type of class it was prior to serialisation. I have a map of constructors indexed by name and the appropriate constructor is retrieved and the data passed as the constructor parameter.
Depending on type there may be children; if so things proceed recursively down the graph.
I have been wondering whether, rather than copy all the property values, I couldn't just assign an appropriate prototype. Bring the mountain to Mahomed, you might say. This would eliminate quite a bit of clutter in my codebase.
As I write it occurs to me that I could use $.extend, but I'm progressively weeding jQuery out of my codebase so this would be a retrograde step.
Is there any known peril in my proposition of diddling the prototype?
Does anyone have a better idea? Other than $.extend, I mean. Something TypeScripty, by preference.
It has been observed in comments that assigning the prototype means the constructor is never called. This is irrelevant. The object state is already set up, all that is required is to make the methods available.
I recently built object with methods which content could be serialized and then reconstructed.
I simply added an argument which could take a JSON object and assign it to itself.
Example using plain object:
function myObject() {
this.valueA = 1;
this.valueB = 2;
this.valueC = 3;
this.add = function() {
return this.valueA + this.valueB + this.valueC;
};
}
var o = new myObject();
console.log(o.add());
console.log(JSON.stringify(o));
If you serialized this you would get:
{"valueA":1,"valueB":2,"valueC":3}
Now, to reconstruct this you can add a Object.assign() to the object like this taking the argument and merge it with self:
function myObject(json) {
this.valueA = 0;
this.valueB = 0;
this.valueC = 0;
this.add = function() {
return this.value1 + this.value2 + this.value3;
};
Object.assign(this, json); // will merge argument with itself
}
If we now pass the parsed JSON object as argument it will merge itself with the object recreating what you had:
var json = JSON.parse('{"valueA":1,"valueB":2,"valueC":3}')
function myObject(json) {
this.valueA = 0;
this.valueB = 0;
this.valueC = 0;
this.add = function() {
return this.valueA + this.valueB + this.valueC;
};
Object.assign(this, json); // will merge argument with itself
}
var o = new myObject(json); // reconstruct using original data
console.log(o.add());
If you now have children via array you simply repeat the process recursively down the chain.
(A bonus is that you can also pass options this way).
Let's say you want to compare two javascript objects or arrays for reference equality.
The most obvious solution would be
if (a === b) { /*...*/ };
The problem with this is that a and b would have to hold the references to actual objects. This can be a problem in same cases. For example, trying to hold actual references in a memoization function would create a memory leak.
One approach is to use JSON.stringify(a) to obtain string representations of objects and compare that. But that can be prohibitively expensive in some use cases. Not to mention that actual reference equality isn't taken into consideration.
That got me wondering. Is there a way to stringify an actual reference instead of the object contents? Obviously, you can't manipulate pointers in javascript, but what if you could get just a pointer representation of some kind. A hash sum? Raw memory location? Guid or integer representation?
I know there are some kinds of object id-s when analyzing memory dump in chrome dev tools. Maybe these things can be accessed during runtime? Using some kind of special switch in node.js?
Can anyone think of a way to get this done in javascript?
If you want to use this for memorization (caching), you should use the ES6 WeakMap. It works from Firefox 6, Chrome 36 and IE 11 on.
The WeakMap object is a collection of key/value pairs in which the keys are objects and the values can be arbitrary values.
[...]
The experienced JavaScript programmer will notice that this API could be implemented in JavaScript with two arrays (one for keys, one for values) shared by the four API methods. Such an implementation would have two main inconveniences. The first one is an O(n) search (n being the number of keys in the map). The second one is a memory leak issue.
It works like a map in Java - you can use arbitrary objects as keys and as values. But the keys won't prevent garbage collection, so you don't have to worry about memory leaks, but still can find the memorized calculation for a given object.
Inspired by #user2864740's remark (dude, set up a name, seriously :-)).
it could also be simply a required method on the objects that will participate in this system
This implementation extends Object prototype to include a utility to generate unique UUID for each object. generateId implementation is taken from here.
Seems to be working for objects and arrays.
Object.prototype._id = function () {
return this.__id || (this.__id = generateId());
function generateId() {
return "xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx".replace(/[xy]/g, function(c) {
var r = Math.random()*16|0, v = c == 'x' ? r : (r&0x3|0x8);
return v.toString(16);
});
}
}
function assert(message, test) {
if (!test) {
throw new Error("ASSERTION FAILED: " + message);
} else {
console.log("PASSED: " + message);
}
}
var x = { name: "same" },
y = { name: "same" },
z = [],
x2 = x;
assert("Id is unchanging", x._id() === x._id());
assert("Id is different for different objects", x._id() !== y._id());
assert("Arrays have id too", z._id() === z._id());
assert("Id is same for same objects", x._id() === x2._id());
Live example
I'm leaving this here to see if there are problems or if maybe someone has a better idea.
In my application, I have a very large array of objects on the front-end, and these objects all have some kind of long ID under the heading ["object_id"]. I'm using UnderscoreJS for all my manipulations of this list. This is a prototype for an application that will eventually be handling most of this effort on the backend.
Merging the list is a big part of the application's requirement. See, the list that I work with initially will have many distinct objects with identical object_ids. Initially I was merging them all in one go with a groupBy and a map-reduce, but now the requirements have changed and I'm merging them one at a time (about a second apart, to simulate a stream of input) into a initially empty array.
My naive implementation was something like this:
function(newObject, objects) {
var obj_id = newObject["object_id"]; //id of object to merge, let's say
var tempObject = null;
var objectToMerge = _.find(objects,
function(obj) {
return obj_id == obj["object_id"];
});
if (objectToMerge) {
tempObject = merge(objectToMerge, newObject);
objects = _.reject(objects, /*same function as findWhere*/ );
} else {
tempObject = newObject;
}
objects.push(tempObject);
return objects;
}
This is ridiculously more efficient than before, when I was remerging from the mock data "source" array every time a new object was supposed to be pushed, so it's down from what I think was O(N^2) at least to O(N), but N here is so large (for JavaScript, anyway!) I'd like to optimize it. Currently worst case, where the object_id is not redundant, is the entire list is traversed twice. So what I'd like is to do a find-and-replace, an operation which would return a new version of the list, but with the merged object in place of the old one.
I could do a map where the iterator returns a new, merged object iff the object_id is the same, but that doesn't have the short-circuit evaluation that _.find has, which means the difference between having a worst-case runtime and having that be the default runtime, and doesn't easily account for pushing the object if there wasn't a match.
I'd also like to avoid mutating the original array in place. I know objects.push(tempObject) does that very thing, but for data-binding reasons I'm ignoring that and returning the mutated list as though it were new.
It's also unavoidable that I'll have to check the array to see if the new object was merged or whether it was appended. Using closures I could keep track of a flag to see if the merge happened, but I'm trying to be as idiomatically LISPy as possible for my own sanity. Also, past a certain point, most objects will be merged, so extra runtime overheard for adding new items isn't a huge problem, as long as it is only incurred when it has to happen.
I'm experiencing an odd behavior (maybe it isn't odd at all but just me not understanding why) with an javascript array containing some objects.
Since I'm no javascript pro, there might very well be clear explanation as to why this is happening, I just don't know it.
I have javascript that is running in a document. It makes an array of objects similar to this:
var myArray = [{"Id":"guid1","Name":"name1"},{"Id":"guid2","Name":"name2"},...];
If I print out this array at the place it was created like JSON.stringify(myArray), I get what I was expecting:
[{"Id":"guid1","Name":"name1"},{"Id":"guid2","Name":"name2"},...]
However, if I try to access this array from a child document to this document (a document in a window opened by the first document) the array isn't an array any more.
So doing JSON.stringify(parent.opener.myArray) in the child document will result in the following:
{"0":{"Id":"guid1","Name":"name1"},"1":{"Id":"guid2","Name":"name2"},...}
And this was not what I was expecting - I was expecting to get the same as I did in teh parent document.
Can anyone explain to me why this is happening and how to fix it so that the array is still an array when addressed from a child window/document?
PS. the objects aren't added to the array as stated above, they are added like this:
function objTemp()
{
this.Id = '';
this.Name = '';
};
var myArray = [];
var obj = new ObjTemp();
obj.Id = 'guid1';
obj.Name = 'name1';
myArray[myArray.length] = obj;
If that makes any difference.
Any help would be much appreciated, both for fixing my problem but also for better understanding what is going on :)
The very last line might be causing the problem, have you tried replacing myArray[myArray.length] = obj; with myArray.push(obj);? Could be that, since you're creating a new index explicitly, the Array is turned into an object... though I'm just guessing here. Could you add the code used by the child document that retrieves myArray ?
Edit
Ignore the above, since it won't make any difference. Though, without wanting to boast, I was thinking along the right lines. My idea was that, by only using proprietary array methods, the interpreter would see that as clues as to the type of myArray. The thing is: myArray is an array, as far as the parent document is concerned, but since you're passing the Array from one document to another, here's what happens:
An array is an object, complete with it's own prototype and methods. By passing it to another document, you're passing the entire Array object (value and prototype) as one object to the child document. In passing the variable between documents, you're effectively creating a copy of the variable (the only time JavaScript copies the values of a var). Since an array is an object, all of its properties (and prototype methods/properties) are copied to a 'nameless' instance of the Object object. Something along the lines of var copy = new Object(toCopy.constructor(toCopy.valueOf())); is happening... the easiest way around this, IMO, is to stringency the array withing the parent context, because there, the interpreter knows it's an array:
//parent document
function getTheArray(){ return JSON.stringify(myArray);}
//child document:
myArray = JSON.parse(parent.getTheArray());
In this example, the var is stringified in the context that still treats myArray as a true JavaScript array, so the resulting string will be what you'd expect. In passing the JSON encoded string from one document to another, it will remain unchanged and therefore the JSON.parse() will give you an exact copy of the myArray variable.
Note that this is just another wild stab in the dark, but I have given it a bit more thought, now. If I'm wrong about this, feel free to correct me... I'm always happy to learn. If this turns out to be true, let me know, too, as this will undoubtedly prove a pitfall for me sooner or later
Check out the end of this article http://www.karmagination.com/blog/2009/07/29/javascript-kung-fu-object-array-and-literals/ for an example of this behavior and explanation.
Basically it comes down to Array being a native type and each frame having its own set of natives and variables.
From the article:
// in parent window
var a = [];
var b = {};
//inside the iframe
console.log(parent.window.a); // returns array
console.log(parent.window.b); // returns object
alert(parent.window.a instanceof Array); // false
alert(parent.window.b instanceof Object); // false
alert(parent.window.a.constructor === Array); // false
alert(parent.window.b.constructor === Object); // false
Your call to JSON.stringify actually executes the following check (from the json.js source), which seems to be failing to specify it as an Array:
// Is the value an array?
if (Object.prototype.toString.apply(value) === '[object Array]') {
//stringify
I've been looking into HTML 5's new local storage and it seems pretty cool so far. I've decided to use JSON to serialize my objects into strings and to parse them back into objects, which all sounds very nice. However, it's easier said than done. I've discovered that you can't just JSON.stringify() an object and expect it to pack nicely for you, but I can't figure out what I have to do instead.
That's not all, though: my object contains two arrays, each of which holds another type of object and one of which is multidimensional. Here's what my rather complex and inter-dependent object architecture looks like:
function Vector2(x, y) {
this.x = x;
this.y = y;
}
function Bar(ID, position) {
this.id = id;
this.position = position;
}
function Goo(state, position) {
this.on = state;
this.position = position;
}
function Foo(name, size) {
this.name = name;
this.size = size;
this.bars = new Array(width)
this.goos = new Array(10);
this.Initialize();
}
Foo.prototype.Initialize() {
for(var x = 0;x<this.size.x;x++) {
this.bars[x] = new Array(this.size.y);
for(var y=0;y<this.size.y;y++) {
this.bars[x][y] = new Bar(x + y, new Vector2(x, y));
}
}
for(var i = 0;i<this.goos.length;i++) {
this.goos[i] = new Goo(on, new Vector2(i, i/2 + 1));
}
}
Each of those objects has plenty of additional functions as well, each added using the same prototype method that I used to add the method to Foo. Like I said, complex. My question is, how do I serialize all this? Do I really need to tack on toJSON() functions to every object?
Finally, once I've packed all this and saved it to localstorage, I know how to retrieve it, but I'm pretty much clueless on how to unpack it with JSON. That's another matter for another time, though, and I suspect it might be a bit easier to figure out on my own once I learn how to pack everything up.
Note: I wouldn't normally such a potentially broad question, but I couldn't really find anything here on SO or with my (admittedly weak) Google-fu that really addresses the issue, and I don't know how to break this question down any further.
Usually, you don't just serialize complex data structures in javascript because the normal serialization doesn't handle multiple difference things all have references to the same object, can't handle circular references, etc...
What I would recommend instead is that you figure out what the real state of your application is. Not the whole instantiated object structure, but what is the minimum amount of information that is actually needed to reconstruct the state of your data. Then, once you've figure that out (it should only be data, no actual objects), then you can create functions or methods to get that data from your data structures or create a new data structure from the data.
In looking at your code, the actual state of a Foo object is a two dimensional array of Bar objects and a one dimensional array of Goo objects and a name and a size. A Bar just has an x, y and id. A Goo just has a state and an x and a y. That would be pretty easy state to write a Foo method to generate and a Foo method to accept that state from saved storage.
Tacking on toJSON and fromJSON functions is probably the right way to do it. You should only be saving the actual unique data for any given object as JSON, it would not be a good idea to serialize the entire instantiated object for several big reasons, #1 being that it's just unnecessary. Also, think of the case where you add or modify functions for one of your objects in the near future: clients who had stored their own serialized objects, will never get your updates. Simply storing the unique instance data, and having functions to convert that data back to a real object (using your most recent definition) is the way to go.
You can pack functions to strings:
var a = function() {return "a"};
a.toString()
// And also:
function b() {return "b"};
b.toString();
So from there on you could hack the JSON source (by Douglas Crockford) and include support for functions. Or something.
I hope I'm not too late here but I just encountered the same problem.
I wanted to persist a promise object (returned from a JQuery AJAX call) to local storage.
Luckily I stumbled upon a little Javascript library called Stash (http://rezitech.github.io/stash/). It makes both the serialization of complex Javascript objects to Strings and the parsing of a String to a Javascript object possible.
The best thing, you don't have to explicitly perform the serialization/parsing because the data types are automatically recognized and transformed when you want to persist/retrieve an object to/from local storage.
Here is a quick test which yielded a positive result
// the api endpoint
var url = "www.api-host.com/api/bla/blub";
// create json call returning promise object
var promise = $.getJSON(url);
// persist promise object in local storage using dash.js
var key = url;
stash.set(key, promise);
// retrieve promise object from local storage
var stashGet = stash.get(key);
// display in console
console.log(stashGet);
Regards, Matthias