I have an object (parse tree) that contains child nodes which are references to other nodes.
I'd like to serialize this object, using JSON.stringify(), but I get
TypeError: cyclic object value
because of the constructs I mentioned.
How could I work around this? It does not matter to me whether these references to other nodes are represented or not in the serialized object.
On the other hand, removing these properties from the object when they are being created seems tedious and I wouldn't want to make changes to the parser (narcissus).
Use the second parameter of stringify, the replacer function, to exclude already serialized objects:
var seen = [];
JSON.stringify(obj, function(key, val) {
if (val != null && typeof val == "object") {
if (seen.indexOf(val) >= 0) {
return;
}
seen.push(val);
}
return val;
});
http://jsfiddle.net/mH6cJ/38/
As correctly pointed out in other comments, this code removes every "seen" object, not only "recursive" ones.
For example, for:
a = {x:1};
obj = [a, a];
the result will be incorrect. If your structure is like this, you might want to use Crockford's decycle or this (simpler) function which just replaces recursive references with nulls:
function decycle(obj, stack = []) {
if (!obj || typeof obj !== 'object')
return obj;
if (stack.includes(obj))
return null;
let s = stack.concat([obj]);
return Array.isArray(obj)
? obj.map(x => decycle(x, s))
: Object.fromEntries(
Object.entries(obj)
.map(([k, v]) => [k, decycle(v, s)]));
}
//
let a = {b: [1, 2, 3]}
a.b.push(a);
console.log(JSON.stringify(decycle(a)))
This is kind of an alternate-answer, but since what a lot of people will come here for is debugging their circular objects and there's not really a great way to do that without pulling in a bunch of code, here goes.
One feature that's not as well-known as JSON.stringify() is console.table(). Simply call console.table(whatever);, and it will log the variable in the console in tabular format, making it rather quite easy and convenient to peruse the variable's contents.
Here is an example of a data structure with cyclic references:
function makeToolshed(){
var nut = {name: 'nut'}, bolt = {name: 'bolt'};
nut.needs = bolt; bolt.needs = nut;
return { nut: nut, bolt: bolt };
}
When you wish to KEEP the cyclic references (restore them when you deserialize, instead of "nuking" them), you have 2 choices, which I'll compare here. First is Douglas Crockford's cycle.js, second is my siberia package. Both work by first "decycling" the object, i.e., constructing another object (without any cyclic references) "containing the same information."
Mr. Crockford goes first:
JSON.decycle(makeToolshed())
As you see, the nested structure of JSON is retained, but there is a new thing, which is objects with the special $ref property. Let's see how that works.
root = makeToolshed();
[root.bolt === root.nut.needs, root.nut.needs.needs === root.nut]; // retutrns [true,true]
The dollar sign stands for the root. .bolt having $ref tells us that .bolt is an "already seen" object, and the value of that special property (here, the string $["nut"]["needs"]) tells us where, see first === above. Likewise for second $ref and the second === above.
Let's use a suitable deep equality test (namely Anders Kaseorg's deepGraphEqual function from accepted answer to this question) to see if cloning works.
root = makeToolshed();
clone = JSON.retrocycle(JSON.decycle(root));
deepGraphEqual(root, clone) // true
serialized = JSON.stringify(JSON.decycle(root));
clone2 = JSON.retrocycle(JSON.parse(serialized));
deepGraphEqual(root, clone2); // true
Now, siberia:
JSON.Siberia.forestify(makeToolshed())
Siberia does not try to mimic "classic" JSON, no nested structure. The object graph is described in a "flat" manner.
Each node of the object graph is turned into a flat tree (plain key value pair list with integer-only values), which is an entry in .forest. At index zero, we find the root object, at higher indices, we find the other nodes of the object graph, and negative values (of some key of some tree of the forest) point to the atoms array, (which is typed via the types array, but we'll skip the typing details here). All terminal nodes are in the atoms table, all non-terminal nodes are in the forest table, and you can see right away how many nodes the object graph has, namely forest.length. Let's test if it works:
root = makeToolshed();
clone = JSON.Siberia.unforestify(JSON.Siberia.forestify(root));
deepGraphEqual(root, clone); // true
serialized = JSON.Siberia.stringify(JSON.Siberia.forestify(root));
clone2 = JSON.Siberia.unforestify(JSON.Siberia.unstringify(serialized));
deepGraphEqual(root, clone2); // true
comparison
will add section later.
note
I'm currently refactoring the package. Central ideas and algorithms are staying the same, but the new version will be easier to use, the top level API will be different. I will very soon archive siberia and present the refactored version, which I'll call objectgraph. Stay tuned, it will happen this month (August 2020)
ah, and ultra short version for the comparison. For a "pointer", I need as much space as an integer takes, since my "pointers to already seen nodes" (as a matter of fact, to all nodes, already seen or not) are just integers. In Mr. Crockford's version, amount needed to store a "pointer" is bounded only by the size of the object graph. That makes the worst case complexity of Mr. Crockford's version extremely horrible. Mr. Crockford gave us "another Bubblesort". I'm not kidding you. It's that bad. If you don't believe it, there are tests, you can find them starting from the readme of the package (will transform them to be benchmark.js compliant also this month, Aug 2020)
much saver and it shows where an cycle object was.
<script>
var jsonify=function(o){
var seen=[];
var jso=JSON.stringify(o, function(k,v){
if (typeof v =='object') {
if ( !seen.indexOf(v) ) { return '__cycle__'; }
seen.push(v);
} return v;
});
return jso;
};
var obj={
g:{
d:[2,5],
j:2
},
e:10
};
obj.someloopshere = [
obj.g,
obj,
{ a: [ obj.e, obj ] }
];
console.log('jsonify=',jsonify(obj));
</script>
produces
jsonify = {"g":{"d":[2,5],"j":2},"e":10,"someloopshere":[{"d":[2,5],"j":2},"__cycle__",{"a":[10,"__cycle__"]}]}
I've created an GitHub Gist which is able to detect cyclic structures and also de- and encodes them: https://gist.github.com/Hoff97/9842228
To transform just use JSONE.stringify/JSONE.parse.
It also de- and encodes functions. If you want to disable this just remove lines 32-48 and 61-85.
var strg = JSONE.stringify(cyclicObject);
var cycObject = JSONE.parse(strg);
You can find an example fiddle here:
http://jsfiddle.net/hoff97/7UYd4/
I create too a github project that can serialize cyclic object and restore the class if you save it in the serializename attribute like a String
var d={}
var a = {b:25,c:6,enfant:d};
d.papa=a;
var b = serializeObjet(a);
assert.equal( b, "{0:{b:25,c:6,enfant:'tab[1]'},1:{papa:'tab[0]'}}" );
var retCaseDep = parseChaine(b)
assert.equal( retCaseDep.b, 25 );
assert.equal( retCaseDep.enfant.papa, retCaseDep );
https://github.com/bormat/serializeStringifyParseCyclicObject
Edit:
I have transform my script for NPM https://github.com/bormat/borto_circular_serialize and I have change function names from french to english.
the nodejs module serialijse provides a nice way to deal with any type of JSON objects containing cycles or javascript class instances.
const { serialize, deserialize } = require("serialijse");
var Mary = { name: "Mary", friends: [] };
var Bob = { name: "Bob", friends: [] };
Mary.friends.push(Bob);
Bob.friends.push(Mary);
var group = [ Mary, Bob];
console.log(group);
// testing serialization using JSON.stringify/JSON.parse
try {
var jstr = JSON.stringify(group);
var jo = JSON.parse(jstr);
console.log(jo);
} catch (err) {
console.log(" JSON has failed to manage object with cyclic deps");
console.log(" and has generated the following error message", err.message);
}
// now testing serialization using serialijse serialize/deserialize
var str = serialize(group);
var so = deserialize(str);
console.log(" However Serialijse knows to manage object with cyclic deps !");
console.log(so);
assert(so[0].friends[0] == so[1]); // Mary's friend is Bob
this serializer supports
cycle in the object definition
reconstruction of class's instance
support for Typed Array, Map, and Set
ability to filter properties to skip during the serialization process.
binary encoding of Typed Array (Float32Array etc ... ) for performance.
function stringifyObject ( obj ) {
if ( _.isArray( obj ) || !_.isObject( obj ) ) {
return obj.toString()
}
var seen = [];
return JSON.stringify(
obj,
function( key, val ) {
if (val != null && typeof val == "object") {
if ( seen.indexOf( val ) >= 0 )
return
seen.push( val )
}
return val
}
);
}
A precondition was missing, otherwise the integer values in array objects are truncated, i.e. [[ 08.11.2014 12:30:13, 1095 ]] 1095 gets reduced to 095.
Related
I have an object with lots of own and inherited methods. I need to make a simple snapshot of it with only primitive properties.
What's the most elegant way how to do it?
Javascript snippet
function copy(source) {
var target;
if (source) {
target = {};
for (var key in source) {
var prop = source[key],
type = typeof prop;
if (type === "string" || type === "number" || type === "boolean") {
target[key] = prop;
}
if (type == "object") {
target[key] = copy(prop);
}
}
}
return target;
}
With Underscore.js
function copy(source) {
var target;
if (source) {
target = {};
_.filter( _.keys(source), function( key ) {
var prop = source[key];
if (!_.isFunction(prop) && !_.isObject(prop)) {
target[key] = prop;
}
if (_.isObject(prop)) {
target[key] = copy(prop);
}
});
}
return target;
}
The problem here is that you are dealing with a mix of types that may or may not be what you want thanks to the loose typing that JavaScript offers. Specifically without walking any given array it is impossible to tell whether it only contains primitives and you might therefore want to keep it or whether it contains objects or functions that you want to filter out.
I realise library recommendations are the oldest hat, but I'm also going to strongly recommend that you use Underscore.js ( or a derivative such as Lodash ) which gives a whole lot of super-convenient functions that I use the whole time. In a very real sense these feel like the missing part of the core JavaScript language - in fact in some cases they are wrapping native functionality, but allowing you to avoid platform inconsistencies.
If you have Underscore included in your library then you can find the non-object property names of any given object like this:
var myObject = {
a: "hello",
b: "goodbye",
c: function() { console.log(this.b);}
d: { hello: "banana" }
};
_.filter( _.keys(myObject), function( key ) {
return ! _.isObject(myObject[key]);
}) // => ["a", "b"]
This could then be tidied into a function to clone the object, but of course it's unnecessary because Underscore offers us _.pick, which returns a copy of the object with only the whitelisted properties:
_.pick( myObject, _.filter( _.keys(myObject), function( key ) { return ! _.isObject(myObject[key]);}))
By flipping the !_.isObject you can retrieve all the properties which are objects, and perform a similar mapping on those if necessary. By wrapping both in a function you can call it recursively on the child objects you want to clone.
This answers your question, but be aware that this might not be the best way to perform undo/redo actions, which you mention in your comments is the use you plan to put this to. It may be easier to record actions and states onto a history stack recording only the actual changes any action performs, where they can be retrieved and reverted safely without needing to keep snapshots of your data.
New ES 6 (Harmony) introduces new Set object. Identity algorithm used by Set is similar to === operator and so not much suitable for comparing objects:
var set = new Set();
set.add({a:1});
set.add({a:1});
console.log([...set.values()]); // Array [ Object, Object ]
How to customize equality for Set objects in order to do deep object comparison? Is there anything like Java equals(Object)?
Update 3/2022
There is currently a proposal to add Records and Tuples (basically immutable Objects and Arrays) to Javascript. In that proposal, it offers direct comparison of Records and Tuples using === or !== where it compares values, not just object references AND relevant to this answer both Set and Map objects would use the value of the Record or Tuple in key comparisons/lookups which would solve what is being asked for here.
Since the Records and Tuples are immutable (can't be modified) and because they are easily compared by value (by their contents, not just their object reference), it allows Maps and Sets to use object contents as keys and the proposed spec explicitly names this feature for Sets and Maps.
This original question asked for customizability of a Set comparison in order to support deep object comparison. This doesn't propose customizability of the Set comparison, but it directly supports deep object comparison if you use the new Record or a Tuple instead of an Object or an Array and thus would solve the original problem here.
Note, this proposal advanced to Stage 2 in mid-2021. It has been moving forward recently, but is certainly not done.
Mozilla work on this new proposal can be tracked here.
Original Answer
The ES6 Set object does not have any compare methods or custom compare extensibility.
The .has(), .add() and .delete() methods work only off it being the same actual object or same value for a primitive and don't have a means to plug into or replace just that logic.
You could presumably derive your own object from a Set and replace .has(), .add() and .delete() methods with something that did a deep object comparison first to find if the item is already in the Set, but the performance would likely not be good since the underlying Set object would not be helping at all. You'd probably have to just do a brute force iteration through all existing objects to find a match using your own custom compare before calling the original .add().
Here's some info from this article and discussion of ES6 features:
5.2 Why can’t I configure how maps and sets compare keys and values?
Question: It would be nice if there were a way to configure what map
keys and what set elements are considered equal. Why isn’t there?
Answer: That feature has been postponed, as it is difficult to
implement properly and efficiently. One option is to hand callbacks to
collections that specify equality.
Another option, available in Java, is to specify equality via a method
that object implement (equals() in Java). However, this approach is
problematic for mutable objects: In general, if an object changes, its
“location” inside a collection has to change, as well. But that’s not
what happens in Java. JavaScript will probably go the safer route of
only enabling comparison by value for special immutable objects
(so-called value objects). Comparison by value means that two values
are considered equal if their contents are equal. Primitive values are
compared by value in JavaScript.
As mentioned in jfriend00's answer customization of equality relation is probably not possible.
Following code presents an outline of computationally efficient (but memory expensive) workaround:
class GeneralSet {
constructor() {
this.map = new Map();
this[Symbol.iterator] = this.values;
}
add(item) {
this.map.set(item.toIdString(), item);
}
values() {
return this.map.values();
}
delete(item) {
return this.map.delete(item.toIdString());
}
// ...
}
Each inserted element has to implement toIdString() method that returns string. Two objects are considered equal if and only if their toIdString methods returns same value.
As the top answer mentions, customizing equality is problematic for mutable objects. The good news is (and I'm surprised no one has mentioned this yet) there's a very popular library called immutable-js that provides a rich set of immutable types which provide the deep value equality semantics you're looking for.
Here's your example using immutable-js:
const { Map, Set } = require('immutable');
var set = new Set();
set = set.add(Map({a:1}));
set = set.add(Map({a:1}));
console.log([...set.values()]); // [Map {"a" => 1}]
Maybe you can try to use JSON.stringify() to do deep object comparison.
for example :
const arr = [
{name:'a', value:10},
{name:'a', value:20},
{name:'a', value:20},
{name:'b', value:30},
{name:'b', value:40},
{name:'b', value:40}
];
const names = new Set();
const result = arr.filter(item => !names.has(JSON.stringify(item)) ? names.add(JSON.stringify(item)) : false);
console.log(result);
To add to the answers here, I went ahead and implemented a Map wrapper that takes a custom hash function, a custom equality function, and stores distinct values that have equivalent (custom) hashes in buckets.
Predictably, it turned out to be slower than czerny's string concatenation method.
Full source here: https://github.com/makoConstruct/ValueMap
Comparing them directly seems not possible, but JSON.stringify works if the keys just were sorted. As I pointed out in a comment
JSON.stringify({a:1, b:2}) !== JSON.stringify({b:2, a:1});
But we can work around that with a custom stringify method. First we write the method
Custom Stringify
Object.prototype.stringifySorted = function(){
let oldObj = this;
let obj = (oldObj.length || oldObj.length === 0) ? [] : {};
for (let key of Object.keys(this).sort((a, b) => a.localeCompare(b))) {
let type = typeof (oldObj[key])
if (type === 'object') {
obj[key] = oldObj[key].stringifySorted();
} else {
obj[key] = oldObj[key];
}
}
return JSON.stringify(obj);
}
The Set
Now we use a Set. But we use a Set of Strings instead of objects
let set = new Set()
set.add({a:1, b:2}.stringifySorted());
set.has({b:2, a:1}.stringifySorted());
// returns true
Get all the values
After we created the set and added the values, we can get all values by
let iterator = set.values();
let done = false;
while (!done) {
let val = iterator.next();
if (!done) {
console.log(val.value);
}
done = val.done;
}
Here's a link with all in one file
http://tpcg.io/FnJg2i
For Typescript users the answers by others (especially czerny) can be generalized to a nice type-safe and reusable base class:
/**
* Map that stringifies the key objects in order to leverage
* the javascript native Map and preserve key uniqueness.
*/
abstract class StringifyingMap<K, V> {
private map = new Map<string, V>();
private keyMap = new Map<string, K>();
has(key: K): boolean {
let keyString = this.stringifyKey(key);
return this.map.has(keyString);
}
get(key: K): V {
let keyString = this.stringifyKey(key);
return this.map.get(keyString);
}
set(key: K, value: V): StringifyingMap<K, V> {
let keyString = this.stringifyKey(key);
this.map.set(keyString, value);
this.keyMap.set(keyString, key);
return this;
}
/**
* Puts new key/value if key is absent.
* #param key key
* #param defaultValue default value factory
*/
putIfAbsent(key: K, defaultValue: () => V): boolean {
if (!this.has(key)) {
let value = defaultValue();
this.set(key, value);
return true;
}
return false;
}
keys(): IterableIterator<K> {
return this.keyMap.values();
}
keyList(): K[] {
return [...this.keys()];
}
delete(key: K): boolean {
let keyString = this.stringifyKey(key);
let flag = this.map.delete(keyString);
this.keyMap.delete(keyString);
return flag;
}
clear(): void {
this.map.clear();
this.keyMap.clear();
}
size(): number {
return this.map.size;
}
/**
* Turns the `key` object to a primitive `string` for the underlying `Map`
* #param key key to be stringified
*/
protected abstract stringifyKey(key: K): string;
}
Example implementation is then this simple: just override the stringifyKey method. In my case I stringify some uri property.
class MyMap extends StringifyingMap<MyKey, MyValue> {
protected stringifyKey(key: MyKey): string {
return key.uri.toString();
}
}
Example usage is then as if this was a regular Map<K, V>.
const key1 = new MyKey(1);
const value1 = new MyValue(1);
const value2 = new MyValue(2);
const myMap = new MyMap();
myMap.set(key1, value1);
myMap.set(key1, value2); // native Map would put another key/value pair
myMap.size(); // returns 1, not 2
A good stringification method for the special but frequent case of a TypedArray as Set/Map key is using
const key = String.fromCharCode(...new Uint16Array(myArray.buffer));
It generates the shortest possible unique string that can be easily converted back. However this is not always a valid UTF-16 string for display concerning Low and High Surrogates. Set and Map seem to ignore surrogate validity.
As measured in Firefox and Chrome, the spread operator performs slowly. If your myArray has fixed size, it executes faster when you write:
const a = new Uint16Array(myArray.buffer); // here: myArray = Uint32Array(2) = 8 bytes
const key = String.fromCharCode(a[0],a[1],a[2],a[3]); // 8 bytes too
Probably the most valuable advantage of this method of key-building: It works for Float32Array and Float64Array without any rounding side-effect. Note that +0 and -0 are then different. Infinities are same. Silent NaNs are same. Signaling NaNs are different depending on their signal (never seen in vanilla JavaScript).
As other guys said there is no native method can do it by far.
But if you would like to distinguish an array with your custom comparator, you can try to do it with the reduce method.
function distinct(array, equal) {
// No need to convert it to a Set object since it may give you a wrong signal that the set can work with your objects.
return array.reduce((p, c) => {
p.findIndex((element) => equal(element, c)) > -1 || p.push(c);
return p;
}, []);
}
// You can call this method like below,
const users = distinct(
[
{id: 1, name: "kevin"},
{id: 2, name: "sean"},
{id: 1, name: "jerry"}
],
(a, b) => a.id === b.id
);
...
As others have said, there is no way to do it with the current version of Set.
My suggestion is to do it using a combination of arrays and maps.
The code snipped below will create a map of unique keys based on your own defined key and then transform that map of unique items into an array.
const array =
[
{ "name": "Joe", "age": 17 },
{ "name": "Bob", "age": 17 },
{ "name": "Carl", "age": 35 }
]
const key = 'age';
const arrayUniqueByKey = [...new Map(array.map(item =>
[item[key], item])).values()];
console.log(arrayUniqueByKey);
/*OUTPUT
[
{ "name": "Bob", "age": 17 },
{ "name": "Carl", "age": 35 }
]
*/
// Note: this will pick the last duplicated item in the list.
To someone who found this question on Google (as me) wanting to get a value of a Map using an object as Key:
Warning: this answer will not work with all objects
var map = new Map<string,string>();
map.set(JSON.stringify({"A":2} /*string of object as key*/), "Worked");
console.log(map.get(JSON.stringify({"A":2}))||"Not worked");
Output:
Worked
I have an object (parse tree) that contains child nodes which are references to other nodes.
I'd like to serialize this object, using JSON.stringify(), but I get
TypeError: cyclic object value
because of the constructs I mentioned.
How could I work around this? It does not matter to me whether these references to other nodes are represented or not in the serialized object.
On the other hand, removing these properties from the object when they are being created seems tedious and I wouldn't want to make changes to the parser (narcissus).
Use the second parameter of stringify, the replacer function, to exclude already serialized objects:
var seen = [];
JSON.stringify(obj, function(key, val) {
if (val != null && typeof val == "object") {
if (seen.indexOf(val) >= 0) {
return;
}
seen.push(val);
}
return val;
});
http://jsfiddle.net/mH6cJ/38/
As correctly pointed out in other comments, this code removes every "seen" object, not only "recursive" ones.
For example, for:
a = {x:1};
obj = [a, a];
the result will be incorrect. If your structure is like this, you might want to use Crockford's decycle or this (simpler) function which just replaces recursive references with nulls:
function decycle(obj, stack = []) {
if (!obj || typeof obj !== 'object')
return obj;
if (stack.includes(obj))
return null;
let s = stack.concat([obj]);
return Array.isArray(obj)
? obj.map(x => decycle(x, s))
: Object.fromEntries(
Object.entries(obj)
.map(([k, v]) => [k, decycle(v, s)]));
}
//
let a = {b: [1, 2, 3]}
a.b.push(a);
console.log(JSON.stringify(decycle(a)))
This is kind of an alternate-answer, but since what a lot of people will come here for is debugging their circular objects and there's not really a great way to do that without pulling in a bunch of code, here goes.
One feature that's not as well-known as JSON.stringify() is console.table(). Simply call console.table(whatever);, and it will log the variable in the console in tabular format, making it rather quite easy and convenient to peruse the variable's contents.
Here is an example of a data structure with cyclic references:
function makeToolshed(){
var nut = {name: 'nut'}, bolt = {name: 'bolt'};
nut.needs = bolt; bolt.needs = nut;
return { nut: nut, bolt: bolt };
}
When you wish to KEEP the cyclic references (restore them when you deserialize, instead of "nuking" them), you have 2 choices, which I'll compare here. First is Douglas Crockford's cycle.js, second is my siberia package. Both work by first "decycling" the object, i.e., constructing another object (without any cyclic references) "containing the same information."
Mr. Crockford goes first:
JSON.decycle(makeToolshed())
As you see, the nested structure of JSON is retained, but there is a new thing, which is objects with the special $ref property. Let's see how that works.
root = makeToolshed();
[root.bolt === root.nut.needs, root.nut.needs.needs === root.nut]; // retutrns [true,true]
The dollar sign stands for the root. .bolt having $ref tells us that .bolt is an "already seen" object, and the value of that special property (here, the string $["nut"]["needs"]) tells us where, see first === above. Likewise for second $ref and the second === above.
Let's use a suitable deep equality test (namely Anders Kaseorg's deepGraphEqual function from accepted answer to this question) to see if cloning works.
root = makeToolshed();
clone = JSON.retrocycle(JSON.decycle(root));
deepGraphEqual(root, clone) // true
serialized = JSON.stringify(JSON.decycle(root));
clone2 = JSON.retrocycle(JSON.parse(serialized));
deepGraphEqual(root, clone2); // true
Now, siberia:
JSON.Siberia.forestify(makeToolshed())
Siberia does not try to mimic "classic" JSON, no nested structure. The object graph is described in a "flat" manner.
Each node of the object graph is turned into a flat tree (plain key value pair list with integer-only values), which is an entry in .forest. At index zero, we find the root object, at higher indices, we find the other nodes of the object graph, and negative values (of some key of some tree of the forest) point to the atoms array, (which is typed via the types array, but we'll skip the typing details here). All terminal nodes are in the atoms table, all non-terminal nodes are in the forest table, and you can see right away how many nodes the object graph has, namely forest.length. Let's test if it works:
root = makeToolshed();
clone = JSON.Siberia.unforestify(JSON.Siberia.forestify(root));
deepGraphEqual(root, clone); // true
serialized = JSON.Siberia.stringify(JSON.Siberia.forestify(root));
clone2 = JSON.Siberia.unforestify(JSON.Siberia.unstringify(serialized));
deepGraphEqual(root, clone2); // true
comparison
will add section later.
note
I'm currently refactoring the package. Central ideas and algorithms are staying the same, but the new version will be easier to use, the top level API will be different. I will very soon archive siberia and present the refactored version, which I'll call objectgraph. Stay tuned, it will happen this month (August 2020)
ah, and ultra short version for the comparison. For a "pointer", I need as much space as an integer takes, since my "pointers to already seen nodes" (as a matter of fact, to all nodes, already seen or not) are just integers. In Mr. Crockford's version, amount needed to store a "pointer" is bounded only by the size of the object graph. That makes the worst case complexity of Mr. Crockford's version extremely horrible. Mr. Crockford gave us "another Bubblesort". I'm not kidding you. It's that bad. If you don't believe it, there are tests, you can find them starting from the readme of the package (will transform them to be benchmark.js compliant also this month, Aug 2020)
much saver and it shows where an cycle object was.
<script>
var jsonify=function(o){
var seen=[];
var jso=JSON.stringify(o, function(k,v){
if (typeof v =='object') {
if ( !seen.indexOf(v) ) { return '__cycle__'; }
seen.push(v);
} return v;
});
return jso;
};
var obj={
g:{
d:[2,5],
j:2
},
e:10
};
obj.someloopshere = [
obj.g,
obj,
{ a: [ obj.e, obj ] }
];
console.log('jsonify=',jsonify(obj));
</script>
produces
jsonify = {"g":{"d":[2,5],"j":2},"e":10,"someloopshere":[{"d":[2,5],"j":2},"__cycle__",{"a":[10,"__cycle__"]}]}
I've created an GitHub Gist which is able to detect cyclic structures and also de- and encodes them: https://gist.github.com/Hoff97/9842228
To transform just use JSONE.stringify/JSONE.parse.
It also de- and encodes functions. If you want to disable this just remove lines 32-48 and 61-85.
var strg = JSONE.stringify(cyclicObject);
var cycObject = JSONE.parse(strg);
You can find an example fiddle here:
http://jsfiddle.net/hoff97/7UYd4/
I create too a github project that can serialize cyclic object and restore the class if you save it in the serializename attribute like a String
var d={}
var a = {b:25,c:6,enfant:d};
d.papa=a;
var b = serializeObjet(a);
assert.equal( b, "{0:{b:25,c:6,enfant:'tab[1]'},1:{papa:'tab[0]'}}" );
var retCaseDep = parseChaine(b)
assert.equal( retCaseDep.b, 25 );
assert.equal( retCaseDep.enfant.papa, retCaseDep );
https://github.com/bormat/serializeStringifyParseCyclicObject
Edit:
I have transform my script for NPM https://github.com/bormat/borto_circular_serialize and I have change function names from french to english.
the nodejs module serialijse provides a nice way to deal with any type of JSON objects containing cycles or javascript class instances.
const { serialize, deserialize } = require("serialijse");
var Mary = { name: "Mary", friends: [] };
var Bob = { name: "Bob", friends: [] };
Mary.friends.push(Bob);
Bob.friends.push(Mary);
var group = [ Mary, Bob];
console.log(group);
// testing serialization using JSON.stringify/JSON.parse
try {
var jstr = JSON.stringify(group);
var jo = JSON.parse(jstr);
console.log(jo);
} catch (err) {
console.log(" JSON has failed to manage object with cyclic deps");
console.log(" and has generated the following error message", err.message);
}
// now testing serialization using serialijse serialize/deserialize
var str = serialize(group);
var so = deserialize(str);
console.log(" However Serialijse knows to manage object with cyclic deps !");
console.log(so);
assert(so[0].friends[0] == so[1]); // Mary's friend is Bob
this serializer supports
cycle in the object definition
reconstruction of class's instance
support for Typed Array, Map, and Set
ability to filter properties to skip during the serialization process.
binary encoding of Typed Array (Float32Array etc ... ) for performance.
function stringifyObject ( obj ) {
if ( _.isArray( obj ) || !_.isObject( obj ) ) {
return obj.toString()
}
var seen = [];
return JSON.stringify(
obj,
function( key, val ) {
if (val != null && typeof val == "object") {
if ( seen.indexOf( val ) >= 0 )
return
seen.push( val )
}
return val
}
);
}
A precondition was missing, otherwise the integer values in array objects are truncated, i.e. [[ 08.11.2014 12:30:13, 1095 ]] 1095 gets reduced to 095.
I'm trying to write a system where the client (browser-based) requests a tree to be returned from the server (app engine, but that's irrelevant). The problem is in converting the tree to JSON: because each object refers to the objects 'below' it, when stringified I end up with an extremely long string which, on parsing, creates new objects for each child instead of references to other nodes.
My current solution would be to write an 'equalTo' and 'toString' function (converting object references to strings) stringify the resulting array and then recreate it on the client side by resolving strings to objects. That solution is making my terrible-algorithm-senses tingle, though, there must be a better way to return such structures through JSON!
EDIT: it just occurred to me that object references could also be converted to array indexes. It's a better solution, but still has that niggling bad-code feel to it.
EDIT2: Right, so I suppose some pseudo-code is in order, then.
var node = {
children : null;
};
var root = Object.create(node);
var level1a = Object.create(node);
var level1b = Object.create(node);
var level2a = Object.create(node);
var level2b = Object.create(node);
root.children = [level1a, level1b];
level1a.children = [level2a, level2b];
So you end up with a tree that looks like this:
stackoverflow won't let me post images
If you have a way to address nodes, you can use JSON revivers and replacers to convert between addresses and references.
For example, if you have two functions like
function addressForNode(node) { ... }
function nodeForAddress(address) { ... }
you could use revivers and replacers that call those when parsing or stringifying
var jsonString = JSON.stringify(tree, function (key, value) {
if (typeof value === 'object') {
var address = addressForNode(value);
if (address !== null) { // value really is a node
return { address: address }
}
}
return value;
});
// And to parse...
var tree = JSON.parse(jsonString, function (key, value) {
if (typeof value === 'object' && typeof value.address === 'string') {
return nodeForAddress(value.address);
}
return value;
});
If you can help it, don't restrict yourself to JSON. Instead consider generating JavaScript code included via linked script, using JSONP techniques if necessary.
Basically, I'm trying to create an object of unique objects, a set. I had the brilliant idea of just using a JavaScript object with objects for the property names. Such as,
set[obj] = true;
This works, up to a point. It works great with string and numbers, but with other objects, they all seem to "hash" to the same value and access the same property. Is there some kind of way I can generate a unique hash value for an object? How do strings and numbers do it, can I override the same behavior?
If you want a hashCode() function like Java's in JavaScript, that is yours:
function hashCode(string){
var hash = 0;
for (var i = 0; i < string.length; i++) {
var code = string.charCodeAt(i);
hash = ((hash<<5)-hash)+code;
hash = hash & hash; // Convert to 32bit integer
}
return hash;
}
That is the way of implementation in Java (bitwise operator).
Please note that hashCode could be positive and negative, and that's normal, see HashCode giving negative values. So, you could consider to use Math.abs() along with this function.
JavaScript objects can only use strings as keys (anything else is converted to a string).
You could, alternatively, maintain an array which indexes the objects in question, and use its index string as a reference to the object. Something like this:
var ObjectReference = [];
ObjectReference.push(obj);
set['ObjectReference.' + ObjectReference.indexOf(obj)] = true;
Obviously it's a little verbose, but you could write a couple of methods that handle it and get and set all willy nilly.
Edit:
Your guess is fact -- this is defined behaviour in JavaScript -- specifically a toString conversion occurs meaning that you can can define your own toString function on the object that will be used as the property name. - olliej
This brings up another interesting point; you can define a toString method on the objects you want to hash, and that can form their hash identifier.
The easiest way to do this is to give each of your objects its own unique toString method:
(function() {
var id = 0;
/*global MyObject */
MyObject = function() {
this.objectId = '<#MyObject:' + (id++) + '>';
this.toString= function() {
return this.objectId;
};
};
})();
I had the same problem and this solved it perfectly for me with minimal fuss, and was a lot easier that re-implementing some fatty Java style Hashtable and adding equals() and hashCode() to your object classes. Just make sure that you don't also stick a string '<#MyObject:12> into your hash or it will wipe out the entry for your exiting object with that id.
Now all my hashes are totally chill. I also just posted a blog entry a few days ago about this exact topic.
What you described is covered by Harmony WeakMaps, part of the ECMAScript 6 specification (next version of JavaScript). That is: a set where the keys can be anything (including undefined) and is non-enumerable.
This means it's impossible to get a reference to a value unless you have a direct reference to the key (any object!) that links to it. It's important for a bunch of engine implementation reasons relating to efficiency and garbage collection, but it's also super cool for in that it allows for new semantics like revokable access permissions and passing data without exposing the data sender.
From MDN:
var wm1 = new WeakMap(),
wm2 = new WeakMap();
var o1 = {},
o2 = function(){},
o3 = window;
wm1.set(o1, 37);
wm1.set(o2, "azerty");
wm2.set(o1, o2); // A value can be anything, including an object or a function.
wm2.set(o3, undefined);
wm2.set(wm1, wm2); // Keys and values can be any objects. Even WeakMaps!
wm1.get(o2); // "azerty"
wm2.get(o2); // Undefined, because there is no value for o2 on wm2.
wm2.get(o3); // Undefined, because that is the set value.
wm1.has(o2); // True
wm2.has(o2); // False
wm2.has(o3); // True (even if the value itself is 'undefined').
wm1.has(o1); // True
wm1.delete(o1);
wm1.has(o1); // False
WeakMaps are available in current Firefox, Chrome and Edge. They're also supported in Node v7 , and in v6 with the --harmony-weak-maps flag.
The solution I chose is similar to Daniel's, but rather than use an object factory and override the toString, I explicitly add the hash to the object when it is first requested through a getHashCode function. A little messy, but better for my needs :)
Function.prototype.getHashCode = (function(id) {
return function() {
if (!this.hashCode) {
this.hashCode = '<hash|#' + (id++) + '>';
}
return this.hashCode;
}
}(0));
For my specific situation I only care about the equality of the object as far as keys and primitive values go. The solution that worked for me was converting the object to its JSON representation and using that as the hash. There are limitations such as order of key definition potentially being inconsistent; but like I said it worked for me because these objects were all being generated in one place.
var hashtable = {};
var myObject = {a:0,b:1,c:2};
var hash = JSON.stringify(myObject);
// '{"a":0,"b":1,"c":2}'
hashtable[hash] = myObject;
// {
// '{"a":0,"b":1,"c":2}': myObject
// }
I put together a small JavaScript module a while ago to produce hashcodes for strings, objects, arrays, etc. (I just committed it to GitHub :) )
Usage:
Hashcode.value("stackoverflow")
// -2559914341
Hashcode.value({ 'site' : "stackoverflow" })
// -3579752159
In ECMAScript 6 there's now a Set that works how you'd like: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set
It's already available in the latest Chrome, FF, and IE11.
The JavaScript specification defines indexed property access as performing a toString conversion on the index name. For example,
myObject[myProperty] = ...;
is the same as
myObject[myProperty.toString()] = ...;
This is necessary as in JavaScript
myObject["someProperty"]
is the same as
myObject.someProperty
And yes, it makes me sad as well :-(
Based on the title, we can generate strong SHA hashes, in a browser context, it can be used to generate a unique hash from an object, an array of params, a string, or whatever.
async function H(m) {
const msgUint8 = new TextEncoder().encode(m)
const hashBuffer = await crypto.subtle.digest('SHA-256', msgUint8)
const hashArray = Array.from(new Uint8Array(hashBuffer))
const hashHex = hashArray.map(b => b.toString(16).padStart(2, '0')).join('')
console.log(hashHex)
}
/* Examples ----------------------- */
H("An obscure ....")
H(JSON.stringify( {"hello" : "world"} ))
H(JSON.stringify( [54,51,54,47] ))
The above output in my browser, it should be equal for you too:
bf1cf3fe6975fe382ab392ec1dd42009380614be03d489f23601c11413cfca2b
93a23971a914e5eacbf0a8d25154cda309c3c1c72fbb9914d47c60f3cb681588
d2f209e194045604a3b15bdfd7502898a0e848e4603c5a818bd01da69c00ad19
Supported algos:
SHA-1 (but don't use this in cryptographic applications)
SHA-256
SHA-384
SHA-512
https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/digest#Converting_a_digest_to_a_hex_string
However, for a simple FAST checksum hash function, made only for collision avoidance, see CRC32 (Content Redundancy Check)
JavaScript CRC32
You might also be interested by this similar method to generate HMAC codes via the web crypto api.
Reference: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol
you can use Es6 symbol to create unique key and access object.
Every symbol value returned from Symbol() is unique. A symbol value may be used as an identifier for object properties; this is the data type's only purpose.
var obj = {};
obj[Symbol('a')] = 'a';
obj[Symbol.for('b')] = 'b';
obj['c'] = 'c';
obj.d = 'd';
Here's my simple solution that returns a unique integer.
function hashcode(obj) {
var hc = 0;
var chars = JSON.stringify(obj).replace(/\{|\"|\}|\:|,/g, '');
var len = chars.length;
for (var i = 0; i < len; i++) {
// Bump 7 to larger prime number to increase uniqueness
hc += (chars.charCodeAt(i) * 7);
}
return hc;
}
My solution introduces a static function for the global Object object.
(function() {
var lastStorageId = 0;
this.Object.hash = function(object) {
var hash = object.__id;
if (!hash)
hash = object.__id = lastStorageId++;
return '#' + hash;
};
}());
I think this is more convenient with other object manipulating functions in JavaScript.
I will try to go a little deeper than other answers.
Even if JS had better hashing support it would not magically hash everything perfectly, in many cases you will have to define your own hash function. For example Java has good hashing support, but you still have to think and do some work.
One problem is with the term hash/hashcode ... there is cryptographic hashing and non-cryptographic hashing. The other problem, is you have to understand why hashing is useful and how it works.
When we talk about hashing in JavaScript or Java most of the time we are talking about non-cryptographic hashing, usually about hashing for hashmap/hashtable (unless we are working on authentication or passwords, which you could be doing server-side using NodeJS ...).
It depends on what data you have and what you want to achieve.
Your data has some natural "simple" uniqueness:
The hash of an integer is ... the integer, as it is unique, lucky you !
The hash of a string ... it depends on the string, if the string represents a unique identifier, you may consider it as a hash (so no hashing needed).
Anything which is indirectly pretty much a unique integer is the simplest case
This will respect: hashcode equal if objects are equal
Your data has some natural "composite" uniqueness:
For example with a person object, you may compute a hash using firstname, lastname, birthdate, ... see how Java does it: Good Hash Function for Strings, or use some other ID info that is cheap and unique enough for your usecase
You have no idea what your data will be:
Good luck ... you could serialize to string and hash it Java style, but that may be expensive if the string is large and it will not avoid collisions as well as say the hash of an integer (self).
There is no magically efficient hashing technique for unknown data, in some cases it is quite easy, in other cases you may have to think twice. So even if JavaScript/ECMAScript adds more support, there is no magic language solution for this problem.
In practice you need two things: enough uniqueness, enough speed
In addition to that it is great to have: "hashcode equal if objects are equal"
https://en.wikipedia.org/wiki/Hash_table#Collision_resolution
Relationship between hashCode and equals method in Java
I combined the answers from eyelidlessness and KimKha.
The following is an angularjs service and it supports numbers, strings, and objects.
exports.Hash = () => {
let hashFunc;
function stringHash(string, noType) {
let hashString = string;
if (!noType) {
hashString = `string${string}`;
}
var hash = 0;
for (var i = 0; i < hashString.length; i++) {
var character = hashString.charCodeAt(i);
hash = ((hash<<5)-hash)+character;
hash = hash & hash; // Convert to 32bit integer
}
return hash;
}
function objectHash(obj, exclude) {
if (exclude.indexOf(obj) > -1) {
return undefined;
}
let hash = '';
const keys = Object.keys(obj).sort();
for (let index = 0; index < keys.length; index += 1) {
const key = keys[index];
const keyHash = hashFunc(key);
const attrHash = hashFunc(obj[key], exclude);
exclude.push(obj[key]);
hash += stringHash(`object${keyHash}${attrHash}`, true);
}
return stringHash(hash, true);
}
function Hash(unkType, exclude) {
let ex = exclude;
if (ex === undefined) {
ex = [];
}
if (!isNaN(unkType) && typeof unkType !== 'string') {
return unkType;
}
switch (typeof unkType) {
case 'object':
return objectHash(unkType, ex);
default:
return stringHash(String(unkType));
}
}
hashFunc = Hash;
return Hash;
};
Example Usage:
Hash('hello world'), Hash('hello world') == Hash('hello world')
Hash({hello: 'hello world'}), Hash({hello: 'hello world'}) == Hash({hello: 'hello world'})
Hash({hello: 'hello world', goodbye: 'adios amigos'}), Hash({hello: 'hello world', goodbye: 'adios amigos'}) == Hash({goodbye: 'adios amigos', hello: 'hello world'})
Hash(['hello world']), Hash(['hello world']) == Hash(['hello world'])
Hash(1), Hash(1) == Hash(1)
Hash('1'), Hash('1') == Hash('1')
Output
432700947 true
-411117486 true
1725787021 true
-1585332251 true
1 true
-1881759168 true
Explanation
As you can see the heart of the service is the hash function created by KimKha.I have added types to the strings so that the sturucture of the object would also impact the final hash value.The keys are hashed to prevent array|object collisions.
eyelidlessness object comparision is used to prevent infinit recursion by self referencing objects.
Usage
I created this service so that I could have an error service that is accessed with objects. So that one service can register an error with a given object and another can determine if any errors were found.
ie
JsonValidation.js
ErrorSvc({id: 1, json: '{attr: "not-valid"}'}, 'Invalid Json Syntax - key not double quoted');
UserOfData.js
ErrorSvc({id: 1, json: '{attr: "not-valid"}'});
This would return:
['Invalid Json Syntax - key not double quoted']
While
ErrorSvc({id: 1, json: '{"attr": "not-valid"}'});
This would return
[]
If you truly want set behavior (I'm going by Java knowledge), then you will be hard pressed to find a solution in JavaScript. Most developers will recommend a unique key to represent each object, but this is unlike set, in that you can get two identical objects each with a unique key. The Java API does the work of checking for duplicate values by comparing hash code values, not keys, and since there is no hash code value representation of objects in JavaScript, it becomes almost impossible to do the same. Even the Prototype JS library admits this shortcoming, when it says:
"Hash can be thought of as an
associative array, binding unique keys
to values (which are not necessarily
unique)..."
http://www.prototypejs.org/api/hash
In addition to eyelidlessness's answer, here is a function that returns a reproducible, unique ID for any object:
var uniqueIdList = [];
function getConstantUniqueIdFor(element) {
// HACK, using a list results in O(n), but how do we hash e.g. a DOM node?
if (uniqueIdList.indexOf(element) < 0) {
uniqueIdList.push(element);
}
return uniqueIdList.indexOf(element);
}
As you can see it uses a list for look-up which is very inefficient, however that's the best I could find for now.
If you want to use objects as keys you need to overwrite their toString Method, as some already mentioned here. The hash functions that were used are all fine, but they only work for the same objects not for equal objects.
I've written a small library that creates hashes from objects, which you can easily use for this purpose. The objects can even have a different order, the hashes will be the same. Internally you can use different types for your hash (djb2, md5, sha1, sha256, sha512, ripemd160).
Here is a small example from the documentation:
var hash = require('es-hash');
// Save data in an object with an object as a key
Object.prototype.toString = function () {
return '[object Object #'+hash(this)+']';
}
var foo = {};
foo[{bar: 'foo'}] = 'foo';
/*
* Output:
* foo
* undefined
*/
console.log(foo[{bar: 'foo'}]);
console.log(foo[{}]);
The package can be used either in browser and in Node-Js.
Repository: https://bitbucket.org/tehrengruber/es-js-hash
If you want to have unique values in a lookup object you can do something like this:
Creating a lookup object
var lookup = {};
Setting up the hashcode function
function getHashCode(obj) {
var hashCode = '';
if (typeof obj !== 'object')
return hashCode + obj;
for (var prop in obj) // No hasOwnProperty needed
hashCode += prop + getHashCode(obj[prop]); // Add key + value to the result string
return hashCode;
}
Object
var key = getHashCode({ 1: 3, 3: 7 });
// key = '1337'
lookup[key] = true;
Array
var key = getHashCode([1, 3, 3, 7]);
// key = '01132337'
lookup[key] = true;
Other types
var key = getHashCode('StackOverflow');
// key = 'StackOverflow'
lookup[key] = true;
Final result
{ 1337: true, 01132337: true, StackOverflow: true }
Do note that getHashCode doesn't return any value when the object or array is empty
getHashCode([{},{},{}]);
// '012'
getHashCode([[],[],[]]);
// '012'
This is similar to #ijmacd solution only getHashCode doesn't has the JSON dependency.
Just use hidden secret property with the defineProperty enumerable: false
It work very fast:
The first read uniqueId: 1,257,500 ops/s
All others: 309,226,485 ops/s
var nextObjectId = 1
function getNextObjectId() {
return nextObjectId++
}
var UNIQUE_ID_PROPERTY_NAME = '458d576952bc489ab45e98ac7f296fd9'
function getObjectUniqueId(object) {
if (object == null) {
return null
}
var id = object[UNIQUE_ID_PROPERTY_NAME]
if (id != null) {
return id
}
if (Object.isFrozen(object)) {
return null
}
var uniqueId = getNextObjectId()
Object.defineProperty(object, UNIQUE_ID_PROPERTY_NAME, {
enumerable: false,
configurable: false,
writable: false,
value: uniqueId,
})
return uniqueId
}