Removing duplicate objects with Underscore for Javascript - javascript

I have this kind of array:
var foo = [ { "a" : "1" }, { "b" : "2" }, { "a" : "1" } ];
I'd like to filter it to have:
var bar = [ { "a" : "1" }, { "b" : "2" }];
I tried using _.uniq, but I guess because { "a" : "1" } is not equal to itself, it doesn't work. Is there any way to provide underscore uniq with an overriden equals function?

.uniq/.unique accepts a callback
var list = [{a:1,b:5},{a:1,c:5},{a:2},{a:3},{a:4},{a:3},{a:2}];
var uniqueList = _.uniq(list, function(item, key, a) {
return item.a;
});
// uniqueList = [Object {a=1, b=5}, Object {a=2}, Object {a=3}, Object {a=4}]
Notes:
Callback return value used for comparison
First comparison object with unique return value used as unique
underscorejs.org demonstrates no callback usage
lodash.com shows usage
Another example :
using the callback to extract car makes, colors from a list

If you're looking to remove duplicates based on an id you could do something like this:
var res = [
{id: 1, content: 'heeey'},
{id: 2, content: 'woah'},
{id: 1, content:'foo'},
{id: 1, content: 'heeey'},
];
var uniques = _.map(_.groupBy(res,function(doc){
return doc.id;
}),function(grouped){
return grouped[0];
});
//uniques
//[{id: 1, content: 'heeey'},{id: 2, content: 'woah'}]

Implementation of Shiplu's answer.
var foo = [ { "a" : "1" }, { "b" : "2" }, { "a" : "1" } ];
var x = _.uniq( _.collect( foo, function( x ){
return JSON.stringify( x );
}));
console.log( x ); // returns [ { "a" : "1" }, { "b" : "2" } ]

When I have an attribute id, this is my preffered way in underscore:
var x = [{i:2}, {i:2, x:42}, {i:4}, {i:3}];
_.chain(x).indexBy("i").values().value();
// > [{i:2, x:42}, {i:4}, {i:3}]

Using underscore unique lib following is working for me, I m making list unique on the based of _id then returning String value of _id:
var uniqueEntities = _.uniq(entities, function (item, key, a) {
return item._id.toString();
});

Here is a simple solution, which uses a deep object comparison to check for duplicates (without resorting to converting to JSON, which is inefficient and hacky)
var newArr = _.filter(oldArr, function (element, index) {
// tests if the element has a duplicate in the rest of the array
for(index += 1; index < oldArr.length; index += 1) {
if (_.isEqual(element, oldArr[index])) {
return false;
}
}
return true;
});
It filters out all elements if they have a duplicate later in the array - such that the last duplicate element is kept.
The testing for a duplicate uses _.isEqual which performs an optimised deep comparison between the two objects see the underscore isEqual documentation for more info.
edit: updated to use _.filter which is a cleaner approach

The lodash 4.6.1 docs have this as an example for object key equality:
_.uniqWith(objects, _.isEqual);
https://lodash.com/docs#uniqWith

Try iterator function
For example you can return first element
x = [['a',1],['b',2],['a',1]]
_.uniq(x,false,function(i){
return i[0] //'a','b'
})
=> [['a',1],['b',2]]

here's my solution (coffeescript) :
_.mixin
deepUniq: (coll) ->
result = []
remove_first_el_duplicates = (coll2) ->
rest = _.rest(coll2)
first = _.first(coll2)
result.push first
equalsFirst = (el) -> _.isEqual(el,first)
newColl = _.reject rest, equalsFirst
unless _.isEmpty newColl
remove_first_el_duplicates newColl
remove_first_el_duplicates(coll)
result
example:
_.deepUniq([ {a:1,b:12}, [ 2, 1, 2, 1 ], [ 1, 2, 1, 2 ],[ 2, 1, 2, 1 ], {a:1,b:12} ])
//=> [ { a: 1, b: 12 }, [ 2, 1, 2, 1 ], [ 1, 2, 1, 2 ] ]

with underscore i had to use String() in the iteratee function
function isUniq(item) {
return String(item.user);
}
var myUniqArray = _.uniq(myArray, isUniq);

I wanted to solve this simple solution in a straightforward way of writing, with a little bit of a pain of computational expenses... but isn't it a trivial solution with a minimum variable definition, is it?
function uniq(ArrayObjects){
var out = []
ArrayObjects.map(obj => {
if(_.every(out, outobj => !_.isEqual(obj, outobj))) out.push(obj)
})
return out
}

var foo = [ { "a" : "1" }, { "b" : "2" }, { "a" : "1" } ];
var bar = _.map(_.groupBy(foo, function (f) {
return JSON.stringify(f);
}), function (gr) {
return gr[0];
}
);
Lets break this down. First lets group the array items by their stringified value
var grouped = _.groupBy(foo, function (f) {
return JSON.stringify(f);
});
grouped looks like:
{
'{ "a" : "1" }' = [ { "a" : "1" } { "a" : "1" } ],
'{ "b" : "2" }' = [ { "b" : "2" } ]
}
Then lets grab the first element from each group
var bar = _.map(grouped, function(gr)
return gr[0];
});
bar looks like:
[ { "a" : "1" }, { "b" : "2" } ]
Put it all together:
var foo = [ { "a" : "1" }, { "b" : "2" }, { "a" : "1" } ];
var bar = _.map(_.groupBy(foo, function (f) {
return JSON.stringify(f);
}), function (gr) {
return gr[0];
}
);

You can do it in a shorthand as:
_.uniq(foo, 'a')

Related

How to sort double objects?

I have an array. The data in the array is in the following format.
var test = [
{
"a" : {
"order" : 100,
}
},
{
"b" : {
"order" : 10,
}
},
{
"c" : {
"order" : 1,
}
},
];
I want to sort this data according to order value. Is there any way to do this?
You can use Object.values to get the first property value and access the order property on that to compare.
let test=[{a:{order:100}},{b:{order:10}},{c:{order:1}}];
test.sort((a, b)=>Object.values(a)[0].order - Object.values(b)[0].order);
console.log(test);
For a more generalized solution, you can create a key extractor function to get the value to compare by.
let test=[{a:{order:100}},{b:{order:10}},{c:{order:1}}];
const getOrder = x => Object.values(x)[0].order;
test.sort((a, b)=>getOrder(a) - getOrder(b));
console.log(test);
You can use JS custom sort from Array.prototype.sort(), reference: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
Then you can sort by comparing the two element's order, but you still need to determine it's key/attribute (e.g.: a or b or c)
Here, you can use Object.keys() function and take the first key in the object, reference: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys
Here's a working example:
var test = [
{
"a" : {
"order" : 100,
}
},
{
"b" : {
"order" : 10,
}
},
{
"c" : {
"order" : 1,
}
},
];
//console.log(test);
test.sort((firstEl, secondEl) => {
var key1 = Object.keys(firstEl)[0];
var key2 = Object.keys(secondEl)[0];
return firstEl[key1].order - secondEl[key2].order
} );
console.log(test);
Output:
[
{
"c": {
"order": 1
}
},
{
"b": {
"order": 10
}
},
{
"a": {
"order": 100
}
}
]

Perform .join on a complex object with array of object dynamically

I have a complex js object, that contains arrays of an object. The problem is some of the main object properties' arrays can have a different property.
var foo = {};
foo.prop1 = [
{name:"test", skill:1},
{name:"test2", skill:2},
];
foo.prop2 = [
{address:"Earth",distance:1},
{address:"Mars", distance:2}
]
My aim is to just replace the main object property value with the joined values for retrieval.
This is what I have right now.
if(Object.keys(foo).length){
Object.keys(foo).forEach(key => {
var x = foo[key];
if(key === "address") {
foo[key] = x.map(function(elem){return elem.address;}).join(";");
} else {
foo[key] = x.map(function(elem){return elem.name;}).join(";");
}
});
}
How can I make it dynamic so that I don't need to use the if statement? I just want to join all the first property of the inner obj.
Result:
foo new values would be:
foo.prop1 = test;test2
foo.prop2 = Earth;Mars
I got it. I just want to join the first property of the sub object.
I replaced the if with this
foo[key] = x.map(function(elem){return elem[Object.keys(elem)[0]]; }).join(";");
I guess you are trying to choose the value with string type
var foo = {};
foo.prop1 = [{
name: "test",
skill: 1
},
{
name: "test2",
skill: 2
},
];
foo.prop2 = [{
address: "Earth",
distance: 1
},
{
address: "Mars",
distance: 2
}
]
function formulate() {
const result = {};
(Object.keys(foo) || []).forEach(function(k) {
result[k] = foo[k].map(function(val) {
str_key = Object.keys(val).filter(function(val_k) {
return typeof val[val_k] === "string";
});
return str_key.map(function(s) {
return val[s];
});
}).join(";");
});
return result;
}
result = formulate()
console.log(result);
I hope, this will work for you
var foo = {};
foo.prop1 = [
{name:"test", skill:1},
{name:"test2", skill:2},
];
foo.prop2 = [
{address:"Earth",distance:1},
{address:"Mars", distance:2}
]
Object.keys(foo).forEach(key => {
foo[key]=foo[key].map(val => { return Object.entries(val)[0][1] } ).toString().split(",").join(";")
});
console.log(foo)

Find difference between two arrays

I have following Plunkr which works perfectly.
https://plnkr.co/edit/WDjoEK7bAVpKSJbAmB9D?p=preview
It uses the _.differenceWith() function of lodash, in order two save all array values, which are not contained by the two arrays.
var result = _.differenceWith(data, test, _.isEqual);
Now I have two problems:
1.) In our project we use an older Lodash Version where the function differenceWith is not implemented
2.) I only need to compare one value of the array. This currently compares the complete objects. I only need to compare the id property.
This will find the objects in arr1 that are not in arr2 based on the id attribute.
var arr1 = [ { "id": "1" }, { "id": "2" }, { "id": "3" } ];
var arr2 = [ { "id": "1" }, { "id": "2" } ];
var result = arr1.filter(o1 => arr2.filter(o2 => o2.id === o1.id).length === 0);
console.log(result);
Note that this example does not require lodash.
If you want to use a different comparison instead of id, you can change the o2.id === o1.id part to a different property.
Here is a more generic solution:
var arr1 = [ { "name": "a" }, { "name": "b" }, { "name": "c" } ];
var arr2 = [ { "name": "a" }, { "name": "c" } ];
function differenceWith(a1, a2, prop) {
return a1.filter(o1 => a2.filter(o2 => o2[prop] === o1[prop]).length === 0);
}
var result = differenceWith(arr1, arr2, 'name');
console.log(result);

Finding an array's objects that are not present in another array by property

I'm looking for a way to find any objects in one array that are not present in another array based upon that object's property. What's the best way to do this with jQuery or underscore?
Given the following example:
"array1":[
{"testProperty":"A"},
{"testProperty":"B"},
{"testProperty":"C"}
]
"array2":[
{"testProperty":"A", "User":"Smith"},
{"testProperty":"B", "User":"Smith"},
]
I would want to return the third object from array1 whose testProperty is "C" since it's not present in array2.
I was able to find several examples of this issue here on stackoverflow, but not when needing to do so using properties from those objects.
I'm not sure if this counts, but if you can use lodash instead of underscore, there is a nice function called differenceBy:
var _ = require("lodash");
var array1 = [
{"testProperty":"A"},
{"testProperty":"B"},
{"testProperty":"C"}
]
var array2 = [
{"testProperty":"A", "User":"Smith"},
{"testProperty":"B", "User":"Smith"}
]
var result = _.differenceBy(array1, array2, function(item) {
return item["testProperty"]
});
console.log(result);
A proposal in plain Javascript with a hash table for look-up.
var data = { "array1": [{ "testProperty": "A" }, { "testProperty": "B" }, { "testProperty": "C" }], "array2": [{ "testProperty": "A", "User": "Smith" }, { "testProperty": "B", "User": "Smith" }, ] },
result = data.array1.filter(function (a) {
return !this[a.testProperty];
}, data.array2.reduce(function (r, a) {
r[a.testProperty] = true;
return r;
}, Object.create(null)));
document.write('<pre>' + JSON.stringify(result, 0, 4) + '</pre>');
You can use filter with map
var a = {'array1': [{"testProperty":"A"}, {"testProperty":"B"}, {"testProperty":"C"}], 'array2': [{"testProperty":"A", "User":"Smith"}, {"testProperty":"B", "User":"Smith"}]};
var result = a.array1.filter(function(e) {
return a.array2.map(el => { return el.testProperty}).indexOf(e.testProperty) == -1;
});
console.log(result);
here's a version in plain es6 js using filter and some method:
array1 = [
{"testProperty":"A"},
{"testProperty":"B"},
{"testProperty":"C"}
];
array2 =[
{"testProperty":"A", "User":"Smith"},
{"testProperty":"B", "User":"Smith"},
]
var r = array1.filter(x =>
! Object.keys(x).some(z =>
array2.some(w =>
Object.keys(w).some(y => y === z && w[y] === x[z])
)));
document.write(JSON.stringify(r))
You could use underscore's reject and some to get what you want:
var result = _.reject(array1, item => _.some(array2, {testProperty: item.testProperty}));
If performance is a concern and testProperty is an unique key of the objects in array2 then you could create a hash using indexBy and check for the result using has:
var hash = _.indexBy(array2, 'testProperty');
var result = _.reject(array1, item => _.has(hash, item.testProperty));

How can I use lodash/underscore to sort by multiple nested fields?

I want to do something like this:
var data = [
{
sortData: {a: 'a', b: 2}
},
{
sortData: {a: 'a', b: 1}
},
{
sortData: {a: 'b', b: 5}
},
{
sortData: {a: 'a', b: 3}
}
];
data = _.sortBy(data, ["sortData.a", "sortData.b"]);
_.map(data, function(element) {console.log(element.sortData.a + " " + element.sortData.b);});
And have it output this:
"a 1"
"a 2"
"a 3"
"b 5"
Unfortunately, this doesn't work and the array remains sorted in its original form. This would work if the fields weren't nested inside the sortData. How can I use lodash/underscore to sort an array of objects by more than one nested field?
I've turned this into a lodash feature request: https://github.com/lodash/lodash/issues/581
Update: See the comments below, this is not a good solution in most cases.
Someone kindly answered in the issue I created. Here's his answer, inlined:
_.sortBy(data, function(item) {
return [item.sortData.a, item.sortData.b];
});
I didn't realize that you're allowed to return an array from that function. The documentation doesn't mention that.
If you need to specify the sort direction, you can use _.orderBy with the array of functions syntax from Lodash 4.x:
_.orderBy(data, [
function (item) { return item.sortData.a; },
function (item) { return item.sortData.b; }
], ["asc", "desc"]);
This will sort first ascending by property a, and for objects that have the same value for property a, will sort them descending by property b.
It works as expected when the a and b properties have different types.
Here is a jsbin example using this syntax.
There is a _.sortByAll method in lodash version 3:
https://github.com/lodash/lodash/blob/3.10.1/doc/README.md#_sortbyallcollection-iteratees
Lodash version 4, it has been unified:
https://lodash.com/docs#sortBy
Other option would be to sort values yourself:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
function compareValues(v1, v2) {
return (v1 > v2)
? 1
: (v1 < v2 ? -1 : 0);
};
var data = [
{ a: 2, b: 1 },
{ a: 2, b: 2 },
{ a: 1, b: 3 }
];
data.sort(function (x, y) {
var result = compareValues(x.a, y.a);
return result === 0
? compareValues(x.b, y.b)
: result;
});
// data after sort:
// [
// { a: 1, b: 3 },
// { a: 2, b: 1 },
// { a: 2, b: 2 }
// ];
The awesome, simple way is:
_.sortBy(data, [function(item) {
return item.sortData.a;
}, function(item) {
return item.sortData.b;
}]);
I found it from check the source code of lodash, it always check the function one by one.
Hope that help.
With ES6 easy syntax and lodash
sortBy(item.sortData, (item) => (-item.a), (item) => (-item.b))
I think this could work in most cases with underscore:
var properties = ["sortData.a", "sortData.b"];
data = _.sortBy(data, function (d) {
var predicate = '';
for (var i = 0; i < properties.length; i++)
{
predicate += (i == properties.length - 1
? 'd.' + properties[i]
: 'd.' + properties[i] + ' + ')
}
return eval(predicate)
});
It works and you can see it in Plunker
If the problem is an integer is converted to a string, add zeroes before the integer to make it have the same length as the longest in the collection:
var maxLength = _.reduce(data, function(result, item) {
var bString = _.toString(item.sortData.b);
return result > bString.length ? result : bString.length;
}, 0);
_.sortBy(data, function(item) {
var bString = _.toString(item.sortData.b);
if(maxLength > bString.length) {
bString = [new Array(maxLength - bString.length + 1).join('0'), bString].join('');
}
return [item.sortData.a, bString];
});
I've found a good way to sort array by multiple nested fields.
const array = [
{id: '1', name: 'test', properties: { prop1: 'prop', prop2: 'prop'}},
{id: '2', name: 'test2', properties: { prop1: 'prop second', prop2: 'prop second'}}
]
I suggest to use 'sorters' object which will describe a key and sort order. It's comfortable to use it with some data table.
const sorters = {
'id': 'asc',
'properties_prop1': 'desc',//I'm describing nested fields with '_' symbol
}
dataSorted = orderBy(array, Object.keys(sorters).map(sorter => {
return (row) => {
if (sorter.includes('_')) { //checking for nested field
const value = row["properties"][sorter.split('_')[1]];
return value || null;
};
return row[sorter] || null;// checking for empty values
};
}), Object.values(sorters));
This function will sort an array with multiple nested fields, for the first arguments it takes an array to modify, seconds one it's actually an array of functions, each function have argument that actually an object from 'array' and return a value or null for sorting. Last argument of this function is 'sorting orders', each 'order' links with functions array by index. How the function looks like simple example after mapping:
orderBy(array, [(row) => row[key] || null, (row) => row[key] || null , (row) => row[key] || null] , ['asc', 'desc', 'asc'])
P.S. This code can be improved, but I would like to keep it like this for better understanding.

Categories