How to dedupe an array of objects by a key value pair? - javascript

// This is a large array of objects, e.g.:
let totalArray = [
{"id":"rec01dTDP9T4ZtHL4","fields":
{"user_id":170180717,"user_name":"abcdefg","event_id":516575,
}]
let uniqueArray = [];
let dupeArray = [];
let itemIndex = 0
totalArray.forEach(x => {
if(!uniqueArray.some(y => JSON.stringify(y) === JSON.stringify(x))){
uniqueArray.push(x)
} else(dupeArray.push(x))
})
node.warn(totalArray);
node.warn(uniqueArray);
node.warn(dupeArray);
return msg;
I need my code to identify duplicates in the array by a key value of user_id within the objects in the array. Right now, my code works to identify identical objects in the array, but I need it to identify dupes based on a key value inside the objects instead. How do I do this? I am struggling to figure out how to path the for each loop to identify the dupe based on the key value instead of the entire object.

Right now, my code works to identify identical objects in the array, but I need it to identify dupes based on a key value inside the objects instead. How do I do this?
Don’t compare the JSON representation of the whole objects then, but only their user_id property specifically.
totalArray.forEach(x => {
if(!uniqueArray.some(y => y.fields.user_id === x.fields.user_id)){
uniqueArray.push(x)
} else(dupeArray.push(x))
})

You could take a Set and push to either uniques or duplicates.
var array = [
{ id: 1, data: 0 },
{ id: 2, data: 1 },
{ id: 2, data: 2 },
{ id: 3, data: 3 },
{ id: 3, data: 4 },
{ id: 3, data: 5 },
],
uniques = [],
duplicates = [];
array.forEach(
(s => o => s.has(o.id) ? duplicates.push(o) : (s.add(o.id), uniques.push(o)))
(new Set)
);
console.log(uniques);
console.log(duplicates);
.as-console-wrapper { max-height: 100% !important; top: 0; }

One way is to keep a list of ids you found so far and act accordingly:
totalArray = [
{ id: 1, val: 10 },
{ id: 2, val: 20 },
{ id: 3, val: 30 },
{ id: 2, val: 15 },
{ id: 1, val: 50 }
]
const uniqueArray = []
const dupeArray = []
const ids = {}
totalArray.forEach( x => {
if (ids[x.id]) {
dupeArray.push(x)
} else {
uniqueArray.push(x)
ids[x.id] = true
}
})
for (const obj of uniqueArray) console.log("unique:",JSON.stringify(obj))
for (const obj of dupeArray) console.log("dupes: ",JSON.stringify(obj))

Related

Efficient way to get unique objects from array of objects, by object property?

Here is my array of objects
const array = [
{id: 1, data: "foo"},
{id: 1, data: "bar"},
{id: 2, data: "baz"}
]
I want to remove all duplicate objects by its id and return only the array of objects that have an unique id.
Expected result:
[
{id: 2, data: "baz"}
]
This is what I have now: O(n^2)
function getUnique(array) {
const newArray = []
for (let obj of array) {
if (array.filter(x => x.id === obj.id).length === 1) {
newArray.push(obj)
}
}
return newArray
}
Whats the more efficient way to achieve this?
Is it possible to get the time-complexity to O(n) or O(n log n)?
const array = [{
id: 1,
data: "foo"
},
{
id: 1,
data: "bar"
},
{
id: 2,
data: "baz"
}
]
let map = {};
array.forEach(x => {
map[x.id] = (map[x.id] || 0) + 1
});
console.log(array.filter(x => map[x.id] === 1))
I would suggest to count the number of occurrences in your array and store them in a Map. Next you filter all element, which count is 1
function getUnique(arr) {
const count = new Map();
arr.forEach((element) => {
count.set(element.id, (count.get(element.id) || 0) + 1);
});
return array.filter((element) => {
return count.get(element.id) === 1;
});
}
This has a runtime of 2(n) since you have to iterate over them twice

How to work with identical, deeply nested data?

For example, has the following data:
let example = {
content: [
...
{ // index = 3
id: "b3bbb2a0-3345-47a6-b4f9-51f22b875f22",
data: {
value: "hello",
content: [
...
{ // index = 0
id: "65b1e224-4eae-4a6d-8d00-c1caa9c7ed2a",
data: {
value: "world",
content: [
...
{ // index = 1
id: "263a4961-efa7-4639-8a57-b20b23a7cc9d",
data: {
value: "test",
content: [
// Nesting unknown.
]
}
}
]
}
}
]
}
}
]
}
And for example an array with indexes leading to the required element(but can be any other):
const ids = [3, 0, 1]
How can you work with an element having this data?
For example, need to change "value" in the element at the specified path in "ids".
You could take an array of indices and get the item of the property content by calling the function again for each missing index.
const
getElement = ({ content }, [index, ...indices]) => indices.length
? getElement(content[index], indices)
: content[index];
If needed, you could add a guard for a missing index and exit early.
You can just recursively loop over your element and change the value, if you got to the last element.
I have written a small example, all you would have to do, is to extend the method for your own data structure (el.data.content):
const el = [[1,2], [3,4], [5,6]];
const changeEl = (el, indexArr, val) => {
if(indexArr.length == 1) {
el[indexArr[0]] = val;
} else {
changeEl(el[indexArr.shift()], indexArr, val);
}
}
changeEl(el, [1, 0], 99);
console.log(el);

How to invert the structure of nested array of objects in Javascript?

I currently have an array that has the following structure:
data = [
{
time: 100,
info: [{
name: "thing1",
count: 3
}, {
name: "thing2",
count: 2
}, {
}]
},
{
time: 1000,
info: [{
name: "thing1",
count: 7
}, {
name: "thing2",
count: 0
}, {
}]
}
];
But I would like to restructure the array to get something like this:
data = [
{
name: "thing1",
info: [{
time: 100,
count: 3
}, {
time: 1000,
count: 7
}, {
}]
},
{
name: "thing2",
info: [{
time: 100,
count: 2
}, {
time: 1000,
count: 0
}, {
}]
}
];
So basically the key would have to be switched from time to name, but the question is how. From other posts I have gathered that using the map function might work, but since other posts had examples to and from different structures I am still not sure how to use this.
There are a number of ways to achieve this however, the key idea will be to perform a nested looping of both data items and their (nested) info items. Doing that allows your algorithm to "visit" and "map" each piece of input data, to a corresponding value in the resulting array.
One way to express that would be to use nested calls to Array#reduce() to first obtaining a mapping of:
name -> {time,count}
That resulting mapping would then be passed to a call to Object.values() to transform the values of that mapping to the required array.
The inner workings of this mapping process are summarized in the documentation below:
const data=[{time:100,info:[{name:"thing1",count:3},{name:"thing2",count:2},{}]},{time:1e3,info:[{name:"thing1",count:7},{name:"thing2",count:0},{}]}];
const result =
/* Obtain array of values from outerMap reduce result */
Object.values(
/* Iterate array of data items by reduce to obtain mapping of
info.name to { time, count} value type */
data.reduce((outerMap, item) =>
/* Iterate inner info array of current item to compound
mapping of info.name to { time, count} value types */
item.info.reduce((innerMap, infoItem) => {
if(!infoItem.name) {
return innerMap
}
/* Fetch or insert new { name, info } value for result
array */
const nameInfo = innerMap[ infoItem.name ] || {
name : infoItem.name, info : []
};
/* Add { time, count } value to info array of current
{ name, info } item */
nameInfo.info.push({ count : infoItem.count, time : item.time })
/* Compound updated nameInfo into outer mapping */
return { ...innerMap, [ infoItem.name] : nameInfo }
}, outerMap),
{})
)
console.log(result)
Hope that helps!
The approach I would take would be to use an intermediate mapping object and then create the new array from that.
const data = [{time: 100, info: [{name: "thing1", count: 3}, {name: "thing2", count: 2}, {}]}, {time: 1e3, info: [{name: "thing1", count: 7}, {name: "thing2", count: 0}, {}]} ];
const infoByName = {};
// first loop through and add entries based on the name
// in the info list of each data entry. If any info entry
// is empty ignore it
data.forEach(entry => {
if (entry.info) {
entry.info.forEach(info => {
if (info.name !== undefined) {
if (!infoByName[info.name]) {
infoByName[info.name] = [];
}
infoByName[info.name].push({
time: entry.time,
count: info.count
});
}
});
}
});
// Now build the resulting list, where name is entry
// identifier
const keys = Object.keys(infoByName);
const newData = keys.map(key => {
return {
name: key,
info: infoByName[key]
};
})
// newData is the resulting list
console.log(newData);
Well, the other guy posted a much more elegant solution, but I ground this one out, so I figured may as well post it. :)
var data = [
{
time: 100,
info: [{
name: "thing1",
count: 3
}, {
name: "thing2",
count: 2
}, {
}]
},
{
time: 1000,
info: [{
name: "thing1",
count: 7
}, {
name: "thing2",
count: 0
}, {
}]
}
];
var newArr = [];
const objInArray = (o, a) => {
for (var i=0; i < a.length; i += 1) {
if (a[i].name === o)
return true;
}
return false;
}
const getIndex = (o, a) => {
for (var i=0; i < a.length; i += 1) {
if (a[i].name === o) {
return i;
}
}
return false;
}
const getInfoObj = (t, c) => {
let tmpObj = {};
tmpObj.count = c;
tmpObj.time = t;
return tmpObj;
}
for (var i=0; i < data.length; i += 1) {
let t = data[i].time;
for (var p in data[i].info) {
if ("name" in data[i].info[p]) {
if (objInArray(data[i].info[p].name, newArr)) {
let idx = getIndex(data[i].info[p].name, newArr);
let newInfoObj = getInfoObj(t, data[i].info[p].count);
newArr[idx].info.push(newInfoObj);
} else {
let newObj = {};
newObj.name = data[i].info[p].name;
let newInfo = [];
let newInfoObj = getInfoObj(t, data[i].info[p].count);
newInfo.push(newInfoObj);
newObj.info = newInfo;
newArr.push(newObj);
}}
}
}
console.log(newArr);
try to use Object.keys() to get the key

Check whether a property of an object exists and only update another property

I have an array with objects that look like this:
let arr = [
{ taxonomy: 'category', id: [ 10, 100 ] },
{ taxonomy: 'post_tag', id: [ 20 ] },
];
I want to be able to push a new object in the array that look like this:
const object = {
taxonomy: 'category',
id: 30
}
What i want is a check if an object in the array with the property value 'taxonomy' already exists, if it does i want to only add the id from the new object in to the existing object. I know how to check if the property already exists but i don't exactly know how to add the new id to the array.
So adding the above object would result in this array:
[
{ taxonomy: 'category', id: [ 10, 100, 30 ] }, // 30 is added
{ taxonomy: 'post_tag', id: [ 20 ] },
];
if it doesn't exist yet it should be added.
Can anybody help me with this?
Use Array.find() to locate an object with the same taxonomy in the array. If one exists, add the id to it. If not, push a clone of the object into the array (after converting the object's id to array):
const addUpdate = obj => {
const current = arr.find(o => obj.taxonomy === o.taxonomy);
if(current) current.id.push(obj.id);
else arr.push({
...obj,
id: [obj.id]
})
};
const arr = [
{ taxonomy: 'category', id: [ 10, 100 ] },
{ taxonomy: 'post_tag', id: [ 20 ] },
];
addUpdate({ taxonomy: 'category', id: 30 });
addUpdate({ taxonomy: 'other', id: 50 });
console.log(arr);
You could find the array and update or push a new object with id as array to the array.
const
array = [{ taxonomy: 'category', id: [ 10, 100 ] }, { taxonomy: 'post_tag', id: [ 20 ] }];
object = { taxonomy: 'category', id: 30 },
item = array.find(({ taxonomy }) => object.taxonomy === taxonomy);
if (item) {
item.id.push(object.id);
} else {
array.push(Object.assign({}, object, { id: [object.id] }));
}
console.log(array);
// remove the last insert
// find item with taxonomy and id
item = array.find(({ taxonomy, id }) => object.taxonomy === taxonomy && id.includes(object.id));
// remove from id by using the index
if (item) item.id.splice(item.id.indexOf(object.id), 1);
console.log(array);
.as-console-wrapper { max-height: 100% !important; top: 0; }
You can use the function forEach for looping and pushing the new id.
let arr = [{taxonomy: 'category',id: [10, 100]},{taxonomy: 'post_tag',id: [20]},],
object = {taxonomy: 'category',id: 30};
arr.forEach(({taxonomy, id}) => {
if (object.taxonomy === taxonomy) {
id.push(object.id);
}
});
console.log(arr);
Using an upsert() function like below, you can accomplish this with a little bit of abstraction, so that it can be re-used for more than just this particular schema. upsert() contains some reasonable defaults for the functions, which would work fine if all the objects are dictionaries with primitive values
function upsert (
array, object, keyFn,
updateFn = (target, object) => Object.assign(target, object),
insertFn = object => object
) {
const key = keyFn(object)
const index = array.findIndex(
value => keyFn(value) === key
)
if (index !== -1) updateFn(array[index], object)
else array.push(insertFn(object))
}
let arr = [
{ taxonomy: 'category', id: [10, 100] },
{ taxonomy: 'post_tag', id: [20] }
]
const obj = { taxonomy: 'category', id: 30 }
upsert(
arr, obj,
o => o.taxonomy, // used to determine if existing object should be updated
(t, o) => t.id.push(o.id), // operation for updating existing object
({ id, ...o }) => ({ ...o, id: [id] }) // return new object to insert
)
console.log(arr)

How can I concat an array to an array that is inside an array of objects?

I am trying to concat an array to an array (productsCategories) inside an array of objects.
So, here's what the productCategories array looks like:
[
{
id: 123,
items: [ { Obj1 }, { Obj2 } ]
},
{
id:456,
items: [ { Obj1 }, { Obj2 } ]
}
]
I have some new array, like [ { Obj3 }, { Obj4 } ] that I want to concat to the productCategories for the object where id = 123.
So to do this,
I've first used lodash's find to find the correct object to update and used concat to join the two arrays:
let nextItems:any = find(productCategories, { id: payload.id });
nextItems = assign({}, nextItems, { items: nextItems.items.concat(payload.items)});
So, nextItems.items has the concatenated items array.
However, I am having trouble now adding this to productCategories array. I need to find the object where id is the same as nextItems.id and then set productCategories.items equal to nextItems.items.
What is the correct way to do this?
Find the index of the object that matches the nextItems.id in the productCategories and assign the new concatenated array to it. You can use the lodash findIndex() method to find the index of the object that matches the id.
var index = _findIndex(productCategories, { id: nextItems.id });
productCategories[index].items = nextItems.items;
You can use plain JavaScript just as well. With ES6 spread syntax it can look like this:
productCategories.filter(x => x.id == payload.id)
.forEach(x => x.items.push(...payload.items));
Here is a snippet with sample data:
// Sample data
var productCategories = [{
id: 123,
items: [ { a: 1 }, { a: 2 } ]
}, {
id: 456,
items: [ { b: 1 }, { b: 2 } ]
}];
var payload = {
id: 123,
items: [ { c: 1 }, { c: 2 } ]
};
// Update with payload
productCategories.filter(x => x.id == payload.id)
.forEach(x => x.items.push(...payload.items));
// Show results
console.log(productCategories);
.as-console-wrapper { max-height: 100% !important; top: 0; }

Categories