We Keep Coding

Nested object of unequal length into array of objects

So I have an interesting problem which I have been able to solve, but my solution is not elegant in any way or form, so I was wondering what others could come up with :)
The issue is converting this response here
const response = {
"device": {
"name": "Foo",
"type": "Bar",
"telemetry": [
{
"timeStamp": "2022-06-01T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-02T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-03T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-04T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-05T00:00:00.000Z",
"temperature": 100,
"pressure": 50
}
]
}
};
Given this selection criteria
const fields = ['device/name', 'device/telemetry/timeStamp', 'device/telemetry/temperature']
and the goal is to return something like this
[
{"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-01T00:00:00.000Z", "device/telemetry/temperature": 100},
{"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-02T00:00:00.000Z", "device/telemetry/temperature": 100},
{"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-03T00:00:00.000Z", "device/telemetry/temperature": 100},
...,
{"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-05T00:00:00.000Z", "device/telemetry/temperature": 100},
]
If you are interested, here is my horrible brute force solution, not that familiar with typescript yet, so please forgive the horribleness :D
EDIT #1
So some clarifications might be needed. The response can be of completely different format, so we can't use our knowledge of how the response looks like now, the depth can also be much deeper.
What we can assume though is that even if there are multiple arrays in the reponse (like another telemetry array called superTelemetry) then the selection criteria will only choose from one of these arrays, never both :)
function createRecord(key: string, value: any){
return new Map<string, any>([[key, value]])
}
function getNestedData (data: any, fieldPath: string, records: Map<string, any[]>=new Map<string, any[]>()) {
let dataPoints: any = [];
const paths = fieldPath.split('/')
paths.forEach((key, idx, arr) => {
if(Array.isArray(data)){
data.forEach(
(row: any) => {
dataPoints.push(row[key])
}
)
} else {
data = data[key]
if(idx + 1== paths.length){
dataPoints.push(data);
}
}
})
records.set(fieldPath, dataPoints)
return records
}
function getNestedFields(data: any, fieldPaths: string[]){
let records: Map<string, any>[] = []
let dataset: Map<string, any[]> = new Map<string, any[]>()
let maxLength = 0;
// Fetch all the fields
fieldPaths.forEach((fieldPath) => {
dataset = getNestedData(data, fieldPath, dataset)
const thisLength = dataset.get(fieldPath)!.length;
maxLength = thisLength > maxLength ? thisLength : maxLength;
})
for(let i=0; i<maxLength; i++){
let record: Map<string, any> = new Map<string, any>()
for(let [key, value] of dataset){
const maxIdx = value.length - 1;
record.set(key, value[i > maxIdx ? maxIdx : i])
}
records.push(record)
}
// Normalize into records
return records
}

As per my understanding you are looking for a solution to construct the desired result as per the post. If Yes, you can achieve this by using Array.map() along with the Array.forEach() method.
Try this :
const response = {
"device": {
"name": "Foo",
"type": "Bar",
"telemetry": [
{
"timeStamp": "2022-06-01T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-02T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-03T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-04T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-05T00:00:00.000Z",
"temperature": 100,
"pressure": 50
}
]
}
};
const fields = ['device/name', 'device/telemetry/timeStamp', 'device/telemetry/temperature'];
const res = response.device.telemetry.map(obj => {
const o = {};
fields.forEach(item => {
const splittedItem = item.split('/');
o[item] = (splittedItem.length === 2) ? response[splittedItem[0]][splittedItem[1]] : obj[splittedItem[2]];
});
return o;
})
console.log(res);

In what follows I will be concerned with just the implementation and runtime behavior, and not so much the types. I've given things very loose typings like any and string instead of the relevant generic object types. Here goes:
function getNestedFields(data: any, paths: string[]): any[] {
If data is an array, we want to perform getNestedFields() on each element of the array, and then concatenate the results together into one big array. So the first thing we do is check for that and make a recursive call:
if (Array.isArray(data)) return data.flatMap(v => getNestedFields(v, paths));
Now that we know data is not an array, we want to start gathering the pieces of the answer. If paths is, say, ['foo/bar', 'foo/baz/qux', 'x/y', 'x/z'], then we want to make recursive calls to getNestedFields(data.foo, ["bar", "baz/qux"]) and to getNestedFields(data.x, ["y", "z"]). In order to do this we have to split each path element at its first slash "/", and collect the results into a new object whose keys are the part to the left of the slash and whose values are arrays of parts to the right. In this example it would be {foo: ["bar", "baz/qux"], x: ["y", "z"]}.
Some important edge cases: for every element of paths with no slash, then we have a key with an empty value... that is, ["foo"] should result in a call like getNestedFields(data.foo, [""]). And if there is an element of paths that's just the empty string "", then we don't want to do a recursive call; the empty path is the base case and implies that we're asking about data itself. That is, instead of a recursive call, we can just return [{"": data}]. So we need to keep track of the empty path (hence the emptyPathInList variable below).
Here's how it looks:
const pathMappings: Record<string, string[]> = {};
let emptyPathInList = false;
paths.forEach(path => {
if (!path) {
emptyPathInList = true;
} else {
let slashIdx = path.indexOf("/");
if (slashIdx < 0) slashIdx = path.length;
const key = path.substring(0, slashIdx);
const restOfPath = path.substring(slashIdx + 1);
if (!(key in pathMappings)) pathMappings[key] = [];
pathMappings[key].push(restOfPath);
}
})
Now, for each key-value pair in pathMappings (with key key and with value restsOfPath) we need to call getNestedFields() recursively... the results will be an array of objects whose keys are relative to data[key], so we need to prepend key and a slash to their keys. Edge cases: if there's an empty path we shouldn't add a slash. And if data` is nullish then we will have a runtime error recursing down into it, so we might want to do something else there (although a runtime error might be fine since it's a weird input):
const subentries = Object.entries(pathMappings).map(([key, restsOfPath]) =>
(data == null) ? [{}] : // <-- don't recurse down into nullish data
getNestedFields(data[key], restsOfPath)
.map(nestedFields =>
Object.fromEntries(Object.entries(nestedFields)
.map(([path, value]) =>
[key + (path ? "/" : "") + path, value])))
)
Now subentries is an array of all the separate recursive call results, with the proper keys. We want to add one more entry correpsonding to data if emptyPathInList is true:
if (emptyPathInList) subentries.push([{ "": data }]);
And now we need to combine these sub-entries by taking their Cartesian product and spreading into a single object for each entry. By Cartesian product I mean that if subentries looks like [[a,b],[c,d,e],[f]] then I need to get [[a,c,f],[a,d,f],[a,e,f],[b,c,f],[b,d,f],[b,e,f]], and then for each of those we spread into single entries. Here's that:
return subentries.reduce((a, v) => v.flatMap(vi => a.map(ai => ({ ...ai, ...vi }))), [{}])
}
Okay, so let's test it out:
console.log(getNestedFields(response, fields));
/* [{
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-01T00:00:00.000Z",
"device/telemetry/temperature": 100
}, {
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-02T00:00:00.000Z",
"device/telemetry/temperature": 100
}, {
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-03T00:00:00.000Z",
"device/telemetry/temperature": 100
}, {
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-04T00:00:00.000Z",
"device/telemetry/temperature": 100
}, {
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-05T00:00:00.000Z",
"device/telemetry/temperature": 100
}] */
That's what you wanted. Even though you said you will never walk into different arrays, this version should support that:
console.log(getNestedFields({
a: [{ b: 1 }, { b: 2 }],
c: [{ d: 3 }, { d: 4 }]
}, ["a/b", "c/d"]))
/* [
{ "a/b": 1, "c/d": 3 },
{ "a/b": 2, "c/d": 3 },
{ "a/b": 1, "c/d": 4 },
{ "a/b": 2, "c/d": 4 }
]*/
There are probably all kinds of crazy edge cases, so anyone using this should test thoroughly.
Playground link to code

Filter some values of big array in Javascript

I have an array fetched from our server which holds 2400 objects (total size is about 7MB) and I want to filter some first values in it. Right now I'm using combination of filter and slice method:
const keyword = 'whatever word';
const recommendList =bigArray.filter(item => item.name.includes(keyword)).slice(0, 5);
What I know is filter method iterates all the element in array and I think it can impact to performance of my app (React Native) cause its large data. So is there any approach to filter the array for some values, without iterating all the elements ?

If you simply want to to break(stop) the loop when you find 5th element then you can do the bellow:
const keyword = 'v';
const bigArray = ['a','v','a','v','a','v','a','v','a','v','a','v','a','v','a','v','a','v'];
const recommendList = [];
for (let i=0; i<bigArray.length; i++) { // loop till you reach end of big array index
if (recommendList.length == 5) // if length is 5 this will break the loop
{
break;
}
if (bigArray[i].includes(keyword)) {
recommendList.push(bigArray[i]); // add if you find
}
}
console.log(recommendList);
If you dont want to use lambda operation can simply use, some, find etc which only works till they return the first response as true
const bigArray = [{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
}
];
const keyword = 'v';
const recommendList = [];
// some operator only iterates till its condition returns true
// so if we get 5 recommended list before the bigArray end we return true and stop the iteration.
bigArray.some(obj => {
if (obj.name.includes(keyword)) {
recommendList.push(obj)
}
return recommendList.length === 5; // return true if 5 values are found, that will terminate the iteration
})
console.log(recommendList);

const keyword = 'whatever word';
const recommendList = [];
for (let i=0; i<bigArray.length; i++) {
if ( recommendList.length >= 5)
break;
const item = bigArray[i];
if (item.name.includes(keyword))
recommendList.push(item);
}

The most efficient approach is to process the array as an iterable sequence.
Example below is based on iter-ops library:
import {pipe, filter, take} from 'iter-ops';
const i = pipe(
bigArray,
filter(item => item.name.includes(keyword)),
take(5)
);
console.log('matches:', [...i]);
This way, you won't be iterating through everything even once, it will stop just as the first 5 matches are found.

In this condition, I think filter (the complexity is O(n)) should be the best solution that lead to minimized performance impact.
Think of that, if you just filter some of the values among those 2400 objects, say 1500. Then you would only get filtered results of 1500 objects, and the rest 900 objects would never be used. So at least one loop is necessary.

How to create an object containing a range of values for a key

I'm trying to create an object that should hold a range of values for a key which I would use as a look-up table. For example, it needs to "catch" values in the ranges: 500-524, 600-650, etc..
e.g.:
const numbers = {
500 to 524: "20",
600 to 650: "25"
}
And I would like to access the value in the following way:
user.list.map(list => numbers[user.points]).
I know I can put all the values from the range as keys but that would be highly inefficient:
const numbers = {
"500": "20",
"501": "20",
"502": "20",
"503": "20",
(...)
}
So, is it possible to include ranges somehow?

You could set it up like this:
numbers = [
{"start": 500, "end": 524, "value": "20"},
...
...
]
function getFromNumbers(num) {
for(let i=0; i<numbers.length; i++) {
numVal = numbers[i];
if(numVal.start <= num && numVal.end >= num)
return numVal.value;
}
}
console.log(user.list.map(list => getFromNumbers(user.points)))

Why don't you create an object like this:
const numbers =
[
{
start : 500
end : 524
value :20
},
{
start : 600
end : 650
value : 25
}
]

I think you should inverse the solution. You can create object, where key is "20" and value is array of values. For example:
const numbers = {
"20": _.range(500, 524)
}

How can I merge JavaScript arrays containing objects, deduplicate and keep the newer object

I'm trying to merge two arrays of objects by checking if the titles are the same, and if they are, then checking for which entry is newer, and discarding the older one. I have found a lot of solutions for discarding true duplicates, but how can I do this in a way where I can decided which to keep based on dates?
const a = [{
"title": "title1",
"date": "2010-08-20T15:51:58"
}, {
"title": "title2",
"date": "2015-09-20T16:45:21"
}]
const b = [{
"title": "title1",
"date": "2015-08-20T15:51:58"
}, {
"title": "title2",
"date": "2015-09-20T16:45:21"
}]
Thanks for any tips you can provide!

Here is ES6 code to do that:
var res = Array.from([...a,...b].reduce ( (hash, v) =>
!hash.has(v.title) || hash.get(v.title).date < v.date ? hash.set(v.title, v) : hash
, new Map()), v => v[1]);
var a = [{
"title": "title1",
"date": "2010-08-20T15:51:58"
}, {
"title": "title2",
"date": "2015-09-20T16:45:21"
}]
var b = [{
"title": "title1",
"date": "2015-08-20T15:51:58"
}, {
"title": "title2",
"date": "2015-09-20T16:45:21"
}]
var res = Array.from([...a,...b].reduce ( (hash, v) =>
!hash.has(v.title) || hash.get(v.title).date < v.date ? hash.set(v.title, v) : hash
, new Map()), v => v[1]);
console.log(res);
Explanation
First the input arrays are concatenated together into one new array with the spread operator:
[...a,...b]
Then an empty Map is created and passed as last argument to reduce:
new Map()
The reduce method calls the arrow function for each element in the concatenated array. The arrow function also receives the above mentioned map as argument (as hash).
The arrow function must return a value. That value is then passed again to subsequent call of this function (for the next element), and so we always return the map, which grows in each function call. It is, as it were, passed from one call to the next. In the last call the returned map becomes the return value of .reduce().
The arrow function itself checks if the current element's title is not yet in the map:
!hash.has(v.title)
If it is in the map already, then the next expression is also evaluated; it checks whether the date in the map entry is before the current element's date.
hash.get(v.title).date < date
If either of the above conditions is true (not in map, or with smaller date), then the map entry is (re)created with the current element as value.
? hash.set(v.title, v)
This set also returns the whole map after setting. Otherwise the map is returned unchanged:
: hash
The result of reduce() is thus a map, keyed by titles. This is really the result you need, but it is in Map format. To get it back to a normal array, the Array.from method is called on it. This changes the Map values into an array of key-value pairs (sub-arrays with 2 elements). Since we are only interested in the values, we apply a function to it:
v => v[1]
This replaces every pair with only the second value. This function is passed as second argument to Array.from which applies it to every pair.
Some remarks
This assumes that your dates are in ISO format, like in your sample: in that case the string comparison gives the correct result for determining whether one date precedes another.
The result will include also the objects that only occur in one of the two input arrays
This is easily extendible to three input arrays: just add a third one like this: [...a,...b,...c]
This runs in O(n) time, where n is the total number of objects present in the input arrays. This is because most JavaScript engines implement Map access operations like .has, .get and .put with O(1) time.

a.forEach(function(elem1,count1){
b.forEach(function(elem2,count2){
//loop trough each a frouvh each b
if(elem1.title==elem2.title){
var date1=new Date(elem1.date);
var date2=new Date(elem2.date);
if(date1.getTime()>date2.getTime()){
//elem from a is older
//delete elem from a
}else{
//elem from b is older
}
}
});
});

var a = [{
"title": "title1",
"date": "2010-08-20T15:51:58"
}, {
"title": "title2",
"date": "2015-09-20T16:45:21"
}]
var b = [{
"title": "title1",
"date": "2015-08-20T15:51:58"
}, {
"title": "title2",
"date": "2015-09-20T16:45:21"
}]
function assign(a, b) {
return a.reduce((acc, itemA) => {
const {title: titleA, date: dateA} = itemA
const itemB = b.find(({title: titleB}) => titleA == titleB)
if (itemB) {
if (new Date(dateA) - new Date(itemB.date) >= 0) {
acc.push(itemA)
} else {
acc.push(itemB)
}
}
return acc
}, [])
}

You can use for loop, new Date().getTime()
var a = [{
"title": "title1",
"date": "2010-08-20T15:51:59"
}, {
"title": "title2",
"date": "2015-09-20T16:45:22"
}];
var b = [{
"title": "title1",
"date": "2015-08-20T15:51:58"
}, {
"title": "title2",
"date": "2015-09-20T16:45:21"
}];
var res = [];
for (var i = 0; i < a.length; i++) {
var curr = a[i];
for (var n = 0; n < b.length; n++) {
if (curr.title === b[n].title) {
if (new Date(curr.date).getTime() > new Date(b[n].date).getTime()) {
res.push(curr)
} else {
res.push(b[n])
}
}
}
}
console.log(res);

How to Optimize Merge of Two Objects That Include Arrays of Objects

I need to merge two objects in a code path that is going to be heavily used. The code works, but I am concerned it is not optimized enough for speed and I am looking for any suggestions to improve/replace what I have come up with. I originally started working off an example at the end of this issue: How can I merge properties of two JavaScript objects dynamically?. That solution works well for simple objects. However, my needs have a twist to it which is where the performance concerns come in. I need to be able to support arrays such that
an array of simple values will look for values in the new object and add those to the end of the existing object and
an array of objects will either merge objects (based off existence of an id property) or push new objects (objects whose id property does not exist) to the end of the existing array.
I do not need functions/method cloning and I don't care about hasOwnProperty since the objects go back to JSON strings after merging.
Any suggestions to help me pull every last once of performance from this would be greatly appreciated.
var utils = require("util");
function mergeObjs(def, obj) {
if (typeof obj == 'undefined') {
return def;
} else if (typeof def == 'undefined') {
return obj;
}
for (var i in obj) {
// if its an object
if (obj[i] != null && obj[i].constructor == Object)
{
def[i] = mergeObjs(def[i], obj[i]);
}
// if its an array, simple values need to be joined. Object values need to be remerged.
else if(obj[i] != null && utils.isArray(obj[i]) && obj[i].length > 0)
{
// test to see if the first element is an object or not so we know the type of array we're dealing with.
if(obj[i][0].constructor == Object)
{
var newobjs = [];
// create an index of all the existing object IDs for quick access. There is no way to know how many items will be in the arrays.
var objids = {}
for(var x= 0, l= def[i].length ; x < l; x++ )
{
objids[def[i][x].id] = x;
}
// now walk through the objects in the new array
// if the ID exists, then merge the objects.
// if the ID does not exist, push to the end of the def array
for(var x= 0, l= obj[i].length; x < l; x++)
{
var newobj = obj[i][x];
if(objids[newobj.id] !== undefined)
{
def[i][x] = mergeObjs(def[i][x],newobj);
}
else {
newobjs.push(newobj);
}
}
for(var x= 0, l = newobjs.length; x<l; x++) {
def[i].push(newobjs[x]);
}
}
else {
for(var x=0; x < obj[i].length; x++)
{
var idxObj = obj[i][x];
if(def[i].indexOf(idxObj) === -1) {
def[i].push(idxObj);
}
}
}
}
else
{
def[i] = obj[i];
}
}
return def;}
The object samples to merge:
var obj1 = {
"name" : "myname",
"status" : 0,
"profile": { "sex":"m", "isactive" : true},
"strarr":["one", "three"],
"objarray": [
{
"id": 1,
"email": "a1#me.com",
"isactive":true
},
{
"id": 2,
"email": "a2#me.com",
"isactive":false
}
]
};
var obj2 = {
"name" : "myname",
"status" : 1,
"newfield": 1,
"profile": { "isactive" : false, "city": "new York"},
"strarr":["two"],
"objarray": [
{
"id": 1,
"isactive":false
},
{
"id": 2,
"email": "a2modified#me.com"
},
{
"id": 3,
"email": "a3new#me.com",
"isactive" : true
}
]
};
Once merged, this console.log(mergeObjs(obj1, obj2)) should produce this:
{ name: 'myname',
status: 1,
profile: { sex: 'm', isactive: false, city: 'new York' },
strarr: [ 'one', 'three', 'two' ],
objarray:
[ { id: 1, email: 'a1#me.com', isactive: false },
{ id: 2, email: 'a2modified#me.com', isactive: false },
{ id: 3, email: 'a3new#me.com', isactive: true } ],
newfield: 1 }

I'd check out: https://github.com/bestiejs/lodash
_.merge is not on the list of 'optimized' functions, but this is a battle tested, battle hardened. He also has a performance suite, could ask how you might contribute to the perf suite to get some visibility into the merge implementation.
https://github.com/bestiejs/lodash/blob/master/lodash.js#L1677-1738
Edit: As an aside, I wouldn't prematurely optimize. I would see if this is actually a problem in your use case and then move on to actual data. I would look at something like: https://github.com/felixge/faster-than-c
Basic tenets:
Collect data
Analyze it
Find problems
Fix them
Repeat
He's got tips on each of those.

If you don't use Lo-Dash, and just want a tool to merge two objects including their arrays, use deepmerge: https://github.com/nrf110/deepmerge
npm install deepmerge