So I have an interesting problem which I have been able to solve, but my solution is not elegant in any way or form, so I was wondering what others could come up with :)
The issue is converting this response here
const response = {
"device": {
"name": "Foo",
"type": "Bar",
"telemetry": [
{
"timeStamp": "2022-06-01T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-02T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-03T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-04T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-05T00:00:00.000Z",
"temperature": 100,
"pressure": 50
}
]
}
};
Given this selection criteria
const fields = ['device/name', 'device/telemetry/timeStamp', 'device/telemetry/temperature']
and the goal is to return something like this
[
{"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-01T00:00:00.000Z", "device/telemetry/temperature": 100},
{"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-02T00:00:00.000Z", "device/telemetry/temperature": 100},
{"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-03T00:00:00.000Z", "device/telemetry/temperature": 100},
...,
{"device/name": "Foo", "device/telemetry/timeStamp": "2022-06-05T00:00:00.000Z", "device/telemetry/temperature": 100},
]
If you are interested, here is my horrible brute force solution, not that familiar with typescript yet, so please forgive the horribleness :D
EDIT #1
So some clarifications might be needed. The response can be of completely different format, so we can't use our knowledge of how the response looks like now, the depth can also be much deeper.
What we can assume though is that even if there are multiple arrays in the reponse (like another telemetry array called superTelemetry) then the selection criteria will only choose from one of these arrays, never both :)
function createRecord(key: string, value: any){
return new Map<string, any>([[key, value]])
}
function getNestedData (data: any, fieldPath: string, records: Map<string, any[]>=new Map<string, any[]>()) {
let dataPoints: any = [];
const paths = fieldPath.split('/')
paths.forEach((key, idx, arr) => {
if(Array.isArray(data)){
data.forEach(
(row: any) => {
dataPoints.push(row[key])
}
)
} else {
data = data[key]
if(idx + 1== paths.length){
dataPoints.push(data);
}
}
})
records.set(fieldPath, dataPoints)
return records
}
function getNestedFields(data: any, fieldPaths: string[]){
let records: Map<string, any>[] = []
let dataset: Map<string, any[]> = new Map<string, any[]>()
let maxLength = 0;
// Fetch all the fields
fieldPaths.forEach((fieldPath) => {
dataset = getNestedData(data, fieldPath, dataset)
const thisLength = dataset.get(fieldPath)!.length;
maxLength = thisLength > maxLength ? thisLength : maxLength;
})
for(let i=0; i<maxLength; i++){
let record: Map<string, any> = new Map<string, any>()
for(let [key, value] of dataset){
const maxIdx = value.length - 1;
record.set(key, value[i > maxIdx ? maxIdx : i])
}
records.push(record)
}
// Normalize into records
return records
}
As per my understanding you are looking for a solution to construct the desired result as per the post. If Yes, you can achieve this by using Array.map() along with the Array.forEach() method.
Try this :
const response = {
"device": {
"name": "Foo",
"type": "Bar",
"telemetry": [
{
"timeStamp": "2022-06-01T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-02T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-03T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-04T00:00:00.000Z",
"temperature": 100,
"pressure": 50
},
{
"timeStamp": "2022-06-05T00:00:00.000Z",
"temperature": 100,
"pressure": 50
}
]
}
};
const fields = ['device/name', 'device/telemetry/timeStamp', 'device/telemetry/temperature'];
const res = response.device.telemetry.map(obj => {
const o = {};
fields.forEach(item => {
const splittedItem = item.split('/');
o[item] = (splittedItem.length === 2) ? response[splittedItem[0]][splittedItem[1]] : obj[splittedItem[2]];
});
return o;
})
console.log(res);
In what follows I will be concerned with just the implementation and runtime behavior, and not so much the types. I've given things very loose typings like any and string instead of the relevant generic object types. Here goes:
function getNestedFields(data: any, paths: string[]): any[] {
If data is an array, we want to perform getNestedFields() on each element of the array, and then concatenate the results together into one big array. So the first thing we do is check for that and make a recursive call:
if (Array.isArray(data)) return data.flatMap(v => getNestedFields(v, paths));
Now that we know data is not an array, we want to start gathering the pieces of the answer. If paths is, say, ['foo/bar', 'foo/baz/qux', 'x/y', 'x/z'], then we want to make recursive calls to getNestedFields(data.foo, ["bar", "baz/qux"]) and to getNestedFields(data.x, ["y", "z"]). In order to do this we have to split each path element at its first slash "/", and collect the results into a new object whose keys are the part to the left of the slash and whose values are arrays of parts to the right. In this example it would be {foo: ["bar", "baz/qux"], x: ["y", "z"]}.
Some important edge cases: for every element of paths with no slash, then we have a key with an empty value... that is, ["foo"] should result in a call like getNestedFields(data.foo, [""]). And if there is an element of paths that's just the empty string "", then we don't want to do a recursive call; the empty path is the base case and implies that we're asking about data itself. That is, instead of a recursive call, we can just return [{"": data}]. So we need to keep track of the empty path (hence the emptyPathInList variable below).
Here's how it looks:
const pathMappings: Record<string, string[]> = {};
let emptyPathInList = false;
paths.forEach(path => {
if (!path) {
emptyPathInList = true;
} else {
let slashIdx = path.indexOf("/");
if (slashIdx < 0) slashIdx = path.length;
const key = path.substring(0, slashIdx);
const restOfPath = path.substring(slashIdx + 1);
if (!(key in pathMappings)) pathMappings[key] = [];
pathMappings[key].push(restOfPath);
}
})
Now, for each key-value pair in pathMappings (with key key and with value restsOfPath) we need to call getNestedFields() recursively... the results will be an array of objects whose keys are relative to data[key], so we need to prepend key and a slash to their keys. Edge cases: if there's an empty path we shouldn't add a slash. And if data` is nullish then we will have a runtime error recursing down into it, so we might want to do something else there (although a runtime error might be fine since it's a weird input):
const subentries = Object.entries(pathMappings).map(([key, restsOfPath]) =>
(data == null) ? [{}] : // <-- don't recurse down into nullish data
getNestedFields(data[key], restsOfPath)
.map(nestedFields =>
Object.fromEntries(Object.entries(nestedFields)
.map(([path, value]) =>
[key + (path ? "/" : "") + path, value])))
)
Now subentries is an array of all the separate recursive call results, with the proper keys. We want to add one more entry correpsonding to data if emptyPathInList is true:
if (emptyPathInList) subentries.push([{ "": data }]);
And now we need to combine these sub-entries by taking their Cartesian product and spreading into a single object for each entry. By Cartesian product I mean that if subentries looks like [[a,b],[c,d,e],[f]] then I need to get [[a,c,f],[a,d,f],[a,e,f],[b,c,f],[b,d,f],[b,e,f]], and then for each of those we spread into single entries. Here's that:
return subentries.reduce((a, v) => v.flatMap(vi => a.map(ai => ({ ...ai, ...vi }))), [{}])
}
Okay, so let's test it out:
console.log(getNestedFields(response, fields));
/* [{
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-01T00:00:00.000Z",
"device/telemetry/temperature": 100
}, {
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-02T00:00:00.000Z",
"device/telemetry/temperature": 100
}, {
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-03T00:00:00.000Z",
"device/telemetry/temperature": 100
}, {
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-04T00:00:00.000Z",
"device/telemetry/temperature": 100
}, {
"device/name": "Foo",
"device/telemetry/timeStamp": "2022-06-05T00:00:00.000Z",
"device/telemetry/temperature": 100
}] */
That's what you wanted. Even though you said you will never walk into different arrays, this version should support that:
console.log(getNestedFields({
a: [{ b: 1 }, { b: 2 }],
c: [{ d: 3 }, { d: 4 }]
}, ["a/b", "c/d"]))
/* [
{ "a/b": 1, "c/d": 3 },
{ "a/b": 2, "c/d": 3 },
{ "a/b": 1, "c/d": 4 },
{ "a/b": 2, "c/d": 4 }
]*/
There are probably all kinds of crazy edge cases, so anyone using this should test thoroughly.
Playground link to code
Related
I'm stuck on this type of situation where the values of the object is changed to a different value. Is there way to shift a value to a key or would simply deleting and adding be better? I tried to loop to see which of the keys overlap in value and using the if statement and conditions i tried adding or deleting using Array methods. However, since the inter data is an object i am sruggling to find the right methods or even the process. I also tried using a function to insert the data and pushing to a new empty array that is returned from the function.
If I have objects in an array like so:
const data = [
{
"date": "12/22",
"treatment": "nausea",
"count": 2
},
{
"date": "12/23",
"treatment": "cold",
"count": 3
},
{
"date": "12/22",
"treatment": "cold",
"count": 2
}
];
and wanting to change the data like so:
const newData = [
{
"date": "12/22",
"cold": 2
"nausea": 2,
},
{
"date": "12/23",
"cold": 3
}
];
try this code using loop and reduce and every time add to new array
const data = [
{
"date": "12/22",
"treatment": "nausea",
"count": 2
},
{
"date": "12/23",
"treatment": "cold",
"count": 3
},
{
"date": "12/22",
"treatment": "cold",
"count": 2
}
];
const newData = [];
const dataByDate = data.reduce((acc, curr) => {
if (!acc[curr.date]) {
acc[curr.date] = { date: curr.date };
}
acc[curr.date][curr.treatment] = curr.count;
return acc;
}, {});
for (let date in dataByDate) {
newData.push(dataByDate[date]);
}
console.log(newData);
We want to reduce the data by unique dates. This can be done with:
An object as a dictionary,
Set or Map, or
Some other custom implementation.
Prefer to use Array.reduce() when reducing an array. This is standardized and more expressive than a custom implementation.
Using a map-like structure as the accumulator allows reduction of the dates by uniqueness and the data itself, simultaneously.
Note: Properties of objects are converted to Strings (except for Symbols). So if you want to use different "keys" that are equal after conversion (e.g. 0 and "0"), you cannot use objects; use Map instead.
(All our dates are Strings already, so this warning does not apply here.)
When using an object we can use the nullish coalescing assignment ??=: This allows us to assign an initial "empty" entry ({ date: dataEntry.date }) when encountering a new unique date.
Further, that assignment evaluates to the dictionary's entry; the entry that was either already present or just assigned.
Then we only need to assign the treatment and its count as a key-value pair to the entry.
const data = [
{ "date": "12/22", "treatment": "nausea", "count": 2 },
{ "date": "12/23", "treatment": "cold", "count": 3 },
{ "date": "12/22", "treatment": "cold", "count": 2 }
];
const newData = reduceByDate(data);
console.log(newData);
function reduceByDate(data) {
const dataByDate = data.reduce((dict, dataEntry) => {
const dictEntry = dict[dataEntry.date] // Get existing or ...
??= { date: dataEntry.date }; // ... just initialized entry.
dictEntry[dataEntry.treatment] = dataEntry.count;
return dict;
}, {});
// Transform dictionary to array of reduced entries
return Object.values(dataByDate);
}
You can make use of reduce() and Object.assign().
First we use reduce to combine objects with the same date into one object and then use assign to merge the values:
const data = [{
"date": "12/22",
"treatment": "nausea",
"count": 2
},
{
"date": "12/23",
"treatment": "cold",
"count": 3
},
{
"date": "12/22",
"treatment": "cold",
"count": 2
}
];
const newData = data.reduce((acc, curr) => {
const dateIndex = acc.findIndex(item => item.date === curr.date);
if (dateIndex === -1) {
acc.push({
date: curr.date,
[curr.treatment]: curr.count
});
} else {
acc[dateIndex] = Object.assign({}, acc[dateIndex], {
[curr.treatment]: curr.count
});
}
return acc;
}, []);
console.log(newData)
I have an array fetched from our server which holds 2400 objects (total size is about 7MB) and I want to filter some first values in it. Right now I'm using combination of filter and slice method:
const keyword = 'whatever word';
const recommendList =bigArray.filter(item => item.name.includes(keyword)).slice(0, 5);
What I know is filter method iterates all the element in array and I think it can impact to performance of my app (React Native) cause its large data. So is there any approach to filter the array for some values, without iterating all the elements ?
If you simply want to to break(stop) the loop when you find 5th element then you can do the bellow:
const keyword = 'v';
const bigArray = ['a','v','a','v','a','v','a','v','a','v','a','v','a','v','a','v','a','v'];
const recommendList = [];
for (let i=0; i<bigArray.length; i++) { // loop till you reach end of big array index
if (recommendList.length == 5) // if length is 5 this will break the loop
{
break;
}
if (bigArray[i].includes(keyword)) {
recommendList.push(bigArray[i]); // add if you find
}
}
console.log(recommendList);
If you dont want to use lambda operation can simply use, some, find etc which only works till they return the first response as true
const bigArray = [{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
},
{
"name": "a"
},
{
"name": "v"
}
];
const keyword = 'v';
const recommendList = [];
// some operator only iterates till its condition returns true
// so if we get 5 recommended list before the bigArray end we return true and stop the iteration.
bigArray.some(obj => {
if (obj.name.includes(keyword)) {
recommendList.push(obj)
}
return recommendList.length === 5; // return true if 5 values are found, that will terminate the iteration
})
console.log(recommendList);
const keyword = 'whatever word';
const recommendList = [];
for (let i=0; i<bigArray.length; i++) {
if ( recommendList.length >= 5)
break;
const item = bigArray[i];
if (item.name.includes(keyword))
recommendList.push(item);
}
The most efficient approach is to process the array as an iterable sequence.
Example below is based on iter-ops library:
import {pipe, filter, take} from 'iter-ops';
const i = pipe(
bigArray,
filter(item => item.name.includes(keyword)),
take(5)
);
console.log('matches:', [...i]);
This way, you won't be iterating through everything even once, it will stop just as the first 5 matches are found.
In this condition, I think filter (the complexity is O(n)) should be the best solution that lead to minimized performance impact.
Think of that, if you just filter some of the values among those 2400 objects, say 1500. Then you would only get filtered results of 1500 objects, and the rest 900 objects would never be used. So at least one loop is necessary.
I am translating some code from python into Javascript (I am inexperienced in JS), as part of it I need to write a JSON, currently when writing in JS it looks like this(just a brief sample):
[
{
"Num": "000000",
"Status": 1,
},
{
"Num": "00001",
"Status": 0,
},
]
However I need it to look like this:
{
"mydata": [
{
"Num": "00000",
"Status": 1,
},
{
"Num": "00001",
"Status": 0,
},
]
}
How can I adapt my code to generate this single main key for the whole JSON, here is what I have so far:
var jsondata = []
for (result in results) {
jsondata.push({
'Num' : idnum,
'Status': results[result].StatusID,
})
}
let data = JSON.stringify(jsondata, null, 2)
fs.writeFileSync('testingjson.json', data)
This code here sits in a for loop, so I cannot just write the key in the push, it would generate the mydata key for every iteration of the loop.
I need to know how I can pre-define the mydata, has anyone got a good method to do this?
Just define mydata as an array, and then at the end, create an object where mydata is one of the keys:
const mydata = []
for (const result in results) {
mydata.push({
'Num' : idnum,
'Status': results[result].StatusID,
});
}
const json = JSON.stringify({ mydata }, null, 2);
If you want to use a different key name, then change it when stringifying, eg:
const json = JSON.stringify({ someKeyName: mydata }, null, 2);
If results is an object, then you can make the code more concise with Object.values and map:
const Num = idnum;
const mydata = Object.values(results)
.map((item => ({ Num, Status: item.StatusID }));
const json = JSON.stringify({ mydata }, null, 2);
fs.writeFileSync('testingjson.json', data)
I've got two arrays that have multiple objects
[
{
"name":"paul",
"employee_id":"8"
}
]
[
{
"years_at_school": 6,
"department":"Mathematics",
"e_id":"8"
}
]
How can I achieve the following with either ES6 or Lodash?
[
{
"name":"paul",
"employee_id":"8"
"data": {
"years_at_school": 6
"department":"Mathematics",
"e_id":"8"
}
}
]
I can merge but I'm not sure how to create a new child object and merge that in.
Code I've tried:
school_data = _.map(array1, function(obj) {
return _.merge(obj, _.find(array2, {employee_id: obj.e_id}))
})
This merges to a top level array like so (which is not what I want):
{
"name":"paul",
"employee_id":"8"
"years_at_school": 6
"department":"Mathematics",
"e_id":"8"
}
The connector between these two is "employee_id" and "e_id".
It's imperative that it's taken into account that they could be 1000 objects in each array, and that the only way to match these objects up is by "employee_id" and "e_id".
In order to match up employee_id and e_id you should iterate through the first array and create an object keyed to employee_id. Then you can iterate though the second array and add the data to the particular id in question. Here's an example with an extra item added to each array:
let arr1 = [
{
"name":"mark",
"employee_id":"6"
},
{
"name":"paul",
"employee_id":"8"
}
]
let arr2 = [
{
"years_at_school": 6,
"department":"Mathematics",
"e_id":"8"
},
{
"years_at_school": 12,
"department":"Arr",
"e_id":"6"
}
]
// empObj will be keyed to item.employee_id
let empObj = arr1.reduce((obj, item) => {
obj[item.employee_id] = item
return obj
}, {})
// now lookup up id and add data for each object in arr2
arr2.forEach(item=>
empObj[item.e_id].data = item
)
// The values of the object will be an array of your data
let merged = Object.values(empObj)
console.log(merged)
If you perform two nested O(n) loops (map+find), you'll end up with O(n^2) performance. A typical alternative is to create intermediate indexed structures so the whole thing is O(n). A functional approach with lodash:
const _ = require('lodash');
const dataByEmployeeId = _(array2).keyBy('e_id');
const result = array1.map(o => ({...o, data: dataByEmployeeId.get(o.employee_id)}));
Hope this help you:
var mainData = [{
name: "paul",
employee_id: "8"
}];
var secondaryData = [{
years_at_school: 6,
department: "Mathematics",
e_id: "8"
}];
var finalData = mainData.map(function(person, index) {
person.data = secondaryData[index];
return person;
});
Sorry, I've also fixed a missing coma in the second object and changed some other stuff.
With latest Ecmascript versions:
const mainData = [{
name: "paul",
employee_id: "8"
}];
const secondaryData = [{
years_at_school: 6,
department: "Mathematics",
e_id: "8"
}];
// Be careful with spread operator over objects.. it lacks of browser support yet! ..but works fine on latest Chrome version for example (69.0)
const finalData = mainData.map((person, index) => ({ ...person, data: secondaryData[index] }));
Your question suggests that both arrays will always have the same size. It also suggests that you want to put the contents of array2 within the field data of the elements with the same index in array1. If those assumptions are correct, then:
// Array that will receive the extra data
const teachers = [
{ name: "Paul", employee_id: 8 },
{ name: "Mariah", employee_id: 10 }
];
// Array with the additional data
const extraData = [
{ years_at_school: 6, department: "Mathematics", e_id: 8 },
{ years_at_school: 8, department: "Biology", e_id: 10 },
];
// Array.map will iterate through all indices, and gives both the
const merged = teachers.map((teacher, index) => Object.assign({ data: extraData[index] }, teacher));
However, if you want the data to be added to the employee with an "id" matching in both arrays, you need to do the following:
// Create a function to obtain the employee from an ID
const findEmployee = id => extraData.filter(entry => entry.e_id == id);
merged = teachers.map(teacher => {
const employeeData = findEmployee(teacher.employee_id);
if (employeeData.length === 0) {
// Employee not found
throw new Error("Data inconsistency");
}
if (employeeData.length > 1) {
// More than one employee found
throw new Error("Data inconsistency");
}
return Object.assign({ data: employeeData[0] }, teacher);
});
A slightly different approach just using vanilla js map with a loop to match the employee ids and add the data from the second array to the matching object from the first array. My guess is that the answer from #MarkMeyer is probably faster.
const arr1 = [{ "name": "paul", "employee_id": "8" }];
const arr2 = [{ "years_at_school": 6, "department": "Mathematics", "e_id": "8" }];
const results = arr1.map((obj1) => {
for (const obj2 of arr2) {
if (obj2.e_id === obj1.employee_id) {
obj1.data = obj2;
break;
}
}
return obj1;
});
console.log(results);
Please check this fiddle example
How can I sort an orderedMap object by the keys sequence in a descending order? I know there is a sortBy method but the document doesn't give a clearer example.
Here's my original object:
var props = {
"item": {
"A": {
"sequence": 5
},
"B": {
"sequence": null
},
"C":{
"sequence": 2
}
}
}
I'd like the result to be like:
var props = {
"item": {
"A": {
"sequence": 5
},
"C":{
"sequence": 2
},
"B": {
"sequence": null
}
}
}
Example Code:
var data = Immutable.fromJS(props,(key, value)=>{
var isIndexed = Immutable.Iterable.isIndexed(value);
return isIndexed ? value.toList() :value.toOrderedMap();
});
var sorted = data.get('item').sortBy((item) => item.get('sequence'));
console.log(sorted.toJS())
The function you are providing to sortBy() is comparator and it needs to return a Number. By adding a minus in front of the item.get('sequence') or following the result with reverse() you will reverse the sort, getting you the output that you want.
Here's an example:
var sorted = data.get('item').sortBy((item) => -item.get('sequence'));
// or var sorted = data.get('item').sortBy((item) => item.get('sequence')).reverse();
// you can now use .map() to go through your OrderedMap which has been
// sortbed by the sequence property and would look like your desired output
Note that you are working with an OrderedMap and that simply calling sorted.toJS() would return a regular JS object where the keys are, obviously, not sorted.
// If you need to iterate through the map and return some
// plain JS you can do something like:
sorted.map(x => x.get('sequence')).toJS()
// => Object {A: 5, C: 2, B: null}