I'm about to show the data I aggregated from the database in a grouped histogram.
The data looks like this:
[
{
"_id": "Gas Separator Filter",
"sellingPrice": 100000,
"quantity": 10
},
{
"_id": "Dry Gas Cartridge",
"sellingPrice": 6005000,
"quantity": 15
}
]
But in order to show it in the chart and for them to be grouped, I need something like this. For each _id in the dataset above I should be able to see two bars in the chart.
[
{
"name": "quantity",
"Gas Separator Filter": 10,
"Dry Gas Cartridge": 15
},
{
"name": "sellingPrice",
"Gas Separator Filter": 100000,
"Dry Gas Cartridge": 6005000
}
]
It's been two hours and I'm not able to think of a good way to do this. What will you suggest?
Here is one solution using old school for loops :)
const transform = (data, nameValues, keyProp) => {
const result = [];
for (const name of nameValues) {
const output = { name };
for (const value of data) {
output[value[keyProp]] = value[name];
}
result.push(output);
}
return result;
};
const result = transform(
[
{
"_id": "Gas Separator Filter",
"sellingPrice": 100000,
"quantity": 10
},
{
"_id": "Dry Gas Cartridge",
"sellingPrice": 6005000,
"quantity": 15
}
],
["sellingPrice", "quantity"],
"_id"
);
console.log(result);
You can use array.reduce to achieve this:
const arrayToArray = (array) => {
var ret = [{
"name": "price"
}, {
"name": "quantity"
}
];
return array.reduce((obj, item, idx, original) => {
ret[0][item._id] = original[idx].sellingPrice;
ret[1][item._id] = original[idx].quantity;
return ret;
}, 0)
}
Like this you set a variable with the base object that you fullfill with couple _id:value for price and quantity.
But is not an "elegant" way to do this. Are you sure you need this objects array structure to show the chart?
I find it hard to explain my solution but here's my take on it (you can add console.logs for different variables to follow the transformation ):
extract all the keys in the object, loop through them, the name would be that key, and use a nested loop to set the other keys and values :
const data = [ { "_id": "Gas Separator Filter", "sellingPrice": 100000, "quantity": 10 }, { "_id": "Dry Gas Cartridge", "sellingPrice": 6005000, "quantity": 15 } ]
const [_id, ...keys] = [...new Set(data.map(e => Object.keys(e)).flat())]
// console.log(keys) : ["sellingPrice", "quantity"]
const result = keys.map(key => {
const values = data.map(obj => ({
[obj._id]: obj[key]
}))
return Object.assign({
name: key
}, ...values);
})
console.log(result)
Related
I'm stuck on this type of situation where the values of the object is changed to a different value. Is there way to shift a value to a key or would simply deleting and adding be better? I tried to loop to see which of the keys overlap in value and using the if statement and conditions i tried adding or deleting using Array methods. However, since the inter data is an object i am sruggling to find the right methods or even the process. I also tried using a function to insert the data and pushing to a new empty array that is returned from the function.
If I have objects in an array like so:
const data = [
{
"date": "12/22",
"treatment": "nausea",
"count": 2
},
{
"date": "12/23",
"treatment": "cold",
"count": 3
},
{
"date": "12/22",
"treatment": "cold",
"count": 2
}
];
and wanting to change the data like so:
const newData = [
{
"date": "12/22",
"cold": 2
"nausea": 2,
},
{
"date": "12/23",
"cold": 3
}
];
try this code using loop and reduce and every time add to new array
const data = [
{
"date": "12/22",
"treatment": "nausea",
"count": 2
},
{
"date": "12/23",
"treatment": "cold",
"count": 3
},
{
"date": "12/22",
"treatment": "cold",
"count": 2
}
];
const newData = [];
const dataByDate = data.reduce((acc, curr) => {
if (!acc[curr.date]) {
acc[curr.date] = { date: curr.date };
}
acc[curr.date][curr.treatment] = curr.count;
return acc;
}, {});
for (let date in dataByDate) {
newData.push(dataByDate[date]);
}
console.log(newData);
We want to reduce the data by unique dates. This can be done with:
An object as a dictionary,
Set or Map, or
Some other custom implementation.
Prefer to use Array.reduce() when reducing an array. This is standardized and more expressive than a custom implementation.
Using a map-like structure as the accumulator allows reduction of the dates by uniqueness and the data itself, simultaneously.
Note: Properties of objects are converted to Strings (except for Symbols). So if you want to use different "keys" that are equal after conversion (e.g. 0 and "0"), you cannot use objects; use Map instead.
(All our dates are Strings already, so this warning does not apply here.)
When using an object we can use the nullish coalescing assignment ??=: This allows us to assign an initial "empty" entry ({ date: dataEntry.date }) when encountering a new unique date.
Further, that assignment evaluates to the dictionary's entry; the entry that was either already present or just assigned.
Then we only need to assign the treatment and its count as a key-value pair to the entry.
const data = [
{ "date": "12/22", "treatment": "nausea", "count": 2 },
{ "date": "12/23", "treatment": "cold", "count": 3 },
{ "date": "12/22", "treatment": "cold", "count": 2 }
];
const newData = reduceByDate(data);
console.log(newData);
function reduceByDate(data) {
const dataByDate = data.reduce((dict, dataEntry) => {
const dictEntry = dict[dataEntry.date] // Get existing or ...
??= { date: dataEntry.date }; // ... just initialized entry.
dictEntry[dataEntry.treatment] = dataEntry.count;
return dict;
}, {});
// Transform dictionary to array of reduced entries
return Object.values(dataByDate);
}
You can make use of reduce() and Object.assign().
First we use reduce to combine objects with the same date into one object and then use assign to merge the values:
const data = [{
"date": "12/22",
"treatment": "nausea",
"count": 2
},
{
"date": "12/23",
"treatment": "cold",
"count": 3
},
{
"date": "12/22",
"treatment": "cold",
"count": 2
}
];
const newData = data.reduce((acc, curr) => {
const dateIndex = acc.findIndex(item => item.date === curr.date);
if (dateIndex === -1) {
acc.push({
date: curr.date,
[curr.treatment]: curr.count
});
} else {
acc[dateIndex] = Object.assign({}, acc[dateIndex], {
[curr.treatment]: curr.count
});
}
return acc;
}, []);
console.log(newData)
I have two JSON files: JSON A has some company properties and the company_id, while JSON B has company names and company ids.
JSON A example:
[
{
"order_name": "Foo",
"company_id": "112233"
},
{
"order_name": "Bar",
"company_id": "123456"
}
]
JSONB example:
[
{
"company_id":"112233",
"name":"ACME company",
},
{
"company_id":"123456",
"name":"John Doe Inc.",
}
]
Which is the most efficient way to do a join by the company_id values? I would like to have the JSON C (merged result) with the company names correctly added, like this:
[
{
"order_name": "Foo",
"company_id": "123456",
"company_name": "John Doe Inc."
},
{
"order_name": "Bar",
"company_id": "112233",
"company_name": "ACME company"
}
]
Is looping and filter for each the only solution? Is there a more efficient way to do this from a performance point of view?
More info:
JSON is not sorted by company_id.
Array A could have more than one object with the same company_id
I'm using Javascript (in a Vue.js app), I don't need to support old browsers
In common modern JavaScript, you can do this as you mentioned with higher-order functions like map, filter, and so on:
const arrayA = [
{
"order_name": "Foo",
"company_id": "112233"
},
{
"order_name": "Bar",
"company_id": "123456"
}
]
const arrayB = [
{
"company_id":"112233",
"name":"ACME company",
},
{
"company_id":"123456",
"name":"John Doe Inc.",
}
]
const mergeAB = arrayA.map( companyA => {
const matched = arrayB.find(companyB => companyB.company_id === companyA.company_id)
if(matched) {
return {...companyA, ...matched}
} else {
// return companyA element or customize it with your case
}
}
)
console.log(mergeAB)
Note 1: Array.find() method complexity is O(n) and Array.map() method complexity is O(n)
Note 2: efficiency is an important thing but not in all situations. sometimes you need to do these types of iteration one time or for a small array size, so no need to worry about the performance.
Note 3: you could compare the answer and find out your best solution since we don't know about your whole code and application.
I hope this will work for you. Let me know if you have any questions.
const arrayOne = [
{
"order_name": "Foo",
"company_id": "112233"
},
{
"order_name": "Bar",
"company_id": "123456"
}
];
const arrayTwo = [
{
"company_id":"112233",
"name":"ACME company",
},
{
"company_id":"123456",
"name":"John Doe Inc.",
}
];
const [source, target] = arrayOne.length > arrayTwo.length
? [arrayOne, arrayTwo]
: [arrayTwo, arrayOne];
const merged = source.map(object =>
{
// Assuming that in the 2nd array, the match is only found 1 time and it EXISTS.
const matched = target.find(element => element.company_id === object.company_id);
// Merge both objects together
return {
...object,
...matched
};
});
console.log(merged);
By having JSONs:
const jsonA = [
{
"order_name": "Foo",
"company_id": "112233"
},
{
"order_name": "Bar",
"company_id": "123456"
}
];
const jsonB = [
{
"company_id":"112233",
"name":"ACME company",
},
{
"company_id":"123456",
"name":"John Doe Inc.",
}
];
you can merge maps into 3rd map with something like this:
const transform = (data, current={}) =>
data.reduce((prev, company) => {
if(!prev[company['company_id']]) prev[company['company_id']] = {};
prev[company['company_id']] = {...prev[company['company_id']], ...company}
return prev;
}, current);
let jsonMap = transform(jsonA, {});
jsonMap = transform(jsonB, jsonMap);
let jsonC = Object.keys(jsonMap).map(companyId => jsonMap[companyId] );
console.log(jsonC);
I need a way to match the closest number of an elasticsearch document.
I'm wanting to use elastic search to filter quantifiable attributes and have been able to achieve hard limits using range queries accept that results that are outside of that result set are skipped. I would prefer to have the closest results to multiple filters match.
const query = {
query: {
bool: {
should: [
{
range: {
gte: 5,
lte: 15
}
},
{
range: {
gte: 1979,
lte: 1989
}
}
]
}
}
}
const results = await client.search({
index: 'test',
body: query
})
Say I had some documents that had year and sales. In the snippet is a little example of how it would be done in javascript. It runs through the entire list and calculates a score, then based on that score it sorts them, at no point are results filtered out, they are just organized by relevance.
const data = [
{ "item": "one", "year": 1980, "sales": 20 },
{ "item": "two", "year": 1982, "sales": 12 },
{ "item": "three", "year": 1986, "sales": 6 },
{ "item": "four", "year": 1989, "sales": 4 },
{ "item": "five", "year": 1991, "sales": 6 }
]
const add = (a, b) => a + b
const findClosestMatch = (filters, data) => {
const scored = data.map(item => ({
...item,
// add the score to a copy of the data
_score: calculateDifferenceScore(filters, item)
}))
// mutate the scored array by sorting it
scored.sort((a, b) => a._score.total - b._score.total)
return scored
}
const calculateDifferenceScore = (filters, item) => {
const result = Object.keys(filters).reduce((acc, x) => ({
...acc,
// calculate the absolute difference between the filter and data point
[x]: Math.abs(filters[x] - item[x])
}), {})
// sum the total diffences
result.total = Object.values(result).reduce(add)
return result
}
console.log(
findClosestMatch({ sales: 10, year: 1984 }, data)
)
<script src="https://codepen.io/synthet1c/pen/KyQQmL.js"></script>
I'm trying to achieve the same thing in elasticsearch but having no luck when using a function_score query. eg
const query = {
query: {
function_score: {
functions: [
{
linear: {
"year": {
origin: 1984,
},
"sales": {
origin: 10,
}
}
}
]
}
}
}
const results = await client.search({
index: 'test',
body: query
})
There is no text to search, I'm using it for filtering by numbers only, am I doing something wrong or is this not what elastic search is made for and are there any better alternatives?
Using the above every document still has a default score, and I have not been able to get any filter to apply any modifiers to the score.
Thanks for any help, I new to elasticsearch links to articles or areas of the documentation are appreciated!
You had the right idea, you're just missing a few fields in your query to make it work.
It should look like this:
{
"query": {
function_score: {
functions: [
{
linear: {
"year": {
origin: 1984,
scale: 1,
decay: 0.999
},
"sales": {
origin: 10,
scale: 1,
decay: 0.999
}
}
},
]
}
}
}
The scale field is mandatory as it tells elastic how to decay the score, without it the query just fails.
The decay field is not mandatory, however without it elastic does not really know how to calculate the new score to documents so it will end up giving a default score only to documents in the range of origin + scale which is not useful for us.
source docs.
I also recommend you limit the result size to 1 if you want the top scoring document, otherwise you'll have add a sort phase (either in elastic or in code).
EDIT: (AVOID NULLS)
You can add a filter above the functions like so:
{
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{
"bool": {
"filter": [
{
"bool": {
"must": [
{
"exists": {
"field": "year"
}
},
{
"exists": {
"field": "sales"
}
},
]
}
}
]
}
},
{
"match_all": {}
}
]
}
},
"functions": [
{
"linear": {
"year": {
"origin": 1999,
"scale": 1,
"decay": 0.999
},
"sales": {
"origin": 50,
"scale": 1,
"decay": 0.999
}
}
}
]
}
}
}
Notice i have a little hack going on using match_all query, this is due to filter query setting the score to 0 so by using the match all query i reset it back to 1 for all matched documents.
This can also be achieved in a more "proper" way by altering the functions, a path i choose not to take.
I want to add more key... value pairs to each of the objects. Is it possible to do that?
Right now I have objects that look like:
{"year":2014,"num":115.5}
{"year":2016,"num":0.0}
{"year":2017,"num":8.28}
{"year":2018,"num":0.0}
I have an array of colors:
let colors = ['#42d4f4','#e6194B','#3cb44b','#911eb4'];
I want to now add these colors to my objects.
I want to make it look like so:
{"year":2014,"num":115.5, "colors": '#42d4f4'}
{"year":2016,"num":0.0, "colors": '#e6194B'}
{"year":2017,"num":8.28, "colors": '#3cb44b'}
{"year":2018,"num":0.0, "colors": '#911eb4'}
Is there a way for me to do that without writing many many if's?
You can use map to loop thru the array. Use spread syntax to shallow copy the object and add the colors property using the index.
let arr = [{
"year": 2014,
"num": 115.5
}, {
"year": 2016,
"num": 0.0
}, {
"year": 2017,
"num": 8.28
}, {
"year": 2018,
"num": 0.0
}];
let colors = ['#42d4f4', '#e6194B', '#3cb44b', '#911eb4'];
let result = arr.map((o, i) => ({ ...o, colors: colors[i] || null }));
console.log(result);
If you want to update the existing variable, you can use forEach
let arr = [{
"year": 2014,
"num": 115.5
}, {
"year": 2016,
"num": 0.0
}, {
"year": 2017,
"num": 8.28
}, {
"year": 2018,
"num": 0.0
}];
let colors = ['#42d4f4', '#e6194B', '#3cb44b', '#911eb4'];
arr.forEach((o, i) => o.colors = colors[i] || null);
console.log(arr);
You can map over the array and return all of the objects with a new key color.
let data =[
{"year":2014,"num":115.5},
{"year":2016,"num":0.0},
{"year":2017,"num":8.28},
{"year":2018,"num":0.0}
];
let colors = ['#42d4f4','#e6194B','#3cb44b','#911eb4'];
let newData = colors.map((color, index) => ({...data[index], color}));
console.log(newData);
If you're not familiar with ES6 syntax, this is the same as:
let data =[
{"year":2014,"num":115.5},
{"year":2016,"num":0.0},
{"year":2017,"num":8.28},
{"year":2018,"num":0.0}
];
let colors = ['#42d4f4', '#e6194B', '#3cb44b', '#911eb4'];
let newData = colors.map(function(colorHex, index) {
return {
year: data[index].year,
num: data[index].num,
color: colorHex
}
})
console.log(newData) // [{"year":2014,"num":115.5, "colors": '#42d4f4'}, ...]
Basically, I have an array with objects and they would need to be grouped together. It is kinda hard to explain, but it might be easier if I just gave you guys an example.
Result data
[
{
"Category": "Préparé",
"Sandwich": "Martino",
"Ingredient": "Ansjovis",
"Price": 3.1
},
{
"Category": "Préparé",
"Sandwich": "Martino",
"Ingredient": "Tabasco",
"Price": 3.1
},
{
"Category": "Veggie",
"Sandwich": "Gezond",
"Ingredient": "Tomaat",
"Price": 2.5
},
{
"Category": "Veggie",
"Sandwich": "Gezond",
"Ingredient": "Kaas",
"Price": 2.5
}
];
This is a basic example of what my array looks like. I cannot change this structure, it is how our API provides the data.
What I actually need is this structure:
[
{
"CategoryName": "Prépare",
"Sandwiches": [
{
"SandwichName": "Martino",
"Price": 3.1,
"Ingredients": ["Ansjovis", "Tabasco"]
}
]
},
{
"CategoryName": "Veggie",
"Sandwiches": [
{
"SandwichName": "Gezond",
"Price": 2.5,
"Ingredients": ["Tomaat", "Kaas"]
}
]
}
]
I have tried some stuff with Underscore and _.groupBy, _.sortBy, _.countBy
But alas, nothing I have tried actually works. Is this even possible with Underscore (or some other library)?
Also on a sidenote, this example might have some JSON structure mistakes, because I wrote it myself. The data provided by the API has a correct format.
The example only has 2 sandwiches, but in real-time, I'll be retrieving multiple categories with each 20 sandwiches and so on. This is just a minified example, but it provides an idea of what I need.
try this in simple js
var map = {};
results.forEach( function(obj){
map[ obj.CategoryName ] = map[ obj.CategoryName ] || [];
map[ obj.CategoryName ].push( obj );
});
var output = Object.keys(map).map( function(key){
var arr = [];
map[key].forEach(function(obj){
arr.push( {
"SandwichName": obj.SandwichName,
"Price": obj.Price,
"Ingredients": obj.Ingredients
});
});
return { "CategoryName" : key , "Sandwiches" : arr };
});
The following piece of code would do the trick for you:
var data = [...]; // this is your json-data
var result = _.map(_.uniq(_.pluck(data, 'Category')), function(category) {
var sandwiches = _.uniq(_.pluck(_.where(data, { Category: category }), 'Sandwich'));
return {
CategoryName: category,
Sandwiches: _.map(sandwiches, function(sandwich) {
return {
SandwitchName: sandwich,
Price: _.findWhere(data, { Category: category, Sandwich: sandwich }).Price,
Ingredients: _.pluck(_.where(data, { Category: category, Sandwich: sandwich }), 'Ingredient')
};
})
};
});