We Keep Coding

ES6 reduce array of objects with conditions

I'm stucked in a (in my opinion) complex reduce method.
Given is an array of objects.
const data =
[
{
"key" : "test1",
"value" : 32,
"type" : "OUT"
},
{
"key" : "test1",
"value" : 16,
"type" : "OUT"
},
{
"key" : "test1",
"value" : 8,
"type" : "IN"
},
{
"key" : "test2",
"value" : 32,
"type" : "OUT"
},
{
"key" : "test2",
"value" : 16,
"type" : "IN"
},
{
"key" : "test2",
"value" : 8,
"type" : "OUT"
},
];
I want to get the sum of values of each object grouped by key attribute. There are two type attributes (IN, OUT) where OUT should be interpreted as negative value.
So in the example above, I'm expecting following result object:
//-32 - 16 + 8 = -40
{
"key" : "test1",
"value" : -40,
"type" : "-"
},
//-32 + 16 - 8 = -24
{
"key" : "test2",
"value" : -24,
"type" : "-"
},
I'm grouping the data using the groupBy function of this SO answer.
Now I'm trying to get the sum using reduce with a filter, like in this SO answer.
However, it delivers me the wrong sums (16 and 8) + since I use filter - only one type is considered.
Here is my code:
const data =
[
{
"key" : "test1",
"value" : 32,
"type" : "OUT"
},
{
"key" : "test1",
"value" : 16,
"type" : "OUT"
},
{
"key" : "test1",
"value" : 8,
"type" : "IN"
},
{
"key" : "test2",
"value" : 32,
"type" : "OUT"
},
{
"key" : "test2",
"value" : 16,
"type" : "IN"
},
{
"key" : "test2",
"value" : 8,
"type" : "OUT"
},
];
//group by key
const groupBy = function(xs, key) {
return xs.reduce(function(rv, x) {
(rv[x[key]] = rv[x[key]] || []).push(x);
return rv;
}, {});
};
const grouped = groupBy(data,"key");
for (const [key, value] of Object.entries(grouped))
{
let x = value.filter(({type}) => type === 'OUT')
.reduce((sum, record) => sum + record.value)
console.log(x);
}
//const filtered = grouped.filter(({type}) => type === 'OUT');
console.log(Object.values(grouped));
Question 1:
Why does the reduce give me the wrong sum for type OUT?
Question 2:
Is there a way to consider both types (IN, OUT) without doing the same procedure again?

You can combine the grouping + counting in 1 reduce() if you set the default value to 0, you can always add (or remove) the value from the current key (type)
const data = [{"key" : "test1", "value" : 32, "type" : "OUT"}, {"key" : "test1", "value" : 16, "type" : "OUT"}, {"key" : "test1", "value" : 8, "type" : "IN"}, {"key" : "test2", "value" : 32, "type" : "OUT"}, {"key" : "test2", "value" : 16, "type" : "IN"}, {"key" : "test2", "value" : 8, "type" : "OUT"}, ];
const res = data.reduce((p, c) => {
(p[c['key']] = p[c['key']] || { ...c, value: 0 });
p[c['key']].value =
(c.type === 'IN')
? (p[c['key']].value + c.value)
: (p[c['key']].value - c.value);
return p;
},{});
console.log(res)
Output:
{
"test1": {
"key": "test1",
"value": -40,
"type": "OUT"
},
"test2": {
"key": "test2",
"value": -24,
"type": "OUT"
}
}

I would break this into two problems:
How to reduce each data value (reduce)
How to evaluate existing/new values (switch)
This way your code is less-coupled and it affords you with greater extensibility. Adding a new operator is as simple as adding a new case in the switch.
const reduceValue = (type, existingValue, newValue) => {
switch (type) {
case 'IN' : return existingValue + newValue;
case 'OUT' : return existingValue - newValue;
default : return existingValue; // or throw new Error(`Unsupported type: ${type}`)
}
};
const processValues = (data) =>
data.reduce((acc, { key, type, value }) => {
acc[key] ??= { key, type: '-', value: 0 };
acc[key].value = reduceValue(type, acc[key].value, value);
return acc;
},{});
const testData = [
{ "key" : "test1", "value" : 32, "type" : "OUT" },
{ "key" : "test1", "value" : 16, "type" : "OUT" },
{ "key" : "test1", "value" : 8, "type" : "IN" },
{ "key" : "test2", "value" : 32, "type" : "OUT" },
{ "key" : "test2", "value" : 16, "type" : "IN" },
{ "key" : "test2", "value" : 8, "type" : "OUT" }
];
console.log(processValues(testData))
.as-console-wrapper { top: 0; max-height: 100% !important; }

I would create 2 functions for applying the sign and store them in a variable.
const applySign = { "IN": nr => +nr, "OUT": nr => -nr };
Then do a simple for...of loop (with object destructuring). If there is no running total at the moment for the current key, set the initial value to 0 (using nullish coalescing assignment ??=). Finally add the current value with applied sign to the running total.
const sums = {};
for (const { key, value, type } of data) {
sums[key] ??= 0;
sums[key] += applySign[type](value);
}
const data = [
{ key: "test1", value: 32, type: "OUT" },
{ key: "test1", value: 16, type: "OUT" },
{ key: "test1", value: 8, type: "IN" },
{ key: "test2", value: 32, type: "OUT" },
{ key: "test2", value: 16, type: "IN" },
{ key: "test2", value: 8, type: "OUT" },
];
const applySign = { "IN": nr => +nr, "OUT": nr => -nr };
const sums = {};
for (const { key, value, type } of data) {
sums[key] ??= 0;
sums[key] += applySign[type](value);
}
console.log(sums);
With a few simple tweaks you can change the above in the output you're looking for:
const sums = {};
for (const { key, value, type } of data) {
sums[key] ??= { key, value: 0 };
sums[key].value += applySign[type](value);
}
const expected = Object.values(sums);
This gives you the base answer, though the type properties that you expect are currently missing. To add them you'll have to do another loop and check the final sum result.
for (const sum of expected) {
sum.type = sum.value < 0 ? "-" : "+";
}
const data = [
{ key: "test1", value: 32, type: "OUT" },
{ key: "test1", value: 16, type: "OUT" },
{ key: "test1", value: 8, type: "IN" },
{ key: "test2", value: 32, type: "OUT" },
{ key: "test2", value: 16, type: "IN" },
{ key: "test2", value: 8, type: "OUT" },
];
const applySign = { "IN": nr => +nr, "OUT": nr => -nr };
const sums = {};
for (const { key, value, type } of data) {
sums[key] ??= { key, value: 0 };
sums[key].value += applySign[type](value);
}
const expected = Object.values(sums);
console.log(expected);
// add type based on the value sign (don't know why)
for (const sum of expected) {
sum.type = sum.value < 0 ? "-" : "+";
}
console.log(expected);
If type is a static "-" and was not supposed to depend on the sign of value, then you can add it when you initially create the sum object.
sums[key] ??= { key, value: 0, type: "-" };

Normalize values for all keys in nested dictionaries and append as new key/value pairs

Say I have a dictionary with nested sub-dictionaries:
let dict =
{
"SEATTLE" : {
"gross_sales" : 106766,
"price" : 584.50,
"dates" : [ {
"date" : "2020-03-13",
"total_sales_to_date" : 2,
"new_sales" : 2,
}
, {
"date" : "2020-03-19",
"total_sales_to_date" : 5,
"new_sales" : 3,
}
]
}
,
"PHOENIX" : {
"gross_sales" : 26691.5,
"price" : 292.25,
"dates" : [ {
"date" : "2020-03-13",
"total_sales_to_date" : 9,
"new_sales" : 9,
}
, {
"date" : "2020-03-19",
"total_sales_to_date" : 19,
"new_sales" : 10,
}
]
}
}
And I would like to normalise each numerical value in the key/value pairs against the other key/values and then append these as new key/value pairs.
For the dates array of time-series data I'd like to normalise each key/value pair in each date against both time (within the array) and against the other locations on the same date (other objects).
For example, this is what I'm seeking after the operation:
{
"SEATTLE" : {
"gross_sales" : 106766,
"normalised_gross_sales" : 1.0,
"price" : 584.50,
"normalised_price" : 1.0,
"dates" : [ {
"date" : "2020-03-13",
"total_sales_to_date" : 2,
"norm_total_sales_over_time" : 0.4,
"norm_total_sales_over_locations" : 0.22222222,
"new_sales" : 2,
}
, {
"date" : "2020-03-19",
"total_sales_to_date" : 5,
"norm_total_sales_over_time" : 1.0,
"norm_total_sales_over_locations" : 0.26315789,
"new_sales" : 3,
}
]
}
,
"PHOENIX" : {
"gross_sales" : 26691.5,
"normalised_gross_sales" : 0.25,
"price" : 292.25,
"normalised_price" : 0.5,
"dates" : [ {
"date" : "2020-03-13",
"total_sales_to_date" : 9,
"norm_total_sales_over_time" : 0.47368421,
"norm_total_sales_over_locations" : 1.0,
"new_sales" : 9,
}
, {
"date" : "2020-03-19",
"total_sales_to_date" : 19,
"norm_total_sales_over_time" : 1.0,
"norm_total_sales_over_locations" : 1.0,
"new_sales" : 10,
}
]
}
}
ie: the total_sales_to_date value for the last date in the array should normalise to 1.0 as norm_total_sales_over_time
and the largest total_sales_to_date value for all objects (SEATTLE, PHOENIX) for the current date in the array should normalise to 1.0 as norm_total_sales_over_locations
I'm finding this very complex to handle in JS. My actual task involves dictionaries with hundreds of sub-dictionaries I need to compare, I'm looking for a scalable solution. In a pandas dataframe this would be trivial, however I'd like to learn how to approach this using modern javascript only as I'm running this process from node.js using an ES6 interpreter.
What is an effective ES6 javascript solution to this?

Here is a solution that returns the normalised values in the manner described:
let dict = {
"SEATTLE": {
"gross_sales": 106766,
"price": 584.50,
"dates": [{
"date": "2020-03-13",
"total_sales_to_date": 2,
"new_sales": 2,
}, {
"date": "2020-03-19",
"total_sales_to_date": 5,
"new_sales": 3,
}]
},
"PHOENIX": {
"gross_sales": 26691.5,
"price": 292.25,
"dates": [{
"date": "2020-03-13",
"total_sales_to_date": 9,
"new_sales": 9,
}, {
"date": "2020-03-19",
"total_sales_to_date": 19,
"new_sales": 10,
}]
}
}
async function normaliseDict(_dict) {
let values = await Object.values(_dict);
// make arrays with values from each key
let all_gross_sales = [];
let all_price = [];
let all_total_sales = {};
values.forEach((element) => {
all_gross_sales.push(element.gross_sales);
all_price.push(element.price);
let most_recent_total_sales_value = element.dates[element.dates.length - 1].total_sales_to_date;
element.dates.forEach((date, idx) => {
date.norm_total_sales_over_time = date.total_sales_to_date / most_recent_total_sales_value;
if (all_total_sales[date.date]) all_total_sales[date.date].push(date.total_sales_to_date);
else {
all_total_sales[date.date] = [];
all_total_sales[date.date].push(date.total_sales_to_date);
}
});
});
const newDict = values.map(ob => {
ob.gross_sales_norm = ob.gross_sales / Math.max(...all_gross_sales);
ob.price_norm = ob.price / Math.max(...all_price);
return ob;
});
values.forEach((element) => {
element.dates.forEach((date, idx) => {
date.norm_total_sales_over_locations_for_this_date = date.total_sales_to_date / Math.max(...all_total_sales[date.date]);
});
});
return await dict;
}
(async () => {
console.log(await normaliseDict(dict))
})()

Firebase functions getChild value

I am having a problem on firebase functions. What I trying to do is when Items's child gets updated, then I want to get the value of Count and do further calculation, But what I am facing is that the firebase log console always shows an erreor "TypeError: Cannot read property 'val' of undefined".
JSON structure
"VTEST" : {
"A" : {
"Count" : 5,
"Items" : {
"item1" : "apple"
},
"NUMBER" : 5
},
"B" : {
"Count" : 8,
"Items" : {
"item1" : "orange;"
},
"NUMBER" : 3
},
"C" : {
"Count" : 10,
"Items" : {
"item1" : "grape"
},
"NUMBER" : 7
},
"D" : {
"Count" : 12,
"Items" : {
"item1" : "grava"
},
"NUMBER" : 10
},
"E" : {
"Count" : 15,
"Items" : {
"item1" : "fish"
},
"NUMBER" : 12
},
"F" : {
"Count" : 18,
"Items" : {
"item1" : "chicken;"
},
"NUMBER" : 8
}
}
My code:
exports.ItemCount = functions.database.ref('/VTEST/{ID}/Items').onUpdate((updateRef, context) => {
var childCount = updateRef.after.numChildren();
var newReference = updateRef.after.ref.parent.child('/Count');
var Count = newReference.val();
Count = Count + childCount;
return updateRef.ref.update({Count})
})
What I expect is the Count's value will be update, but it always show error : "TypeError: Cannot read property 'val' of undefined"
Can anyone tell me what am I doing wrong here, I don't get it.

The problem comes from the fact that a Reference does not have a val() method. You need to use the once() method to get the value of the corresponding database location.
The following adapted code should work:
exports.ItemCount = functions.database
.ref('/VTEST/{ID}/Items')
.onUpdate((updateRef, context) => {
var childCount = updateRef.after.numChildren();
var newReference = updateRef.after.ref.parent.child('/Count');
return newReference.once('value').then(dataSnapshot => {
var Count = dataSnapshot.val();
Count = Count + childCount;
return newReference.parent.update({ Count: Count });
});
});
However, depending on your exact requirements, you may decide to use a Transaction, see https://firebase.google.com/docs/database/web/read-and-write#save_data_as_transactions

Can I make Mongo map reduce count multiple values of an object?

I have a collection that I am trying to map reduce by id and date to produce a graph for sales of a product in store vs online. A new object is created for each transaction, so I would like to reduce them to a total count for a given day. An object looks something like this:
object
{
"ProductID": 1
"Purchase Method": In Store
"Date": 2018-01-16
"Count": 5
}
What I am trying to achieve as output is to have in store and online purchases combined into 1 object with a key being the id and the date and then the value being the counts of each method as shown below:
ProductID: 1
Date: 2018-01-16
[
{Name: "In store", Count: 3}
{Name: "Online", Count: 2}
]
My current method was to map the objects by Id, date, and Purchase Method so the reduce would get the total count for that id on that date using that method, but this leads to having two entries for an id and date, 1 for in store and 1 for online. This is the current state of my functions:
var mapDailySales = function() {
var sale = this;
/*Converts timestamp to just date */
var pad = function pad(n, width, z) {
z = z || '0';
n = n + '';
return n.length >= width ? n : new Array(width - n.length + 1).join(z) + n;
};
var d = sale.Date;
var date = d.getFullYear() + "-" + pad(d.getMonth() + 1, 2, 0) + "-" + pad(d.getDate(), 2, 0);
emit({ProductId: sale.ProductID, Date:date, Method: sale.PurchaseMethod},
{Name: sale.PurchaseMethod, Count: 1})
};
var reduceDailySales = function(key, value) {
var res = {Name: 0, Count: 0};
value.forEach(function(value){
res.Name = value.Name;
res.Count += value.Count;
});
return res;
};
Current Output looks something like this:
{
"_id" : {
"ProductId" : 1,
"Date" : "2018-01-16",
"Method" : "Online"
},
"value" : {
"Name" : "Online",
"Count" : 3
}
}
Is there a way to achieve my desired output without map reducing again on the current output?

You can use aggregation pipeline to get the results instead of mapReduce, $group by ProductID and Date, with $project you can map counts to an array
added $out to write the results to new collection, removing it will return a cursor
db.prod.aggregate([
{$group : {
_id : {ProductID : "$ProductID", Date : "$Date"},
onlineCount : {$sum : {$cond : [{$eq : ["$PurchaseMethod", "Online"]}, "$Count" , 0]}},
storeCount : {$sum : {$cond : [{$eq : ["$PurchaseMethod", "In Store"]}, "$Count" , 0]}}
}
},
{$project : {
_id : 0,
ProductID : "$_id.ProductID",
Date : "$_id.Date",
counts : [{Name: "In Store", Count: "$storeCount"},{Name : "Online", Count: "$onlineCount"}]
}},
{$out : "count_stats"}
]).pretty()
collection
> db.prod.find()
{ "_id" : ObjectId("5a98ce4a62f54862fc7cd1f5"), "ProductID" : 1, "PurchaseMethod" : "In Store", "Date" : "2018-01-16", "Count" : 5 }
{ "_id" : ObjectId("5a98ce4a62f54862fc7cd1f6"), "ProductID" : 1, "PurchaseMethod" : "Online", "Date" : "2018-01-16", "Count" : 2 }
>
result
> db.count_stats.find()
{ "_id" : ObjectId("5a98d3366a5f43b12a39b4ac"), "ProductID" : 1, "Date" : "2018-01-16", "counts" : [ { "Name" : "In Store", "Count" : 5 }, { "Name" : "Online", "Count" : 2 } ] }
>
if you want to use mapReduce, you can use finalize to reduce or transform the result further
db.prod.mapReduce(
<map>,
<reduce>,
{
out: <collection>,
finalize: <function>
}
)

Mongo Map Reduce to Return an Array of Objects

I've been trying to use the MapReduce functionaltity in Mongo 2.6.11 to help in returning an array of objects as a value to a corresponding key.
For example, given the following input to a MapReduce job:
{ IncidentNumber : 123, StartLoc : 5, EndLoc : 10, },
{ IncidentNumber : 123, StartLoc : 10, EndLoc : 15, },
{ IncidentNumber : 123, SStartLoc : 10, EndLoc : 15, },
{ IncidentNumber : 321, StartLoc : 0, EndLoc : 5, },
{ IncidentNumber : 321, StartLoc : 10, EndLoc : 20, }
I would like to get output that looks like this:
{ IncidentNumber : 123, Locations : [{StartLoc : 5, EndLoc : 10},{StartLoc : 10, EndLoc : 15}, {StartLoc : 10, EndLoc : 15}],
{ IncidentNumber : 321, Locations : [{StartLoc : 0, EndLoc : 5},{StartLoc : 10, EndLoc : 20}]
My current map and reduce functions for this is as follows:
var mapFunction = function() {
emit(this.IncidentNumber, {StartLoc : this.StartLoc, EndLoc: this.EndLoc} )
}
var reduceFunction = function(key, values) {
var out = []
return out.push(values);
}
This is giving me results that look like:
{ "_id" : 50144, "value" : 1 }
{ "_id" : 68971, "value" : { "startLoc" : 10, "endLoc" : 5} }
{ "_id" : 108294, "value" : 1 }
{ "_id" : 165130, "value" : 1 }
{ "_id" : 194016, "value" : 1 }
{ "_id" : 210018, "value" : 1 }
{ "_id" : 210195, "value" : 1 }
{ "_id" : 212069, "value" : 1 }
I know my reduce function is not correct and I'm not sure if this is an odd use case as I'm not really doing any actual 'reduction'. But I'm not sure exactly what the problem is. Any help would be much appreciated.
Thanks.
After some more searching I found a way of a getting this to work based on this post MongoDB Map/Reduce Array aggregation question. For anyone who is trying to solve the same problem The map/reduce functions I've used are as follows:
var mapFunction = function() {
emit( {
incident: this.IncidentNumber,
} , {
data: [ { start : this.startLoc, end : this.endLoc} ]
} );
}
var reduceFunction = function(key,values) {
var out = [];
values.forEach(function(d){
Array.prototype.push.apply(out, d.data);
});
return { data: out };
}