Summarize & Group By with Lodash - javascript

I'm new to Lodash and I'm trying to perform a complex sum with group by as SQL but I don't find any solution. I have tried to use/combine multiple Lodash functions without success.
My requirement is like this. I have a JSON response:
input =
[{"quantity":1067,"gross_revenue":4094.2,"date":"03","company":"Cat1","product":"Car"},
{"quantity":106,"gross_revenue":409,"date":"02","company":"Cat2","product":"Car"},
{"quantity":106,"gross_revenue":85,"date":"03","company":"Cat2","product":"House"},
{"quantity":106,"gross_revenue":100,"date":"02","company":"Cat3","product":"House"},
{"quantity":20,"gross_revenue":150,"date":"03","company":"Cat5","product":"Technology"},
{"quantity":40,"gross_revenue":100,"date":"01","company":"Cat5","product":"Technology"},
{"quantity":20,"gross_revenue":15,"date":"01","company":"Cat5","product":"Car"},
{"quantity":20,"gross_revenue":18,"date":"01","company":"Cat5","product":"House"},
{"quantity":20,"gross_revenue":2,"date":"01","company":"Cat2","product":"House"},
{"quantity":20,"gross_revenue":25,"date":"01","company":"Cat3","product":"House"}]
I need to generate a result as below to populate the series for a HighChart:
[{ name: 'Car', data: [15, 409, 4094.2] },
{ name: 'House', data:[45, 100, 85] },
{ name: 'Techonology', data:[100, null, 150] }]
Those values are the result from:
Make a group by using Product with the tag name
Based on following procedure, generate an array with the tag data
2.1 Sum the gross revenue based on Product and date (all existing dates)
2.2 Include a null value if there doesn't exist gross revenue for any existing day
2.3 Sort the results for gross revenue based on date, ascending order
Is this possible? Or is there another solution for this?
Thanks.

Here's one way to do it - certainly not the only solution...
var input = [
{"quantity":1067,"gross_revenue":4094.2,"date":"03","company":"Cat1","product":"Car"},
{"quantity":106,"gross_revenue":409,"date":"02","company":"Cat2","product":"Car"},
{"quantity":106,"gross_revenue":85,"date":"03","company":"Cat2","product":"House"},
{"quantity":106,"gross_revenue":100,"date":"02","company":"Cat3","product":"House"},
{"quantity":20,"gross_revenue":150,"date":"03","company":"Cat5","product":"Technology"},
{"quantity":40,"gross_revenue":100,"date":"01","company":"Cat5","product":"Technology"},
{"quantity":20,"gross_revenue":15,"date":"01","company":"Cat5","product":"Car"},
{"quantity":20,"gross_revenue":18,"date":"01","company":"Cat5","product":"House"},
{"quantity":20,"gross_revenue":2,"date":"01","company":"Cat2","product":"House"},
{"quantity":20,"gross_revenue":25,"date":"01","company":"Cat3","product":"House"}
];
var result = [];
var groupedByProduct = _.groupBy(input, "product");
// get the set of unique dates
var dates = _.uniq(_.map(input, 'date'));
// for each product, perform the aggregation
_.forEach(groupedByProduct, function(value, key) {
// initialize the data array for each date
data = [];
for (var i = 0; i < dates.length; i++) {
data.push(null);
}
// aggregate gross_revenue by date
_.forEachRight(_.groupBy(groupedByProduct[key], "date"), function(dateValue, dateKey) {
// use the date as an array index
data[parseInt(dateKey) - 1] = _.sumBy(dateValue, function(o) {
return o.gross_revenue
});
});
// push into the result array
result.push({"name": key, "data": data});
});
document.getElementById("result").innerHTML = JSON.stringify(result);
<script src="https://cdn.jsdelivr.net/lodash/4.11.1/lodash.min.js"></script>
<pre id="result"></pre>

Related

Convert an array into an object to create a report javascript

I have an array that represents a table like this:
table = [['john', 'male', 24, '12/12/12'], ['jane', 'female', 24, 12/12/12]]
I want to let the user choose which column they want, so later they can make a pdf report with the columns they chose, I think making an object like this is the best way to get that data, I might be wrong of course haha.
Let's say the user wants the following data on the report header: name, age, date, I want an object like this:
userHeader = { name: 'John', age: 24, date: '12/12/12'}
So I can make the next report:
Report #1234
|-------------------------------|
|Name: John Date: 24/12/12 | <-Header
|Age: 24 |
|-------------------------------|
|some data | <--Body
| .... |
|-------------------------------|
I have an array with the columns the user wants that stores it index, for example if the user wants the columns 1 and 2, the array is this:
var userColumns = [1,2]
How can I approach this problem? How would you do it?
EDIT: I put the wrong table,. this are the tables:
table1 = [['john', 'male', 24, '12/12/12', 1], ['john', 'male', 24, 01/05/12, 1]]
table2 = [['john', 'male', 24, '12/07/12', 2], ['john', 'male', 24, 05/05/12, 2]]
To get some context, I have a CSV file with multiple columns and rows, each row has a different codeItem, this codeItem can be repeated in multiple rows or not, what i do is create multiple tables that have the same code report, for example if the CSV data has 10 rows, 5 with an codeItem:1 and the other 5 with codeItem: 2, I create 2 tables, one with all the rows that have the codeItem 1 and another with a codeItem 2, then I would make a report for each codeItem, in this case 2 reports, so each table has some rows that have the same data on some columns.
The user columns is what columns the user chose to appear on the report, I have an array with the header columns:
var headers = ['name', 'sex', 'age', 'date', 'codeReport']
What I do is match the index on the header array to the userColumns, lets say the user wants the name and age headers, the user header is:
userHeader = [0, 2]
I know it sounds confusing and it really is.
First of all, if you want to use objects for storing data from an given array, you need I routine to convert them. Therefor I allways create an empty object o = {}, and with o["prop"] = value can this object be filled. The same as o.prop = value.
let headers = ['name', 'sex', 'age', 'date', 'codeReport'];
function createObjectFromArray(array, indexes)
{
let result = {};
for(let index of indexes)
{
result[headers[index]] = array[index];
}
return result;
}
let recordObject = createObjectFromArray(['john', 'male', 24, '12/12/12', 1], [1, 2]);
//Object {sex: "male", age: 24}
With the help of the ECMAScript 6 class Map, it is possible to link an object to any data. Or you can use IndexedDB.
let reportDataBase = new Map();
reportDataBase.set(recordObject, "somedata");
reportDataBase.get(recordObject); // "somedata"
If you want to iterate through all tables (table 1 has code item 1, table 2 has code item 2, ...), you need an object, which is iterable. I recommend an array.
let tables = [table1, table2];
let selectedColumns = [1, 2];
for(var report = 0; report != tables.length; report++)
{
console.log("report : " + (report + 1));
tables[report].forEach(function(item)
{
console.log(createObjectFromArray(item, selectedColumns));
});
}
I think a better way to storage in-memory data and generate different reports, is to use a data structure like this:
var reports = [
[['john', 24, "somedata1"], ['lara', 22, "somedata2"]],
[['mark', 21, "somedata3"], ['eve', 25, "somedata4"]]
];
But its a bad idea, to storage all personal data in a open browser. To much ressources for showing one record and the question is: Wants the person that his data is public?
One solution is: frontend <-> node.js
You just need an association of table headers with array of user data like
ReportHeaderAssociation = {
"0":Name",
"1":Age",
....
}
So when you got columns that users want( here [1,2] ) you can get title from association object and value by accessing array element.
Like , userdata[1] will give you second element from data which is gender. Likewise all elements.
In short: a single object with relativity will do the job no need to run long loops to convert array to object.
You need a mapping for the column names, for example with the same indexes as the data and a filter for which columns you want.
After that map and reduce are your friends ;)
var table = [
['John', 'male', 24, '12/12/12'],
['Jane', 'female', 22, '11/11/11']
];
var columnMapping = ['name', 'gender', 'age', 'date'];
var getData = function(data, columnNames, columnFilter) {
return data.map(function(item) {
return item.reduce(function(obj, item, idx) {
if (columnFilter.indexOf(idx) > -1) {
obj[columnNames[idx]] = item;
}
return obj;
}, {});
});
}
// age and gender
console.log(getData(table, columnMapping, [1, 2]));
// name and date
console.log(getData(table, columnMapping, [0, 3]));
// name, age and date
console.log(getData(table, columnMapping, [0, 2, 3]));
// name, age and date only if name is John
console.log(getData(table, columnMapping, [0, 2, 3]).filter(function(item) {
return item.name === 'John';
}));

javascript sort key value pair on frequency in different array

I would like to store product information in a key, value array, with the key being the unique product url. Then I would also like to store the visit frequency of each of these products. I will store these objects as window.localStorage items, but that's not very important.
The thing I had in mind was two key value arrays:
//product information
prods["url"] = ["name:product_x,type:category_x,price:50"]
//product visits frequency
freq["url"] = [6]
Then I would like to sort these prods based on the frequency.
Is that possible?
Hope you guys can help! Thanks a lot
Well you seem to have made several strange choices for your data format/structure. But assuming the format of the "prod" is beyond your control but you can choose your data structure, here's one way to do it.
Rather than two objects both using url as a key and having one value field each I've made a single object still keyed on url but with the product and frequency information from each in a field.
Objects don't have any inherent order so rather than sorting the table object I sort the keys, your "url"s ordered by ascending frequency.
To show that it's sorted that way I print it out (not in the same format).
For descending frequency, change data[a].freq - data[b].freq to data[b].freq - data[a].freq
var data = {
"url": {
prod: "name:product_x,type:category_x,price:50",
freq: 6
},
"url2": {
prod: "name:product_y,type:category_y,price:25",
freq: 3
}
};
var sorted = Object.keys(data).sort((a, b) => data[a].freq - data[b].freq);
console.log(sorted.map(k => [data[k].freq, k, data[k].prod]));
There's more than one way to format the data, which would change the shape of the code here.
maybe something like this:
var prods = [
{url:1, val:[{name:'a',type:'x',price:60}]},
{url:2, val:[{name:'b',type:'x',price:30}]},
{url:3, val:[{name:'c',type:'x',price:50}]},
{url:4, val:[{name:'c',type:'x',price:20}]},
{url:5, val:[{name:'c',type:'x',price:10}]},
{url:6, val:[{name:'c',type:'x',price:40}]}
];
var freq = [
{url:1, freq:6},
{url:2, freq:3},
{url:3, freq:5},
{url:4, freq:2},
{url:5, freq:1},
{url:6, freq:4}
];
prods.sort(function (a, b) {
var aU = freq.filter(function(obj) {
return obj.url === a.url;
});
var bU = freq.filter(function(obj) {
return obj.url === b.url;
});
if (aU[0].freq > bU[0].freq) {
return 1;
}
if (aU[0].freq < bU[0].freq) {
return -1;
}
return 0;
});

js: array with in an associative array

I am trying to set up an associative array for the following data:
name date alpha beta
Andrew 12/08/07 2.3 1.4
5/12/07
26/03/08
____________________________________
Fred 3/09/07 2.1 1.1
23/01/08
____________________________________
Basically, each patient would have a name and alpha , beta value but multiple dates on which they visited doctor. I was thinking of something like following where name is the primary key and dates are stored in an array and alpha, beta is a float value associated with the key.
var info = [{ name: [{ dates: [ ], alpha: float, beta: float }] }];
and then this info array would be populated on reading the csv file. What would be the right format for initialising such an associative array? or what other data structure would be a good approach for representing such a data?
Thanks in advance!
Edit: Since each patient has a unique name, instead of using an array, you should consider using a single object where each patient is an object identified by the object key, for example:
var patientList = {
andy: {},
bob: {}
};
To get your data from your CSV file into this structure you might consider something like this:
var csv = 'Andrew\t12/08/07\t1.2\t3.4\nAndrew\t15/09/08\t1.2\t3.4\nAndrew\t14/08/07\t1.2\t3.4\t\nBob\t18/09/08\t1.2\t3.4\nAndrew\t21/08/07\t1.2\t3.4\nDavid\t31/09/08\t1.2\t3.4\nAndrew\t22/08/07\t1.2\t3.4\t\nSam\t26/09/08\t1.2\t3.4';
// Split the CSV file at the carriage return.
var data = csv.split('\n');
// Recursive over the data with `map`, splitting each line up
// on the tabs and returning a patient object for each.
data = data.map(function (el) {
var patient = el.split('\t');
return {
name: patient[0],
date: patient[1],
alpha: patient[2],
beta: patient[3]
}
});
function getListOfPatientNames(arr) {
var newarr = [];
// For each patient object return the patient name only
newarr = arr.map(function (patient) {
return patient.name;
});
// A quick way of filtering out duplicates
return newarr.filter(function(elem, pos) {
return newarr.indexOf(elem) == pos;
});
}
// Return a unique list of names, and sort them.
var names = getListOfPatientNames(data).sort();
var patientList = {};
for (var i = 0, l = data.length; i < l; i++) {
var name = data[i].name;
// If the patient name doesn't exist in patientList yet
if (!patientList[name]) {
// Add a new patient object using the name as the key
var newPatient = {
dates: [data[i].date],
alpha: data[i].alpha,
beta: data[i].beta
};
patientList[name] = newPatient;
} else {
// If the patient already exists push the date to the dates array
patientList[name].dates.push(data[i].date);
}
}
Demo
The term "associative array" is almost never used wrt JavaScript; you use objects (sometimes called "maps" or "dictionaries") for name/value information, or arrays for ordered data.
It looks like you want an array of patient objects, like this:
var patients = [
{
name: "Andrew",
dates: [/*...the dates...*/],
alpha: 2.3,
beta: 1.4
},
{
name: "Fred",
dates: [/*...the dates...*/],
alpha: 2.1,
beta: 1.1
}
];
You might or might not want to use a constructor function to create those objects, depending on your needs, but with the simple data you've given there's probably no need.

Trouble Pivoting data with Map Reduce

I am having trouble pivoting my dataset with map reduce. I've been using the MongoDB cookbook for help, but I'm getting some weird errors. I want to take the below collection and pivot it so that each user has a list of all of the review ratings.
My collection looks like this:
{
'type': 'review',
'business_id': (encrypted business id),
'user_id': (encrypted user id),
'stars': (star rating),
'text': (review text),
}
Map function (wrapped in Python):
map = Code(""""
function(){
key = {user : this.user_id};
value = {ratings: [this.business_id, this.stars]};
emit(key, value);
}
""")
The map function should return an array of values associated with the key...
Reduce function (wrapped in Python):
reduce = Code("""
function(key, values){
var result = { value: [] };
temp = [];
for (var i = 0; i < values.length; i++){
temp.push(values[i].ratings);
}
result.value = temp;
return result;
}
""")
However, the results return one less rating than total. In fact, some users have None returned, which can't happen. Some entries look like the following:
{u'_id': {u'user: u'zwZytzNIayFoQVEG8Xcvxw'}, u'value': [None, [u'e9nN4XxjdHj4qtKCOPQ_vg', 3.0], None, [...]...]
I can't pinpoint what in my code is causing this. If there are 3 reviews, they all have business IDs and ratings in the document. Plus, using 'values.length + 1' in my loop condition breaks values[i] for some reason.
Edit 1
I've embraced the fact that reduce gets called multiple times on itself, so below is my new reducer. This returns an array of [business, rating, business, rating]. Any idea how to output [business, rating] arrays instead of one giant array?
function(key, value){
var result = { ratings:[] };
var temp = [];
values.forEach(function(value){
value.ratings.forEach(function(rating){
if(temp.indexof(rating) == -1){
temp.push(rating);
}
});
});
result. rartings = temp;
return result;
}
Heres a test example:
1) Add some sample data:
db.test.drop();
db.test.insert(
[{
'type': 'review',
'business_id': 1,
'user_id': 1,
'stars': 1,
},
{
'type': 'review',
'business_id': 2,
'user_id': 1,
'stars': 2,
},
{
'type': 'review',
'business_id': 2,
'user_id': 2,
'stars': 3,
}]
);
2) Map function
var map = function() {
emit(this.user_id, [[this.business_id, this.stars]]);
};
Here we set the results as we want them to look like at the end of the process. Why? because if there is only ever a single review by a user (the key we are grouping by) then the results won't go through a reduce phase.
3) Reduce function
var reduce = function(key, values) {
var result = { ratings: [] };
values.forEach(function(value){
result.ratings.push(value[0]);
});
return result;
};
Here we collect up all the values, remembering we nested them in the map method, so we can just pick out the first value for each set of results.
4) Run the map reduce:
db.test.mapReduce(map, reduce, {finalize: final, out: { inline: 1 }});
Alternative - use the aggregation framework:
db.test.aggregate({
$group: {
_id: "$user_id",
ratings: {$addToSet: {business_id: "$business_id", stars: "$stars"}}
}
});

Getting corresponding values from two different objects

I have returned an object from an ajax call that is the combination of two different arrays' of objects. One is Jobs and the second is the corresponding Customer Records for those jobs. The returned object is formatted like so..
{ "jobs" : [
{jobID: 1,
jobLocation: here
},
{jobID: 2,
jobLocation: there
}
],
"customers" : [
{customerID:1,
customerName:Tom
},
{customerID:2,
customerName:Sally
}
]
}
The items in the job array are sequentially related to the customers in the customer array. (i.e. the first customer owns the first job) How can I iterate or parse then iterate over this object to make list objects<li>'s that have are composed of a field from the jobs array and a field from the corresponding object of the customers array? thank you
A plain, old for-loop might do the job:
var customers = obj.customers;
var jobs = obj.jobs;
var $ul = $("<ul></ul>");
for (var i = 0; i < customers.length; i++) {
var customer = customers[i];
var job = jobs[i];
var $li = $("<li></li>").text(customer.customerName + " - " + job.jobLocation);
$li.appendTo($ul);
}
$("#result").append($ul);
See http://jsfiddle.net/XpQms/

Categories