Related
I have a JavaScript array of data, which currently looks like this
2021-09-01 00:10:00,11,10,'John',23
2021-09-01 13:20:00,12,20,'Sarah',55
2021-09-01 18:50:00,34,5,'Garry',12
2021-09-02 09:01:00,22,40,'Vicki',53
2021-09-02 14:23:00,34,2,'Peter',123
2021-09-02 16:20:00,21,10,'Victor',03
2021-09-02 18:00:00,88,40,'Nelly',3
You will notice that the first part is a date, for example "2021-09-01 00:10:00"
How can I create a new javascript array from that data which groups the rows by date, so in my example it would be 2 rows
2021-09-01
2021-09-02
And then add up the 3rd comma-separated value of each row (the values just before the persons name), so in my example it would output
2021-09-01,35
2021-09-02,92
And then give a total number of rows found for each group (there are 3 rows for 2021-09-01 and 4 rows for 2021-09-02)
2021-09-01,35,3
2021-09-02,92,4
Then return an average for each row (for example, first row = 35 / 3 = 11.66)
2021-09-01,35,3,11.66
2021-09-02,92,4,23
I then want to do a bit of maths on each row, but hopefully I can figure that out
Is this at all possible?
This would be a big help if anyone has an idea
Thank you
==================
WOW thank you everyone for your suggestions, maybe I was a little confused by saying it was an array, but when I did a console.log of my "sampledata" data, it was showing it as array
I'm basically using this, and want to run my original question on the sampledata output
d3.queue()
.defer(d3.csv, "test.csv")
.await(function(error, sampledata)
{
if (error) throw error;
console.log(sampledata);
});
if this helps
I like reduce for this
const csv = `2021-09-01 00:10:00,11,10,'John',23
2021-09-01 13:20:00,12,20,'Sarah',55
2021-09-01 18:50:00,34,5,'Garry',12
2021-09-02 09:01:00,22,40,'Vicki',53
2021-09-02 14:23:00,34,2,'Peter',123
2021-09-02 16:20:00,21,10,'Victor',03
2021-09-02 18:00:00,88,40,'Nelly',3`;
const arr = csv.split("\n"); // split the csv on newline
const res = arr.reduce((acc,cur) => {
let [date,ignore,num] = cur.split(","); // regex is needed if there are quoted commas before the interesting data
date = date.split(" ")[0]; // take the date part
acc[date] = acc[date] || {sum:0,cnt:0, avg:0 }; // reuse if existing, create new if not
acc[date].sum += +num;
acc[date].cnt++;
acc[date].avg = +(acc[date].sum/acc[date].cnt).toFixed(2); // remove decimals more than 2
return acc
},{})
console.log(res)
First, you'll need to create an object to map dates to their respective values, then, you'll iterate over your array in groups of 5.
const itemsByDate = {};
for (let i = 0; i < items.length; i += 5) {
const date = items[i].toISOString().slice(0, 10);
const value = items[i + 2];
if (!itemsByDate[date]) {
itemsByDate[date] = {
value,
count: 1
}
}
else {
itemsByDate[date].value += value;
itemsByDate[date].count += 1;
}
}
Then you iterate over the values of itemsByDate and calculate the average
for (const item of Object.values(itemsByDate)) {
item.mean = item.value / item.count;
}
If you really want to restructure your data as an array again, which I don't think is a good idea if you want to treat that data with javascript, you can do that with
const myArray = Object.entries(itemsByDate).flatMap(([date, item]) => {
return [date, item.value, item.count, item.mean];
});
Using only JavaScript, I need to
Group by code.
Get latest modifieddate.
Display total grouped code as Count.
Starting JSON Result
[
{"ID":1,"code":"AAA","modifieddate":"2019-06-01","user":"John"},
{"ID":2,"code":"AAA","modifieddate":"2019-06-02","user":"Jane"},
{"ID":3,"code":"AAA","modifieddate":"2019-06-03","user":"Sue"},
{"ID":4,"code":"BBB","modifieddate":"2019-06-10","user":"Rick"},
{"ID":5,"code":"CCC","modifieddate":"2019-06-11","user":"Joe"}
]
Desired JSON Result set
[
{"ID":3,"code":"AAA","modifieddate":"2019-06-03","user":"Sue","Count":"3"},
{"ID":4,"code":"BBB","modifieddate":"2019-06-10","user":"Rick","Count":"1"},
{"ID":5,"code":"CCC","modifieddate":"2019-06-11","user":"Joe","Count":"1"}
]
Tried using reduce method.
I do not have access to modify the server side API code.
I am using Aurelia JS.
You can use Array.reduce to group the result set by each item's code property, incrementing Count as needed, then take the values from the accumulation object. Along the way, we perform a date comparison to determine which the most recent entry to include in the result.
const data = [ {"ID":1,"code":"AAA","modifieddate":"2019-06-01","user":"John"}, {"ID":2,"code":"AAA","modifieddate":"2019-06-02","user":"Jane"}, {"ID":3,"code":"AAA","modifieddate":"2019-06-03","user":"Sue"}, {"ID":4,"code":"BBB","modifieddate":"2019-06-10","user":"Rick"}, {"ID":5,"code":"CCC","modifieddate":"2019-06-11","user":"Joe"} ];
const result = Object.values(data.reduce((a, e) => {
if (!a[e.code]) {
a[e.code] = {...e, Count: 0};
}
if (Date.parse(e.modifieddate) > Date.parse(a[e.code].modifieddate)) {
a[e.code] = {...e, Count: a[e.code].Count};
}
a[e.code].Count++;
return a;
}, {}));
console.log(result);
By the way, this is just a plain JS array we're working with, not JSON.
This should get you:
let array = [
{"ID":1,"code":"AAA","modifieddate":"2019-06-01","user":"John"},
{"ID":2,"code":"AAA","modifieddate":"2019-06-02","user":"Jane"},
{"ID":3,"code":"AAA","modifieddate":"2019-06-03","user":"Sue"},
{"ID":4,"code":"BBB","modifieddate":"2019-06-10","user":"Rick"},
{"ID":5,"code":"CCC","modifieddate":"2019-06-11","user":"Joe"}
]
let result = array.reduce(function(total, currentValue, currentIndex, arr) {
let index = total.findIndex(function(entry) { return entry.code == currentValue.code; })
if (index >= 0) { // entry already exists
// check modified
if (total[index].modifieddate > currentValue.modifieddate) { // already have most recent of the two
total[index].Count += 1;
} else { // need to replace with more recent
currentValue.Count = total[index].Count + 1;
total[index] = currentValue;
}
} else { // first record for this code
currentValue.Count = 1;
total.push(currentValue);
}
return total;
}, []);
console.log(result);
Here is a working js-fiddle
Note: Comments are made in code block
I have a JSON object like this...
{
"tasks":[
{
"id":"task_3",
"taskName":"Task A",
"assignee":"Barrack Obama",
"timeReqOptimisitic":"4",
"timeReqNormal":"8",
"timeReqPessimistic":"14",
"timeUnit":"Days",
"timeReq":"8.33",
"positionX":493,
"positionY":101,
"lockStatus":"unlocked"
}
],
"milestones":[
{
"id":"task_1",
"milestoneName":"Start",
"positionX":149,
"positionY":109,
"lockStatus":"unlocked",
"milestoneDate":"2015-04-07"
},
{
"id":"task_2",
"milestoneName":"Finish",
"positionX":989,
"positionY":367,
"lockStatus":"unlocked",
"milestoneDate":"2015-04-22"
}
],
"connections":[
{
"connectionId":"con_10",
"pageSourceId":"task_1",
"pageTargetId":"task_3"
},
{
"connectionId":"con_20",
"pageSourceId":"task_3",
"pageTargetId":"task_2"
}
]
}
...this is a minimal version. In practice, there are numerous items in "tasks", "milestones" and "connections".
I need to iterate through the object and determine the "id" of the "milestones" item with the lowest/earliest "milestoneDate", then identify the "connections" item that has the same value for its "pageSourceId" and return its "pageTargetId".
So in the above example:
Step 1) Iterate through the object and determine the "id" of the "milestones" item with the lowest/earliest "milestoneDate".
Answer: milestones.id = "task_1"
Step 2) Identify the "connections" item that has the same value for its "pageSourceId".
Answer: connections.pageSourceId = "task_1"
Step 3) Return its "pageTargetId".
Answer: "task_3"
I have a working example here. However, I would like to know if there is a way to accomplish this without using the extremely high start date and also in one loop.
As you are not parsing the same array on these two loops, there is no way to merge your loops.
Anyway, you can yet remove the loops to access to the arrays:
http://jsfiddle.net/gael/sruvtwre/2/
$.each(object.milestones, function( index, value ) {
if(startDate > parseDate(value.milestoneDate)) {
startDate = parseDate(value.milestoneDate);
id = value.id
}
});
$.each(object.connections, function( index, value ) {
if(id == value.pageSourceId) {
pageTargetId = value.pageTargetId;
}
});
May be also sorting, and indexing your datas. Then you would need no loops:
Elements in milestones should be sorted, so the earliest milestones element would be milestones[0].
Elements in connections should be indexed by their pageTargetId property, so the requested element should be connections[id].
Your two loops would become:
var pageTargetId= object.connections[ object.milestones[0].id ].pageTargetId;
http://jsfiddle.net/gael/sruvtwre/4/
As said in comments, sorting is not an optimal solution, even if that does not really matter for small sets.
Roughly, there is no no needs to sort all the datas, just the latest matters.
You can use array reduce method, as an comparable alternative to a simple loop:
var latestMilestone= object.milestones.reduce(function(milestone1, milestone2){
if( parseDate(milestone1.milestoneDate) > parseDate(milestone2.milestoneDate) )
return milestone1;
else
return milestone2;
//convert date to timestamp
function parseDate(date) {
var parts = date.split('-');
return Date.UTC(parts[0], parts[1]-1, parts[2]); // Note: months are 0-based
}
});
How about this:
Assuming you get the milestones.id = "task_1" in first loop; outside the loop we can have use jQuery grep. As connections will have unique pageSourceId, grep will return an array with only one object.
var filteredData = jQuery.grep('CONNECTIONS_ARRAY', function(element, index){
return element.pageSourceId == 'MILESTONES_ID'; // Which you get in the loop earlier
});
Then we can access pageTargetId like this:
if(filteredData.length){
filteredData[0].pageTargetId;
}
Try
var dates = []
, ids = []
, filtered = $.map(data.milestones, function(value, index) {
dates.push(new Date(value.milestoneDate).getTime());
ids.push(value.id);
if (dates.length === data.milestones.length) {
var id = ids[$.inArray(Math.min.apply(Math, dates), dates)]
, res = $.grep(data.connections, function(task, key) {
return task.pageSourceId === id
})[0].pageTargetId;
return res
}
})[0]; // `"task_3"`
var data = {
"tasks":[
{
"id":"task_3",
"taskName":"Task A",
"assignee":"Barrack Obama",
"timeReqOptimisitic":"4",
"timeReqNormal":"8",
"timeReqPessimistic":"14",
"timeUnit":"Days",
"timeReq":"8.33",
"positionX":493,
"positionY":101,
"lockStatus":"unlocked"
}
],
"milestones":[
{
"id":"task_1",
"milestoneName":"Start",
"positionX":149,
"positionY":109,
"lockStatus":"unlocked",
"milestoneDate":"2015-04-07"
},
{
"id":"task_2",
"milestoneName":"Finish",
"positionX":989,
"positionY":367,
"lockStatus":"unlocked",
"milestoneDate":"2015-04-22"
}
],
"connections":[
{
"connectionId":"con_10",
"pageSourceId":"task_1",
"pageTargetId":"task_3"
},
{
"connectionId":"con_20",
"pageSourceId":"task_3",
"pageTargetId":"task_2"
}
]
};
var dates = []
, ids = []
, filtered = $.map(data.milestones, function(value, index) {
dates.push(new Date(value.milestoneDate).getTime());
ids.push(value.id);
if (dates.length === data.milestones.length) {
var id = ids[$.inArray(Math.min.apply(Math, dates), dates)]
, res = $.grep(data.connections, function(task, key) {
return task.pageSourceId === id
})[0].pageTargetId;
return res
}
})[0];
document.write(filtered);
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js">
</script>
New to crossfilter. I've a flat data which is given below:
id,name,patientId,conditionId,isPrimary,age,gender,race,Status,CGI
1,M1,1,c1,Y,33,Male,White,Discharged,0
2,M2,1,c1,N,33,Male,White,Discharged,0
3,M3,1,c2,N,33,Male,White,Discharged,0
4,M4,1,c2,N,33,Male,White,Discharged,0
5,M5,1,c3,N,33,Male,White,Discharged,0
6,M6,1,c3,N,33,Male,White,Discharged,0
25,M1,5,c1,Y,33,Male,White,Discharged,1
26,M7,5,c2,N,33,Male,White,Discharged,1
27,M4,5,c4,N,33,Male,White,Discharged,1
28,M4,5,c1,N,33,Male,White,Discharged,1
29,M4,5,c2,N,33,Male,White,Discharged,1
30,M5,5,c4,N,33,Male,White,Discharged,1
29,M2,6,c1,Y,33,Male,White,Discharged,1
30,M2,7,c1,Y,33,Male,White,Discharged,1
I want to do a count on conditionId but since there are multiple records belonging to the same person as identified by patientId, the count of value c1 should be 4 (belonging to patientId 1, 5, 6, 7) - because same patient may have multiple records (for eg. patientId of 1 is repeated 6 times and two of them has c1 which should be counted only once) . I'm struggling to write a group.reduce on conditionId but could not even start.
Thanks in advance.
Here's one way of doing it. In the example I assumed that the first value was the patientId and the second value the conditionId. The code keeps track of grouping keys (concatenation of the patientId and the conditionId) that were seen already and ignores them.
var countMap = [
[1, 'c1'],
[1, 'c1'],
[2, 'c1'],
[2, 'c2']
].reduce(function (r, v) {
var condition = v[1],
groupKey = v[0] + condition;
if (!r.seen[groupKey]) {
r.seen[groupKey] = true;
r.count[condition] = (r.count[condition] || 0) + 1;
}
return r;
}, {seen: {}, count: {}}).count;
countMap.c1; //2
countMap.c2; //1
I do not know about crossfilter or dc.js, that's why I gave you a vanilla JS solution.
It's a little complicated to do this in Crossfilter, but the solution is similar to that provided by #plalx.
Here is a helper function I am using in one of my projects. It's not perfect, and is a bit optimized to reduce dictionary lookups, so it's not the most readable. The basic idea is you need to keep a dictionary of values seen before for each group. You only need to remember patients, because the condition is already known based on the group your are in:
function reduceHelper(accessorFunction) {
var internalCount;
return {
add: function (p, v) {
if(p.unique.has(accessorFunction(v))) {
internalCount = p.unique.get(accessorFunction(v));
p.unique.set(accessorFunction(v), internalCount + 1);
} else {
p.unique.set(accessorFunction(v), 1);
++p.count;
}
return p;
},
remove: function (p, v) {
if(p.unique.has(accessorFunction(v))) {
internalCount = p.unique.get(accessorFunction(v));
if(internalCount == 1) {
p.unique.remove(accessorFunction(v));
--p.count;
} else {
p.unique.set(accessorFunction(v), internalCount - 1);
}
}
return p;
},
init: function () {
return {unique: d3.map(), count: 0};
}
};
}
You'll need to create a Crossfilter (xfilter) on your data and then:
var helperFunctions = reduceHelper(function(d) { return d.patientId; });
var dim = xfilter.dimension(function (d) { return d.conditionId; };
var group = dim.group()
.reduce(helperFunctions.add, helperFunctions.remove, helperFunctions.init);
Your group will now count the number of patients that have each condition. If a condition appears more than once for a given patient, that patient will still only be counted once. At least, it will if my solution works properly :-)
How do I user the JavaScript "filter" attribute as filter my JavaScript object?
I've been reading the following StackOverflow post, and am in a similar situation.
I have the following JavaScript object:
{
'cars' :
[{
"car_id" : "1",
"price" : "42999",
"make_id" : "050",
"year_built" : "2007",
"color_id" : "832"
},
..........
]}
I'm using JQuery to display controls to allow people to filter based on: Price, Make, Year Built, Color
Per that other post, I can use the following code:
// if using an old browser, define the 'filter' attribute
if (!Array.prototype.filter)
{
Array.prototype.filter = function(fun /*, thisp*/)
{
var len = this.length >>> 0;
if (typeof fun != "function")
throw new TypeError();
var res = new Array();
var thisp = arguments[1];
for (var i = 0; i < len; i++)
{
if (i in this)
{
var val = this[i]; // in case fun mutates this
if (fun.call(thisp, val, i, this))
res.push(val);
}
}
return res;
};
}
then to perform the actual filter, I can do:
result = cars.
filter(function(p) { return p.price >= 15000 }).
filter(function(p) { return p.price <= 40000 }).
filter(function(p) { return p.year_built >= 2000 }) etc
What I don't understand is, how do I use my JQuery controls to dynamically change the filter once the filter has already been set? Meaning, let's say I have the filter applied from above, then the user changes there mind and wants to increase the maximum they are willing to pay for a car from $40,000 to $50,000.
How would I problematically modify my filter from :
filter(function(p) { return p.price <= 40000 }).
to:
filter(function(p) { return p.price <= 50000 }).
how do I use my JQuery controls to dynamically change the filter once the filter has already been set?
You don't set a filter. You call filter() with a filter function and get a filtered array back; you can't change the filter that was applied to the array afterwards. Instead you must call filter() again, and pass a different filter function.
Or the same filter function with a closure over a variable that has changed:
var minprice= 10000;
var minpricefilter= function(p) { return p.price>=minprice };
result= cars.filter(minpricefilter);
minprice= 20000;
result= cars.filter(minpricefilter);
You could use a function generator.
function createPriceFilter(price)
{
filter = function(){ return p.price >= price };
return filter;
}
Then, when you filter, always use a function generator.
cars
.filter( createPriceFilter( mySelectedPrice ) )
.filter( createSomethingFilter(...) )
. (...)
Instead of filter, how about a plain old loop:
var min_year = 2000;
var min_price = 15000;
var max_price = 40000;
function fillTable() {
clearTheTable();
for (i = 0; i < cars.length; i++) {
var car = cars[i];
if (p.price >= min_price && p.price <= max_price && p.year_built >= min_year)
addCarToTable(car);
}
}
Each time your parameters change, just call fillTable() again to regenerate the whole table.
(There are much cleverer things you can do but this is the simplest thing I could think of.)
Forget callback based filtering. Enter jOrder: http://github.com/danstocker/jorder.
Filtering by iterating over your entire table is tedious and slow. With jOrder, you search by index:
var table = jOrder(json.cars)
.index('id', ['car_id'])
.index('price', ['price'], { grouped: true, ordered: true, type: jOrder.number })
.index('maker', ['maker_id'], { grouped: true })
.index('year', ['year_built'], { grouped: true, ordered: true, type: jOrder.number })
.index('color', ['color_id'], { grouped: true });
Then you get the records you want by:
var filtered = table.where([{ price: { lower: 15000, upper: 40000 } }], { mode: jOrder.range });
Note that you can only apply one inequality filter at a time. To do more, use filtered as an input for a different jOrder table, put only the necessary index on it, and perform the second inequality filter on that one. And so on. Even if you stack up a couple of filters like this, it will be still faster than iteration by a factor of about 10 to 100 depending on the size of your table.