We Keep Coding

Group and count json data in queue() in d3.js

I am trying to change the format of a json file sourcing a D3 map. The original source is World Choropleth by palewire.
EDIT: working code thanks to below answer in the Plunker:
http://embed.plnkr.co/JYTATyrksAg3OJ0scHAp/
The original json is in a nested format with a count for each value:
{
"id": "IQ-G02-D009",
"rate": "1"
},
{
"id": "IQ-G05-D030",
"rate": "4"
},
{
"id": "IQ-G07-D047",
"rate": "5"
}
]
The new json will use a flat format, something like:
[
{
"id": "IQ-G02-D009"
},
{
"id": "IQ-G05-D030"
},
{
"id": "IQ-G05-D030"
},
{
"id": "IQ-G05-D047"
},
{
"id": "IQ-G07-D047"
}
]
It seems using a rollup and nest function would be the way forward to get the new json in a similar form of the former one, but am getting stuck at implementing those in a queue().
The rollup should be implemented at the .defer level it seems:
queue()
.defer(d3.json, "map.json")
.defer(d3.json, "data.json")
.await(ready);
Can I use the nest and rollup function directly in the queue?

You had some typo's working against you. The general answer to your question, as cal_br_mar said: apply nest and roll up when your data is loaded. So:
queue()
.defer(d3.json, "map.json")
.defer(d3.json, "data.json")
.await(ready)
function ready(error, country, activities) {
activities = d3.nest()
.key(function(d) {
return d.id;
})
.rollup(function(v) {
return v.length;
})
.entries(activities);
}
You had a typo in your last line: it should be activities, not expenses (wrong copy and paste, I guess).
And in the code that follows it should be:
activities.forEach(function(d) {
rateById[d.key] = +d.values;
});
The nest() and rollup puts your data in key and values properties, as you can see in your console output of the nested data:
[{"key":"IQ-G02-D009","values":1},{"key":"IQ-G05-D030","values":2},{"key":"IQ-G05-D047","values":1},{"key":"IQ-G07-D047","values":1}]
Change those fields and you should be fine.

Angular2 filter nested array of objects

I have an array of objects as below. I am trying to achieve search functionality using .filter() method in my angular2 application.
[{
"label": "new",
"result": [{
"label": "new",
"fname": "abc",
"lname": "xyz"
}, {
"label": "new",
"fname": "abc1",
"lname": "xyz1"
}]},
{
"label": "old",
"result": [{
"label": "old",
"fname": "abc2",
"lname": "xyz2"
}]
}]
I am able to achieve parent/one level filtering using below code:
this.data.filter(item => (item.label.toLowerCase().indexOf(inputText) !== -1);
This is returning the object that matches value of label. I want to have filter on 'fname' and 'lname' also.

Your data structure is not well suited for doing an arbitrary string lookup from a 3 level data structure like this.
Your are going to end up having to iterate all three levels - the array of objects, the results array in each object, and all the properties of the objects in the results array.
This is an O(n^3) operation and would likely perform unsatisfactorily with a large data set. You may need to think about how you can better structure this data for this use case.

I would when I get the data from the server create a lookup variable inside every item which contains all text you want to be able to search.
But make sure this is only done once, since it's wouldn't be very performant if you did it for every keystroke.
// Add searchable string to users
users.forEach(u => {
u.filterTerm = `${(u.id || '').toLowerCase()} ${(u.email || '').toLowerCase()} ${(u.firstName || '').toLowerCase()} ${(u.lastName || '').toLowerCase()}`;
});
Then you can use the includes method to filter
let filtered = allUsers.filter(u => {
return u.filterTerm.includes(filterTerm);
});

D3 - Update chart based on data keys

I am new to D3 and I would like to train myself on some scatterplot chart (based on the one from the NYT : Student debt.
I managed to recreate a graph like this one : Oklahoma Colleges.
Now, I have much more entries than the Oklahoma chart, so my chart is not very readable and I would like to filter the data based on a button with which I can select to display only the "public" colleges or the "private" ones.
I have read many tutorials about the ENTER-UPDATE-EXIT methods but I still have some trouble in applying it practically on my case.
Assuming the following JSON file :
[ {
"Id": 1,
"Name": "College A",
"Type": "Public" }, {
"Id": 2,
"Name": "College B",
"Type": "Private" }, {
"Id": 3,
"Name": "College C",
"Type": "Public" }, {
"Id": 4,
"Name": "College D",
"Type": "Public" }, {
"Id": 5,
"Name": "College E",
"Type": "Private" }, {
"Id": 6,
"Name": "College F",
"Type": "Private" }, ]
I would like to achieve the following algorithm :
button.on("change"){
If value == "public" :
display data where data.type == "public"
Else
display data where data.type == "private"
}
My first solution was to create a SVG each time I push the button (and erase the previous SVG) with the new dataset. But I think there is a much nicer way to do this :)
Can you help me ?
Thank you !
EDIT : following #sapote warrior answer -
Here what I do when I load the data :
d3.json("data.json", function(data) {
//Coerce string to numbers
...
dataset = data;
...
//Add axis and legend
}
And when I click to one of the two button :
function update(input){
var data = [];
for(i in dataset) {
if(dataset[i]["Type"] == input)
data.push(dataset[i]);
}
test = data; //Global scope variable, to check if new data is here on the console
circles = svg.selectAll(".circle")
.data(data);
circles.exit().remove();
circles.enter().append("circle")
.attr("class","circle")
...
}
But this doesn't work perfectly. Circles appear correctly when first click to any button, then they not all disappear when I click to the second button, and the new data doesn't seem to be correctly appended.
Hum, still have some issue understanding the enter-update-exit process ^^
EDIT : Ok problem solved ! I have just made some mistakes when implementing the enter-update-exit methods. Did it with a reduced dataset to understand the issue and now it's clear in my mind :)

I think I may be able to help you. Assuming that your circles are already displayed on the SVG, one way to do it is build a new array of values when your button is clicked that are of type "Public" or "Private". Something like this:
publicButton.on("click", function() {
newData = [];
for(i in existingDataArray) {
if(existingDataArray[i]["Type"] == "Public")
newData.push(existingDataArray[i]);
}
Now you can use this new data with the .data().enter().exit().remove() methods that you mentioned to append new data to your circle objects. After that you can remove those circles that aren't in the new data, and keep those that are. Those that you keep you can then doing something to them like update their color or nothing at all if you like. Sort of like this:
var circles = svg.selectAll("circle").data(newData);
circles.exit().remove();
circles.enter().attr("fill", ...);
}
Hopefully this helps some, let me know if you have any questions.

Why does the country with no data always go on top?

I have a GeoJSON map and I am trying to hook it up in Highmaps. This map consists of 4 small squares ("properties": {"name":"shape1"}) that will have data and 1 big square that will never have data ("properties": {"name":"base"}) The purpose of this big square is to be a background shape for the others.
The problem is that the big shape will always be on top of the small ones. I have tried both of these arrangements in the GeoJSON file:
placing the big shape before the small ones
placing the big shape after the small ones
Please see this example
Is there any way to make the shapes with data stay on top of the ones with no data?

Simply for that base element set null value, like this: http://jsfiddle.net/91ut26vz/1 - note it need to be first element in data array.
// Prepare random data
var data = [
{
"name": "base",
"value": null
},{
"name": "shape1",
"value": 728
},
{
"name": "shape2",
"value": 710
},
{
"name": "shape3",
"value": 963
},
{
"name": "shape4",
"value": 541
}
];

Should my API include both the number and the string representation of an enum?

Backgound info
In a product I am working on we have a lot of enumerated values for storing the "type" of a piece of data. For example (this stuff lives in C in a microcontroller):
typedef enum
{
Temperature = 100,
InputVoltage = 200,
AverageVoltage = 201,
Current = 300,
} ReadingType_t;
This allows us to handle "readings" generically, kind of like this:
typedef struct
{
ReadingType_t type;
sint32_t value;
} Reading_t;
Reading_t readings[5];
readings[0].type = Temperature;
readings[0].value = 25; // in degrees C
readings[1].type = InputVoltage;
readings[1].value = 4321; // in milliVolts
The Problem
I am trying to specify a JSON-based API for outputting information of this form. Some of the possible formats I have include:
readings = [
{
"type": "Temperature",
"value": 25
},
{
"type": "InputVoltage",
"value": 4321,
}
]
or
readings = [
{
"type": 100,
"value": 25
},
{
"type": 200,
"value": 4321,
}
]
or
readings = [
{
"type": {
"code": 100,
"name": "Temperature"
},
"value": 25
},
{
"type": {
"code": 200,
"name": "InputVoltage"
},
"value": 4321,
}
]
Assumptions
I am assuming that the API generates JSON.
I am assuming the API will be consumed by a Javascript client application and/or consumed directly for debugging.
Questions
The questions that I have are:
Should I include the enumeration "code", "title" or both? Is there an accepted practice for this?
If I include the "title" should it be in a regular language format? i.e. should I have "InputVoltage" or "Input Voltage"?
How should I go about supporting Internationalisation vs ease of consumption? i.e.:
If I send the numeric "code" there is no confusion about what the type is BUT a lookup table has to be used to get a human usable string (with a different table being used for internationalisation)
If I send the string "title" in regular language format the API must do the internationalisation (which is fine) BUT then we cannot do any logic based on that value (because the "title" will change based on the language whereas the code will not).

I would strongly advise against including the code as well as the name in the output. It is redundant, it ties you too much to both the code and the name being valid as long as you want to support reading/writing of the data.
I would recommend outputting the name to the file. The code is an implementation detail that could change in the future. Today you have an enum, tomorrow you could have a hash table. But the name is a meaningful information that, most likely, won't change.