I am currently working with the Treemap visualization of D3.js and was hoping to understand how the flare.json used in the example has been organized. Does the format of the json input file need to be in the exact same structure as used in the example. I have an input file from a web crawler with a list of URLs and their respective parent URLs. I tried using something like the following but it won't work and am not sure if it's just the structure that's different or something else.
listURLs.json:
{
"name": "flare",
"children": [
{"children":"http:\/\/a.wholelottanothing.org","name":"http:\/\/buzz.blogger.com"},
{"children":"http:\/\/www.bitworking.org","name":"http:\/\/buzz.blogger.com"},
{"children":"http:\/\/blog.computationalcomplexity.org","name":"http:\/\/buzz.blogger.com"},
{"children":"http:\/\/www.blogactionday.org","name":"http:\/\/buzz.blogger.com"},
{"children":"http:\/\/www.wikipaintings.org","name":"http:\/\/littlegreeniguana.blogspot.com"}
]
}
I know this looks very different from the flare.json used in the example but can this work? Also, the input that I am using doesn't include the 'size' parameter which is also probably why the output is blank. How do I use the size here? Can it be dynamically adjusted later in the code? Any help will be most appreciated, I am a D3 novie!
The hierarchical data format expected by tree, pack and other D3 hierarchical layouts is expecting "children" to be an array of objects, and traverses that hierarchical data in preparation for formatting your objects for display using layouts. So, you don't want to use "children" to store a single link, instead, you want it to store an array of objects formatted just like the parent object (even if there is only one thing in that array). It's a bit hard to grasp what you're trying to display in your dataset, but my guess is all those websites are under buzz.blogger.com, except the last one, in which case properly formatted hierarchical data would look like this (Note that everything is nested in a root node, which you can name whatever you want):
{
"name": "root node",
"children": [
{"name":"http:\/\/buzz.blogger.com", "children": [
{"name": "http:\/\/www.bitworking.org"},
{"name": "http:\/\/blog.computationalcomplexity.org"},
{"name": "http:\/\/www.blogactionday.org"}
]
},
{"name":"http:\/\/littlegreeniguana.blogspot.com", "children": [
{"name": "http:\/\/www.wikipaintings.org"}
]
}
]
}
Related
This is a problem that is so hard for me to find using just keywords -- I've scrolled pages after pages and all the hits are on getting data from json structure, instead of getting data structure from json.
If you don't know what goal I'm trying to achieve, here are tools to get data structure from json to Go:
https://mholt.github.io/json-to-go/
This tool instantly converts JSON into a Go type definition
https://json2struct.mervine.net/
Convert JSON in to a useful struct.
https://transform.tools/json-to-go
online playground to convert JSON to Go Struct
For one specific application, I'm getting data with all kinds of slightly different json data structures, which makes my data extraction failing all the times. I need to compare those data structure-wise, as each individual json data are surely different.
It looks like you're struggling to find what you want because vanilla JavaScript doesn't have the type concepts you're trying to use.
You probably want to generate a json schema or a typescript interface, and searching "JSON to schema npm" will get you more useful results.
One such example is to-json-schema
import toJsonSchema from 'to-json-schema';
toJsonSchema({ "foo": "bar", "x": [1, 2] });
/* returns
{
"type": "object",
"properties": {
"foo": {
"type": "string"
},
"x": {
"type": "array",
"items": {
"type": "integer"
}
}
}
}
*/
For more specific answers you'd need to provide some minimal sample of input and output, e.g. what would you expect from { "foo": "bar", "x": [1, 2] }
I'm using Vega JS for building a tree chart. In general, my question is the following:
Vega documentation has a great example of tree layout. How can I extend it with an ability to collapse & expand its nodes?
To be more specific, let's consider an example of tree chart that I'm building here in Vega Editor.
If you click on the nodes, they will toggle (expand or collapse), allowing you to see particular branches of the tree. This feature works fine unless you try to collapse the top-level node (region) while keeping the second-level nodes (districts) expanded. In that case the tree will look like this:
It happens because of the way I handle this interaction:
When you click on a node, toggledNode signal is triggered, which in turn triggers toggle action in the expandedNodes data array. I.e. by clicking on a node, I add or remove that node to the expandedNodes array (more precisely, we add/remove a reduced object with only name property)
Thus expandedNodes data contains information about which nodes are explicitly expanded. But it doesn't know if those expanded nodes are inside of a collapsed parent node.
Then, to find out which nodes are actually visible, I use visibleNodes data. There I apply filter transform with the following expression: !datum.parent || indata('expandedNodes', 'name', datum.parent). I.e. I check only one level up: if the node's parent is present in the expandedNodes array , I consider the node as visible.
The problem is the following: I can't find any way of extending this functionality across multiple levels.
Probably I could write some hooks to check the same condition across 2 or 3 levels, e.g:
!datum.parent ||
indata('expandedNodes', 'name', datum.parent) &&
indata('expandedNodes', 'name', datum.myCustomFieldWithParentNode.parent) &&
indata('expandedNodes', 'name', datum.myCustomFieldWithParentNode.myCustomFieldWithParentNode.parent)
But it seems too complex for such a simple problem, and also it's not a final solution. In theory, a tree may contain dozens of nesting levels: what to do then?
I found one useful expression in Vega: treeAncestors. I could easily write a solution in JavaScript, where I have loops and array methods such as .some() and .every(). But apparently Vega doesn't support any expressions to iterate over an array. So even though I can get an array of tree node ancestors with treeAncestors function, I can't do anything with it to verify that all ancestors are expanded.
Either my approach is wrong, and somebody can find a better algorithm for doing the same, which doesn't require iterating over arrays (except for data and indata expressions) - or it's a current limitation of Vega.
You can use treeAncestors and then use a flatten transform to get a dataset that you can query. In your case it would look something like:
{
"transform": [
{
"as": "treeAncestors",
"type": "formula",
"expr": "treeAncestors('tree', datum.id, 'root')"
}
],
"name": "tree-ancestors",
"source": "tree"
},
{
"transform": [{"fields": ["treeAncestors"], "type": "flatten"}],
"name": "tree-ancestors-flatt",
"source": "tree-ancestors"
},
{
"transform": [
{
"type": "filter",
"expr": "indata('selected', 'value', datum.treeAncestors.id)"
}
],
"name": "filtered",
"source": "tree-ancestors-flatt"
},
{
"transform": [{"type": "aggregate", "groupby": ["id"]}],
"name": "filtered-aggregate",
"source": "filtered"
},
{
"transform": [
{
"type": "filter",
"expr": "indata('filtered-aggregate', 'id', datum.id) "
}
],
"name": "filtered-tree",
"source": "tree"
}
Vega doesn't seem to have a recursive way of solving the problem for your question" hey, if all my parents are expanded, then I am visible as a node ".
You can check indeed conditions for all levels you wish to define.
{
"type": "filter",
"expr": "!datum.parent || indata('expandedNodes','name',datum.parent)&&datum.depth==1||(indata('expandedNodes','name',datum.firstParent)&&indata('expandedNodes','name',datum.secondParent)&&datum.depth==2)||(indata('expandedNodes','name',datum.firstParent)&&indata('expandedNodes','name',datum.secondParent)&&indata('expandedNodes','name',datum.thirdParent)&&datum.depth==3)"
}
The code above says to VEGA : " hey check if all of my defined parents are expanded and filter me if any of my parents exist but are not expanded
To see the full solution with your case, please check :
spec
I am a completely new to D3.
I'd like to visualize easy results of tests in bubbles, e.g. Name: mathematics, completed-tests: 340, name: Latin, completed tests: 550.
When I look at the example on https://bl.ocks.org/mbostock/4063269, I can see that only a CSV file is loaded.
According to the documentation it is also possible to load a JSON file, but I didn't find the correct "semantic" for the JSON file.
What should the JSON file look like and how should it be called in the JavaScript section of the index.html?
Note: I want to use version 4.0. Does it matter?
Your question here is not about the correct structure of the JSON file, since there is no correct structure: it depends on the code.
Your question here is, given that specific code you linked, how can you change the CSV for a JSON.
To answer that, you have to understand how d3.csv creates an array of objects. In the Bostock's code you linked, each object in that array has an id and a value key/value pair.
Therefore, your JSON need to have a structure like this:
[{
"id": "flare.analytics.cluster.AgglomerativeCluster",
"value": 1938
}, {
"id": "flare.analytics.cluster.CommunityStructure",
"value": 3812
}, {
"id": "flare.analytics.cluster.HierarchicalCluster",
"value": 2714
}, {
"id": "flare.analytics.cluster.MergeEdge",
"value": 1743
}]
Since you are creating your own JSON you don't need to mind the row function, which removes the objects without value and coerces their values to a number.
Here is the updated bl.ocks using a JSON file (with just some objects): https://bl.ocks.org/anonymous/4ca57ea4393a37bc92091325eba295dd
Have in mind that, if you use the data structure you described in your question...
[{"Name": "mathematics", "completed-tests": 340}, etc...]
... you will have to make several changes in Bostock's code, both in d3.hierarchy and in the node selection.
I have a string that I need to parse into a graph (DAG) data structure using javascript. Included in the data structure are a few attributes I should store, such as the node's id, name, and a label that is given to the link if one exists to another node. So, an example would be
Node1 (id: 1, name: 'first') --('link name')--> Node2 (id:....)
and so forth. Once the data structure is created I do not need to do any more operations on it other than read it (I will later use it to render a visualization with d3). The amount of nodes will not be very many, as several of them are shared.
I am imagining an adjacency list but am not sure how I would encode that in javascript. For instance, I know a json object can have a "field" : "value" structure but can I do that with Object : [list of adjacent Objects]?
you can use lists (arrays) in json. E.g. I could represent a simple directed graph as
{
"NodeA": {"name": "NodeA", "adjacentTo": ["NodeB", "NodeC"]},
"NodeB": {"name": "NodeB", "adjacentTo": ["NodeC", "NodeD"]},
"NodeC": {"name": "NodeC", "adjacentTo": ["NodeA"]},
"NodeD": {"name": "NodeD", "adjacentTo": []}
}
This would be the graph:
C
^^
| \
| \
A -> B -> D
The name field really isn't needed, but you can associate any attributes you want with a node that way.
JavaScript objects must have string keys, but can store any type of value. Of course, the entire point in an id is to allow you to represent a complex type wirh a simple one.
var adjacentTo = {};
adjacentTo[node1.id] = [node2, node3]
While following numerous D3 examples, data usually gets formatted in the format given in flare.json:
{
"name": "flare",
"children": [
{
"name": "analytics",
"children": [
{
"name": "cluster",
"children": [
{"name": "AgglomerativeCluster", "size": 3938},
:
I have an adjacency list as follows:
A1 A2
A2 A3
A2 A4
which I want to convert to the above format. Currently, I am doing this on the server-side but is there a way to achieve this using d3's functions? I found one here, but the approach seems to require modification of the d3 core library which I am not in favor due to maintainability. Any suggestions?
There's no prescribed format, as you can usually redefine your data through various accessor functions (such as hierarchy.children) and array.map. But the format you quoted is probably the most convenient representation for trees because it works with the default accessors.
The first question is whether you intend to display a graph or a tree. For graphs, the data structure is defined in terms of nodes and links. For trees, the input to the layout is the root node, which may have an array of child nodes, and whose leaf nodes have an associated value.
If you want to display a graph, and all you have is a list of edges, then you'll want to iterate over the edges in order to produce an array of nodes and an array of links. Say you had a file called "graph.csv":
source,target
A1,A2
A2,A3
A2,A4
You could load this file using d3.csv and then produce an array of nodes and links:
d3.csv("graph.csv", function(links) {
var nodesByName = {};
// Create nodes for each unique source and target.
links.forEach(function(link) {
link.source = nodeByName(link.source);
link.target = nodeByName(link.target);
});
// Extract the array of nodes from the map by name.
var nodes = d3.values(nodeByName);
function nodeByName(name) {
return nodesByName[name] || (nodesByName[name] = {name: name});
}
});
You can then pass these nodes and links to the force layout to visualize the graph:
http://bl.ocks.org/2949937
If you want to produce a tree instead, then you'll need to do a slightly different form of data transformation to accumulate the child nodes for each parent.
d3.csv("graph.csv", function(links) {
var nodesByName = {};
// Create nodes for each unique source and target.
links.forEach(function(link) {
var parent = link.source = nodeByName(link.source),
child = link.target = nodeByName(link.target);
if (parent.children) parent.children.push(child);
else parent.children = [child];
});
// Extract the root node.
var root = links[0].source;
function nodeByName(name) {
return nodesByName[name] || (nodesByName[name] = {name: name});
}
});
Like so:
http://bl.ocks.org/2949981
D3 doesn't require a specific format. It all depends on your application. You can certainly convert an adjacency list to the format used in flare.json, but this again would be application-specific code. In general, you can't do that as adjacency lists as such don't have "head" or "root" elements you would need to build a tree. In addition, you would need to handle cycles, orphans etc. separately.
Given that you're currently doing the conversion on the server side, I'd be tempted to say that "if it ain't broken, don't fix it" ;)