The problem: Looking for quite some time for a js graph library to create a directed graph (e.g. using dagre layout), with the constrain of non-overlapping edges.
Steps until now
Started with cytoscape.js but as it seems, such a feature doesn't exist.
Continued on with an svg based solution (considering that all elements are in the DOM), d3.js using the dagre-d3, but still the information in the DOM is the path route.
Objective
Find a way to detect edge overlapping, either canvas or svg based.
Create a custom layout to respect this constrain. Will use this as a metric for my convergence algorithm.
Graphical Representation
Below a graphical representation of the objective.
I want to detect that edges 0>1 and 2>3 are overlapping.
Any ideas, thoughts are welcome.
If there is something wrong with my logic, corrections/suggestions are more than welcome.
Finding edge crossings (line intersections) is a fairly simple bit of geometry which is explained here --> https://stackoverflow.com/a/18234609/368214
But then minimising such edge crossings in a graph (zero edge crossings are only possible in planar graphs) is one of the great research challenges of graph layout - https://cs.stackexchange.com/questions/14901/how-to-reduce-the-number-of-crossing-edges-in-a-diagram
Some graph layouts for specific graph types like DAGS such as Sugiyama aim to reduce crossings and similar cytoscape layouts are available at yfiles if that helps (i.e. the hierarchic layout) --> http://apps.cytoscape.org/apps/yfileslayoutalgorithms
Related
I designed an ad hoc library to write SVG images. I use it for the graphical visualisation of large sparse matrices : homology matrices of the comparison of two genomes.
In my case, a usual homology matrix is 20 000 x 20 000 with around 30 000 scattered dots (representing homologies), for instance this homology matrix.
My ad hoc library is not mature enough for new developpements and I am interested by the 3D.js library. I also found the graphical library Cairo and its binding pyCairo interesting.
My needs are :
the possibility to output SVG, PNG or PDF images of the matrix.
when the mouse get over a dot of the homology matrix I would like to show specific informations.
a plus would be the ability to zoom in an area of the matrix, and show new details above a zoom threshold.
I know that 3D.js has dynamic features (interesting for navigating/zooming through the sparse matrix) but I wonder if it will display fast enough my data. It might be a better idea to use Cairo in an application and recompute the image for different zoom levels.
In brief I feel that I have 2 solutions :
write a HTML file, with a 3D.js script that adds SVG elements of the homology matrix and interactivity. 3D.js would be great for sharing matrices from a server.
write a dedicated app (e.g. with GTK) and draw the matrix inside the app with Cairo. Interactions would then be added by asking Cairo to re-draw each time the user interacts with the app.
Do I miss something ? Which solution would you use ?
I'm looking at plotting a scatterplot with a large number of points (500,000 and upwards).
Currently, we're doing this in Python with Matplotlib. It plots the points, and it provides controls to pan and zoom. I don't believe it provides any clustering or points, it just plots them all - doesn't make much sense at the zoomed out view, I suppose, but you can zoom in and they're all there.
I was looking at doing the chart in JavaScript, to make it a bit easier to distribute. I was looking at D3.js, to see if something similar is feasible there. I did find this example of a basic scatterplot:
http://bl.ocks.org/mbostock/3887118
Firstly, would you be able to plot that number of points? (500,000 and upwards) I was under the impression you couldn't due to the overhead of all the DOM objects? Are there ways around this?
Secondly, is there any kind of clustering available, either a library or even just an example of this being done in D3.js?
Thirdly, if anybody knows any good examples of pan/zoom functionality and clustering, or even just a packaged JS library that handles it, that would be awesome.
Fourth, it would be also nice to have click handlers for each point - and to display some text either in a overlay, or even just in a separate window. Any thoughts on this?
Can you draw half a million points with D3? Sure, but not with SVG. You'll have to use canvas (here's a simple example with 10,000 points that includes brush-based selection: http://bl.ocks.org/emeeks/306e64e0d687a4374bcd) and that means that you no longer have individual elements to assign click handlers to. You will not be able to render half a million points with SVG, because all those DOM elements will choke your interface, as you mentioned.
D3 does include quadtree support that can be leveraged for clustering. It's in use in the above example to speed up search but you could use it to nest elements in proximity at certain scales.
Ultimately, your choices are:
1) Some other library/custom implementation that renders in canvas and polls the mouse position to give you the data element rendered at that point.
2) A sophisticated custom D3 approach that nests elements in proximity and only renders SVG elements appropriate at the zoom level and canvas position (pan) you're at.
Yes, D3.js can be made to work with million scale data with two things:
pre-rendering on the server side. For more see here: https://mango-is.com/blog/engineering/pre-render-d3-js-charts-at-server-side/
By aggregating (or clustering) part of the data so that user can interact and expand the graph if need be. For this use collapsible nodes if you can (http://bl.ocks.org/mbostock/1062288).
Also avoid using force layout. It takes time to settle and converge to a stable positioning.
For clustering libraries, I would pick one up off the shelf. I would choose the scikits library from python, there are many in JavaScript but they are not very robust as they mostly cover k-means or hierarchical clustering. I would precalculate the coordinates using scikits by clustering and then render it using D3.
D3 handles Pan and zoom. Again click handlers and text display are available in D3. (http://bl.ocks.org/robschmuecker/7880033)
I'd like to visualize a 20K node dependency graph in d3. Force-directed graphs such as http://bl.ocks.org/mbostock/1153292 are too slow to render in the browser for this number of nodes.
Basically I want to represent nodes containing text and directed edges from one node to another, and add zooming and panning functionality. How can I go about doing this in d3?
Here's an alternative which doesn't seem to use force to lay out the nodes - there's no springing, performs well, and has built in upload/download facility. Its license is MIT/X:
Interactive tool for creating directed graphs using d3.js
directed-graph-creator
Operation:
drag/scroll to translate/zoom the graph
shift-click on graph to create a node
shift-click on a node and then drag to another node to connect them with a directed edge
shift-click on a node to change its title
click on node or edge and press backspace/delete to delete
The zoom behaviour (and pan) you would get basically for free through the zoom behaviour. The layout you would have to do yourself though -- the force layout is pretty much the only thing in D3 you can use to lay out a graph of this kind.
Regardless of what you're using, with 20K nodes anything dynamic is going to be pretty slow -- simply rendering all the elements is going to take quite some time during which the browser will seem unresponsive. An alternative you may want to consider is to pre-render the graph using something more suitable for large amounts of data, save the result as an image (or even static SVG) and add a little bit of D3 code on top for zoom/pan.
Good morning,
just starting with the awesome d3js library ...
I want to show only a portion of a graph with the force directed layout. The idea is to have one node declared as the "center" and show all nodes within a distance of two (for example) from this center node, the neighbors of the center node and the neighbors of the neighbors. If the user clicks on one of the displayed nodes it becomes the "new" center node and a different "subgraph" is displayed. I wonder if there is an example around implementing this kind of subgraph layout and if some kind of a "node-distance" algorithm is already implemented in d3js.
Thanks a lot
martin
UPDATE:
Just found the example Modifying a Force Layout which illuminates how to add and remove nodes and edges from a force directed layout.
I just uploaded a "proof of concept level" of an interactive force directed subgraph.
http://justdharma.com/d3/sub-graph/
In this example I use backbonejs under the hood. Being the first time I implement something with backbonejs, I for sure use it in a very crude manner. While this example might illuminate one way how an interactive sub-graph can be achieved it is for sure not a template how to do it - as said just a proof of concept hack ...
This isn't implemented in D3, and I'm not aware of any examples. What you would have to do is the following:
Set the fixed attribute of the new center node to true to prevent the force layout from changing its position.
Set the px and py attributes of that same node to the center position.
For each node in your force layout, compute the shortest path to the new center node.
Depending on the length of the path in each case, either remove the node or keep it.
The trickiest part here is the computation of the path from each node to the new center, but even this is a pretty standard algorithmic problem. Another thing to keep in mind is that you need to modify the data structures that contain the nodes and links of the force layout in place, i.e. you can't set new nodes and links for the force layout and expect everything to work smoothly.
While learning graph algorithms and canvas HTML element it stuck me that i should have a small graph library of my own in javascript which shows the graphs using canvas element can you point me in right direction so as i can read some code[js/python] as on how to display the graphs and write my own little lib.
PS : i mean edges and Node grapth not the bars and pie charts.
There are quite a few ways to display graphs using various layout algorithms. Two of the primary ones are a Force-Directed layout algorithm, which can be a continuous layout algorithm using a push-pull principle. Next is the GEM layout algorithm which uses a rudimentary physics approach to determine the best "fit".
Here are some examples:
http://vis.stanford.edu/protovis/ex/force.html
bitbucket.org/bwalenz/algorithms/src/
Protovis has probably the most functional. The bitbucket link is an implementation of the GEM algorithm but with no visualization. It just lays out the nodes/edges.
Not really using canvas, but quite a good JS solution for graphs: Raphael Charting plugin
The good thing about Raphael is that it works in all browsers, IE included.
Great lib, for graphics in canvas: processing.js:
http://processingjs.org/
this is a graphic library, that allows you to write build amazing visualization using simple javascript code. have a look.