I have been trying to develop a web based application to help in the graphing of nodes and their interactions.
I have attempted to use the Sigma.Js with the Force Atlas extension.
For my simple tests (few nodes) the results are quite good-looking, however with an additional thousand nodes the result becomes quite a mess.
Is there any such way to make the result more view able? (easier on the eyes/not just 1 big blob) How would I go about doing this? Are there any algorithms already written(that I may implement?)
You can try the Fruchterman-Reingold force layout (for which there is a sigma plugin). It specifically minimises the number of links that cross each other, so it is in general more suitable for large graphs (unless all the nodes have lots of connections).
In addition, the fisheye plugin may help to make more sense of the graph after it has been drawn.
sigma.layout.forceAtlas2 scales much better, however it won't do miracles if the graph has a strong density of connections.
Related
I'd like to achieve something similar to this diagram using d3:
https://gojs.net/latest/samples/records.html
I was trying to look it up on the web but I hardly found any resources to get started with.
I'm quite new to d3 so I don't know how to approach this task, any help or guidance would be appreciated!
This is another one of those questions where people need to implement a sophisticated diagram visualization and they think just because D3 is a lot about visualization, it will be a good fit.
IMHO it is not.
D3 could be a part of the solution, but it certainly cannot be the solution alone: It's almost like asking whether JavaScript can be used to create this kind of diagram. Of course it can! D3 is just a very thin (but very useful) layer on top of the DOM+JavaScript, that can help you with crunching numbers, work with colors, coordinate systems, create DOM elements, and simplifies working with the DOM. Think of it as jquery for DOM and data plus a lot of very nice demos. But the value very often is in the demos, rather than in D3 itself: You need to implement a lot of things to get from a simple mapping from data to dom elements to a sophisticated diagram visualization like the one you are referring to.
If you don't want to implement and maintain most of the low-level diagram logic yourself, you should rather be looking at a diagramming solution, rather than a tool that will help you create SVG elements elegantly with less code.
Look at this question to see a list of graph and diagram visualization software. Agreed, D3 is also on this list (for the same reason you are asking this question), but there are also far more capable tools on that list that you should be looking at, my recommendation being either the one that you've found already or preferrably this one if your requirements are more sophisticated.
I have looked at d3.js, but it seems to not work well for large data sets. I might try to switch from svg to a canvas and see the effect, does anybody have any insight on how well this works for large data sets.
Or should I look into vivagraph.js or another graphing engine?
Additionally I saw similar posts on the same question, but the answers were from 3 years ago, so I'm looking to see if anything new is better.
Thanks
Depending on your definition of large, D3 might still work for you. If you leverage canvas, you can get to the low thousands of nodes and 10,000 or so edges:
http://bl.ocks.org/emeeks/e330141d1279c6115a3a
http://emeeks.github.io/introspect/block_block.html
If you're dealing with more than that in the browser, there are some interesting solutions on the horizon that use WebCL and WebGL like https://github.com/Superconductor/superconductor that can do massive networks and edge bundling and such.
I am trying to build a network graph (like a network for brain) to display millions of nodes. I would like to know to what extent I can push the d3 js to in terms of adding more network nodes on one graph?
Like for example, http://linkedjazz.org/network/ and http://fatiherikli.github.io/programming-language-network/#foundation:Cappuccino
I am not that familiar with d3.js (though I am a JS dev), I just want to know if d3.js is the right tool to build a massive network visualization (one million nodes +) before I start looking at some other tools.
My requirements are simply: build a interactive web based network visualization that can scale
Doing a little searching myself, I found the following D3 Performance Test.
Be Careful, I locked up a few of my browser tabs trying to push this to the limit.
Some further searching led me to a possible solution where you can pre-render the d3.js charts server side, but I'm not sure this will help depending on your level of interaction desired.
That can be found here.
"Scaling" is not really an abstract question, it's all about how much you want to do and what kind of hardware you have available. You've defined one variable: "millions of nodes". So, the next question is what kind of hardware will this run on? If the answer is "anything that hits my website", the answer is "no, it will not scale". Not with d3 and probably not with anything. Low cost smartphones will not handle millions of nodes. If the answer is "high end workstations" the answer is "maybe".
The only way to know for sure is to take the lowest-end hardware profile you plan to support and test it. Can you guarantee users have access to a 64GB 16 core workstation? An 8GB 2 core laptop? Whatever it is, load up a page with whatever the maximum number of nodes is and sketch in something to simulate the demands of the type of interaction you want and see if it works.
How much d3 scales is very dependent on how you go about using it.
If you use d3 to render lots of svg elements, browsers will start to have performance issues in the upper thousands of elements. You can render up to about 100k elements before the browser crashes, but at that point user interaction is basically useless.
It is possible, however, to render lots and lots of lines or circles with a canvas. In canvas, everything is rendered in a single image file. Rather than creating a new element for each node or line, you draw a line in the image file for it. The downside of this is that animation is a bit more difficult, since you can't move elements in a canvas, only draw on top of a canvas or redraw the whole thing. This isn't impossible, but would be computationally expensive with a million nodes.
Since canvas doesn't have nodes, if you want to use the enter/exit/update paradigm with it, you have to put placeholder elements in the DOM. Here's a good example of how to do that: DOM-to-canvas with D3.
Since the memory costs of canvas don't scale with the number of nodes, it makes for a very scalable solution for large visualizations, but workarounds are required to get it to be interactive.
I'm about to develop a UI for medical research application. I need to make a time series line graph. Main issue is the amount of data:
5,000 points per graph, with a few of them displayed simultaneity. I’m expecting 50,000 points processed all together.
The question is what presentation library?
The main features I’m looking for are: Handles huge data sets, Zoom, annotations, live update.
I’m already looking into http://dygraphs.com/ and http://meteorcharts.com/
I wouldn't want any library that renders the data as DOM elements, or that uses SVG (from performance perspective)
Well, I think I'll give everyone my own answer to my question:
Dygraphs (http://dygraphs.com/) seems to be on the spot. The documentation, although a lot of apparent efforts, leaves a lot to be desired. But from performance, features and flexibility, it's the best I've seen. At least for my current project needs.
Way to go, Dygraphs!
Have you checked out D3? I'm using that for a lot of graph visualization. It looks like this example renders to svg.
My stuff renders to a SVG for force graph visualizations too, but it looks like D3 might be able to use either a canvas or SVG, but I'm not positive about what all can be rendered to which. Honestly, I'd give D3 a try even with SVG, it might be fast enough. I remember reading something about someone simulating thousands of particles using D3's force graph visualizations without issues. It's SUPER easy to get your data into the right format for it to use.
Good luck!
I am developing a very similar application.
I am currently using Flot for the chart rendering. It handles annotations and zoom, take a look at their plugin library.
I recommend this downsampling plugin which will speed up graph rendering. Rendering 5000 points on your graph is useless: you have less vertical pixels on your screen than that! so this library will only render those that actually have a visual importance.
This only gives you the graph. You may want some kind of dashboard to present all that... I am currently looking at Grafana, which is used for a totally different purpose but makes awesome dashboards. It may be possible to "rip out" their dashboarding features (it uses Flot as well).
Another option is Hightcharts, but that's not free.
Check raphael js Library
Raphaël is a small JavaScript library that should simplify your work with vector graphics on the web. If you want to create your own specific chart or image crop and rotate widget, for example, you can achieve it simply and easily with this library.
With d3js it is possible to visualize node graphs, like this example
Now I would like to let the user interact with the graph. It must be possible to:
create/delete nodes and edges
attach data to a node/edge, by clicking on it.
push the data to the server, so that it can be made persistent
Is it possible to do that with d3js? Is there any other tool which offers this kind of functionality?
Sure, all of these things are relatively easy to do with d3. You can see lots of different examples of force directed graphs at http://bl.ocks.org/mbostock. Here are a couple that you might find useful:
Adding nodes and edges to a graph:
http://bl.ocks.org/mbostock/929623
http://bl.ocks.org/rkirsling/5001347
Loading data from an XML file to create a layout
http://bl.ocks.org/mbostock/1080941
Listening for mouse over
http://bl.ocks.org/mbostock/2706022
From personal experience I would say that d3 would be the best tool for doing this sort of thing. I'd suggest playing around with it a bit and then asking more specific questions when you have them.