In order to have a reasonable performance with a lot of svg paths, svg text and svg textpath elements on a leaflet map, I wonder how D3 handles elements which are currently not on screen.
So for example when I zoom in to an area such as Washington State, 99.9% of the world is not shown - is D3 default behaviour to draw all the other elements regardless?
I am basing my project on Mike Bostocks d3 + leaflet example. There are no viewports/ viewbox attributes used - is it done somewhere else? Thanks for your input.
I think there's two parts to this question
Drawing of SVG DOM elements that are off screen
As #LarsKotthoff mentions, it's probably not worth worrying about these, as the browser will probably do a better job than you of optimising them away.
Processing of data that will result in SVG DOM elements being drawn off screen.
I think this is where you can make a difference. If you have data manipulation/processing that is expensive, then processing things that will not be displayed seems like a waste of cycles. The only way I can think of improving this situation is to determine as early as possible whether something will be off screen or not. If it is going to be off screen, then ignore it when doing any further data processing.
In these situations though, you need a way to detect when it moves into view or out of view and either process or not, as appropriate. This may result in some additional overhead that makes it not worth doing.
Your individual situation will determine how effective this can be for you, but if you have a specific example, then users here may be able to assist with re-factoring to help performance.
There are also other things you can do, like re-thinking the visualisation to require less elements in the first place. In my experience performance has not really been an issue until such a point that there is so much information on screen that the value of the visualisation has been diminished. Removing the extraneous information has resulted in improved performance and improved comprehension of the visualisation. Of course, this is my particular experience and there are definitely times when that won't apply.
Related
I'm developing a D3 application that utilizes a lot of text. Since there is alot of text elements, panning around using D3 zoom causes some lag. Is there a way in which I can improve the performance of my application? I'm happy to hear any suggestions, you guys might have. I've been thinking about paginating my data and detecting pan events but my UI is very complex and the user has the freedom of placing things the way he wants so I am not sure how to go about implementing a pan/pagination solution.
Such an open question is bound to get very opinion-based answers, and without more context you're also bound to get some useless ones, but here you go: it depends on what you want and what you mean by "a significant amount".
Supposing that significant amount is > 1000 elements, consider using canvas instead of SVG. Sure, if you can click on individual text nodes, that's more of a hassle, but it should be really good at panning/zooming.
If that is not possible, look at your code. Are you repositioning all text nodes individually? If so, place them all inside one g node and give that node a transform and zoom. In other words, make that node responsible for all global movement, and place the text nodes only relative to each other.
If that is also not possible, consider removing text nodes if they're outside the bounds of the SVG. Repositioning invisible nodes takes a lot of computation power, be smart about it. This is probably the most complex solution, so try it last.
I have a SVG generated map for the game I am developing. I have no problems with the game being open-source and it uses open web technologies such as HTML and SVG. No problems there.
But at the same time I want the players not to be able to see or reverse engineer a map of the whole world (to retain true exploration). For now I generate map using a seed that is secret and not version controlled. So even though the algorithm is known curious players can use open-sourced code to generate "game-like worlds" but not that exact one. This solves the "global" problem.
But since SVG is rendered on a page as a single Voronoi diagram all the data (I don't mind the coordinates of points) would be extractable. Data like resources, land types, biomes, climate etc. could be fetched from SVG to gain an upper hand in finding good locations for settlements.
Any idea how to prevent that? Players have limited vision so I thought about either:
not rendering the whole Voronoi diagram at all (just the visible part), but that could be potentially tricky to do (maybe, haven't looked into it yet),
inserting the resource/land tile data into SVG graph only to visible locations
I can see the benefits of both approaches and if done correctly it could even boost the performance (not rendering the whole thing/rendering with less data) and lead to bigger worlds without impacting performance.
Any other ideas/programming/architectural approaches to help with the issue?
(I am using Vue.js, d3.js, svg-pan-zoom and Laravel backend just in case it helps.)
The ideas that you gave are perfect, but for implementing them, you need to make hard work, and spend much time.
I have a suggestion. Is will work for most of the users. Maybe some users will "hack" it. But I believe it will work for 95% of the times.
You can create a very big rectangle, from the top left point 0,0 until the right bottom point. The rectangle will be white, and it will be over all other shapes.
This way if someone will download the SVG, we will see nothing. Just a big white rectangle.
In you game HTML, you can add a CSS selector, to hide this rectangle.
If you following this method, most of the users (who don't have a photo editing software) will not be able to see the map.
Users who knows how to inspect elements in HTML may see the map. But I believe that most of them who will see a white box, will not believe that there is something behind.
I think that this is a simple temporary approach that you can do, before doing other more defensive ways.
I have a huge SVG which has lots of graphical elements, on mouse wheel zoom event gets triggered which re-paints the whole SVG.
With some profiling I could break down the time taken in such event,
Calculations needed to transform the SVG are taking 1-2ms.
Upon applying the transform, the paint operation is taking 500ms-1.5sec(very sluggish user experience).
As I understand, the paint operation would depend on the render tree of the SVG/HTML which is quite big in this case.
I thought of trying the following ways to improve the user experience
Use 'debouncing' so that I will coalesce mulitple scroll events into
one signal within a given interval.
Possibly use 'requestAnimationFrame' with 'debouncing'.
Look for ways to optimize the render tree of the SVG although I feel
this is difficult in my case because of other constraints.
Are these the right ways or am I moving in the wrong direction?
What other techniques could be used for a smoother user experience in such cases when re-painting is inevitable?
I am trying to build a network graph (like a network for brain) to display millions of nodes. I would like to know to what extent I can push the d3 js to in terms of adding more network nodes on one graph?
Like for example, http://linkedjazz.org/network/ and http://fatiherikli.github.io/programming-language-network/#foundation:Cappuccino
I am not that familiar with d3.js (though I am a JS dev), I just want to know if d3.js is the right tool to build a massive network visualization (one million nodes +) before I start looking at some other tools.
My requirements are simply: build a interactive web based network visualization that can scale
Doing a little searching myself, I found the following D3 Performance Test.
Be Careful, I locked up a few of my browser tabs trying to push this to the limit.
Some further searching led me to a possible solution where you can pre-render the d3.js charts server side, but I'm not sure this will help depending on your level of interaction desired.
That can be found here.
"Scaling" is not really an abstract question, it's all about how much you want to do and what kind of hardware you have available. You've defined one variable: "millions of nodes". So, the next question is what kind of hardware will this run on? If the answer is "anything that hits my website", the answer is "no, it will not scale". Not with d3 and probably not with anything. Low cost smartphones will not handle millions of nodes. If the answer is "high end workstations" the answer is "maybe".
The only way to know for sure is to take the lowest-end hardware profile you plan to support and test it. Can you guarantee users have access to a 64GB 16 core workstation? An 8GB 2 core laptop? Whatever it is, load up a page with whatever the maximum number of nodes is and sketch in something to simulate the demands of the type of interaction you want and see if it works.
How much d3 scales is very dependent on how you go about using it.
If you use d3 to render lots of svg elements, browsers will start to have performance issues in the upper thousands of elements. You can render up to about 100k elements before the browser crashes, but at that point user interaction is basically useless.
It is possible, however, to render lots and lots of lines or circles with a canvas. In canvas, everything is rendered in a single image file. Rather than creating a new element for each node or line, you draw a line in the image file for it. The downside of this is that animation is a bit more difficult, since you can't move elements in a canvas, only draw on top of a canvas or redraw the whole thing. This isn't impossible, but would be computationally expensive with a million nodes.
Since canvas doesn't have nodes, if you want to use the enter/exit/update paradigm with it, you have to put placeholder elements in the DOM. Here's a good example of how to do that: DOM-to-canvas with D3.
Since the memory costs of canvas don't scale with the number of nodes, it makes for a very scalable solution for large visualizations, but workarounds are required to get it to be interactive.
1.) I found a canvas API called EaselJS, it does an amazing job of creating a display list for each elements you draw. They essentially become individually recognizable objects on the canvas (on one single canvas)
2.) Then I saw on http://simonsarris.com/ about this tutorial that can do drag and drop, it makes use of a hidden canvas concept for selection.
3.) And the third approach, a working approach, http://www.lucidchart.com/ , which is exactly what I'm trying to achieve, basically have every single shape on a different canvas, and use to position them. There's a huge amount of canvas.
The question is, what is the easiest way to achieve interactive network diagram as seen on http://www.lucidchart.com/
A side question is, is it better to get text input through positioning on canvas or using multiple canvas (one for rendering text) as in LucidChart
I'm the person who made the tutorials in 2. There's a lot going on here, so I'll try to explain a bit.
I use a hidden canvas for selection simply because it is easy to learn and will work for ANY kind of object (text, complex paths, rectangles, semi-transparent images). In the real diagramming library that I am writing, I don't do anything of the sort, instead I use a lot of math to determine selection. The hidden-canvas method is fine for less than 1000 objects, but eventually performance starts to suffer.
Lucidchart actually uses more than one canvas per object. And it doesn't just have them in memory, they are all there the DOM. This is an organizational choice on their part, a pretty weird one in my opinion. SVG might have made their work a lot easier if thats what they are going to do, as if seems they are doing a lot of footwork just to be able to emulate how SVG works a bit. There aren't too many good reasons to have so many canvases in the DOM if you can avoid it.
It seems to me that the advantage of them doing it that way is that if they have 10,000 objects, when you click, you only have to look at the one (small) canvas that is clicked for selection testing, instead of the entire canvas. So they did it to make their selection code a little shorter. I'd much rather only have one canvas in the DOM; their way seems needlessly messy. The point of canvas is to have a fast rendering surface instead of a thousand divs representing objects. But they just made a thousand canvases.
Anyway, to answer your question, the "easiest" way to achieve interactive network diagrams like lucidchart is to either use a library or use SVG (or an SVG library). Unfortunately there aren't too many yet. Getting all the functionality yourself in Canvas is hard but certainly doable, and will afford you better performance than SVG, especially if you plan on having more than 5,000 objects in your diagrams. Starting with EaselJS for now isn't too bad of an idea, though you'll probably find yourself modifying more and more of it as you get deeper into your project.
I am making one such interactive canvas diagramming library for Northwoods Software, but it won't be done for a few more months.
To answer the question that is sort-of in your title: The fastest method of doing interactiveness such as hit-testing is using math. Any high-performance canvas library with the features to support a lot of different types of objects will end up implementing functions like getNearestIntersectionPoint, getIntersectionsOnRect, pathContainsPoint, and so on.
As for your side question, it is my opinion that creating a text field on top of the canvas when a user wants to change text and then destroying it when the user is done entering text is the most intuitive-feeling way to get text input. Of course you need to make sure the field is positioned correctly over the text you are editing and that the font and font sizes are the same for a consistent feel.
Best of luck with your project. Let me know how it goes.
Using SVG (and maybe libraries as Raphael)!!
Then any element can receive mouse events.