I am rendering an SVG using D3 that shows circles from JSON data. I want to support zooming and dragging. The JSON structure can get very large. Here is my main issue:
Appending circles for all JSON entries doesn't really work. The page becomes way too slow as there might be thousands of <circle> elements in the DOM.
How I'm solving it:
I keep a reduced copy of the data set that I update in the drag function. On each dragging event, I declare an empty data set:
var reducedData = [];
I go over the entire data set and only push to reducedData the circles that have center coordinates that are visible given the current axis. I then erase the SVG and redraw it using reducedData. I do the same process on every zoom event, only appending to reducedData the circles that have a radius greater than 5 pixels given the current zoom ratio.
Although page is very responsive and seems to work well, it's very inefficient and I'm sure this isn't the best way to do this. What are some alternative solutions to my problem? Thanks.
Of course there's always room for improvement, but I think that your approach is already good enough and may it doesn't get so much better than that. However here are some points for you to consider and/or test by yourself if you wish so...
First off I would recommend you to check if any optimization is really needed. In latest versions of Google Chrome, in its DevTools under the "performance" tab, you can use CPU throttling to simulate a slower device. Then using the timeline tool, you can verify if either your data reduction or DOM manipulations are causing any bottlenecking and dropping your frame rate. If not, don't sweat it, you're good to go.
If from your analysis, you find that the data reduction is slowing down your rendering, you can use the timeline tool to find exactly where it is slow and research for faster alternatives.
In the other hand if it is your DOM manipulation that is causing any trouble, make sure that you're using the general update pattern which ensures that you're creating or removing elements only when really needed. Furthermore you may speed up the creation of circles by duplicating them instead of creating new ones.
Usually when too many data items needs to be visualized, as a last resort we switch from SVG to a canvas based visualization, but I think that would be overkill for your context.
Hope it helps and let us know if you have any questions.
I ended up using Crossfilter.js for fast filtering of the data. That way I don't need to manually keep a reduced copy of the data set. I can simply filter it quickly on each drag and zoom event. Thank you for everyone that answered.
How I solved this issue was to only update visible svg elements as the user is panning/zooming.
function pointInDomain(d, domain) {
return d.x < domain[1] && d.x > domain[0]
}
function zoomed() {
xz = d3.event.transform.rescaleX(x);
xGroup.call(xAxis.scale(xz));
var domain = xz.domain();
clippedArea.selectAll("circle")
.style("visibility", d => pointInDomain(d, domain) ? "visible" : "hidden")
.filter(d => pointInDomain(d, domain))
.attr("cx", d => xz(d.x));
}
JSFiddle
Related
I'm developing a D3 application that utilizes a lot of text. Since there is alot of text elements, panning around using D3 zoom causes some lag. Is there a way in which I can improve the performance of my application? I'm happy to hear any suggestions, you guys might have. I've been thinking about paginating my data and detecting pan events but my UI is very complex and the user has the freedom of placing things the way he wants so I am not sure how to go about implementing a pan/pagination solution.
Such an open question is bound to get very opinion-based answers, and without more context you're also bound to get some useless ones, but here you go: it depends on what you want and what you mean by "a significant amount".
Supposing that significant amount is > 1000 elements, consider using canvas instead of SVG. Sure, if you can click on individual text nodes, that's more of a hassle, but it should be really good at panning/zooming.
If that is not possible, look at your code. Are you repositioning all text nodes individually? If so, place them all inside one g node and give that node a transform and zoom. In other words, make that node responsible for all global movement, and place the text nodes only relative to each other.
If that is also not possible, consider removing text nodes if they're outside the bounds of the SVG. Repositioning invisible nodes takes a lot of computation power, be smart about it. This is probably the most complex solution, so try it last.
My page is running a touchmove event which captures the position of the user's finger on the screen via:
xPos = e.originalEvent.touches[0].pageX;
yPos = e.originalEvent.touches[0].pageY;
The page has many layers (created with position:absolute divs) and at this point, I want to calculte how many such layers exist below the user's current position on the screen.
The only method I can think of is to have an array of all the layers' positions and loop through that. However that seems rather processor intensive when there may be hundreds of layers on screen at once.
Is there a simple way in js or JQuery to count the items that exist in a position, or a better practise way to do it than my array suggestion.
As far as I know, there is no such way. My approach would be to give all layers a certain class, select them all and iterate through them. Depending on what you are trying to achieve with this and how often you'll have to perform this calculation, it may be possible to use caching and approximation (e.g. not checking a certain pixel but an area of x^2 pixels and caching the result, etc) to make things run faster.
That being said, I encourage you to first try the solution that you've thought of and see how fast it actually runs. Browsers are pretty fast for such standard operations (think layer drag & drop with boundary checks), so I'm pretty confident that it won't be as slow as you think it will be :)
In order to have a reasonable performance with a lot of svg paths, svg text and svg textpath elements on a leaflet map, I wonder how D3 handles elements which are currently not on screen.
So for example when I zoom in to an area such as Washington State, 99.9% of the world is not shown - is D3 default behaviour to draw all the other elements regardless?
I am basing my project on Mike Bostocks d3 + leaflet example. There are no viewports/ viewbox attributes used - is it done somewhere else? Thanks for your input.
I think there's two parts to this question
Drawing of SVG DOM elements that are off screen
As #LarsKotthoff mentions, it's probably not worth worrying about these, as the browser will probably do a better job than you of optimising them away.
Processing of data that will result in SVG DOM elements being drawn off screen.
I think this is where you can make a difference. If you have data manipulation/processing that is expensive, then processing things that will not be displayed seems like a waste of cycles. The only way I can think of improving this situation is to determine as early as possible whether something will be off screen or not. If it is going to be off screen, then ignore it when doing any further data processing.
In these situations though, you need a way to detect when it moves into view or out of view and either process or not, as appropriate. This may result in some additional overhead that makes it not worth doing.
Your individual situation will determine how effective this can be for you, but if you have a specific example, then users here may be able to assist with re-factoring to help performance.
There are also other things you can do, like re-thinking the visualisation to require less elements in the first place. In my experience performance has not really been an issue until such a point that there is so much information on screen that the value of the visualisation has been diminished. Removing the extraneous information has resulted in improved performance and improved comprehension of the visualisation. Of course, this is my particular experience and there are definitely times when that won't apply.
I wanted to produce a visualization that contains a good deal of nodes with the d3 force layout (more than 500 hundred nodes). While it is working correctly with as much as 200 hundred nodes it gets very slow with about 500, in the sense that the layout hiccups from one frame to the next and events like mouseover on nodes are far from being responsive. This made me ask several questions.
Is there some kind of limit in the number of nodes after which it is not recommended to use the force layout ? If so, is there any other library that could handle the job ?
If I wanted to speed up the process with d3, which parts should be optimized ? I tried keeping the use of css/attributes markup minimal (just gave a radius and a fill color to nodes + stroke-width and stroke color to links) and reduce the use of interaction (mouseover events) but could there be any more optimization done to the force object which holds all of the information ? The size of data must play a certain role...
Thank you for your input !
One way of doing this would be to handle not every tick event, but only a fraction of them, e.g. skipping a specified number or dynamically adapting the number of events depending on other considerations.
If you want smooth movements, add a transition between the positions set in the handled tick events. You can of course combine these ideas as well and skip events while the transition is running, handling the first one after it has been completed.
I would like to make a scatter plot using D3 with the ability of only looking at a small section at a time using some sort of slider across the x-axis. Is there a method in javascript where I can efficiently buffer the data and quickly access the elements as the user scrolls left or right?
My goal is similar to this protovis example here, but with 10 times the amount of data. This example chokes when I make that many data points.
I have done a scatterplot with around 10k points where I needed to filter sections of the plot interactively.
I share a series of tips that worked for me, which I hope some may hopefully help you too:
Use a key function for your .data() operator as it is done at the end of this tutorial. The advantage of using keys is that you do not need to update elements that do not change.
Not related to d3, but I divided my data space into a grid, so that each data point is associated to a single cell (in other words each cell is an index to a set of points). In this way, when I needed to access from, let's say, from x_0 to x_1, I knew what cells I needed, and hence I could access a much more refined set of possible data points (avoiding iterating along all points).
Avoid transitions: from my personal experiences the .transition() is not very smooth when thousand of SVG elements are selected (it may be better now in newer versions or with faster processors)
In my case it was more convenient to make points invisible (.attr("display","none")) or visible rather than removing and creating SVG elements (I wonder if this is more time efficient too)