I'm trying to improve my D3 force directed graph's performance. Currently it uses SVG elements but as soon as the number of nodes reaches 500 and links ~ 2000 it becomes almost impossible to use. I'm looking at some alternative ways of rendering the graph.
Canvas seems to be a nice option :
http://bl.ocks.org/mbostock/3180395
But is it possible to attach images to nodes on canvas as it's done here:
http://bl.ocks.org/eesur/be2abfb3155a38be4de4
Thanks
You can use drawImage(). Here's some documentation
And yeah, canvas is a good approach or speeding things up — within reason. The force layout itself takes a bunch of CPU cycles no matter how you render it, and your numbers are already quite high for that. Also, while rendering 500 circles into a canvas at 60fps (fps = frames per second) should be doable, rendering 2000 links in addition will already start slowing things down too. Still, it'll be much better than SVG.
In order to know whether — and how much — your optimizations are improving performance, consider using something like stats.js
Related
I've just started out with SVG and have gathered enough information to see that animating these objects is not the same as animating canvas objects. With canvas objects, the canvas is redrawn on every frame, but it seems like with SVG you aren't supposed to be thinking in terms of frames and FPS, but rather in terms of seconds and delays by using the built in <animate> tag.
While I appreciate all the out of the box functionality for SVG animations, I've built up quite an understanding (and library) for how to get my animations working together on canvas using the frames and a timeline paradigm.
I know there is timeline and scheduling support for SVG with libraries like GSAP or SVG.js but I much prefer thinking in terms of FPS and and frameCount.
I was wondering if I might be able to continue using this paradigm, but instead simply update the attributes of my SVG objects on each frame iteration and let the DOM re-render the positions instead of figuring out a way to describe the animation in the <animate> tag.
I'm completely open to suggestions if using the <animate> tag isn't as finicky as I think it is for scheduling my animations/getting them to move together and any suggestions would be much appreciated.
My animation needs aren't to the likes of high performance websites, I just want to step through them to show people some math concepts as they progress.
Updating SVG element attributes is a completely fine way to go about animating them. It can be more performant than clearing all elements in your main SVG tag, and redrawing - but that depends on the drawing, browser, and animation details. But, SVG renders surprisingly quickly, so if you are just stepping through tailored frames to demonstrate a concept, and if it would be more convenient to think of each frame independently, you should be completely fine to also clear and redraw. If your project is intended for a website, make sure to test your implementation on Chrome and Firefox at least as they have slight differences.
I am trying to build a network graph (like a network for brain) to display millions of nodes. I would like to know to what extent I can push the d3 js to in terms of adding more network nodes on one graph?
Like for example, http://linkedjazz.org/network/ and http://fatiherikli.github.io/programming-language-network/#foundation:Cappuccino
I am not that familiar with d3.js (though I am a JS dev), I just want to know if d3.js is the right tool to build a massive network visualization (one million nodes +) before I start looking at some other tools.
My requirements are simply: build a interactive web based network visualization that can scale
Doing a little searching myself, I found the following D3 Performance Test.
Be Careful, I locked up a few of my browser tabs trying to push this to the limit.
Some further searching led me to a possible solution where you can pre-render the d3.js charts server side, but I'm not sure this will help depending on your level of interaction desired.
That can be found here.
"Scaling" is not really an abstract question, it's all about how much you want to do and what kind of hardware you have available. You've defined one variable: "millions of nodes". So, the next question is what kind of hardware will this run on? If the answer is "anything that hits my website", the answer is "no, it will not scale". Not with d3 and probably not with anything. Low cost smartphones will not handle millions of nodes. If the answer is "high end workstations" the answer is "maybe".
The only way to know for sure is to take the lowest-end hardware profile you plan to support and test it. Can you guarantee users have access to a 64GB 16 core workstation? An 8GB 2 core laptop? Whatever it is, load up a page with whatever the maximum number of nodes is and sketch in something to simulate the demands of the type of interaction you want and see if it works.
How much d3 scales is very dependent on how you go about using it.
If you use d3 to render lots of svg elements, browsers will start to have performance issues in the upper thousands of elements. You can render up to about 100k elements before the browser crashes, but at that point user interaction is basically useless.
It is possible, however, to render lots and lots of lines or circles with a canvas. In canvas, everything is rendered in a single image file. Rather than creating a new element for each node or line, you draw a line in the image file for it. The downside of this is that animation is a bit more difficult, since you can't move elements in a canvas, only draw on top of a canvas or redraw the whole thing. This isn't impossible, but would be computationally expensive with a million nodes.
Since canvas doesn't have nodes, if you want to use the enter/exit/update paradigm with it, you have to put placeholder elements in the DOM. Here's a good example of how to do that: DOM-to-canvas with D3.
Since the memory costs of canvas don't scale with the number of nodes, it makes for a very scalable solution for large visualizations, but workarounds are required to get it to be interactive.
I'm making a little game with HTML5 and MooTools and I have performance problems on firefox. I have implemented a counter to determine how often my update method gets called and it returns 64 times per second. The result seems much much slower (like 30 FPS). I think my problem is actually described on this article http://blog.sethladd.com/2011/03/measuring-html5-browser-fps-or-youre.html. I couldn't find out a way to solve this directly, but I think I can optimize the perforance.
I think one big problem in my logic is that I draw every single object on the canvas directly. I have done some games in Java before and I had great performance improvements with manipulating an image (drawing in the memory) and drawing just the final image. This way the browser would have much less requests to draw something and perhaps draw faster.
Is there a way to do this? I have found some libraries for image manipulation in JavaScript, but I would like to do it myself.
I can't show you the full code, because the project is for school and because it is way too big (~1500 lines of code).
http://www.html5rocks.com/en/tutorials/canvas/performance/
Maybe this will help. It shows you how to improve performance by using an offscreen canvas to render your scene.
I am working on a mobile web dice simulator. The initial prototype is here: http://dicewalla.com
I currently have one large canvas where I draw all the dice. I am planning to re-write the code in way that is more MVC and easier for me to update. I think it would be easier for me to generate a small canvas for each die object than to draw all of the dice on the big canvas and keep updating that big canvas.
My question is if there is a bad performance hit in having the browser create lots of little canvases vs one big one. It's hard to test locally, I was hoping someone here might know what the best practice is.
Multiple canvases usually allow for better performance as you're able to selectively re-render.
If you have only one canvas and want to update one die, you'll typically have to redraw the entire canvas. On the other hand, multiple canvases allow you to update only the dice that need to be redrawn. That's an increase in efficiency.
Further, you shouldn't see any noticeable difference in loading 1 canvas versus 100.
In terms of performance, like was mentioned earlier, there should be little difference between 1-100 canvas elements if you're not updating the graphics on a regular basis. (ie: static graphics / no animation)
Most of the references around the net regarding multiple canvases tend to deal with cases where you have multiple layers and need to handle drawing on top of other things with transparency.
That being said, what you're doing with dicewalla doesn't look like it will gain anything from having multiple canvases.
Also you can selectively redraw the regions of a single canvas to get better performance if updating the entire canvas is a bottleneck. This gives you the performance benefits of having multiple canvases without having to deal with managing and creating those elements.
I'm investigating the possibility of producing a game using only HTML's canvas as the display media. To take an example task I need to do, I need to construct the game environment from a number of isometric tiles. Of course, working in 2D means they by necessity come in rectangular packages so there's a large overlap between tiles.
I'm old enough that the natural solution to this problem is to call BitBltMasked. Oh wait, no, an HTML canvas doesn't have something as simple and as pleasing as BitBlt. It seems that the only way to dump pixel data in to a canvas is either with drawImage() which has no useful drawing modes that ignore the alpha channel or to use ImageData objects that have the image data in an array.. to which every. access. is. bounds. checked. and. therefore. dog. slow.
OK, that's more of a rant than a question (things the W3C like tend to provoke that from me), but what I really want to know is how to draw fast to a canvas? I'm finding it very difficult to ditch the feeling that doing 100s of drawImages() a second where every draw respects the alpha channel is inherently sinful and likely to make my application perform like arse in many browsers. On the other hand, the only way to implement BitBlt proper relies heavily on a browser using a hotspot-like execution technique to make it run fast.
Is there any way to draw fast across every possible implementation, or do I just have to forget about performance?
This is a really interesting problem, and there's a few interesting things you can do to solve it.
First, you should know that drawImage can accept a Canvas, not just an image. The "sub-Canvas"es don't even need to be in the DOM. This means that you can do some compositing on one canvas, then draw it to another. This opens a whole world of optimization opportunities, especially in the context of isometric tiles.
Let's say you have an area that's 50 tiles long by 50 tiles wide (I'll say meters for the sake of my own sanity). You might divide the area into 10x10m chunks. Each chunk is represented by its own Canvas. To draw the full scene, you'd simply draw each of the chunks' Canvas objects to the main canvas that's shown to the user. If only four chunks (a 20x20m area), you would only perform four drawImage operations.
Of course, each of those individual chunks will need to render its own Canvas. On game ticks where nothing happens in the chunk, you simply don't do anything: the Canvas will remain unchanged and will be drawn as you'd expect. When something does change, you can do one of a few things depending on your game:
If your tiles extend into the third dimension (i.e.: you have a Z-axis), you can draw each "layer" of the chunk into its own Canvas and only update the layers that need to be updated. For example, if each chunk contains ten layers of depth, you'd have ten Canvas objects. If something on layer 6 was updated, you would only need to re-paint layer 6's Canvas (probably one drawImage per square meter, which would be 100), then perform one drawImage operation per layer in the chunk (ten) to re-draw the chunk's Canvas. Decreasing or increasing the chunk size may increase or decrease performance depending on the number of update you make to the environment in your game. Further optimizations can be made to eliminate drawImage calls for obscured tiles and the like.
If you don't have a third dimension, you can simply perform one drawImage per square meter of a chunk. If two chunks are updated, that's only 200 drawImage calls per tick (plus one call per chunk visible on the screen). If your game involves very few updates, decreasing the chunk size will decrease the number of calls even further.
You can perform updates to the chunks in their own game loop. If you're using requestAnimationFrame (as you should be), you only need to paint the chunk Canvas objects to the screen. Independently, you can perform game logic in a setTimeout loop or the like. Then, each chunk could be updated in its own tick between frames without affecting performance. This can also be done in a web worker using getImageData and putImageData to send the rendered chunk back to the main thread whenever it needs to be updated, though making this work seamlessly will take a good deal of effort.
The other option that you have is to use a library like pixi.js to render the scene using WebGL. Even for 2D, it will increase performance by decreasing the amount of work that the CPU needs to do and shifting that over to the GPU. I'd highly recommend checking it out.
I know that GameJS has blit operations, and I certainly assume any other html5 game libraries do as well (gameQuery, LimeJS, etc etc). I don't know if these packages have addressed the specific array-bounds-checking concern that you had, but in practice their samples seem to work plenty fast on all platforms.
You should not make assumptions about what speedups make sense. For example, the GameJS developer reports that he was going to implement dirty rectangle tracking but it turned out that modern browsers do this automatically---link.
For this reason and others, I suggest to get something working before thinking about the speed. Also, make use of drawing libraries, as the authors have presumably spent some time optimizing performance.
I have no personal knowledge about this, but you can look into the appMobi "direct canvas" HTML element which is allegedly a much faster version of normal canvas, link. I'm confused about whether this works in all browsers or just webkit browsers or just appMobi's own special browser.
Again, you should not make assumptions about what speedups make sense without a very deep knowledge of web browser internal processes. That webpage about "direct canvas" mentions a bunch of things that slow down canvas-drawing: "Reflowing text, mapping hot spots, creating indexes for reference links, on and on." Alpha-blending and array-bounds-checking are not mentioned as prominent causes of slowness!
Unfortunately, there's no way around the alpha composition overhead. Clipping may be one solution, but I doubt there would be much, if any, performance gain. Not to mention how complicated such a route would be to implement on irregular shapes.
When you have to draw the entire display, you're going to have to deal with the performance hit. Although afterwards, you have a whole screen's worth of pre-calculated alpha imagery and you can draw this image data at an offset in one drawImage call. Then, you would only have to individually draw the new tiles that are scrolled into view.
But still, the browser is having to redraw each pixel at a different location in the canvas. Which is quite expensive. It would be nice if there was a method for just scrolling pixels, but no luck there either.
One idea that comes to mind is that you could implement multiple canvases, translating each individual canvas instead of redrawing the pixels. This would allow the browser to decide how to redraw those pixels, in a more native way, at least in theory anyway. Then you could render the newly visible tiles on a new, or used/cached, canvas element. Positioning it to match up with the last screen render.
But that's just my two blits... I mean bits... duh, I mean cents :]