CanvasRenderingContext2D.drawImage() very slow in Chrome on large canvas - javascript

I'm currently developing a web client (HTML5 and JavaScript) for my city building game project. The client interacts through SignalR with a web server written in C#/.NET, on which all the game logic resides.
The implementation requires quite a large map which is implemented by a set of canvas elements that represent different layers. The actual drawing of the map consists of drawing 25x25px cells, some of which are animated. That means that there's a lot of small 'drawImage'-invocations taking place on the '2D contexts'.
The current implementation works fine and smooth in Mozilla Firefox, Internet Explorer and Edge. It is however tremendously slow on Google Chrome, most likely due to the fact that the implementation does not play nice with its hardware accelerated rendering.
Acquisition of the tile cell .PNG-images are done through downloading them from the web server and storing them in in-memory 'Image'-objects. From there, I draw them directly to the canvas when necessary. If my current research is done correctly this is where the bottleneck resides; the source 'Image'-objects reside in CPU-memory whereas the target Canvas-element is optimized for GPU-memory access, resulting in a lot of swapping.
I have tried moving the 'Image'-objects to a large off-screen 'buffer' canvas (large enough so that the hardware acceleration is supposed to kick in on Chrome) but this does not produce a noticable difference:
https://github.com/Miragecoder/Urbanization/commit/86ac62a785b233eea28c53b8a7d474ef92ffc283
I have also tried implementing deferred invocations of the 'drawImage'-function through requestAnimationFrame but this too did not produce noticeable differences.
I have the following questions:
Do I understand the problem correctly?
What can I do to improve the performance of the web client?
Some links to questions I have researched but so far with no result:
HTML5 Canvas slow on Chrome, but fast on FireFox
https://gamedev.stackexchange.com/questions/32221/huge-performance-difference-when-using-drawimage-with-img-vs-canvas
https://code.google.com/p/chromium/issues/detail?id=170021
Google Chrome hardware acceleration making game run slow

Your main problem seems to be the number of different Image-objects you use to draw to the canvas. You absolutely should use a Textureatlas, putting as many graphics as possible in a single image.
You should then render your sprites from as few main-images as possible by specifying the relevant rectangle:
void ctx.drawImage(image, sx, sy, sWidth, sHeight, dx, dy, dWidth, dHeight);
instead of the whole image:
void ctx.drawImage(image, dx, dy);
see CanvasRenderingContext2D.drawImage() for details. This way, you avoid too many context switches. As Blindman67 mentioned, you should be careful to switch between texture atlases as few times as possible - for example, you may want to use a single texture atlas for all your rendering to canvas1, another one for all your images for canvas2 etc.

Related

p5.js sketch very slow on mobile devices

I'm building a creative simulation in p5.js by employing double pendula. Here is the link to the stripped-down version of the code (I'm not sure it can be made smaller):
https://github.com/amzon-ex/amzon-ex.github.io/tree/master/dp-sketch-stripped
Here is where you can see the animation: https://amzon-ex.github.io/dp-sketch-stripped/dp-sketch.html
When I run this on my laptop (Windows 10, MS Edge 88) I get about 55-60 fps, which is what I want. However, when I run the same sketch on a mobile device (Android, Chrome 88) the performance is very poor and I hardly get 5-6 fps. I do not understand what is complicated about my code that contributes to such a poor performance. I have ran other kinds of p5.js sketches before on mobile and they've worked well.
Any insight is appreciated.
Since I have a few files in the project link I provided, here's a quick guide (there is absolutely no need to read everything):
sketch.js is the key file, which builds the animation. It takes an image, builds an array filled with brightness values from the image (array lumamap, in setup()). Then I draw trails for my pendula where the brightness of the trail at any pixel corresponds to the brightness value in lumamap.
dp-sketch.html is the HTML container for the p5 sketch.
libs/classydp.js houses the DoublePendulum class which describes a double pendulum object.
As I've found out with some help, the issue is tied to varying pixel density on different devices. As mobile devices have higher pixel density compared to desktops/laptops, my 500x500 canvas becomes much bigger on those displays, hence leading to many more pixels to render. Coupled with the fact that mobile processors are weaker in comparison to desktop processors, the lag/low framerate is expected.
This can be avoided by setting pixelDensity(1) in setup(); which avoids rendering larger canvases on dense-pixel devices.

How can I speed up this slow canvas DrawImage operation? WebGL? Webworkers?

I am currently combining two high resolution images into an html5 canvas element.
The first image is a JPEG and contains all color data, and the second image is an alpha mask PNG file.
I then combine the two to produce a single RGBA canvas.
The app is dealing with 2048x2048 resolution images, that need to also maintain their alpha channel. Therefore by using this method as ooposed to simply using PNG's, I have reduced the average filesize from around 2-3mb to 50-130kb plus a 10kb png alpha mask.
The method I use is as follows:
context.drawImage(alpha_mask_png, 0, 0, w, h);
context.globalCompositeOperation = 'source-in';
context.drawImage(main_image_jpeg, 0, 0, w, h);
Unfortunately this operation takes around 100-120ms. And is only carried out once for each pair of images as they are loaded. While this wouldn't normally be an issue, in this case an animation is being rendered to another main visible canvas (of which these high res images are the source art for) which suffers from a very noticable 100ms judder (mostly perceptible in firefox) whenever new source art is streamed in, loaded, and combined.
What I am looking for is a way to reduce this.
Here is what I have tried so far:
Implemented WebP for Google chrome, removing the need to combine the JPEG and PNG alpha mask altogether. Perfect in Chrome only but need a solution mostly for Firefox (IE 10/11 seems to perform the operation far quicker)
I have tried loading the images in a webworker and decoding them both, followed by combining them. All in pure javascript. This works but is far too slow to be of use.
I have also tried using WebP polyfills. Weppy is very fast and when ran in a webworker does not effect the main loop. However it does not support alpha transparency so is of no use which is a real shame as this method is very close. libwebpjs works okay within a webworker but like my manual decoding of the JPEG/PNG, is far too slow.
EDIT: To further clarify. I have tried transferring the data from my webworkers using transferrable objects and have even tried turning the result into a blob and creating an objectURL which can then be loaded by the main thread. Although there is no lag in the main thread any more, the images simply take far too long to decode.
This leaves me with WebGL. I have literally no understanding of how WebGL works other than I realise that I would need to load both the JPEG and PNG as seperate textures then combine them with a shader. But I really wouldn't know where to begin.
I have spent some time playing with the code from the following link:
Blend two canvases onto one with WebGL
But to no avail. And to be honest I am concerned that loading the images as textures might actually take longer than my original method anyway.
So to summarise, I am really looking for a way of speeding up the operation on high resolution images. (1920x1080 - 2048x2048) be it with the use of WebGL or indeed any other method.
Any help or suggestions would be greatly appreciated.

HTML5 Canvas fillRect slow

I'm trying to write a simple raycaster using an HTML5 canvas and I'm getting abysmal framerates. Firefox's profiler reports 80% of my runtime is spent in context2d.fillRect(), which I'm using per column for floors and ceilings and per pixel on the textured walls. I came across this question, and found that fillRect was faster than 1x1 px pictures by 40% on chrome and 4% on firefox. They mention how it's still calculating alpha stuff, although I'd assume if the alpha is 1 it'd have a pass through for opaque rendering? Are there any other methods for doing a lot of rectangle and pixel blitting with javascript I should try out?
A solution I have found to this is to put the fillRect() call inside a path each time you call it:
canvasContext.beginPath();
canvasContext.rect(1, 1, 10, 10);
canvasContext.fill();
canvasContext.closePath();
It seems that the call to rect() adds to the path of the previous rect() call, if used incorrectly this can lead to a memory leak or a buildup of resource usage.
There are two solutions that you could try in order to reduce the CPU usage of your renderings.
Try using requestAnimationFrame in order for your browser to render your canvas graphics whenever it is ready and especially only when the user has the canvas tab opened in the browser. More info: https://developer.mozilla.org/en-US/docs/Web/API/window.requestAnimationFrame
Second solution, depending on whether part or all of your content is dynamic, is to pre-render portions of your canvas on other "hidden" canvas in memory with JavaScript (for instance you split your main canvas in 4 sub canvas then only have to draw 4 elements on screen).
PS: If you're on a mac using Firefox combined with multiple canvas renderings will boost your CPU usage compared to Chrome
Hope this helps

Drawing lots of hexagons and distorting them

I'm starting a web game, and I would like to know the best way to do it.
The game is a field of hexagons seen from the above, that can be rotated/inclined.
What's the best way to do it? What I was going to do was use a canvas and use 2d transforms to simulate 3d rotations, the problem is that:
The game must work on smartphones
For every rotation i must redraw all the canvas, and there could be 200 hexagons on the screen to redraw many times, so i think canvas are too expensive in terms of resources...
There's two ways I can think of:
Using canvas only. This means quite wide browser support, will have few quirks and generally "just work". Potential problems: Performance on low-end machines like you mentioned. But is rotating the game critical to gameplay? If not, you could consider turning that off on slow smartphones. Users will most likely not miss that feature if it's not important to gameplay.
A combination of canvas and CSS3 transforms (rotating cube example). Could give you better performance than pure canvas solution. Potential problems: requires "mixing" of techniques, which always means a risc of running into unexpected problems or quirks.
You should look at www.kineticjs.com
Canvas is the way to go for drawing 100's of shapes because it doesn't write to the DOM for every path like SVG does.
It has several examples where shapes are programmatically drawn in the 1000s. I recently used it for a similar animation here:
http://movable.pagodabox.com

How does Javascript CANVAS works?

Javascript CANVAS is amazing: it allows us to draw something like lines, polygons on the browser screen.
I wonder how does Javascript CANVAS works. For example to draw a line, does it use a series aligned tiny images to simulate the line or some other approach?
Thanks in advance.
Any reasonable implementer would just use a bitmap (stored internally in the browser), and draw to that using OS native drawing commands.
Why does it matter? It's not at all related to HTML+CSS, if that's what you're wondering.
More detail, for detail's sake:
When the browser's HTML parser sees a canvas element (of a given width & height) it needs to allocate an onscreen pixmap to cover that area. It either does this manually (i.e. malloc()) or it calls into some OS native drawing API to create a surface to draw on. The OS native API could be Windows, Gtk, Kde, Qt, or any other drawing library that the implementer of the browser chose. Also, it's highly dependent on the operating system. Internet Explorer probably calls into some Windows native library (i.e. DirectX or WinFooBarMethod()).
Once the drawing surface is created, it's made accessible to the internal guts of the JavaScript interpreter, likely via a pointer or handle to the constructed drawing surface. Then, when the JS interpreter sees an invocation of one of the canvas methods, it turns this into a call to the appropriate OS native command.
So, using the Windows 3.1 style metaphor:
"new canvas(width, height)" = "WinCreatePixmap(width, height)"
"canvas.setPixel(x,y,color)" = "WinSetPixel(x,y,color)"
And using a manually managed pixmap:
"new canvas(width, height)" = "malloc(width * height * sizeof(Pixel))"
"canvas.setPixel(x,y,color)" = "canvas[x][y] = color;"
Again, it shouldn't matter to the JavaScript developer how these methods are implemented. The only people who need to care are the ones who are writing HTML5 compliant web browsers with canvas support.
If you know C++, you can go to the source.
For example, in Firefox, the "graphics context" object is implemented by the class nsCanvasRenderingContext2D. But that class doesn't actually modify the pixels directly. Instead, it asks a separate object, called Thebes, to do that. Thebes in turn delegates this work to a graphics library called Cairo, which typically asks a library provided by your operating system to do the actual pixel work. I imagine it's a similar story everywhere.
At the very bottom, the canvas has a two-dimensional array of pixels. Each pixel is a 32-bit integer. A pixel is set by assigning a value to an element of the array. Somewhere there's a bit of code that determines which pixels to paint and assigns the appropriate values to the appropriate array elements.
In theory, the pixels might be drawn by your video card, but I have heard that graphics cards generally can't be trusted to do 2D graphics, because the hardware is aggressively tuned for 3D gaming and trades away too much accuracy for speed.
If you're interested in how line drawing works, check out Bresenham's Line Drawing Algorithm.
Surely that's implementation-specific to the JavaScript engine browser in question?
You're thinking too much, it's simple:
A canvas is like an image that can be drawn on to the browser.
I think implementation is important. Why does it matter? Look at flash. When you use the drawing API to create complex fractal artwork it is actually creating vector artwork and making every line and curve a child of the object being drawn on, thus it rerenders the vector artwork every frame.. CRASH! or chug... chug........ chug..............
So for complex fractals or art that records equations, I have to use a Bitmap or the render engine CACKS. It DOES make a difference, since now I am trying to transfer some of my flash multimedia to Javascript and encountering differences among browsers.

Categories