I've been working on trying to make some visual effects for a javascript game I'm creating, and I noticed a piece of code that I'm using to modulate the color of my sprites makes my browsers memory usage go up and up, seemingly without limit.
You can see the code and the memory leak here: http://timzook.tk/javascript/test.html
This memory leak only happens in my updateimage() function when I call getImageData from my canvas context every frame (via setInterval) in order to make a new ImageData object to recolor. I would have thought that javascript's garbage collector would be destroying the old one, but if not I have no idea how to destroy it manually. Any help figuring out why it does this or how to fix it would be appreciated.
My question is very similar to this one: What is leaking memory with this use of getImageData, javascript, HTML5 canvas However, I NEED my code to run every frame in the functioned called by setInterval, his solution of moving it outside the setInterval function isn't an option for me, and I can't leave a comment asking if he found some other method of solving it.
Note to people testing it out: Since this example uses getImageData, it can't be tested out locally just by throwing it in a .html file, a web server is required. Also it obviously uses HTML5 elements so some browsers won't work with it.
Edit: *SOLVED* Thanks, the solution below fixed it. I didn't realize that you could use a canvas element the same way as you could use an image in drawImage(), I restructured my code so it now uses significantly less memory. I uploaded this changed code to the page linked above, if anyone wants to see it.
You aren't getting a memory leak from your calls to getImageData(). The source of your problem is this line:
TempImg.src = ImgCanvas.toDataURL("image/png");
Effectively, every time that line of code is executed the browser "downloads" another image and stores it in memory. So, what you actually end up with is a cache that is growing very quickly. You can easily verify this by opening the site in Chrome and checking the resources tab of the developer tools (ctrl+shift+i).
You can work around this by making a TempImgCanvas and storing your image data on that canvas instead of keeping an image object updated after every call to your updateimage() loop.
I have to step away for a while, but I can work up an example in a few hours when I get back, if you would like.
Edit: I restructured things a bit and eliminated your caching issue. You just have to make two changes. The first is replacing your updateimage() function with this one:
function updateimage() {
var TempImgData = ImgContext.getImageData(0, 0, ImgCanvas.width, ImgCanvas.height);
var NewData = TempImgData.data;
var OrigData = ImgData.data;
//Change image color
var len = 4*ImgData.width*ImgData.height-1;
for(var i=0;i<=len;i+=4) {
NewData[i+0] = OrigData[i+0] * color.r;
NewData[i+1] = OrigData[i+1] * color.g;
NewData[i+2] = OrigData[i+2] * color.b;
NewData[i+3] = OrigData[i+3];
}
//Put changed image onto the canvas
ImgContext.putImageData(TempImgData, 0, 0);
}
The second is updating the last line in draw() to read as follows:
drawImg(ImgCanvas, Positions[i].x, Positions[i].y, Positions[i].x+Positions[i].y);
Using this updated code, we simply refer to the original base (white) image data and calculate new values based on that every time we go through the updateimage() function. When you call getImageData() you receive a copy of the image data on the canvas, so if you edit the canvas or the data, the other one remains untouched. You were already grabbing the original base image data to begin with, so it made sense to just use that instead of having to re-acquire it every time we updated.
Also, you'll notice that I modified your loop that changes the image color slightly. By obtaining a handle to the actual data array that you want to access/modify, you save yourself having to resolve the array location from the parent object every time you go through the loop. This technique actually results in a pretty nice performance boost. Your performance was fine to start with, but it can't hurt to be more efficient since it's pretty straight-forward.
Related
I simply want to restart a canvas on a website. I do not want to reload the page, so the restart has to be done dynamically. Problem : after few restarts the animation gets slower and the memory increases.
Actually i face a more complex situation : I use a javascript framework to load pages, MeteorJS with IronRouter. Changing the URL only changes the DOM (nodes are removed) but the 'page' is not really left. So i thought that coming back to my page with the canvas can be assimilated to a dynamic canvas restart, since i have no solution to that too.
That is why i describe the problem from two points of view (same problem on both) :
first a simple page where a button asks the canvas to restart
then applying the solution to MeteorJS with IronRouter and try to come back on my page many times.
Note : my canvas content is some WebGL (in ThreeJS).
1. Deleting and re-initializing canvas dynamically.
In this case i face those two problems :
animation not smooth
memory increase. To be precise :
(source: hostingpics.net)
Page loaded, playing with canvas / 2. Restarting the canvas 30-40 times (i did not stop restarting so i do not know why the increase is not linear) / 3. Playing with canvas / 4. Closing the tab (reloading the page only frees a small fraction of what has been mobilized)
Few questions on SO yet asked how to solve those issues, but reproducing the solutions partially solves the problem. I have read two main things :
the canvas-related parts (nodes, variables, functions) have to be cleaned from the DOM (That is something Meteor does. But in this first part it stays relevant and yet is not enough to prevent the issues).
if requestAnimationFrame has been called, it needs to get assigned to a variable so cancelAnimationFrame(variable) can stop it. That indeed seems to prevent the frame rate drop. While memory stays high, at least the movements remain fluent.
With those solutions the memory problem still happens.
Here is an example : http://codepen.io/Astrak/pen/RNYLxd
2. In MeteorJS : i cannot get this partial solution work.
Here is what i wrote in my app, with id=requestAnimationFrame(myAnimation) declared globaly :
Template.myTemplate.destroyed=function(){
cancelAnimationFrame(id);
document.body.removeChild(document.querySelector('canvas'));//useful in Meteor ?
};
Despite those instructions, my animation gets slow much quicker with MeteorJS than in the previous example : it becomes unusable after 3-4 page change. And same memory problem.
Thanks for every comment and answer and happy coding :)
Related questions :
Performance drop on multiple restart of scene from threejs
Performance drops when trying to reset whole scene with Three.js
How do we handle webgl context lost event in Three.js
Pause, resume and restart Canvas animations with JS
How can I restart my function?
JavaScript restart canvas script
For memory problem, I'd recommend Chrome's javascript profiler - you can compare snapshots before and after canvas is reloaded and see which objects are not freed from memory.
Problem usually comes from event handlers and/or using closures. If you, for example, hook a function to window object's onresize event, all objects referenced in this function will stay in memory as long as window exists. So always remove event handlers from global objects if you want to refresh page content without actually refreshing the browser session.
Similar with closures. They will create link between function's scope and referenced variables preventing GC to collect them. Check this https://www.meteor.com/blog/2013/08/13/an-interesting-kind-of-javascript-memory-leak
I am now entering Kinetic and it has made it far easier for me to draw on canvas. However, building a game app, I need to clear the rectangle on every animation request. They're controlled by an fps cap script, but still, there are about 50 updates per second.
Kinetic's .removeChildren() method not only clears the canvas, it deletes the canvas node from the DOM. Doing so not only makes DOM queries inconsistent by intervals of .02 second, but also drops my FPS rate by about 60% in comparison to stock HTML5 canvas handling on every machine I ran the game on.
Is there a KineticJS method for clearing the canvas in a manner such as clearRect()'s?
Edit:
I have also made sure it's not a problem on any other part of the program. Call stack doesn't overflow, the FPS drop is just due to DOM changing twice every .02 second.
Edit 2:
I have tried the following:
Ignore the layer before and create a blank rectangle to fill up the visible part of the canvas. It dropped my frame rate to about 14 FPS;
Use the .clear() method. It solved the DOM consistency problem but the frame rate got even lower than before.
It seems the only solution would be calling the default HTML5 clearRect() method, but that would mean creating the canvas element by hand (and possibly making Kinetic useless as a library for my app).
Edit 3:
As for the app, I've started using standard HTML5 canvas since I have a deadline. I'd still like to see a Kinetic solution though - it might be helpful in the future. It surprises me to see such a simple thing is so hard, if not impossible, in a popular library like KineticJS.
You can use layer.clear with a bounding area to clear just the "dirty" part of your layer.
// tell layer not to auto-clear on layer.draw
layer.setClearBeforeDraw(false);
// clear the "dirty" portion of the canvas
layer.clear(100, 100, 150, 150);
// adjust some animation values and
// just draw the element that has changed
myRect.draw();
You should try to create new Layer for example:
var newLayer = new Kinetic.Layer();
Or call this function:
Canvas.clear();
Kinetic makes it very easy to draw using layers, groups and shapes.
If your view is properly make of these items you can easily remove them and they will be removed from the stage.
Perhaps you need to rewrite you code to make it work better in kinetic.
if you think your code is properly written you can try (as a workaround) to create kinetic rectangle and fill it with whatever you want to simulate a clear.
I'm writing a game in javascript canvas, and I experience lag sometimes. I think this is because I draw pretty much to the canvas.
My background for example isn't static, it moves from the right to the left, so each time, I have to clear the entire canvas. And I'm not clearing with just a color, I'm clearing with an image that moves every cycle of the gameloop.
I think this is an expensive operation and I was thinking if there is something to make this operation less expensive, like for example using double buffering for clearing the background (no idea if you can use double buffering just for clearing the background?).
Or should I use double buffering in the entire game?
(I don't think so because I've read that the browser already does that for you)
Since you experience your lag 'sometimes', and it's not a low frame rate issue, i would rather move my eyes towards the evil garbage collector : whenever it triggers, the application will freeze for up to a few milliseconds, and you'll get a frame miss.
To watch for this, you can use google profiling tools, in TimeLine / Memory / press the record button : you can see that a GC occurred when there's a sudden drop in the memory used.
Beware when using this tool, that it slows down a bit the app, and it creates garbage on its own (!)...
Also, since any function call for example creates a bit of garbage, so you can't have a full flat memory line.
Below i show the diagram of a simple game i made before i optimized memory use,
there's up to 5 GC per second :
And here the diagram of the very same game after memory optimisation : there's something like 1 GC per second. Because of the limitations i mentioned above, it is in fact the best we can get, and the game suffers no frame drop and feels more responsive.
To avoid creating garbage
- never create object, arrays or functions.
- never grow or reduce an array size.
- watch out for hidden object creation : Function.bind or Array.splice are two examples.
You can also :
- Pool (recycle) your objects
- use a particle engine to handle objects that are numerous / short lived.
I made a pooling lib (here) and a particle engine (here) you might use if you're interested.
But maybe first thing to do is to seek where you create objects, and have them created only once. Since js is single-threaded, you can in fact use static objects for quite a few things without any risk.
Just one small expl :
function complicatedComputation(parameters) {
// ...
var x=parameters.x, y = parameters.y, ... ...
}
// you can call creating an object each time :
var res = complicatedComputation ( { x : this.x, y : this.y, ... ... } );
//... or for instance define once a parameter object :
complicatedComputation.parameters = { x :0, y:0, ... ... };
// then when you want to call :
var params = complicatedComputation.parameters;
params.x = this.x ; params.y = this.y; ... ...
var res = complicatedComputation(params);
It has the drawback that previous calls parameters remains, so you don't get undefined if you don't
set them, but previous value, so you might have to change bit your function.
But on the other hand, if you call several times the function with similar parameters, it comes very handy.
Happy memory hunting !
Double buffering is a technique to reduce flickering: You draw to one buffer while another buffer is displayed, and then swap them out in a single operation, so the user doesn't see any of the drawing in a partial state.
However, it does not really help for performance problems at all, as there is still the same amount of drawing. I would not try to use double buffering if you don't have a problem with flickering, as it requires more memory and flickering may be implicitly prevented by the system by similar or other means.
If you think drawing the background is too expensive, there are several things that you could look into:
Do you scale the background image down while drawing? If so, create a downscaled version once and use this for clearing the background, reducing the drawing costs per iteration.
Remember the "dirty" areas and just draw the portions of the background that were obscured. This will add some management overhead but reduce the number of pixels that need to be touched significantly
Make the background the background image of the canvas DOM element and just clear the canvas to transparency in each iteration. If the canvas is very large, you could make this faster by remembering the "dirty" areas and just clearing them.
Are you sure painting the background is the main cause of lag? Do you still have lag when you only repaint the background and do not do much else?
I am trying to convert the canvas element on this page to a png using the following snippet (e.g. enter in JavaScript console):
(function convertCanvasToImage(canvas) {
var image = new Image();
image.src = canvas.toDataURL("image/png");
return image;
})($$('canvas')[0]);
Unfortunately, the png I get is completely blank. Notice also that the original canvas goes blank after resizing the page.
Why does the canvas go blank? How can I convert this canvas to a png?
Kevin Reid's preserveDrawingBuffer suggestion is the correct one, but there is (usually) a better option. The tl;dr is the code at the end.
It can be expensive to put together the final pixels of a rendered webpage, and coordinating that with rendering WebGL content even more so. The usual flow is:
JavaScript issues drawing commands to WebGL context
JavaScript returns, returning control to the main browser event loop
WebGL context turns drawing buffer (or its contents) over to the compositor for integration into web page currently being rendered on screen
Page, with WebGL content, displayed on screen
Note that this is different from most OpenGL applications. In those, rendered content is usually displayed directly, rather than being composited with a bunch of other stuff on a page, some of which may actually be on top of and blended with the WebGL content.
The WebGL spec was changed to treat the drawing buffer as essentially empty after Step 3. The code you're running in devtools is coming after Step 4, which is why you get an empty buffer. This change to the spec allowed big performance improvements on platforms where blanking after Step 3 is basically what actually happens in hardware (like in many mobile GPUs). If you want work around this to sometimes make copies of the WebGL content after step 3, the browser would have to always make a copy of the drawing buffer before step 3, which is going to make your framerate drop precipitously on some platforms.
You can do exactly that and force the browser to make the copy and keep the image content accessible by setting preserveDrawingBuffer to true. From the spec:
This default behavior can be changed by setting the preserveDrawingBuffer attribute of the WebGLContextAttributes object. If this flag is true, the contents of the drawing buffer shall be preserved until the author either clears or overwrites them. If this flag is false, attempting to perform operations using this context as a source image after the rendering function has returned can lead to undefined behavior. This includes readPixels or toDataURL calls, or using this context as the source image of another context's texImage2D or drawImage call.
In the example you provided, the code is just changing the context creation line:
gl = canvas.getContext("experimental-webgl", {preserveDrawingBuffer: true});
Just keep in mind that it will force that slower path in some browsers and performance will suffer, depending on what and how you are rendering. You should be fine in most desktop browsers, where the copy doesn't actually have to be made, and those do make up the vast majority of WebGL capable browsers...but only for now.
However, there is another option (as somewhat confusingly mentioned in the next paragraph in the spec).
Essentially, you make the copy yourself before step 2: after all your draw calls have finished but before you return control to the browser from your code. This is when the WebGL drawing buffer is still in tact and is accessible, and you should have no trouble accessing the pixels then. You use the the same toDataUrl or readPixels calls you would use otherwise, it's just the timing that's important.
Here you get the best of both worlds. You get a copy of the drawing buffer, but you don't pay for it in every frame, even those in which you didn't need a copy (which may be most of them), like you do with preserveDrawingBuffer set to true.
In the example you provided, just add your code to the bottom of drawScene and you should see the copy of the canvas right below:
function drawScene() {
...
var webglImage = (function convertCanvasToImage(canvas) {
var image = new Image();
image.src = canvas.toDataURL('image/png');
return image;
})(document.querySelectorAll('canvas')[0]);
window.document.body.appendChild(webglImage);
}
Here's some things to try. I don't know whether either of these should be necessary to make this work, but they might make a difference.
Add preserveDrawingBuffer: true to the getContext attributes.
Try doing this with a later tutorial which does animation; i.e. draws on the canvas repeatedly rather than just once.
toDataURL() read data from Buffer.
we don't need to use preserveDrawingBuffer: true
before read data, we also need to use render()
finally:
renderer.domElement.toDataURL();
I do have a Paper.path() in Raphael that is filled with a simple texture:
var fill = screen.path(Iso.topFacePath(top)).attr({
fill: 'url(http://www.example.com/mytexture.jpg)',
});
The path can be altered by the user via drag and drop. For this I use Element.drag() to bind the handlers.
The problem that I encouter now is that while the onmove-handler function is called the element in question will be recalculated and has to be drawn again. Apparently this is "too much" for raphael and the fill pattern will disappear randomly (flicker) and appear again some time later (at latest onend).
The actual code I use is a little too much to post here but I built a fiddle where you can see what's going on (you can drag the upper sides of the quadrangle).
Is there a simple fix to this?
I am used to canvas much more than raphael (actually this is the first time I really use raphael) so maybe my approach of redrawing everything everytime sth changes is plain wrong?
EDIT: I just found out that seems to be somehow browser-related as well. Chrome and Firefox will produce the flicker where Safari seems to do everything just fine.
This seems to be caching issue (raphael.js does not cache the bitmap fill and will reload it on every change) and is fixed (for me) by this pull request on GitHub that is (as of 08/14/2012) still pending.
Raphael is pretty hard / impossible to build oneself as the make file points to local and/or inexistent files, but you can either concatenate everything by hand, modify the build script or use the modified build that is used in the example to get hold of the fix.
Let's hope it will find its way into a future release of Raphael.