many programmers believe javascript closures starts leaking memory.
I want to see some examples where this happens.
I dont want to see closure being added as events to dom elements and removing dom elements in IE. As in link below
function foo(value) {
var bar = document.getElementById("selector");
bar.attachEvent("onclick",
// closure
function() {
alert(value);
}
);
}
i want to see some other cases where closures may start to leak.
To sum up on jAndy's link (for the full picture still follow the link).
Closures them selfs are not a big problem, the problems comes when you don't understand that the everything in the inherited scopes continues to live on as long as the closure exists.
Example
function doAmazingGrace() {
// gigantic list of stuff!
var list = [...];
var result = magicComputationOnTheList();
// readOnlyResults for whatever reason....
return {
get: function() {
return result;
}
}
}
result itself is not a problem since it has to live on in order for the closure to work, but list will be "leaked", unless you loose the reference to the closure it will go away so it's not technically a leak.
But as long as the closure lives on, you keep a reference to list in there, which isn't needed.
So in order to fix it, it would be wise to use delete list or list = null before returning the object, so that the Array can be garbage collected.
The second issue are circular references, but that's not a closure issue, you can always introduce circular references and whether and how long they leak is also dependent on the garbage collectors ability to get rid of them. V8 does a good job at that, old IE versions had extreme problems with it though.
Related
If I have the following code:
function MyClass() {
this.data = {
// lots of data
};
}
var myClassInstace = new MyClass();
var myobj = {
num:123,
str:"hello",
theClass:myClassInstance
};
I know it's absolutely necessary to do:
myobj.theClass = null;
To free up myClassInstance and its data property for GC. However, what should I do with myobj.num and myobj.str? Do I have to give them a value of null too? Does the fact that they're primitive change anything regarding GC?
The JavaScript runtime that implements garbage collection will be able to collect items as soon as values are no longer reachable from code. This is true for object references as well as primitives. The details of the exact moment the item is collected varies by implementation, but it is not even necessary to set your object references to null (as you state) unless you need the object cleaned up sooner than the natural termination of the current function.
This all ties into the fundamental concept of "scope" and the Scope Chain. When an item is no longer in any other objects scope chain it can be collected. Understanding this clearly will answer this question and also help to understand closures, which are scenarios where items stay in memory longer than you might have expected.
There are a lot of "it depends here", ranging from what your code is doing to what browser you're running in. However, if your object is JIT compiled to not use a map for its attributes, then the number should be an 8 byte double stored inline inside the object. Nulling it will do nothing.
The string and the myclass instance will be a pointer to memory allocated outside the object (since a string can be arbitarily many bytes, it can't be stored inside the object. A compiler could conceivably store one instance of the string in memory and never free it, however). Nulling them can allow the garbage collector to free them before the main object goes out of scope.
However, the real question is why you're worried about this. Unless you have profiled your code and identified garbage collection or memory leaks as a problem, you should not be trying to optimize GC behavior. In particular, unless your myobj object is itself going to be live for a long time, you should not worry about nulling fields. The GC will collect it when it goes out of scope.
setting to undefined (not null) will work however delete is better example delete myobj.theClass
Just to avoid misunderstanding I will say that there is no way to really delete an object from memory in JavaScript. you delete it's references or set them to undefined so that the GC can do it's work and really delete.
Coming from iOS, I'm used to having weak references between objects, so I can access data from wherever and not worry about reference cycles.
Not 100% sure on how Javascript works with something like this:
function One () {
this.hello = 'hi from one';
this.two = new Two(this);
}
function Two (one) {
this.one = one;
}
Two.prototype.sayHi = function () {
console.log(this.one.hello);
}
var o = new One();
o.two.sayHi(); // Outputs: hi from one
This does work, but is there a better way to do this? Should I be worried about memory here? Doing the same thing carelessly in iOS would get us into trouble.
Bonus: If this is actually safe, is there a way Javascript can leak memory? I know it's GC under the hood, but is there anything I should be watching out for that it might not catch?
What you are afraid of here are cycles in reference graph.
Garbage collectors can deal with cycles and are able to release objects which reference each other but does not have references to any of them from outside.
This is different to reference-counted memory management where cycles keep the memory and must be broken with weak references.
This does not, however, mean they you don't have to care about memory leaks as there are other ways to introduce them.
I'm a novice to this kind of javascript, so I'll give a brief explanation:
I have a web scraper built in Nodejs that gathers (quite a bit of) data, processes it with Cheerio (basically jQuery for Node) creates an object then uploads it to mongoDB.
It works just fine, except for on larger sites. What's appears to be happening is:
I give the scraper an online store's URL to scrape
Node goes to that URL and retrieves anywhere from 5,000 - 40,000 product urls to scrape
For each of these new URLs, Node's request module gets the page source then loads up the data to Cheerio.
Using Cheerio I create a JS object which represents the product.
I ship the object off to MongoDB where it's saved to my database.
As I say, this happens for thousands of URLs and once I get to, say, 10,000 urls loaded I get errors in node. The most common is:
Node: Fatal JS Error: Process out of memory
Ok, here's the actual question(s):
I think this is happening because Node's garbage cleanup isn't working properly. It's possible that, for example, the request data scraped from all 40,000 urls is still in memory, or at the very least the 40,000 created javascript objects may be. Perhaps it's also because the MongoDB connection is made at the start of the session and is never closed (I just close the script manually once all the products are done). This is to avoid opening/closing the connection it every single time I log a new product.
To really ensure they're cleaned up properly (once the product goes to MongoDB I don't use it anymore and can be deleted from memory) can/should I just simply delete it from memory, simply using delete product?
Moreso (I'm clearly not across how JS handles objects) if I delete one reference to the object is it totally wiped from memory, or do I have to delete all of them?
For instance:
var saveToDB = require ('./mongoDBFunction.js');
function getData(link){
request(link, function(data){
var $ = cheerio.load(data);
createProduct($)
})
}
function createProduct($)
var product = {
a: 'asadf',
b: 'asdfsd'
// there's about 50 lines of data in here in the real products but this is for brevity
}
product.name = $('.selector').dostuffwithitinjquery('etc');
saveToDB(product);
}
// In mongoDBFunction.js
exports.saveToDB(item){
db.products.save(item, function(err){
console.log("Item was successfully saved!");
delete item; // Will this completely delete the item from memory?
})
}
delete in javascript is NOT used to delete variables or free memory. It is ONLY used to remove a property from an object. You may find this article on the delete operator a good read.
You can remove a reference to the data held in a variable by setting the variable to something like null. If there are no other references to that data, then that will make it eligible for garbage collection. If there are other references to that object, then it will not be cleared from memory until there are no more references to it (e.g. no way for your code to get to it).
As for what is causing the memory accumulation, there are a number of possibilities and we can't really see enough of your code to know what references could be held onto that would keep the GC from freeing up things.
If this is a single, long running process with no breaks in execution, you might also need to manually run the garbage collector to make sure it gets a chance to clean up things you have released.
Here's are a couple articles on tracking down your memory usage in node.js: http://dtrace.org/blogs/bmc/2012/05/05/debugging-node-js-memory-leaks/ and https://hacks.mozilla.org/2012/11/tracking-down-memory-leaks-in-node-js-a-node-js-holiday-season/.
JavaScript has a garbage collector that automatically track which variable is "reachable". If a variable is "reachable", then its value won't be released.
For example if you have a global variable var g_hugeArray and you assign it a huge array, you actually have two JavaScript object here: one is the huge block that holds the array data. Another is a property on the window object whose name is "g_hugeArray" that points to that data. So the reference chain is: window -> g_hugeArray -> the actual array.
In order to release the actual array, you make the actual array "unreachable". you can break either link the above chain to achieve this. If you set g_hugeArray to null, then you break the link between g_hugeArray and the actual array. This makes the array data unreachable thus it will be released when the garbage collector runs. Alternatively, you can use "delete window.g_hugeArray" to remove property "g_hugeArray" from the window object. This breaks the link between window and g_hugeArray and also makes the actual array unreachable.
The situation gets more complicated when you have "closures". A closure is created when you have a local function that reference a local variable. For example:
function a()
{
var x = 10;
var y = 20;
setTimeout(function()
{
alert(x);
}, 100);
}
In this case, local variable x is still reachable from the anonymous time out function even after function "a" has returned. If without the timeout function, then both local variable x and y will become unreachable as soon as function a returns. But the existence of the anonymous function change this. Depending on how the JavaScript engine is implemented, it may choose to keep both variable x and y (because it doesn't know whether the function will need y until the function actually runs, which occurs after function a returns). Or if it is smart enough, it can only keep x. Imagine that if both x and y points to big things, this can be a problem. So closure is very convenient but at times it is more likely to cause memory issues and can make it more difficult to track memory issues.
I faced same problem in my application with similar functionality. I've been looking for memory leaks or something like that. The size of consumed memory my process has reached to 1.4 GB and depends on the number of links that must be downloaded.
The first thing I noticed was that after manually running the Garbage Collector, almost all memory was freed. Each page that I downloaded took about 1 MB, was processed and stored in the database.
Then I install heapdump and looked at the snapshot of the application. More information about memory profiling you can found at Webstorm Blog.
My guess is that while the application is running, the GC does not start. To do this, I began to run application with the flag --expose-gc, and began to run GC manually at the time of implementation of the program.
const runGCIfNeeded = (() => {
let i = 0;
return function runGCIfNeeded() {
if (i++ > 200) {
i = 0;
if (global.gc) {
global.gc();
} else {
logger.warn('Garbage collection unavailable. Pass --expose-gc when launching node to enable forced garbage collection.');
}
}
};
})();
// run GC check after each iteration
checkProduct(product._id)
.then(/* ... */)
.finally(runGCIfNeeded)
Interestingly, if you do not use const, let, var, etc when you define something in the global scope, it seems be an attribute of the global object, and deleting returns true. This could cause it to be garbage collected. I tested it like this and it seems to have the intended impact on my memory usage, please let me know if this is incorrect or if you got drastically different results:
x = [];
process.memoryUsage();
i = 0;
while(i<1000000) {
x.push(10.5);
}
process.memoryUsage();
delete x
process.memoryUsage();
this is my code, I do not know if it good for prevent leaking memory ? help and how can I test for leaking memory?
var Test = function () {
this.ar = [];
this.List = function () {
return this.ar;
}
this.Add = function (str) {
this.ar.push(str);
}
}
use:
var t = new Test();
t.Add("One");
t.Add("Two");
t.Add("Three");
alert(JSON.stringify(t.List()));
t = undefined;
alert(JSON.stringify(t.List() ));
Setting t to undefined will clear that reference to the object. If there are no other references to that object in your code, then the garbage collector will indeed free up that Test() object. That's how things work in javascript. You don't delete an object, you just clear any references to it. When all references are gone, the object is available for garbage collection.
The actual delete keyword in javascript is used only to remove a property from an object as in delete t.list.
Different browsers have different tools available for keeping track of memory usage. The most universal, blackbox way I know of for a test is to run a cycle over and over again where you assign very large objects (I often use big strings) into your test (to consume noticable amounts of memory) with some sort of setTimeout() in between some number of runs (to let the garbage collector have some cycles) and then just watch the overall memory usage of the browser. As long as the overall memory usage doesn't keep going up and up as you keep doing more and more runs then you must not have a noticeable leak.
Individual browsers may have more comprehensive measuring tools available. Info here for Chrome.
I have been using jQuery for over a couple of months and read up on Javascript memory leaks for a few days.
I have two questions regarding memory leaks and jQuery:
When I bind (using .bind(...)) do I have to unbind them (.unbind()) if I leave the page/refresh to avoid memory leaks or does jQuery remove them for me?
Concerning closures, I read that they can lead to memory leaks if used incorrectly. If I do something such as:
function doStuff( objects ){ //objects is a jQuery object that holds an array of DOM objects
var textColor = "red";
objects.each(function(){
$(this).css("color", textColor );
});
}
doStuff( $( "*" ) );
I know that the above code is stupid (better/simpler r ways of doing this) but I want to know if this causes circular references/closure problems with .each and if it would cause a memory leak. If it does cause a memory leak, how would I rewrite it (usually similar method) to avoid a memory leak?
Thanks in advance.
Edit: I have another case similar to question 2 (which I guess makes this part 3).
If have something like this:
function doStuff( objects ){ //iframe objects
var textColor = "red";
function innerFunction()
{
$(this).contents().find('a').css("color", textColor );
}
objects.each(function(){
//I can tell if all 3 are running then we
//have 3 of the same events on each object,
//this is just so see which method works/preferred
//Case 1
$(this).load(innerFunction);
//Case 2
$(this).load(function(){
$(this).contents().find('a').css("color", textColor );
});
//Case 3
$(this).load(function(){
innerFunction();
});
});
}
doStuff( $( "iframe" ) );
There are 3 cases above and I would like to know which method (or all) would produce a memory leak. Also I would like to know which is the preferred method (usually I use case 2) or better practice (or if these are not good what would be better?).
Thanks again!
1) No. The browser clears everything between page loads.
2) In its current form there will be no memory leak, since jquery's .each() function doesn't bind anything, so once its execution is finished, the anonymous function it was passed is no longer reachable, and therefore the environment it closed (i.e. the closure as a whole) is also not reachable. So the garbage collection engine can clean everything up - including the reference to objects.
However, if instead of .each() you had something harmless, like $('div:eq(0)').bind() (I'm trying to emphasise that it needn't be a reference to the large objects variable, enough that it is even a single unrelated element), then since the anonymous function sent to .bind() closes the objects variable, it will remain reachable, and therefore not garbage collected, allowing memory leaks.
A simple way to avoid this problem, is to objects = null; at the end of the executing function.
I should note that I'm not familiar with the JS garbage collection engines, so it is possible that there are reasonably intelligent optimisations. For example it can be checked whether or not the anonymous function tries to access any variables closed in with it, and if not, it might pass them to the garbage collector, which would solve the problem.
For further reading, look for references to javascript's memory model, specifically the environment model and static binding.
There are some subtle leak patterns that you may not even recognize. Check out a question I asked some time ago about a similar issue
jQuery 1.5 Memory leak in IE8
If you create a closure around an object which references the dom node, a leaking reference loop is made, and unfortunately, it can not be corrected by simply unbinding. You will have to set the reference to the DOM node in the object to null.
In regards to 1., no, you definitely do not have to .unbind() when leaving a page. I am not entirely sure about two.
About 1, sometimes you have to free your resources especially when you have circular references and using Internet Explorer (because of some bugs, because in theory you shouldn't have to do that) ;) Google maps had a function in their v2 to prevent that that we had to call on document.onunload (GUnload).
About 2, you don't have circular references. It consumes a lot of memory since the this must have its own context of execution to access textColor among others..
A circular reference is achieved when an object references to itself or a closure calls itself. Maybe there's other situations..
Hope this helps