We Keep Coding

Huge node leak in web app under Chrome - developer tools not reporting cause

I have a rather complicated web-application that is leaking nodes at an incredibly fast rate. Using windows task manager, I can see the amount of memory used by the chrome process increasing by around 3MB a second.
My initial approach was to disable parts of the application to isolate the area causing the problem. After a little trial and error discovered the cause were some table updates in the document. The app requests data from the server every second to update several tables of data. Each table is updated in the same way. All rows are removed from the table and new rows containing the updated data are inserted. I could not see a problem when inspecting the common code that does this.
To solve this I obviously turned to the Chrome Developer Tools. The problem I have is that the tools are giving me conflicting information.
The Timeline tool shows that the number of nodes is increasing by around 28000 each second. Forced garbage collection does not reduce this back to its original level.
The Timeline tool shows the JS Heap size fluctuating over time. A forced garbage collection returns the heap to its original size (plus or minus a few 100K)
Using the "three-snapshot" technique the Profiler - Heap Snapshot tool shows no HTML or Text nodes created between snapshot 1 and snapshot 2 that are present in snapshot 3.
Comparing the snapshots shows the creation and deletion of many HTMLTableRowElement, HTMLTableCellElement, HTMLInputElement and Text nodes. No increase in node count is reported.
The Profiler - Heap Allocation tool verifies the results of the Heap Snapshot tool. No leaks of any Node type are reported at any point.
The Heap tools show small increases in the (compiled code), (array) and (system) types.
The Heap Profile tools report heap sizes of around 12MB for my "raw" javascript version of the app, and around 7MB for my closure compiler - compiled version of the application. These values do not grow much over time.
This leaves me a little confused. There is obviously a memory leak. It is reported by Windows Task Manager, and by the Timeline tool as a node leak, but the Heap profiling tools and JS Heap Timeline do not show the issue.
As far as I can tell, the HTMLTableRowElements are referenced only in two places, in the document and in an object used for lookup by value. The object is always cleared when the table is cleared. I can "cure" the issue by changing my code to create all the nodes, but never inserting them into the document, just referencing them in the object. Obviously this is not a fix because the users cannot see the data.
After 2 days of testing and investigation I am now at a loss on how to proceed. The plot thickens if you throw in IE and Firefox. These browsers do not appear to have the same memory/node leak. I also believe the problem did not previously exist with Chrome. Unfortunately there does not seem to be a way to go back to previous versions of chrome to see if it is a bug in Chrome.
Does anyone have any advice on this. Am I missing some thing, or misinterpreting the output of the developer tools? Is there a way to go back to previous chrome versions? Does this sound like a bug in Chrome? All comments welcome.
This is the code used to insert items using Google's Closure API:
/**
* Inserts and/or updates a row in the table.
*
* #param {string|number} rowId The identifier of the row in the rows_ map
* #param {boolean} insertTop If true the row is inserted at the top of the document
* #param {...goog.dom.Appendable} var_args The items to add to each cell in the row
*/
sm.ui.DataTable.prototype.updateRow = function(rowId, insertTop, var_args) {
var dom = this.getDomHelper();
// Insert the new session data
if(!this.rows_[rowId]) {
// There is no row present (simple case of create one)
// Create the table row
var row = this.rows_[rowId] = dom.createDom('tr', {'style' : 'display: none;'});
var colView = this.colView_;
var colNames = this.columnNames_;
for(var i = 0; i < colNames.length; i++) {
var cell = dom.createDom('td');
if(!colView[i]) {
goog.style.showElement(cell, false);
}
row.appendChild(cell);
}
// Add to the table if element exists
var element = this.getElement();
var tBody = element.tBodies[0];
if(element) {
this.showPage_(this.currentPage_, false);
if(insertTop) {
// Insert a row at the top of the table
tBody.insertBefore(row, tBody.rows[0] || null);
} else {
// Append to the end if insert top no set true
tBody.appendChild(row);
}
this.showPage_(this.currentPage_, true);
// Update the footer to as it may need displaying or changing
this.updateFooter_();
}
}
// Loop over the var args and set the content of each cell
// arguments will be string, Node, array of strings and Nodes
for (var i = 0; i < arguments.length - 2; i++) {
var row = this.rows_[rowId];
dom.removeChildren(row.cells[i]);
dom.append(row.cells[i], arguments[i + 2]); // Removing this line "cures" the problem.
}
};
/**
* Removes all rows from the table.
* #param {string=} opt_message Message to display in the instead of data (reset table again to clear)
*/
sm.ui.DataTable.prototype.reset = function (opt_message) {
var element = this.getElement();
var tBody = element.tBodies[0];
while(tBody.rows.length > 0) {
tBody.deleteRow(0);
}
// Reset the rows and pages
this.rows_ = {};
this.currentPage_ = 1;
this.updateFooter_();
if(opt_message) {
// Create the row to inset in the table
// Ensure it spans all the columns
var dom = this.getDomHelper();
var messageCell = dom.createDom('tr', null,
dom.createDom('td', {'colspan' : this.columnNames_.length}, opt_message));
goog.dom.append(tBody,messageCell);
}
};

This very much looks like the classic event purge bug.
If an event is attached to a node, the node is not disposed of until the event handlers are detached.
jQuery, for example, intrinsicly detaches all events from removed nodes.
The code to correctly remove a node (along with all of her descendants) is:
function walkTheDOM(node, func)
{
func(node);
node = node.firstChild;
while (node)
{
walkTheDOM(node, func);
node = node.nextSibling;
}
}
function purgeEventHandlers(node)
{
walkTheDOM(node, function (n) {
var f;
for (f in n)
{
if (typeof n[f] === "function")
{
n[f] = null;
}
}
});
}
// now you can remove the node from the DOM
This technique is directly taken from Crockford's Good Parts.
Alternatively, use jQuery which will take care of it in the very same way.

How can I make Qooxdoo virtual list scalable?

I need to show list of data , at least 1 million rows (Big data , machine learning).
I do not need to show at once , remotetablemodel of qooxdoo table works fine but instead of table i choose list as design choice.
Below is a test i've made.
//create the model data, 1mil items
var rawData = [];
for (var i = 0; i < 1000000; i++) {
rawData[i] = "Item No " + i;
}
var model = new qx.data.Array(rawData);
//create the list
var list = new qx.ui.list.List(model);
this.getRoot().add(list);
I understand the point that it will take long to generate rawdata and assign it to list.
But the problem is after assigning the list , the virtual list itself is almost non-responsive.
Scrolling is very slow , navigating by down arrow freezes a few secs too.
Qooxdoo virtual infrastructure is suppose to render only visible items if i understand correctly? But in above test case it is so slow.
I expect to work like remote table model .
Tested with qooxdoo latest 4.0.0 and 3.5.1 , on Chrome 35 stable.

I can reproduce you issue only with the source version and not with the build version. I found the reason why the performance is so slow. There is an runtime check in an internal method from the SingleValueBinding which has a huge performance impact on the rendering.
I opened a bug report for that:
http://bugzilla.qooxdoo.org/show_bug.cgi?id=8439
But as I sad this issue only occurs with your developer version. So your customers are not effected.
You can disable the check if you want. Just remove the check block:
https://github.com/qooxdoo/qooxdoo/blob/master/framework/source/class/qx/data/SingleValueBinding.js#L915
You can also load your model data in parts to improve the model creation. You can maybe load the next part when the user has scrolled to the end of the list. You can use the example you have already seen:
Infinite scroll in qooxdoo with virtual list

ExtJs 4.2 ref-selector vs Ext.getcmp() vs Ext.ComponentQuery.query()

I was asked to develop a tab panel with 6 tabs, each having 30 to 40 elements. Each tab is acting as a form in accumulating the details of a person and the last tab is a Summary page which displays all the values entered in the first five tabs. I was asked to provide summary as a tab because, the user can navigate to summary tab at any instance and look at the details entered by him/ or glace the summary. i am following ExtJs MVC pattern. Payload is coming from / going to Spring MVC Application. (JSON)
Using tab change event in controller and if the newtab is summary I am rendering the page with show hide functionality.
Method 1 :In controller I have used Ext.getCmp('id of each element inside the tabs') and show hide the components in summary tab based on the value entered by the user. This killed my app in IE8 popping a message saying that the "script is slow and blah blah..." i had to click on NO for 5 to 6 times for the summary tab to render and display the page.
Method 2 :In controller I used ref and selectos to acccess all the items in tabs. I have used itemId for each and every field in summary tab. like this.getXyz().show(). I thought it would be fast. Yes it was in Google chrome. but my app in IE8 is slow compared to goolge chrome/firefox
Any suggestions regarding this and plan to decrease page render time. The summary page has more than 1000 fields. Please feel free to shed ur thoughts or throw some ideas on this.
thank you!!

I've got a few suggestions you can try. First, to answer your title, I think the fastest simple way to lookup components in javascript is to build a hash map. Something like this:
var map = {};
Ext.each(targetComponents, function(item) {
map[item.itemId] = item;
});
// Fastest way to retrieve a component
var myField = map[componentId];
For the rendering time, be sure that the layout/DOM is not updated each time you call hide or show on a child component. Use suspendLayouts to do that:
summaryTabCt.suspendLayouts();
// intensive hide-and-seek business
// just one layout calculation and subsequent DOM manipulation
summaryTabCt.resumeLayouts(true);
Finally, if despite your best efforts you can't cut on the processing time, do damage control. That is, avoid freezing the UI the whole time, and having the browser telling the user your app is dead.
You can use setTimeout to limit the time your script will be holding the execution thread at once. The interval will let the browser some time to process UI events, and prevent it from thinking your script is lost into an infinite loop.
Here's an example:
var itemsToProcess = [...],
// The smaller the chunks, the more the UI will be responsive,
// but the whole processing will take longer...
chunkSize = 50,
i = 0,
slice;
function next() {
slice = itemsToProcess.slice(i, i+chunkSize);
i += chunkSize;
if (slice.length) {
Ext.each(slice, function(item) {
// costly business with item
});
// defer processing to give time
setTimeout(next, 50);
} else {
// post-processing
}
}
// pre-processing (eg. disabling the form submit button)
next(); // start the loop

up().down().action....did the magic. I have replaced each and every usage of Ext.getCmp('id'). Booooha... it's fast and NO issues.
this.up('tabpanel').down('tabName #itemIdOfField').actions.
actions= hide(), show(), setValues().

Try to check deferredRender is true. This should only render the active tab.
You also can try a different hideMode. Especially hideMode:'offsets ' sounds promising
Quote from the sencha API:
hideMode: 'offsets' is often the solution to layout issues in IE specifically when hiding/showing things
As I wrote in the comment, go through this performance guide: http://docs.sencha.com/extjs/4.2.2/#!/guide/performance
In your case, this will be very interesting for you:
{
Ext.suspendLayouts();
// batch of updates
// show() / hide() elements
Ext.resumeLayouts(true);
}

Sencha Touch Ext.navigation.View pop to root

I have an Ext.navigation.View in which I have pushed a few views. Certain user interactions require that I go directly back to the top level of the navigation view -- the equivalent of popToRootViewControllerAnimated: on a UINavigationController in iOS.
I have tried various things like:
while(navigationView.getItems().getCount() > 1)
navigationView.pop();
and
while(navigationView.canPop())
navigationView.pop();
Neither work. The first example seems to put me into an infinite loop which isn't too surprising. The second example only seems to pop one view off.
So the question: What is the proper way to pop to the root view in an Ext.navigation.View in Sencha Touch (version 2 developer preview)?

There has been a number of interim methods for achieving this.
The one I was using was to pop a number higher than the number of levels you would ever have e.g.
navigationView.pop(10);
and that worked fine, but I was never happy with that, but I see they have now introduced a reset method. Which you can call thus...
navigationView.reset();
Internally within the Sencha source code (see below) you can see that it does a similar job to what #Mithralas said, but just easier to write.
// From the Sencha source code
this.pop(this.getInnerItems().length);

popToRoot: function() {
this.pop(this.getItems().length - 1);
}

Ensure to use the NavigationView.reset() method. To make it clear, if your main navigation view is Main, you'd do something like this in the controller:
this.getMain().reset();

The solution turned out to be extending the navigation view with the following:
popToRoot: function(destroy)
{
var navBar = this.getNavigationBar(),
stackLn = this.stack.length,
stackRm;
//just return if we're at root
if(stackLn <= 1) return;
//just return if we're already animating
if(navBar && navBar.animating) return;
//splice the stack to get rid of items between top and root
stackRm = this.stack.splice(1, stackLn-2);
//remove views that were removed from the stack if required
if(destroy) {
stackRm.forEach(function(val, idx, arr) {
this.remove(val, true);
});
}
//clear out back button stack
navBar.backButtonStack = [];
//now we can do a normal pop
this.pop();
}

How do I prevent the Javascript InfoVis SpaceTree `ST.select()` method from collapsing nodes?

I'm using the JavaScript InfoVis Toolkit and in particular the SpaceTree visualisation.
I need to expand all of the tree and then show a path from a particular leaf node back to the root.
I've got the tree to expand just fine but it's the selection of a leaf node and highlighting the path back to the root that's causing me some problems.
I'm using the ST.select(node, onComplete) function to select the leaf node I'm interested in and indeed the path back to the root (lines and nodes) are highlighted.
To do this I implemented the onBeforePlotNode and onBeforePlotLine ST.Controller methods to allow me to highlight the nodes back to the root and their plotlines:
onBeforePlotNode: function(node){
//add some color to the nodes in the path between the
//root node and the selected node.
if (node.selected) {
node.data.$color = "#dddddd";
} else {
delete node.data.$color;
}
},
onBeforePlotLine: function(adj){
if (adj.nodeFrom.selected && adj.nodeTo.selected) {
adj.data.$color = "#33CC33";
adj.data.$lineWidth = 5;
} else {
delete adj.data.$color;
delete adj.data.$lineWidth;
}
}
The problem is that when I call ST.select() to highlight the leaf node all child nodes beneath this level are collapsed/hidden.
To see this in action I've uploaded a couple of examples:
Full tree expansion - leaf not selected
Leaf selected - path shown, but all children below node N2 missing
You may need to scroll down if your browser window is a bit small.
So my question is, how do I show nodes from a leaf node back to the root node in JavaScript InfoVis without collapsing level 3's children (level 1 being the root)?
If there was a way to find my leafe node and walk the tree back to the root (setting styles on the way) then I'd be happing doing that

OK after digging through all that code, cluttering it with console.log() calls and breakpoints, I found it.
It has to do with the inital onClick call, the fact that the graph as an update loop that's running in the background and the fact that everything besides onClick seems to ignore the busy state of the graph.
What happens
onClick gets called and triggers a chain of events, part of them is asynchronous
select is being called which is more or less synchronous and does its work
onClick finally gets done and one of it's side effects is that it re-expands the graph
select has set clickedNode and now onClick uses the newly set value of it and screws up
Solution
We need to redesign select so it respects the busy state of the graph:
select: function(id, onComplete) {
var that = this;
if (this.busy) {
window.setTimeout(function() {
that.select(id, onComplete);
}, 1);
return;
}
// original select code follows here (remove the old var that = this; assignment)
That's all, we simply check for the busy state and delay select until it's false.
This should also be applied to all other function besides onClick that are called from the outside, the library designer here did a bad job of indicating what has side effects and what has not though.

Did you try setting "constrained: false" in ST's properties? That solved it for me.
http://thejit.org/static/v20/Docs/files/Visualizations/Spacetree-js.html