I'm working on a pretty simple web application that displays top users hitting a web page. I've written some code in d3 to convert the data to a bar graph and have written a simple php script to pull the data from a database. I use an ajax request to pull the data once every 5 seconds and update the bar graph. Without fail if I leave the page open in the background it will get to the point of the old
aw snap google chrome has run out of memory
I've gone through a bunch of sites on memory leaks and done what I could to prevent them but there is a decent enough chance I messed something up. The problem is other than that error coming without fail on this page (and I've written plenty of javascript applications where this doesn't happen) there is absolutely no evidence the leak is happening. The data I'm retrieving every 5 seconds is 212kb, and when I do a heap recording heap memory peaks at 25mb and doesn't really increase (it generally bounces between 10 and 25mb), so it seems like that 212kb is being garbage collected and is not accumulating in the heap. Similarly I've looked at the task manager and when I make that the only tab open there seems to be a good amount of fluctuation but definitely not a trend upwards. I've taken heap snapshots and they tend to be in the 10-15mb range and I really don't understand how to read the snapshot, but here's what it looks like after running for 15 minutes or so:
It's just getting extremely frustrating, I've spent 20-30 hours on this but it seems like a case of a watched pot never boils. If I look for evidence of it happening I can never find any, but if I leave it open while I leave my computer for a few hours, without fail the page crashes. It's almost like the garbage collector just waits for me to leave my computer before deciding to just stop running.
Any thoughts on what could be the culprit here or next steps to attempt to identify the culprit? This is my first experience with a memory leak and it's driving me crazy. Thanks.
Related
Relative newbie to nodejs here trying to figure out a performance issue in a newly built application.
I am running a performance test on my node 0.12.7 app and I find the server hanging intermittently. It needs a restart upon reaching that state. After confirming there is no memory leak (the process heap does not exceed 500 MB whereas the default max-heap size is 1.4GB I understand), we moved to checking CPU profile. I have used this snippet of code with a dependency on v8-profiler to get regular profiles
Here is one of the charts that we encountered from jmeter (although the server didn't hang)
We plotted flame graphs in Chrome by loading the CPU profiles. I was expecting to find the JS stuck somewhere at this point, but I find that exactly in that time range, the node server is idle for a long time. Could anyone help me understand what could be the probable causes for the server to stay idle while being bombarded with client requests, and eventually recovering to continue operations after 10 minutes?
I unfortunately have lost the data to check if the responses between 16:48:10 and 16:57:40 are error or success, but it is very likely that they are error responses from the proxy since node didn't have a care in the world.
Here are the flame charts seen in Chrome
Before 16.47 hrs,
Around 16.47 hrs
A couple of minutes after 16.47 hrs
There could be multiple reasons around this.
Server is not accepting the requests. Do you see drop in throughput after you reach the peak?
Have you checked the server logs to see if any exceptions are logged?
Try plotting trends of response time and throughput for your test duration.
You may want to see any IO bound operations in your code.
Check the processor queue length. You should see it building up if processes are not getting enough CPU.
So, I'm trying to profile an existing large-ish node.js webapp.
It seems to run OK, when running normally, I added a debug launch configuration.
My first attempt, I left it running, attempting to load a very complex page - came back to find a 4GB profile log had filled the remaining space on the small partition it (and my system) is on causing all sorts of problems.
Did a quick test, running the server for only a few seconds, got a ~15MB profile of it loading some modules - but it works.
A second attempt - watching much closer, I started loading a simpler page as soon as the server was accepting connections (watching the profile log grow at ~3mb/s) - by the time the page had loaded, I have a 600MB log, stop the server to see the results.
After waiting for quite a long time, phpstorm complains it does not have enough memory - limited to 750MB.
I've increased it to 1500MB - but thought I should ask for some wisdom here before another test, as this profiler really does not seem practical!
Am I just expecting too much for this profiler to give me info on a complex app(and I should use a different profiler)?
Does this indicate some kind of problem with the app?
Perhaps I am doing something wrong?
All answers welcome!
Edit: the command phpstorm runs is /usr/bin/node --prof --logfile=v8-23-11-15_00-37-35-0.log --log-timer-events bin/www
OK, removing --log-timer-events makes the log a reasonable size, and the server runs at a decent speed! There is less info - but it still seems to contain much of what I am hoping/expecting to see.
Not marking this answer as accepted, as I feel I am still floundering in the dark a little more than I'd like.
In one of our apps, we're finding that when usage spikes, RSS (as reported by top(1) for example) increases, hits a plateau and starts oscillating, but never comes back down. After a bit of a break, stressing the server again results in another higher plateau. This trend continues into the many gigabytes of RAM.
At first we suspected a memory leak on the Javascript side, and so started tracking the statistics returned by process.memoryUsage():
A few phases here:
After starting the server, we hit it with 100 concurrent long running requests for a few minutes and then left the server running for the evening. RSS stayed at around 320MB for the night; and heap usage just kept oscillating. We were emitting traces every minute so maybe this explained the movement in the heap.
At around 9a, we hit it with another round of concurrent requests and transitioned to a workflow involving 2 threads of rolling long running requests (downloads of 10MB of data). We kept these running until around 2p and stopped them. During this phase, after the initial bump in RSS, it again started to level off and stay between around 400 and 450MB.
The last bit is another round of concurrent requests. Here, RSS grew to between 480MB and 500MB
I'm wondering if there are ways to figure out what's going on outside the heap that could be consuming all of this memory and not releasing it. Short of that, are there any hidden options for limiting memory usage outside of the heap?
And I have to ask. Are we making too much of this? Is this about normal for Node?
This is more of a concept question as I am trying to learn more about long-polling, in Javascript/jQuery specifically. I have a web app where I am long-polling (websockets are not an option right now) to a json file. I have run some tests and after leaving the app open for some time, it starts to slow down and later seems to start getting stuck. Using Chrome, I've checked the developer tools and the memory starts going through the roof, as with the listeners (>5000) around 1 1/2 hours of up time. I've searched and searched but can't find forums that pinpoint this problem and solution. In my code, I am using setInterval every 30 seconds. What do I need to do in order to keep the memory and listeners count low, and make sure the app does not overload and get slow? The function of the app requires it to stay up for long periods of time.
Thank you for your help.
Lately, I've been playing around with Ramsey's theorem for R(5,5). You can see some examples of previous attempts here: http://zacharymaril.com/thoughts/constructionGraph.html
Essence: find all the k4's in a graph/its complement and then connect another point in such a way that no k5's are formed (I know with one type of choice, mathematically it becomes improbable that you would get past 14. But there are ways around that choice and I've gotten it to run as far as 22-23 without bricking my browser.)
With new ideas, I started playing around with storing information from batch to batch. The current construction graph goes through and searches for all the k4's in a graph every time it sees the graph. I thought this was overkill, since the k4's will stay the same in the previous graph and only new k4's could show up in the connections produced by the addition of the new point. If you store the previous k4's each time you find them and then only search in the frontier boundaries that were newly created, then you reduce the number of comparisons you have to do from (n 4) to (n-1 3).
I took a shot at implementing this last night and got it to work without obvious errors. While I am going to go back after this and comb through it for any problems, the new method makes the program much much slower. Before, the program was only ~doubling in terms of time it took to do all the comparisons. Now, it is going up in what looks to be factorial time. I've gone back through and tried to ferret out any obvious errors, but I am wondering whether the new dependence on memory could have created the whole slow down.
So, with that long intro, my main question is how are the memory and speed of a program related in a web browser like chrome? Am I slowing down the program by keeping a bunch of little graphs around as JSON objects? Should it not matter in theory how much memory I take up in terms of speed? Where can I learn more about the connection between the two? Is there a book that could explain this sort of thing better?
Thank you for any advice or answers. Sorry about the length of this: I am still buried pretty deep in the idea and its hard to explain it shortly.
Edit:
Here are the two webpages that show each algorithm,
With storage of previous finds:
http://zacharymaril.com/thoughts/constructionGraph.html
Without storage of previous find:
http://zacharymaril.com/thoughts/Expanding%20Frontier/expandingFrontier.html
They are both best viewed with Chrome. It is the browser I have been using to make this, and if you open up the dev panel with ctrl shift i and type "times", you can see a collection of all the times so far.
Memory and speed of a program are not closely interrelated.
Simple examples:
Computer with almost no ram but lots
of hard drive space is going to be
thrashing the hard drive for virtual
memory. This will slow things down
as hard drives are significantly
slower than ram.
A computer built out
of all ram is not going to do the
same thing. It won't have to go to the hard drive so will stay quicker.
Caching usually takes up a lot of
ram. It also significantly increases
the speed of an application. This is
how memcache works.
An algorithm may take a long time but
use very little ram. Think of a
program that attempts calculating PI.
It will never finish, but needs very
little ram.
In general, the less ram you use (minus caching) the better for speed because there's less chance you're going to run into constraints imposed by other processes.
If you have a program that takes considerable time to calculate items that are going to be referenced again. It makes sense to cache them memory so you don't need to recalculate them.
You can mix the two by adding timeouts to the cached items. Every time you add another item to the cache, you check the items there and remove any that haven't been accessed in a while. "A while" is determined by your need.