I am using HAPI.JS framework with NodeJS and created a proxy. Think that proxy means i am just maintaining session in redis. Other than that i am not doing anything in the code. May be only thing is i am using setInterval to log my process.memoryUsage() for every 3 mintues.
My Questions:
Why my Memory Keeps on Increasing?
Will it get down?
Is this occurs due to setInterval keeps on logging the process usage?
Is this occurs due to console logging of every request and response?
My Redis Database is kept open till my server crashes, it this causes this ?
Do i need use process mananger like new relic or strong loop to identify this?
So how long this memory will keep on increasing, at some point it must stop (i want to know which point is that?)
I am using sequelize of MSSQL transaction using pooling concept? Does pooling makes this?
P.S I am new to node JS.
Why my Memory Keeps on Increasing?
You got a memory leak
Will it get down?
Sometimes GC kicks in and cleans up some things (that are not leaking)
Is this occurs due to setInterval keeps on logging the process usage?
Usually not, but w/o seeing the code I can't say this for sure
Is this occurs due to console logging of every request and response?
Usually not, but w/o seeing the code I can't say this for sure
My Redis Database is kept open till my server crashes, it this causes this ?
Should not be a problem.
Do i need use process mananger like new relic or strongloop to identify this?
It is one way to do it ... but there are also others.
So how long this memory will keep on increasing, at some point it must stop (i want to know which point is that?)
Depends on the server setup. How much RAM + what else is running etc.
I am using sequelize of MSSQL transaction using pooling concept? Does pooling makes this?
Usually not, but w/o seeing the code I can't say this for sure
Maybe this post helps you find the leak:
https://www.nearform.com/blog/how-to-self-detect-a-memory-leak-in-node/
Related
I have a python websocket server attempting to communicate with a javascript websocket client (embedded in HTML). The events are being emited from the server immediately, but it takes upwards of 30 seconds for the server to send the event trigger, despite both the client and server being locally hosted.
Here is the relavent code for the server:
sio = socketio.AsyncServer(cors_allowed_origins='*')
app = web.Application() #aiohttp web server
loop = asyncio.get_event_loop()
sio.attach(app)
async def index(request):
with open('./index.html') as f:
return web.Response(text=f.read(), content_type='text/html')
app.router.add_get('/', index)
app.router.add_get('/index.html', index)
if __name__ == '__main__':
web.run_app(app)
the event is being fired like so (edit, this must be done with event loops, as emit is an asynchronous function being run from a synchronous one.):
print('Starting event')
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(sio.emit('ChangeProgressState'))
loop.close()
print('Event has been fired.')
However, the print statements show up immediately. On the client end, I am connecting and trying to consume the event like this:
const socket = io.connect("http://localhost:8080", {
transports: ['websocket']
})
socket.on("ChangeProgressState", function (data) {
console.log("got event.")
//some code here...
});
However, from the time it takes for the event to fire, and the time it takes for the javascript socket to notice can be a very long time, from 30 seconds to sometimes a few minutes. Is there something I'm doing wrong here?
It should be noted, there are very little (2%-5%) resources being consumed (both memory and CPU), so I do not currently think that is the issue. Any help would be much appreciated.
EDIT 11/15/2019: I have tried looking at the networking tab of the application (chromium-browser on raspberry pi). It seems to show the initial socket connection, but it doesn't show anything in terms of communication between sockets, even after the event eventually fires.
EDIT 2: This definitely seems to be an issue server-side. I can send events from the JS client to the python server essentially immediately, but going in the other direction is when it takes a long time to arrive. I'm not quite sure why though.
Ah ok, so my gut said it sounds like the client is long polling. Many socket libraries first establish long-polling and then upgrade to ws connections.
After taking a look at Socket.io:
... which first establishes a long-polling connection, then tries to upgrade to better transports that are “tested” on the side, like WebSocket. ...
So I don't believe you're doing anything wrong, it's just the initialization process of establishing the WebSocket connection.
As for the python part, I'll be honest that's a tad more fuzzy to me. My first guess is that the loop code doesn't block the print statement from being executed -- but I'm more familiar with JavaScript than Python, so not completely certain on that front. My second guess is that I do know from other pub/sub libraries that the server side engine sometimes makes use of a middle layer of sorts (sometimes a cache, sometimes a queue) that helps ensure messages are sent/received, that's also a possibility.
extra tidbit: I suspect if you look at the network tab of your browser's dev tools, it'd display that behavior, some form of HTTP requests, and then eventually you'd see the socket connection. Playing around with turning your Python server/service off/on would also demonstrate the robustness of socket.io in the browser and edge cases for how it handles unstable networking when communicating with respect to various internet communication protocols.
Thank you to everyone that helped answering this question! I finally found a solution that is a bit unorthodox, so I'll explain the whole situation here.
Essentially, in order to run an async method in a synchronous context, you must use the asyncio's run_until_complete method on an event loop. This is how I was doing it when this question was asked. However, after talking to the creator of the python-socketio library, it seems that you must run this in the same event loop as the one the server is running in.
However, this creates a different problem. If an event loop is already running, python does not allow you to use run_until_complete on it, giving you an error: RuntimeError: This event loop is already running.
So, this things sound contradictory right? And you would be correct. However, this problem is prevalent enough that another library exists for the sole purpose of monkey-patching the python asyncio library to fix this problem. I found this library here .
After installing and utilizing that library, I can now do this, which fixes my problem completely:
main_event_loop = asyncio.get_event_loop()
main_event_loop.run_until_complete(sio.emit("ChangeProgressState"))
Now the program runs as expected, and the messages are being sent/arriving immediately.
I have a performance problem in Javascript causing a crash lately at work. With the objective modernising our applications, we are looking into running our applications as webservers, onto which our client would connect via a browser (chrome, firefox, ...), and having all our interfaces running as HTML+JS webpages.
To give you an overview of our performance needs, our application run image processing from camera sources, running in some cases at more than 20 fps, but in the most case around 2-3fps max.
Basically, we have a Webserver written in C++, which HTTP requests, and provides the user with the HTML pages of the interface and the corresponding JS scripts of the application.
In order to simplify the communication between the two applications, I then open a web socket between the webpage and the c++ server to send formatted messages back and forth. These messages can be pretty big, up to several Mos.
It all works pretty well as long as the FPS stays relatively low. When the fps increases the following two things happen.
Either the c++ webserver memory footprint increases pretty fast and crashes when no more memory is available. After investigation, this happens when the network usage full, and the websocket cache fills up. I think this is due to the websocket TCP-IP way of doing stuff, as the socket must wait for the message to be sent and received to send the next one.
Or the browser crashes after a while, showing the Aw snap screen (see figure below). It seems in that case that the same thing more or less happen but it seems this time due to the garbage collection strategy. The other figure below shows the printscreen of the memory usage when the application is running, clearly showing saw pattern. It seems to indicate that garbage collection is doing its work at intervals that are further and further away.
I have trapped the problem down to very big messages (>100Ko) being sent at fast rate per second. And the bigger the message, the faster it happens. In order to use the message I receive, I start a web worker, pass the blob i received to the web worker, the webworker uses a FileReaderSync to convert the message as an ArrayBuffer, and passes it back to the main thread. I expect this to have quite a lot of copies under the hood, but I am not so well versed in JS yet so to be sure of this statement. Also, I initialy did the same thing without the webworker (FileReader), but the framerate and CPU usage were really bad...
Here is the code I call to decode the messages:
function OnDataMessage(msg)
{
var webworkerDataMessage = new Worker('/js/EDXLib/MessageDecoderEvent.js'); // please no comments about this, it's actually a bit nicer on the CPU than reusing the same worker :-)
webworkerDataMessage.onmessage = MessageFileReaderOnLoadComManagerCBack;
webworkerDataMessage.onerror=ErrorHandler;
webworkerDataMessage.postMessage(msg.data);
}
function MessageFileReaderOnLoadComManagerCBack(e)
{
comManager.OnDataMessageReceived(e.data);
}
and the webworker code:
function DecodeMessage(msg)
{
var retMsg = new FileReaderSync().readAsArrayBuffer(msg);
postMessage(retMsg);
}
function receiveDecodingRequest(e)
{
DecodeMessage(e.data);
}
addEventListener("message", receiveDecodingRequest, true);
My question are the following:
Is there a way to make the GC not have to collect so much memory, by for instance telling some of the parts I use to reuse buffers instead of recreating them, or keeping the GC work intervals fixed ? This is something I know how to do in C++, but in JS ?
Is there another method I should use for my big payloads? Keep in mind that the transmission should be as fast as possible.
Is there another method for reading blob data as arraybuffers that would faster than what I did?
I thank you in advance for you help/comments.
As it turns out, the memory problem was due to the new WebWorker line and the new FileReaderSync line in the WebWorker.
Removing these greatly improved the performances!
Also, it turns out that this decoding operation is not necessary if I want to use the websocket as array buffer. I just need to set the binaryType attribute of websockets to "arraybuffer"...
So all in all, a very simple solution to a pain in the *** problem :-)
I'm researching about php7 and node.js to decide which one is better suited for my tasks. I read about node.js needs a server restart when a error gets thrown.
So lets say I use many libraries in my website, so a error is plausible.
I read in node.js I can store data in variables instead of in a database and use that data from the variables in the next call. Correct me if I'm wrong I never used node.js so far.
Now error gets thrown and cause of this server needs to be restarted.
Then I read there are tools that do that they restart the server eg. the tool called "forever". But now my questions -->
Does the next instance of my server can maintain the state of the old instance or does the data in the variables get lost?
Or do i have to pass this data via some tools like "forever" in the constructor or something of the next instance of the server? I guess this would be spaghetti code.
And if a error gets thrown in cause of wrong requests and there other requests still processing and the server shuts down cause of the error, will all requests time out or return something?
Thank you very much for making stuff clear for me
I read in node.js I can store data in variables instead of in a database and use that data from the varaibles in the next call. Correct me if I'm wrong I never used node.js so far.
You are wrong. Though you can store data in variables and reuse them, node doesn't work the way you are thinking.
Does the next instance of my server can maintain the state of the old instance or does the data in the variables get lost?
It gets lost
Or do i have to pass this data via some tools like "forever" in the constrctor or something of the next instance of the server? I guess this would be spaghetti code.
You need a datastore, a database like mysql or redis for example
And if a error gets thrown in cause of wrong requests and there other requests still processing and the server shuts down cause of the error, will all requests time out or return something?
They will be killed.
You have to add error handling like in every other program you're writing. A properly written program should shutdown very rarely to never, because you catch all your errors
I am working in real time trading application using Node.js(v0.12.4) and Socket.io(1.3.2). In that, I am facing some time delay nearly (100ms) when the response emitting from Node.js to GUI(Socket.Io).
I don't have a clue why the time delay is there while emitting data from Node.js to GUI (Socket.IO).
This happening in Production Site. And we tried to debug this in production server location also because of network latency. But same result.
Please anyone help me on this?
One huge thing to note before doing the following.
When calculating timing from back-end(server side) to front end
(client side) you need to run this on the same computer that uses the same
timing crystal.
quartz crystal-driven timing even on high quality motherboards deffer
from one another.
If you find no delay when calculating time delay from back-end(server side) to front end (client side) on the same pc then the delay you originally found was caused by either the network connection or the deference in the motherboards timing crystals. Which would eliminate Node.js and Socket.io as the cause of the time delay.
Basically you need to find out where the delay is happening before you can solve the problem.
What you need to do is find out what is causing the largest performance hit in your project. In order to do this you will need to isolate the time each process takes. You also need to measure the time delay from the initial retrieval of the data to the release of the data. Then measure the time delay of each function, method and process. Try to isolate the problem. You need to ask what is taking the most time to do?
In order to find out where your performance hit is coming from you need to do the following.
Find out how long it takes for your code to get the information it needs.
Find out how long each information manipulation process takes before it sends out the data i.e function/methods...
After the information is sent out find how long it takes to get the information to the client side and load.
Add all of the times up and make sure it is equal to your performance delay in order to insure you have all the data you require to isolate the performance leak.
Order every method, function, process… by its time delay most time consuming to least time consuming. When you find what processes are causing the largest delays you will then be able to fix the problem... Or at least explore tangible solutions...
Here are some tools you can use to measure the performance throughout your code.
First: Chrome has a really good tool that lets you see the performance of each piece of executed code.
https://developers.google.com/web/tools/chrome-devtools/profile/evaluate-performance/timeline-tool?hl=en
https://developer.chrome.com/devtools/docs/timeline
Second: performance-now node package. You can use it for dev performance testing.
Third: Stack overflow post on measuring time/performance in js.
Fourth: you can use things like console.time()
https://nodejs.org/api/console.html
https://developer.mozilla.org/en-US/docs/Web/API/Console/time
https://developer.chrome.com/devtools/docs/console-api
I have found out that time delay where it is happening.
Once, I have emitted the data from Client Socket to Node, I will show the some alert message ("Data Processed"). The alert message taking time to render in GUI.
This alert message blocking the response data from the Node to Socket.
Thanks for your help Guys.
I'd like to monitor any node.js process that pops up on my machine and eventually print out the maximum memory used by the process when the process dies. Is there any monitoring software that does this already and if not, what is the best way to go about doing this?
Since I know which type of Node.js processes I want to monitor, I could put a check (setInterval) inside each process and then just log the maximum memory used that way. I believe I can access the amount of memory used via process.memoryUsage() and just store the maximum amount and log that upon exit using process.on('exit').
However, is there a way to monitor a group of node.js processes somehow? How do I know which group I am looking at? In other words, how can I mark certain node.js processes so that I can monitor different groups?
Check StrongLoop Arc: https://strongloop.com/strongblog/node-js-performance-heap-profiling-tip/
You can add title to your processes using process.title.
You may also want to check memwatch.
And finally, check this blog post: http://blog.yld.io/2015/08/10/debugging-memory-leaks-in-node-js-a-walkthrough/
Hope this helps.