I'm researching about php7 and node.js to decide which one is better suited for my tasks. I read about node.js needs a server restart when a error gets thrown.
So lets say I use many libraries in my website, so a error is plausible.
I read in node.js I can store data in variables instead of in a database and use that data from the variables in the next call. Correct me if I'm wrong I never used node.js so far.
Now error gets thrown and cause of this server needs to be restarted.
Then I read there are tools that do that they restart the server eg. the tool called "forever". But now my questions -->
Does the next instance of my server can maintain the state of the old instance or does the data in the variables get lost?
Or do i have to pass this data via some tools like "forever" in the constructor or something of the next instance of the server? I guess this would be spaghetti code.
And if a error gets thrown in cause of wrong requests and there other requests still processing and the server shuts down cause of the error, will all requests time out or return something?
Thank you very much for making stuff clear for me
I read in node.js I can store data in variables instead of in a database and use that data from the varaibles in the next call. Correct me if I'm wrong I never used node.js so far.
You are wrong. Though you can store data in variables and reuse them, node doesn't work the way you are thinking.
Does the next instance of my server can maintain the state of the old instance or does the data in the variables get lost?
It gets lost
Or do i have to pass this data via some tools like "forever" in the constrctor or something of the next instance of the server? I guess this would be spaghetti code.
You need a datastore, a database like mysql or redis for example
And if a error gets thrown in cause of wrong requests and there other requests still processing and the server shuts down cause of the error, will all requests time out or return something?
They will be killed.
You have to add error handling like in every other program you're writing. A properly written program should shutdown very rarely to never, because you catch all your errors
Related
I have a python websocket server attempting to communicate with a javascript websocket client (embedded in HTML). The events are being emited from the server immediately, but it takes upwards of 30 seconds for the server to send the event trigger, despite both the client and server being locally hosted.
Here is the relavent code for the server:
sio = socketio.AsyncServer(cors_allowed_origins='*')
app = web.Application() #aiohttp web server
loop = asyncio.get_event_loop()
sio.attach(app)
async def index(request):
with open('./index.html') as f:
return web.Response(text=f.read(), content_type='text/html')
app.router.add_get('/', index)
app.router.add_get('/index.html', index)
if __name__ == '__main__':
web.run_app(app)
the event is being fired like so (edit, this must be done with event loops, as emit is an asynchronous function being run from a synchronous one.):
print('Starting event')
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(sio.emit('ChangeProgressState'))
loop.close()
print('Event has been fired.')
However, the print statements show up immediately. On the client end, I am connecting and trying to consume the event like this:
const socket = io.connect("http://localhost:8080", {
transports: ['websocket']
})
socket.on("ChangeProgressState", function (data) {
console.log("got event.")
//some code here...
});
However, from the time it takes for the event to fire, and the time it takes for the javascript socket to notice can be a very long time, from 30 seconds to sometimes a few minutes. Is there something I'm doing wrong here?
It should be noted, there are very little (2%-5%) resources being consumed (both memory and CPU), so I do not currently think that is the issue. Any help would be much appreciated.
EDIT 11/15/2019: I have tried looking at the networking tab of the application (chromium-browser on raspberry pi). It seems to show the initial socket connection, but it doesn't show anything in terms of communication between sockets, even after the event eventually fires.
EDIT 2: This definitely seems to be an issue server-side. I can send events from the JS client to the python server essentially immediately, but going in the other direction is when it takes a long time to arrive. I'm not quite sure why though.
Ah ok, so my gut said it sounds like the client is long polling. Many socket libraries first establish long-polling and then upgrade to ws connections.
After taking a look at Socket.io:
... which first establishes a long-polling connection, then tries to upgrade to better transports that are “tested” on the side, like WebSocket. ...
So I don't believe you're doing anything wrong, it's just the initialization process of establishing the WebSocket connection.
As for the python part, I'll be honest that's a tad more fuzzy to me. My first guess is that the loop code doesn't block the print statement from being executed -- but I'm more familiar with JavaScript than Python, so not completely certain on that front. My second guess is that I do know from other pub/sub libraries that the server side engine sometimes makes use of a middle layer of sorts (sometimes a cache, sometimes a queue) that helps ensure messages are sent/received, that's also a possibility.
extra tidbit: I suspect if you look at the network tab of your browser's dev tools, it'd display that behavior, some form of HTTP requests, and then eventually you'd see the socket connection. Playing around with turning your Python server/service off/on would also demonstrate the robustness of socket.io in the browser and edge cases for how it handles unstable networking when communicating with respect to various internet communication protocols.
Thank you to everyone that helped answering this question! I finally found a solution that is a bit unorthodox, so I'll explain the whole situation here.
Essentially, in order to run an async method in a synchronous context, you must use the asyncio's run_until_complete method on an event loop. This is how I was doing it when this question was asked. However, after talking to the creator of the python-socketio library, it seems that you must run this in the same event loop as the one the server is running in.
However, this creates a different problem. If an event loop is already running, python does not allow you to use run_until_complete on it, giving you an error: RuntimeError: This event loop is already running.
So, this things sound contradictory right? And you would be correct. However, this problem is prevalent enough that another library exists for the sole purpose of monkey-patching the python asyncio library to fix this problem. I found this library here .
After installing and utilizing that library, I can now do this, which fixes my problem completely:
main_event_loop = asyncio.get_event_loop()
main_event_loop.run_until_complete(sio.emit("ChangeProgressState"))
Now the program runs as expected, and the messages are being sent/arriving immediately.
I have several scenarios to be executed. I introduce some test data in the database (using beforeAll) before executing these scenarios and remove such data after executing the scenarios.
The problem is that if a scenario fails, the code present within afterAll is not being executed. Therefore, test data is not removed from the data base. Is there any other way to accomplish this?
Thanks in advance.
First, You should mock the database connection, there are many libraries to do so, for instance if you are using mongodb have a look at Mockgoose
Mockgoose provides test database by spinning up mongod on the back when mongoose.connect call is made. By default it is using in memory store which does not have persistence.
For the afterAll hook that never runs (Which is the default behavior in case a test fails):
I suggest you truncate everything in the beforeAll hook, so whenever you start running the tests you have an empty database, even if you have some data from the last run (which will not be the case if you use Mockgoose or similar)
I am using HAPI.JS framework with NodeJS and created a proxy. Think that proxy means i am just maintaining session in redis. Other than that i am not doing anything in the code. May be only thing is i am using setInterval to log my process.memoryUsage() for every 3 mintues.
My Questions:
Why my Memory Keeps on Increasing?
Will it get down?
Is this occurs due to setInterval keeps on logging the process usage?
Is this occurs due to console logging of every request and response?
My Redis Database is kept open till my server crashes, it this causes this ?
Do i need use process mananger like new relic or strong loop to identify this?
So how long this memory will keep on increasing, at some point it must stop (i want to know which point is that?)
I am using sequelize of MSSQL transaction using pooling concept? Does pooling makes this?
P.S I am new to node JS.
Why my Memory Keeps on Increasing?
You got a memory leak
Will it get down?
Sometimes GC kicks in and cleans up some things (that are not leaking)
Is this occurs due to setInterval keeps on logging the process usage?
Usually not, but w/o seeing the code I can't say this for sure
Is this occurs due to console logging of every request and response?
Usually not, but w/o seeing the code I can't say this for sure
My Redis Database is kept open till my server crashes, it this causes this ?
Should not be a problem.
Do i need use process mananger like new relic or strongloop to identify this?
It is one way to do it ... but there are also others.
So how long this memory will keep on increasing, at some point it must stop (i want to know which point is that?)
Depends on the server setup. How much RAM + what else is running etc.
I am using sequelize of MSSQL transaction using pooling concept? Does pooling makes this?
Usually not, but w/o seeing the code I can't say this for sure
Maybe this post helps you find the leak:
https://www.nearform.com/blog/how-to-self-detect-a-memory-leak-in-node/
Is there any way i can do some database updation things whenever my node.js server crashes or stopped. Like try{}catch(){}finally(){} in JAVA. I am a bit newbie here.
Is there any events will node emit before it going shutdown. If so i can write my function there.
I have scenario,if i stop the server manually,i need to update some fields in the database.
The same is for Unhandled crashes also.
i here about domain in Node.js. But i have no idea how to monitor a whole server using domain.
An event is emitted when the node process is about to exit:
process.on('exit', function(code) {
console.log('About to exit with code:', code);
});
http://nodejs.org/api/process.html#process_event_exit
You can't query the database here though, since this handler can only perform synchronous operations. Some possible alternatives:
use database transactions so you never need to do "database updation things" when your app crashes
use a tool like Upstart to automatically restart your process, and then do database fixup stuff whenever your process starts
When you are using node JS it's bad practice to use try / catch, because of big number of asynchronous calls. The best practice here to use "promises" review next link, there you can find a good explanation: https://www.promisejs.org/
I've got a question about data flow that is summarized best by the image below:
I've got the data path from the UI (WaveMaker) down to the hardware working perfectly. The question I have is whether I'm missing something in the connection from the Java Service to Wavemaker.
I'm trying to provide information back to Wavemaker from the HW. The specifics of shared memory and semaphore signaling are worked out already. Where I'm running into a problem is how to get the data from the Java Service back to WaveMaker, when it hasn't specifically requested it. My plan was to generate events when the Java Service returned, but another engineer here insists that it won't work, since there's no direct call from Wavemaker and we don't want to poll.
What I proposed was to call the function after the page loaded, allow the blocking to occur at the .so level, as shown below, and then handle the return string when the call returned. We would then call the function again. That has the serious flaw of blocking out interaction with the user interface.
Another option put forth would be to use a hidden control, somehow pass it into Java, and invoke an event on it from Java, which could then be made to execute a script to update the UI with the HW response. That keeps the option of using threads alive, and possibly resolves the issue. Is there some more elementary way of getting information from Java->JavaScript->UI without it having been asked for?