I have been reading a pretty good book on node and I am on the topic of the framework where Node.js is non blocking. I come from a VB background so I am use to seeing code go in sequence. For the code below, with regards to none blocking Asynchronous framework. What is the event here, shouldn't the event loop pick up on the "for" meaning this is the event and going in sequence node should not do anything until i++ = var i?
Reason why I ask is that I am thinking of an SNMP server side application and I just can not get in my head around what node.js will do if I tell it to ping 10 devices. If the 7th IP is offline I would have to wait for the snmp timeout to occur before going to the 8th, is this correct?
var http = require('http'),
urls = ['shapeshed.com', 'www.bbc.co.uk', 'edition.cnn.com'];
function fetchPage(url) {
var start = new Date();
http.get({ host: url }, function(res) {
console.log("Got response from: " + url);
console.log('Request took:', new Date() - start, 'ms');
});
}
for(var i = 0; i < urls.length; i++) {
fetchPage(urls[i]);
}
Coming from a VB background you have an advantage: VB is event driven too! Have you ever needed to can DoEvents() in your VB code? That's telling VB to run the pending events in the event queue.
The difference is that in VB the events are typically user triggered and UI based, mouse clicks and the like. Node, being primarily server side, has events primarily around I/O.
Your code never gets interrupted or blocked (unless you do it deliberately). In the code snippet above, for example, the call to http.get means "go get this URL, and call this callback when you're done." This kicks off the http request and returns immediately. So your for loop will spin through all your URLs, kicking off all the get operations, and then be finished.
At that point you return from your function, and node goes back to the event loop. Once a request is done, node schedules that's request's callback onto the event loop and the callback will eventually run.
One thing to think about: what if one of the http requests finished before the for loop did? In that case the callback would be scheduled on the event loop. But you're not back to the event loop yet, you're still running your for loop. None of the callbacks will execute until you return from whatever function is currently running.
This kind of "don't do too much in an event handler" advice should sound very familiar to a VB programmer.
No. Asynchronous means that I/O (like a HTTP request) does not block; it is transparently handled on separate threads. The call to http.get returns immediately. Thus, your for loop actually completes (in real time) before a single byte goes over the wire.
In the case of the http module, requests are actually queued in the background for you via the Agent class. By default, node will only open 5 concurrent HTTP requests. You can change this by using a custom Agent.
Related
In general is an event loop only for IO ? and what exactly is an IO job?
For example let's say that a request comes into NodeJs which is then making an outbound http request to an API to get some data while not blocking the user in the meantime.
Is that an IO job and how would NodeJs handle it? what if instead of the http request I wanted to asynchronously make a lengthy calculation and then return a value to the user? Is that handled by the event loop too despite being CPU bound?
In general is an event loop only for IO ?
I wouldn't count timers (setTimeout, setInterval) and sheduling (setImmeadiate, process.nextTick) as IO but one could generally say that the events in the event loop are coming from the outside
and what exactly is an IO job?
That depends on the context you are talking about. Every program gets a certain Input by the user and generates a certain Output. In a terminal for example the input are your keypresses and the output is what is displayed. Whe talking about nodejs IO, one usually refer to network / file operations, or more generally: code not written in js.
For example let's say that a request comes into NodeJs which is then making an outbound http request to an API to get some data while not blocking the user in the meantime.
Is that an IO job and how would NodeJs handle it?
Nodejs would spawn a background thread that makes the request, the main process continues with other stuff in the meantime (continues with other events on the event queue). Then if the async request is done the background process pushes the result onto the event queue, the event loop will pull it from there and execute callbacks etc.
what if instead of the http request I wanted to asynchronously make a lengthy calculation and then return a value to the user?
You have to spawn another thread in nodejs, lengthy calculations are synchronous otherwise.
Is that handled by the event loop too despite being CPU bound?
Everything somewhen lands on the event loop, and everything gets executed on the CPU ...
I have read the difference between Multi thread mechanism and NodeJS Single thread mechanism here. I know less about thread concept.
My question
The above article says that all the Non Blocking I/O is handled using single thread in Event loop.
I have read through questions posted in this forum, but all it says is just the overview of how single thread is working and not the deeper mechanism. says something like...
Starts processing the Client Request
If that Client Request Does Not requires any Blocking IO Operations, then process everything, prepare response and send it back to client.
If there are like 2 or more Non Blocking requests in Event Queue, Event loop takes each requests and processes it.
First request enter Event Pool and starts processing and does not wait or hold till the response and meanwhile request 2 enters and starts processing without wait.
Now,since the 2nd request has taken the thread for processing (and all request is handled using single thread) , currently what is handling the 1st request process, If there is thread sharing , how is it happening ?
Is the first request process released when handling 2nd request and later comes back to 1st request ? if so how is it happening in thread perspective ?
how does single thread processes 2 or more request concurrently as basically thread will be assigned to a request until all it's process is finished
and how is single thread handled for both Input and Output operation at same time ?
is there any topic i am missing to read so that i'm getting this single thread event loop mechanism ?
First off, "single threaded" applies only to one thread running your Javascript. The node.js has other native threads for implementing some of the functions in the built-in library. For example, file I/O uses a thread pool in order to implement asynchronous file I/O. But, what's most important to understanding how your own Javascript runs is that there is only one thread of your Javascript.
Let's imagine that you have a simple web server like this:
const http = require('http');
function sendFile(res, filename) {
if (filename.startsWith("/")) {
filename = filename.slice(1) + ".html";
}
fs.readFile("1.html", (err, data) => {
if (err) {
res.writeHead(404);
res.end("not found");
} else {
res.writeHead(200, {'Content-Type': 'text/html'});
res.write(data);
res.end();
}
});
}
const server = http.createServer((req, res) => {
if (req.url === "/1" || req.url === "/2" || req.url === "/3") {
sendFile(req.url);
} else {
res.writeHead(404);
res.end("not found");
}
});
server.listen(80);
This web server responds to requests for three URLs /1, /2 and /3.
Now imagine that three separate clients each request one of those URLs. Here's the sequence of events:
Client A requests http://myserver.com/1
Client B requests http://myserver.com/2
Client C requests http://myserver.com/3
Server receives incoming connection from client A, establishes the connection, client sends the request for /1 and the server reads and parses that request.
While the server is busy reading the request from client A, the requests from both client B and client C arrive.
The TCP stack handles incoming connections at the OS level (using other threads i.e. kernel level thread).
Notifications of the arriving connections are put in the node.js event queue. Because the node.js server is busy running Javascript for the client A connection, those two events sit in the event queue for now.
At the same time as those other connections are arriving, the node.js server is starting to run the request handler for /1. It finds a match in the first if statement and calls sendFile("/1").
sendFile() calls fs.readFile() and then returns. Because fs.readFile() is asynchronous, that file operation is started, but is handed over to the I/O system inside of node.js and then the function immediately returns. When sendFile() returns, it goes back to the http server request handler which also then returns. At this point, there's nothing else for this request to do. Control has been returned back to the interpreter to decide what to do next.
The node.js interpreter checks the event queue to see if there is anything in their to process. It finds the incoming request from client B and that request starts processing. This request goes through the same 8 and 9 steps until it returns with another fs.readFile() operations initiated.
Then step 10 is repeated for the incoming request from client C.
Then, some short time later, one of the three fs.readfile() operations that were previously initiated completes and places a completion callback into the Javascript event queue. As soon as the Javascript interpreter has nothing else to do, it finds that event in the event queue and begins to process it. This calls the callback that was passed to fs.readFile() with the two parameters that that function expects and the code in the callback starts to execute.
Assuming the fs.readFile() operation was successful, it calls res.writeHead(), then res.write(), then res.send(). Those three calls all send data to the underlying OS TCP stack where it is then sent back to the client.
After res.end() returns, control is returned back to the interpreter and it checks the event queue for the next event. If another fs.readFile() callback is already in the event queue, then it is pulled out of the event queue and processed like the previous one. If the event queue is empty, then the interpreter waits until something is put in the event queue.
If there are like 2 or more Non Blocking requests in Event Queue, Event loop takes each requests and processes it.
node.js only runs one at a time. But, the key is that asynchronous code in the request handler allows the handler to return control back to the system so that other events can be processed while that first request was waiting for its asynchronous operation to complete. This is a form of cooperative, non-pre-emptive multi-tasking. It's not multiple threads of Javascript. The first request handler actually starts and asynchronous operation and then returns (as if it was done). When it returns, the next event in the queue can start processing. At some later time when the asynchronous operation completes, it will insert its own event into the event queue and it will get back in line to use the single thread of Javascript again.
First request enter Event Pool and starts processing and does not wait or hold till the response and meanwhile request 2 enters and starts processing without wait.
Most of this has already been described above. If the Javascript thread is busy why request 2 enters the event queue, that request will sit in the event queue until the Javascript thread is no longer busy. It may have to wait a short period of time. But, it won't have to wait until request 1 is done, only until request 1 returns control back to the system and is, itself, waiting for some asynchronous operation to complete.
Now,since the 2nd request has taken the thread for processing (and all request is handled using single thread) , currently what is handling the 1st request process, If there is thread sharing , how is it happening ?
While the 2nd request is using the Javascript thread, the 1st request is not running any Javascript. It's native code asynchronous operations may be running in the background (all asynchronous operations require some native code), but there is only one piece of Javascript running at any given time so if the 2nd request is running some Javascript, then the first request is either waiting for its asynchronous operation to finish or that operation has already finished and an event is sitting in the event queue waiting for the 2nd request to be done so that event can get processed.
Is the first request process released when handling 2nd request and later comes back to 1st request ? if so how is it happening in thread perspective ?
This all works through the event queue. 1st request runs until it returns. 2nd request runs until it returns. When async operation from 1st request completes it inserts an item in the event queue. When the JS interpreter is free, it pulls that event from the event queue and runs it. There may be threads involved in native code implementations of asynchronous operations, but there is still only one thread of Javascript.
how does single thread processes 2 or more request concurrently as basically thread will be assigned to a request until all it's process is finished
It never actually runs multiple pieces of Javascript concurrently. The Javascript from each different operation runs until it returns control back to the interpreter. Asynchronous operations (such as file I/O or networking operations) can run concurrently and those are managed by native code, sometimes using additional threads and sometimes not. File I/O uses a thread pool to implement non-blocking, asynchronous file I/O. Networking uses OS event notifications (select, epoll, etc...), not threads.
and how is single thread handled for both Input and Output operation at same time ?
It doesn't in your Javascript. It would typically read, then write. It doesn't do both "at the same time". Now, the TCP stack may be doing some actual parallel work inside the OS, but that's all managed by the OS and even that probably gets serialized at the network interface at some point.Requests are handled by single thread where as input output processes are managed by os level threads created per each process by OS
is there any topic i am missing to read so that i'm getting this single thread event loop mechanism ?
Read every thing you can find about the Javascript event queue. Here are some references to get you started:
How does JavaScript handle AJAX responses in the background?
Where is the node.js event queue?
Node.js server with multiple concurrent requests, how does it work?
Asynchronous process handler in node
How does a single thread handle asynchronous code in Node.js?
When using $http we can set the timeout for it and it will look something like this:
$http.get(url,{timeout: 5000}).success(function(data){});
What is that timeout mean? Is it mean the connection (data download) must be completed within the timeout period? Or it is mean the delay time to receive respond from the server? What would be the best general minimal timeout setting for mobile connection?
If the http request does not complete within the specified timeout time, then an error will be triggered.
So, this is kind of like saying the following to the $http.get() function:
I'd like you to fetch me this URL and get me the data
If you do that successfully, then call my .success() handler and give me the data.
If the request takes longer than 5000ms to finish, then rather than continue to wait, trigger a timeout error.
FYI, it looks to me like AngularJS has converted to using standard promise syntax, so you should probably be doing this:
$http.get(url,{timeout: 5000}).then(function(data){
// successfully received the data here
}, function(err) {
// some sort of error here (could be a timeout error)
});
What is that timeout mean? Is it mean the connection (data download) must be completed within the timeout period?
Yes. If not completed within that time period, it will return an error instead. This avoids waiting a long time for a request.
Or it is mean the delay time to receive respond from the server?
No, it is not a delay time.
What would be the best general minimal timeout setting for mobile connection?
This is hard to say without more specifics. Lots of different things might drive what you would set this to. Sometimes, there is no harm in letting the timeout be a fairly long value (say 120 seconds) in case the server or some mobile link happens to be particularly slow some day and you want to give it as much chance as possible to succeed in those circumstances. In other cases (depending upon the particular user interaction), the user is going to give up anyway if the response time is more than 5 seconds so there may be no value in waiting longer than that for a result the user will have already abandoned.
timeout – {number|Promise} – timeout in milliseconds, or promise that should abort the request when resolved.
Source
Timeout means "perform an action after X time", in JS anyway.
I have been working in Node.js and I am wondering what exactly does listen method do, in terms of eventloop. If I had a long running request, does it mean that server will never listen since it can only do one work at a time.
var http = require('http');
function handleRequest(request, response) {
response.end('Some Response at ' + request.url);
}
var server = http.createServer(handleRequest);
server.listen(8083, function() {
console.log('Listening...')
})
Is server.listen listening to some event?
You can think of server.listen() as starting your web server so that it is actually listening for incoming requests at the TCP level. From the node.js http documentation for .listen():
Begin accepting connections on the specified port and hostname.
The callback passed to server.listen() is optional. It is only called once to indicate that the server has been successfully started and is now listening for incoming requests. It is not what is called on every new incoming request. The callback passed to .createServer() is what is called for every new incoming request.
Multiple incoming requests can be in process at the same time though due to the single-threaded nature of node.js only one request is actually executing JS code at once.
But, a long running request is generally idle most of the time (e.g. waiting for database I/O or disk I/O or network I/O) so other requests can be processed and run during that idle time. This is the async nature of node.js and why it is important to use asynchronous I/O programming with node.js rather than synchronous I/O processing because asynchronous I/O allows other requests to run during the time when node.js is just waiting for I/O.
Yes, it basically binds an event listener to that port; similar to how event listeners work in your own code. Going more in depth would involve sockets, etc...
https://nodejs.org/api/net.html#net_server_listen_port_host_backlog_callback
The other answers are essentially correct, but I wanted to add more detail.
When you call createServer, the handler you pass in is what gets called on every incoming HTTP connection. But that is merely setting that up: it does not actually start the server or start listening for those connections. That doesn't happen until you call listen.
The (optional) callback for listen is just what gets called when the server has successfully started and is now listening for connections. Most of the time, it's simply used to log to the console that the server is started. You could also use it to record server start time for uptime monitoring. That callback is NOT invoked for every HTTP request: only once on server startup.
You don't even have to supply the callback for listen; it works fine without it. Here are some common variations (note that it's a good practice to let the port be specified by an environment variable, usually PORT; if that environment variable isn't set, there is a default):
// all in one line, no startup message
var server = http.createServer(handler).listen(process.env.PORT || 8083);
// two lines, no startup message
var server = http.createServer(handler); // server NOT started
server.listen(process.env.PORT || 8083); // server started, no confirmation
// most typical variation
var server = http.createServer(handler);
server.listen(process.env.PORT || 8083, function() {
// server started, startup confirmed - note that this only gets called once
console.log('server started at ' + Date.now());
});
I have a Web Worker. I wish to make periodic network requests with it. One thing I particularly want is to make these requests even if the main JS execution thread is blocked (eg by a window.alert). I'm using Chrome 38.
However, when I attempt to make network requests in the worker, the requests appear to be blocked by the UI thread. Here is a contrived example to illustrate the problem:
base.js:
var worker = new Worker("/worker.js");
setTimeout(function() {
console.log("begin blocking");
var startDt = new Date();
var blockPeriod = 5000;
var a;
// Obviously we'd never actually do this, but this while loop
// is a convenient way to create the problem case (a blocked main
// thread).
while ((new Date() - startDt) < blockPeriod) {
a = 0;
}
console.log("stop blocking");
}, 3000);
worker.js:
var requestInterval = 1000;
var sendRequest = function() {
console.log("Send request interval");
var request = new XMLHttpRequest();
request.open("GET", "/ping", true);
request.onload = function() {
if (request.status === 200){
console.log(request.responseText)
} else {
console.log(request.status)
}
};
request.onerror = function() {
console.log("error")
};
request.send();
setTimeout(sendRequest, requestInterval);
}
sendRequest();
The result I'm seeing is that we see successful HTTP requests for three seconds, until the blocking begins. At this point, we don't see anything logged to the console until the blocking ends, at which point we see five "Send request interval"s followed by 5 logs of the response, like so:
Send request interval
{"pong": true}
Send request interval
{"pong": true}
Send request interval
{"pong": true}
Send request interval
{"pong": true}
begin blocking
stop blocking
5x Send request interval
5x {"pong": true}
Send request interval
{"pong": true}
I also see in my server logs that no requests are made in that blocking time, then those five requests are all received roughly simultaneously at the end of the blocking period.
Given that "Send request interval" occurs five times in a row, the worker is evidently continuing to execute: if it weren't, it wouldn't make it through to queue up the next iteration. I've also found that if I block by triggering a window.alert instead of spinning in a loop, I get the log messages from the beginning of sendRequest at 1 second intervals, and then get the response handler log messages in a large batch as soon as I stop blocking.
In Firefox, the background thread seems to stop entirely in this case (I don't get that same batch of five requests queued up during the blocked period). However, I'm only targeting Chrome in this case (and I ultimately want to use WebSockets which don't even work in Firefox Workers), so I'm not really interested in that.
All added together, this leads me to believe that there are some classes of activity in Web Workers which are blocked by the spawning thread, and some which are not (I originally saw this same behavior with WebSockets). Concretely, I'd like to know (if anyone does know):
What Worker activity is blocked by the main thread in Chrome?
Is there a way to work around this? I'd very much like to be able to able to establish a WebSocket connection in a Worker, and then continue to PING/PONG back and forth, even if something (such as an alert/confirm) does block the main thread.
Is this all nonsense, and am I just doing something stupid?
Your observation is correct. When the UI thread is blocked, network calls aren't dispatched.
Even worse, Chrome has the best behavior of the bunch. When a worker makes a XHR request when the UI thread is blocked:
Chrome: all requests are queued. The browser will not actually issue the requests until the UI thread unblocks. On the plus side, the worker thread is still free to run.
Firefox: new XMLHttpRequest() blocks until the UI thread unblocks.
IE: xhr.open() blocks until the UI thread unblocks.
While Chrome fortunately does not cause a worker thread to stop and wait (even though it won't get any data), Firefox and IE will cause a worker thread to wait on the UI thread when you try to make a XHR request.
There is no way to work around this; you're beholden to the browser to make requests on your behalf. I haven't done any testing with WebSockets, but they may deliver events even if the UI thread is blocked. At worst, the received messages would queue until the UI thread unblocks.
In case anyone stumbles across this, this behavior is confirmed as a bug (to the loose definition of "bug" as "does not behave as it ought to") in Blink, as of February 2015:
https://code.google.com/p/chromium/issues/detail?id=443374
I'm having the same issue, with a Web Worker that performs a sort of a keep-alive: it periodically pings the server, to inform that the page is still alive. I'm also using console.log in the worker, but i'm sure this is not the cause.
After some investigations, I can state that the problem can address two different situations:
a long-time UI operation is blocking main thread and worker's requests are not executed. This is the case of sample illustrated by heliotrope. As of September 2018, the problem occurs only on Firefox and IE, since Chrome and Edge can correctly handle activities on worker when the main thread is blocked.
To solve this issue, I'm thinking at sending a special ping to the server before starting any long-time operation, to inform that he won't receive anything from me, until the operation is finished.
a long-time asynchronous ajax call is being performed by main thread and worker's requests are queued. Some setting on the server is preventing from multiple ajax calls to run in parallel: other requests are queued until the first ajax call completes. The problem is server-specific, hence it does not depend on any browser in particular.
This problem is, in my case, due to ASP.NET session state locking: the ASP.NET pipeline will not process requests belonging to the same session concurrently but queues them, and executes them serially. Here is a detailed link: http://tech-journals.com/jonow/2011/10/22/the-downsides-of-asp-net-session-state.
Marking controller's session state as ReadOnly will solve the problem, while completely disabling session state in Web.config (<sessionState mode="Off" />) will seriously improve performance of the whole application.