I have found strange delay of http.request function. Here is my code
var express = require('express');
var http = require('http');
app.set('port', process.env.PORT || 3000);
var app = express();
app.get('/aaa',function(req,res) {
setTimeout(function(){
res.json({"a":1});
},500);
});
app.get('/bbb',function(req,res){
var options = {
host: '127.0.0.1',
port: 3000,
path: '/aaa',
method: 'GET'
};
var request = http.request(options, function(result) {
result.on("data",function(){
});
res.json({"b":2});
});
request.on('error', function() {
res.json({"b":2});
});
request.end();
});
http.createServer(app).listen(app.get('port'), function(){
});
Client call /bbb, then it's handler call /aaa and within 500ms result returns back to client.
I tried to measure response time in different situations using Apache Bench:
1) 1000 requests with 1 concurrent requests.
Average response time: 500ms
2) 1000 requests with 50 concurrent requests.
Average response time: 5000ms
3) 1000 requests with 100 concurrent requests.
Average response time: 10000ms
Why response time is growing?
It's okay when I call /aaa directly
It's not unusual behaviour. The HTTP Client used in the callback to /bbb (http.request) is limited to 5 concurrent sockets per host. In other words, it can only make 5 HTTP requests in parallel. You can find reference to this here in the documentation
Just to confirm you're hitting the limit, you should run your tests using 5 and 6 concurrent requests. You'll see (as I did) average response time drops significantly at 6 concurrent requests. This is because the 6th concurrent request will be queued until one of the 5 preceeding requests to /aaa is completed.
To answer your question about why response time grows: The more concurrency you add in your benchmark, the more average response time will go up because each request has to wait for more requests in the queue to finish before it can get a socket.
You can increase the number of concurrent sockets your HTTP client can handle by modifying the default agent like this:
var http = require("http");
http.globalAgent.maxSockets = 10;
You can also circumvent pooling altogether by passing agent:false to http.get like so:
http.get({hostname:'localhost', port:80, path:'/', agent:false}, function (res) {
// Do stuff
})
Update (8th Feb 2015)
An important change regarding this answer has come up in Node v 0.12.0.
maxSockets are no longer limited to 5. The default is now set to
Infinity with the developer and the operating system given control
over how many simultaneous connections an application can keep open to
a given host.
I had same issue and it is resolved by keeping it very simple get request as below
var req = http.get(requestUrl)
req.end();
Related
andI'm trying to work on a caching solution for inflight requests to Koa,
Let's say that i have 100 seperate users hitting the same endpoint concurrently, but the endpoint takes ~4-5 seconds to return a response.
For example:
GET http://mykoa.application.com/getresults
In my router middleware is it possible to cache all of the concurrent inbound requests and then once the response has been generated return the same result to all of them? Potentially something similar to the example below?
const inflight = {};
router.use(async function(ctx, next) {
// Create a cache 'key'
const hash = `${ctx.request.url}-${ctx.state.user?.data?.id}-${JSON.stringify(ctx.request.body)}`;
// Check if there is already a request inflight
if (inflight[hash]) {
// If there is then wait for the promise resolution
return await inflight[hash];
}
// Cache the request resolution for any other identical requests
inflight[hash] = next();
await inflight[hash];
// Clean it up so that the next request will be fresh
inflight[hash].then(function(res) {
delete inflight[hash];
}, function(err) {
delete inflight[hash];
})
})
In my head this should work, and the expectation would be that all 100 concurrent requests would resolve at the same time (after the first one has resolved) however in my tests each request is still being run separately and taking 4-5 seconds each in sequence.
In my express app, I have defined 2 endpoints in my application. One for is-sever-up check and one for simulating a blocking operation.
app.use('/status', (req, res) => {
res.sendStatus(200);
});
app.use('/p', (req, res) => {
const { logger } = req;
logger.info({ message: 'Start' });
let i = 0;
const max = 10 ** 10;
while (i < max) {
i += 1;
}
res.send(`${i}`);
logger.info({ message: 'End' });
});
I am using winston for logging and PM2 for clustering using the following command
$ pm2 start bin/httpServer.js -i 0
It has launched 4 instances.
Now, when I visit the routes /p, /p, /status in order in different tabs with around 1 second delay between (request 1 and request 2) and (request 2 and request 3), I expected to get response for request 1 and request 2 after some time but with around 1 second delay and response for request 3 should come instantly.
Actual: The response for request 3 did come instantly but something weird happened with request 1 and request 2. The request 2 didn't even start until request 1 was completed. Here are logs that I got. You can see the time stamp for the end of request 1 and start of request 2.
{"message":"Start","requestId":"5c1f85bd-94d9-4333-8a87-30f3b3885d9c","level":"info","timestamp":"2020-12-28 07:34:48"}
{"message":"End","requestId":"5c1f85bd-94d9-4333-8a87-30f3b3885d9c","level":"info","timestamp":"2020-12-28 07:35:03"}
{"message":"Start","requestId":"f1f86f68-1ddf-47b1-ae62-f75c7aa7a58d","level":"info","timestamp":"2020-12-28 07:35:03"}
{"message":"End","requestId":"f1f86f68-1ddf-47b1-ae62-f75c7aa7a58d","level":"info","timestamp":"2020-12-28 07:35:17"}
Why did the request 1 and request 2 not start at the same time (with 1 second delay, of course)? And if they are running synchronously, why did request 3 respond instantly and not wait for request 1 and 2 to complete?
That's because connection of the header is keep-alive in the response which your node server respond in default. So, connection will be reused when you use browser (curl also could simulate the reused connection situtation). That means multiple request is served by the same instance within a specified time. Even you have multiple node instances.
Note: You could see specified time in response header like this Keep-Alive: timeout=5
If you use browser, open network tab to see response headers.
If you use curl, add -v options to see response headers
You could try to use multiple separated curl command at the same time in terminal. Separated curl command means connection will not be reused. So, you'll get your expected results. You could add a console.log("status test") in /status router. Then, use pm2 logs to see which instance serve the request like following format (these logs are produced by accessing endpoint with browser).
0|server | status test
0|server | status test
0 means the first instance, you will see this is all the same instance to serve request when you use browser to access endpoint. But, if you use curl, you'll find out the number is always changed which mean every request is served by different node instance.
You could see I sent two request at the same time with curl in terminal. Then, different node instance to serve the request. So, the start and end time of console.log are same. In this example, I have 8 event-loop so I could deal with 8 long-processing (synchronous code) request at the same time.
And, you could use curl to simulate the keep-alive situation. Then, you'll see the request is served by same node instance.
curl http://localhost:8080/status http://localhost:8080/status -v -H "Connection: keep-alive"
You also could use connection close to see the request is served by different node instance.
curl http://localhost:8080/status http://localhost:8080/status -v -H "Connection: close"
You could see the different here.
If you want to close the connection in server side, you could use following code.
res.setHeader("Connection", "close")
This is my test code.
const express = require("express")
const app = express();
const port = 8080;
app.use('/status', (req, res) => {
console.log("status tests");
res.sendStatus(200);
});
app.use('/p', (req, res) => {
console.log(new Date() + " start");
let i = 0;
const max = 10 ** 10;
while (i < max) {
i += 1;
}
res.send(`${i}`);
console.log(new Date() + " end");
});
app.listen(port, () => {
return console.log(`server is listening on ${port}`);
});
var data = import('./blahblah/data')
app.get('/data', data.getData)
app.listen(3000)
// blahblah/data:
exports.getData = function(req, res) {
setTimeout(function(){
res.send('test')
}, 10000)
};
//code I use to test
for(var i = 0;i<10;i++){
$.ajax({
url: 'http://127.0.0.1:3000/data',
success: function(l){console.log(l);}
});
}
If I send 10 simultaneous requests to this endpoint, then they only receive a reply one at a time 10 seconds apart, with the last request coming 110 seconds after the first request was sent. Aren't nodejs and express supposed to let other requests process when the other requests are running asynchronous code?
Your code is fine, probably something is wrong with the way you're testing it.
Please, make sure that you're sending your requests in parallel.
I tested your code with loadtest utility
loadtest -n 10 -c 10 http://localhost:3000/data
and got expected results
Target URL: http://localhost:3000/data
Max requests: 10
Concurrency level: 10
Agent: none
Completed requests: 10
Total errors: 0
Total time: 10.050681130000001 s
Here is a code snipped I tested:
var app = require('express')();
app.get('/data', function(req, res) {
setTimeout(function(){
res.send('test')
},10000)
});
app.listen(3000)
Update:
Web browsers usually limit the number of parallel requests per host. I just tested it in my Google Chrome and it limited the number of parallel requests by 6:
Here is a detailed timing for one of delayed requests:
Looks like your browser have an even harsher limits.
Noob question on using callbacks as a control flow pattern with Node and the http class. Based on my understanding of the event loop, all code is blocking, i/o is non-blocking and using callbacks, here's the a simple http server and a pseudo rest function:
// Require
var http = require("http");
// Class
function REST() {};
// Methods
REST.prototype.resolve = function(request,response,callback) {
// Pseudo rest function
function callREST(request, callback) {
if (request.url == '/test/slow') {
setTimeout(function(){callback('time is 30 seconds')},30000);
} else if (request.url == '/test/foo') {
callback('bar');
}
}
// Call pseudo rest
callREST(request, callback);
}
// Class
function HTTPServer() {};
// Methods
HTTPServer.prototype.start = function() {
http.createServer(function (request, response) {
// Listeners
request.resume();
request.on("end", function () {
// Execute only in not a favicon request
var faviconCheck = request.url.indexOf("favicon");
if (faviconCheck < 0) {
//Print
console.log('incoming validated HTTP request: ' + request.url);
//Instantiate and execute on new REST object
var rest = new REST();
rest.resolve(request,response,function(responseMsg) {
var contentType = {'Content-Type': 'text/plain'};
response.writeHead(200, contentType); // Write response header
response.end(responseMsg); // Send response and end
console.log(request.url + ' response sent and ended');
});
} else {
response.end();
}
});
}).listen(8080);
// Print to console
console.log('HTTPServer running on 8080. PID is ' + process.pid);
}
// Process
// Create http server instance
var httpServer = new HTTPServer();
// Start
httpServer.start();
If I open up a browser and hit the server with "/test/slow" in one tab then "/test/foo" in another, I get the following behavior - "foo" responds with "Bar" immediately and then 30 secs late, "slow" responds with "time is 30 seconds". This is what I was expecting.
But if I open up 3 tabs in a browser and hit the server with "/test/slow" successively in each tab, "slow" is being processed and responds serially/synchronously so that the 3 responses appear at 30 second intervals. I was expecting the responses right after each other if they were being processed asynchronously.
What am I doing wrong?
Thank you for your thoughts.
This is actually not the server's fault. Your browser is opening a single connection and re-using it between the requests, but one request can't begin until the previous finishes. You can see this a couple of ways:
Look in the network tab of the Chrome dev tools - the entry for the longest one will show the request in the blocking state until the first two finish.
Try opening the slow page in different browsers (or one each in normal and incognito windows) - this prevents sharing connections.
Thus, this will only happen if the same browser window is making multiple requests to the same server. Also, note that XHR (AJAX) requests will open separate connections so they can be performed in parallel. In the real world, this won't be a problem.
I've created a node.js script, that scans network for available HTTP pages, so there is a lot of connections i want to run in parallel, but it seems that some of the requests wait for previous to complete.
Following is the code fragment:
var reply = { };
reply.started = new Date().getTime();
var req = http.request(options, function(res) {
reply.status = res.statusCode;
reply.rawHeaders = res.headers;
reply.headers = JSON.stringify(res.headers);
reply.body = '';
res.setEncoding('utf8');
res.on('data', function (chunk) {
reply.body += chunk;
});
res.on('end', function () {
reply.finished = new Date().getTime();
reply.time = reply.finished - reply.started;
callback(reply);
});
});
req.on('error', function(e) {
if(e.message == 'socket hang up') {
return;
}
errCallback(e.message);
});
req.end();
This code performs only 10-20 requests per second, but i need 500-1k requests performance. Every queued request is made to a different HTTP server.
I've tried to do something like that, but it didn't help:
http.globalAgent.maxSockets = 500;
Something else must be going on with your code. Node can comfortably handle 1k+ requests per second.
I tested with the following simple code:
var http = require('http');
var results = [];
var j=0;
// Make 1000 parallel requests:
for (i=0;i<1000;i++) {
http.request({
host:'127.0.0.1',
path:'/'
},function(res){
results.push(res.statusCode);
j++;
if (j==i) { // last request
console.log(JSON.stringify(results));
}
}).end();
}
To purely test what node is capable of and not my home broadband connection the code requests from a local Nginx server. I also avoid console.log until all the requests have returned because it is implemented as a synchronous function (to avoid losing debugging messages when a program crash).
Running the code using time I get the following results:
real 0m1.093s
user 0m0.595s
sys 0m0.154s
That's 1.093 seconds for 1000 requests which makes it very close to 1k requests per second.
The simple code above will generate OS errors if you try to make a lot of requests (like 10000 or more) because node will happily try to open all those sockets in the for loop (remember: the requests don't start until the for loop ends, they are only created). You mentioned that your solution also runs into the same errors. To avoid this you should limit the number of parallel requests you make.
The simplest way of limiting number of parallel requests is to use one of the Limit functions form the async.js library:
var http = require('http');
var async = require('async');
var requests = [];
// Build a large list of requests:
for (i=0;i<10000;i++) {
requests.push(function(callback){
http.request({
host:'127.0.0.1',
path:'/'
},function(res){
callback(null,res.statusCode);
}).end()
});
}
// Make the requests, 100 at a time
async.parallelLimit(requests, 100,function(err, results){
console.log(JSON.stringify(results));
});
Running this with time on my machine I get:
real 0m8.882s
user 0m4.036s
sys 0m1.569s
So that's 10k request in around 9 seconds or roughly 1.1k/s.
Look at the functions available from async.js.
I've found solution for me, it is not very good, but works:
childProcess = require('child_process')
I'm using curl:
childProcess.exec('curl --max-time 20 --connect-timeout 10 -iSs "' + options.url + '"', function (error, stdout, stderr) { }
This allows me to run 800-1000 curl processes simultaneously. Of course, this solution has it's weekneses, like requirement for lots of open file decriptors, but works.
I've tried node-curl bindings, but that was very slow too.