Let's say my browser post a HTTP request to a domain, before this request finish, another different request (by ajax) was send to the same domain. Since the first request still on-going and not yet terminated, will that mean second request will have to wait first request to finish in order to use the persistent connection that being used by first request? If it is, how to prevent this? If I have a long streaming connection in the first request, does that mean second request will need to hang around for long time?
(Let's assume the maximum persistent connection for browser is one. Actually I don't really understand what this "max persistent connection" does. Does it mean when the persistent connection is over the maximum number, the rest of connection will become non-persistent ? Confusing...)
Can anyone explain this?
Since the first request still on-going and not yet terminated, will that mean second request will have to wait first request to finish in order to use the persistent connection that being used by first request?
No. The two requests are still asynchronous and in parallel (unless the server limits this).
HTTP Keep Alive only means that they are faster because both requests can use the same connection, especially when pipelining them.
However, if there is no pipelining, the browser could also decide to open a second connection for the second request, instead of waiting for the first request to finish and reusing its connection. See Under what circumstances will my browser attempt to re-use a TCP connection for multiple requests? for details.
I don't really understand what this "max persistent connection" does. Does it mean when the persistent connection is over the maximum number, the rest of connection will become non-persistent?
No. When the limit is reached, new requests will have to wait until a connection from the pool becomes usable again.
Related
As of about 2017, it has been possible to abort a fetch and immediately move-on in front-end javascript. I'm having trouble finding any information on what this does to the HTTP connection, though. Does it close it prematurely, or does the browser keep the request open, and just discard the response as it arrives?
I have a use-case where a user is making a (potentially) expensive database calls from a webapp frontend. Sometimes, they notice that a request is taking too long, and manually cancel it. I would love to be able to take that signal and cancel my expensive database query, since they're no longer interested in the results.
Is there any way that my REST server can tell that the fetch has been aborted? (My server is a Java Jersey/Grizzly.)
It's generally supposed to:
https://fetch.spec.whatwg.org/#http-network-fetch step 17.2.3/4
If aborted, then:
If fetchParams is aborted, then:
Set response’s aborted flag.
If stream is readable, then error stream with the result of
deserialize a serialized abort reason given fetchParams’s controller’s
serialized abort reason and an implementation-defined realm.
Otherwise, if stream is readable, error stream with a TypeError.
If connection uses HTTP/2, then transmit an RST_STREAM frame.
Otherwise, the user agent should close connection unless it would be
bad for performance to do so.
For instance, the user agent could keep the connection open if it
knows there’s only a few bytes of transfer remaining on a reusable
connection. In this case it could be worse to close the connection and
go through the handshake process again for the next fetch.
However, Firefox currently does not (https://bugzilla.mozilla.org/show_bug.cgi?id=1568422).
I learned that under HTTP1.1, the max number of default simultaneous persistent connections per host name (origin?) is going to be 6, at least for chrome. I am not asking about the exact number of the limit since I know it varies from browser to browser. I am more curious about when we will open a new connection for new requests - does the browser reuse the same TCP connection somehow or it always starts a new TCP connection unless if it hasn't reached the limit of concurrent requests?
Let's say we are using HTTP1.1 and we have Connection: Keep-Alive
if in the html we have
<script src="https://foo/foo1.js"></script>
<script src="https://foo/foo2.js"></script>
<script src="https://foo/foo3.js"></script>
<script src="https://foo/foo4.js"></script>
<script src="https://foo/foo5.js"></script>
<script src="https://foo/foo6.js"></script>
<script src="https://foo/foo7.js"></script>
will each one of the scripts result in a new TCP connection established or all the subsequent requests will reuse the first TCP connection established by the first script tab? And if each one of these script result in a new TCP connection established, given the browser's limit for concurrent requests being 6, does the 7th request have to wait until the 6th request to be finished in order to establish the connection?
The above example is about initiating requests from HTML tags. What about api calls made from JavaScript? Let's in our javascript we have
const result1 = apiCall1()
const result2 = apiCall2()
const result3 = apiCall3()
const result4 = apiCall4()
const result5 = apiCall5()
const result6 = apiCall6()
const result7 = apiCall7()
And assume the endpoint that those API calls are hitting is all api.foo.com/v1/tasks, my questions are, again: will each one of the api call result in a new TCP connection established or all the subsequent requests will reuse the first TCP connection established by the first api call? And if each one of these api call result in a new TCP connection established, given the browser's limit for concurrent requests being 6, does the 7th request have to wait until the 6th request to be finished in order to establish the connection?
My last question is, compared to http1.1, does http2 address this problem by allowing sending many requests at the same time over one single TCP connection?
will each one of the scripts result in a new TCP connection established or all the subsequent requests will reuse the first TCP connection established by the first script tab?
Yes it would download them one by one, and start to open up more TCP connections to do that, up to the maximum of 6. The 7th request would have to wait for one of the connections to free up before it could be downloaded.
But the reality is, that the first request may have finished by the time later TCP connections are opened so it might not quite reach the 6 limit for only 6 or 7 requests.
What about api calls made from JavaScript? Let's in our javascript
Exact same thing. Limit of 6 per origin. Though one thing to note is certain CORS requests sent without credentials effectively counts as another origin (even though it’s the same actual origin) and so get another 6 connections.
My last question is, compared to http1.1, does http2 address this problem by allowing sending many requests at the same time over one single TCP connection?
Basically yes. Not quite at the same time due to the way TCP works, but as near as possible. See my answer here: What does multiplexing mean in HTTP/2
The process is simple, if you assign keep-alive the connection is remembered for faster handshake so a user can make many requests without having to re-open a costly secure connection.
Now there will always be the syn/ack process to make requests with the server. For the server to respond to every item your user requested a new connection is needed. There's bypassing this a little with cache to help your bandwidth and lessen the requests to server. All connections are ended upon request served.
So in a scenario 100 browsers want to hit your site, each request looks like 1.js 2.js... The output should be in order but this can greatly depend on a lot of things. Your language you're coding in server-sided, how it's handled, serves and if you manage any queues. If you make a request that requires longer processing (will get back to you in the future) other requests could go ahead as long as you're not blocking the event loop (comes down to your server).
Below you can see the process to establish a connection to the server, this is engaged each and every request. The cost to TLS can be improved but initial request is expensive.
I know that different browsers have different amounts of concurrent connections they can handle to the same hostname, but what exactly happens to a new request when that limit is hit?
Does it automatically wait and retry again later or is there something I need to do to help this process along?
Specifically, if this is a XMLHttpRequest executed via JavaScript and not just some assets being loaded by the browser from markup, could that automatically try again?
I have a client side library that makes multiple API requests and occasionally it tries to send too many too quickly. When this happens, I can see server side API errors, but this doesn't make sense. If the concurrency limit stops requests, then they would have never hit the server, would they?
Update: Thanks to #joshstrike and some more testing, I've discovered that my actual problem was not related to concurrent HTTP request limits in the browser. I am not sure these even apply to JavaScript API calls. I have a race condition in the specific API calls I'm making, which gave an error that I initially misunderstood.
The browser will not retry any request on its own if that request times out on the server (for whatever reason - including if you exceed the API's limits). It's necessary to check the status of each request and handle retrying them in some way that's graceful to the application and the user. For failed requests you can check the status code. However for requests which simply hang for a long time it may be necessary to attach a counter to your request, and "cancel" it after a delay... Then if a result comes back bearing the number of one that has already been canceled, ignore that result if a newer one has already returned. This is what typically happens in a long-polling application that is hitting a server constantly and not knowing whether some pings will return later or never return at all.
When the limit on the Chrome is reached it pauses anymore requests. Once one request has been responded to, the browser sends the next request. On Chrome that limit is six for me.
I am creating a question answering application using Node.js + Express for my back-end. Front-end sends the question data to the back-end, which in turn makes requests to multiple third-party APIs to get the answer data.
Problem is, some of those third-party APIs take too long to respond, since they have to do some intense processing and calculations. For that reason, i have already implemented a caching system that saves answer data for each different question. Nevertheless, that first request each time might take up to 5 minutes.
Since my back-end server waits and does not respond back to the front-end until data arrives (the connections are being kept open), it can only serve 6 requests concurrently (that's what I have found). This is unacceptable in terms of performance.
What would be a workaround to this problem? Is there a way to not "clog" the server, so it can serve more than 6 users?
Is there a design pattern, in which the servers gives an initial response, and then serves the full data?
Perhaps, something that sets the request to "sleep" and opens up space for new connections?
Your server can serve many thousands of simultaneous requests if things are coded properly and it's not CPU intensive, just waiting for network responses. This is something that node.js is particularly good at.
A single browser, however, will only send a few requests at a time (it varies by browser) to the same endpoint (queuing the others until the earlier ones finish). So, my guess is that you're trying to test this from a single browser. That's not going to test what you really want to test because the browser itself is limiting the number of simultaneous requests. node.js is particularly good at having lots of request in flight at the same time. It can easily do thousands.
But, if you really have an operation that takes up to 5 minutes, that probably won't even work for an http request from a browser because the browser will probably time out an inactive connection still waiting for a result.
I can think of a couple possible solutions:
First, you could make the first http request be to just start the process and have it return immediately with an ID. Then, the client can check every 30 seconds of so after that sending the ID in an http request and your server can respond whether it has the result yet or not for that ID. This would be a client-polling solution.
Second, you could establish a webSocket or socket.io connection from client to server. Then, send a message over that socket to start the request. Then, whenever the server finishes its work, it can just send the result directly to the client over the webSocket or socket.io connection. After receiving the response, the client can either keep the webSocket/socket.io connection open for use again in the future or it can close it.
I'm developing a single-page application, which sends multiple AJAX request to the server.
The system works with polling, because some data-request can take about 10-20minutes to calculate.
client asks server for data
server hands out a job-id
client asks server every few seconds for the result
The polling algorithm lowers the polling frequency over time, stopping at intervals of 10seconds.
But when a client sends different data requests in a short time, he ends up with about 10-20 job-ids and starts polling for all of them.
Is it better to simply do it this way and let the browser handle those requests in parallel or should I schedule every request and serialize them all?
Would it bring performance benefits to serialize them?
If each initial request returns a unique id and each page has a unique user id then you can poll on what information for each request.
In the JSON I would return the results for any completed request, and the current status of those that haven't completed, such as whether it has started being processed, and perhaps a percentage of completion, or how many requests are ahead of that request.
This will simplify the work as you won't be making several polling calls, but just one, getting back a complex result to give feedback to the user the status of each request.
I find it useful to give some information on status for long-running queries otherwise the user may think the request was lost.
Some months ago, I faced performance issues due to multiple ajax calls, but I haven't investigated deeper this topic since then : High latencies loading stores in an ExtJS 4.1 MVC application.