How Websockets are implemented?
What is the algorithm behind this new tech (in comparison to Long-Polling)?
How can they be better than Long-Polling in term of performance?
I am asking these questions because here we have a sample code of Jetty websocket implementation (server-side).
If we wait long enough, a timeout will occur, resulting in the
following message on the client.
And that is definately the problem I'm facing when using Long-polling. It stops the process to prevent server overload, doesn't it ?
How Websockets are implemented?
webSockets are implemented as follows:
Client makes HTTP request to server with "upgrade" header on the request
If server agrees to the upgrade, then client and server exchange some security credentials and the protocol on the existing TCP socket is switched from HTTP to webSocket.
There is now a lasting open TCP socket connecting client and server.
Either side can send data on this open socket at any time.
All data must be sent in a very specific webSocket packet format.
Because the socket is kept open as long as both sides agree, this gives the server a channel to "push" information to the client whenever there is something new to send. This is generally much more efficient than using client-driven Ajax calls where the client has to regularly poll for new information. And, if the client needs to send lots of messages to the server (perhaps something like a mnulti-player game), then using an already open socket to send a quick message to the server is also more efficient than an Ajax call.
Because of the way webSockets are initiated (starting with an HTTP request and then repurposing that socket), they are 100% compatible with existing web infrastructure and can even run on the same port as your existing web requests (e.g. port 80 or 443). This makes cross-origin security simpler and keeps anyone on either client or server side infrastructure from having to modify any infrastructure to support webSocket connections.
What is the algorithm behind this new tech (in comparison to
Long-Polling)?
There's a very good summary of how the webSocket connection algorithm and webSocket data format works here in this article: Writing WebSocket Servers.
How can they be better than Long-Polling in term of performance?
By its very nature, long-polling is a bit of a hack. It was invented because there was no better alternative for server-initiated data sent to the client. Here are the steps:
The client makes an http request for new data from the client.
If the server has some new data, it returns that data immediately and then the client makes another http request asking for more data. If the server doesn't have new data, then it just hangs onto the connection for awhile without providing a response, leaving the request pending (the socket is open, the client is waiting for a response).
If, at any time while the request is still pending, the server gets some data, then it forms that data into a response and returns a response for the pending request.
If no data comes in for awhile, then eventually the request will timeout. At that point, the client will realize that no new data was returned and it will start a new request.
Rinse, lather, repeat. Each piece of data returned or each timeout of a pending request is then followed by another ajax request from the client.
So, while a webSocket uses one long-lived socket over which either client or server can send data to the other, the long-polling consists of the client asking the server "do you have any more data for me?" over and over and over, each with a new http request.
Long polling works when done right, it's just not as efficient on the server infrastructure, bandwidth usage, mobile battery life, etc...
What I want is explanation about this: the fact Websockets keep an
open connection between C/S isn't quite the same to Long Polling wait
process? In other words, why Websockets don't overload the server?
Maintaining an open webSocket connection between client and server is a very inexpensive thing for the server to do (it's just a TCP socket). An inactive, but open TCP socket takes no server CPU and only a very small amount of memory to keep track of the socket. Properly configured servers can hold hundreds of thousands of open sockets at a time.
On the other hand a client doing long-polling, even one for which there is no new information to be sent to it, will have to regularly re-establish its connection. Each time it re-establishes a new connection, there's a TCP socket teardown and new connection and then an incoming HTTP request to handle.
Here are some useful references on the topic of scaling:
600k concurrent websocket connections on AWS using Node.js
Node.js w/1M concurrent connections!
HTML5 WebSocket: A Quantum Leap in Scalability for the Web
Do HTML WebSockets maintain an open connection for each client? Does this scale?
Very good explanation about web sockets, long polling and other approaches:
In what situations would AJAX long/short polling be preferred over HTML5 WebSockets?
Long poll - request → wait → response. Creates connection to server like AJAX does, but keep-alive connection open for some time (not long though), during connection open client can receive data from server. Client have to reconnect periodically after connection is closed due to timeouts or data eof. On server side it is still treated like HTTP request same as AJAX, except the answer on request will happen now or some time in the future defined by application logic. Supported in all major browsers.
WebSockets - client ↔ server. Create TCP connection to server, and keep it as long as needed. Server or client can easily close it. Client goes through HTTP compatible handshake process, if it succeeds, then server and client can exchange data both directions at any time. It is very efficient if application requires frequent data exchange in both ways. WebSockets do have data framing that includes masking for each message sent from client to server so data is simply encrypted. support chart (very good)
Overall, sockets have much better performance than long polling and you should use them instead of long polling.
Related
I'm building a website, and one of the features is a public chat that anyone online can use. When a message is entered it is sent to the server and then saved to a SQL database. How could I relay this information to all the online clients without them making a request to the server? I've thought about having all clients make a request to the server every 500ms or so but I feel that would be incredibly inefficient. Any suggestions?
What you're looking for is typically called "server push" where the server can unilaterally send data to the client without the client having to "poll" or repeatedly ask for new info.
The two general technologies for server push these days are webSockets and server-sent events (SSE). In both cases, the client initiates a connection to the server and that connection is held open so that the server can send data to the client whenever it wants to without the client having to specifically poll for that data.
A webSocket is a full, two-way data channel. Either client or server can send data in either direction.
SSE is a one-way channel, the server can send data to a listening client.
You can see these articles on comparing the pros/cons of each.
WebSockets vs Server-Sent Events - ably.com
Server-sent events vs. WebSockets - logrocket.com
Difference between server sent events and Websockets in HTML5 - geeksforgeeks.org
And, there are dozens of other articles here.
You may also want to be aware of socket.io which is a widely used layer built on top of webSockets that adds more features than either of these have (a named message layer, auto-reconnect, message acknowledgement, direct message response, built-in JSON support, etc...).
Any of these can do what you're asking for. Which of these to choose really depends upon the details of your requirements.
Try making a WebSocket server. There is ws package for Node.js, and alternatively socket.io. However if your client is a web client, you can use socket.io for easy-use and setup.
I am creating a question answering application using Node.js + Express for my back-end. Front-end sends the question data to the back-end, which in turn makes requests to multiple third-party APIs to get the answer data.
Problem is, some of those third-party APIs take too long to respond, since they have to do some intense processing and calculations. For that reason, i have already implemented a caching system that saves answer data for each different question. Nevertheless, that first request each time might take up to 5 minutes.
Since my back-end server waits and does not respond back to the front-end until data arrives (the connections are being kept open), it can only serve 6 requests concurrently (that's what I have found). This is unacceptable in terms of performance.
What would be a workaround to this problem? Is there a way to not "clog" the server, so it can serve more than 6 users?
Is there a design pattern, in which the servers gives an initial response, and then serves the full data?
Perhaps, something that sets the request to "sleep" and opens up space for new connections?
Your server can serve many thousands of simultaneous requests if things are coded properly and it's not CPU intensive, just waiting for network responses. This is something that node.js is particularly good at.
A single browser, however, will only send a few requests at a time (it varies by browser) to the same endpoint (queuing the others until the earlier ones finish). So, my guess is that you're trying to test this from a single browser. That's not going to test what you really want to test because the browser itself is limiting the number of simultaneous requests. node.js is particularly good at having lots of request in flight at the same time. It can easily do thousands.
But, if you really have an operation that takes up to 5 minutes, that probably won't even work for an http request from a browser because the browser will probably time out an inactive connection still waiting for a result.
I can think of a couple possible solutions:
First, you could make the first http request be to just start the process and have it return immediately with an ID. Then, the client can check every 30 seconds of so after that sending the ID in an http request and your server can respond whether it has the result yet or not for that ID. This would be a client-polling solution.
Second, you could establish a webSocket or socket.io connection from client to server. Then, send a message over that socket to start the request. Then, whenever the server finishes its work, it can just send the result directly to the client over the webSocket or socket.io connection. After receiving the response, the client can either keep the webSocket/socket.io connection open for use again in the future or it can close it.
How twitter page queries/receives notifications, information about new tweets?
I'd like to implement something like this mechanism for my html+js client->webservice
I don't know what Twitter uses exactly but there are few techniques to handle server notifications.
You can use long-polling (your client issues the same ajax request every few seconds to get new information):
http://techoctave.com/c7/posts/60-simple-long-polling-example-with-javascript-and-jquery/
Or there is the "new" standard called Websocket. A good start to how to write a websocket client is this mozilla tutorial.
There are multiple ways to implement real-time notifications:
HTTP Long Polling : The client initiates a request. The server checks if it has any new notifications. Irrespective of whether or not it has new notifications appropriate response is send and connection is closed. After time X client initiates another request (+ Very easy to implement - notifications are not real time. They depend on X since data retrieval is client initiated. As X decreases overhead on server increases )
HTTP Streaming: This is very similar to HTTP Long Polling however the connection is not closed. The server sends chunked response. So as soon as server receives new notification that it wants to push it can simply write to the socket. ( + lower latency than long polling and almost real time behaviour / overhead of closing connection and re opening reduced - memory usage client side keeps on piling up / ugly hacks etc )
WebSocket: TCP based protocol provides true two way communication. The server can push data to client any time. ( + ve: true real time - some older browsers dont support it ). Read more about it WebSocket.org | About WebSocket
I have a JavaScript/html web socket client. It receives images from my server quite frequently. I am hosting my web socket server in a C# console application.
The longest my client has to wait for an image is 10 seconds.
Should I also be sending 'keep-alive' packets from my client (JavaScript) to my Server or is the fact that it receives regular data packets from my server enough?
Thanks
You can send ping messages from the server, and the client will answer with a pong message. It is part of the WebSocket protocol, not sure if your server implementation supports it. However, from the client you have no way to know if you got a ping message in while, so the client may be let hanging there.
Or you can do it at application level using your own messages. Basically create a "ping" message in JSON and send it regularly from the client to the server, then if you do not get a response in X time, reconnect.
If you don´t, the only problem is that the connection maybe half-open, and your client may be hanging there waiting for a image that will never come.
Cheers.
This is more of a n00b question, but I've never really known the answer.
so why do we need the websockets protocol?
and, what are the advantages over comet-style/long poll/hanging GET-style use of HTTP?
Comet and Ajax can both deliver end-user experiences that provide desktop-like functionality and low user-perceived latency, only Web Sockets lives up to the promise of providing a native means to accurately and efficiently stream events to and from the browser with negligible latency.
With polling it makes unnecessary requests and, as a result, many connections are opened and closed needlessly in low-message-rate situations.(as with polling it sends HTTP requests at regular intervals and immediately receives a response.)
Web Sockets remove the overhead and dramatically reduce complexity.
1-WebSocket is a naturally full-duplex,
bidirectional, single-socket connection. With WebSocket, your HTTP request becomes a
single request to open a WebSocket connection and reuses the same connection
from the client to the server, and the server to the client.
2-WebSocket reduces latency. For example, unlike polling,
WebSocket makes a single request. The server does not need to wait for a request from
the client. Similarly, the client can send messages to the server at any time. This single
request greatly reduces latency over polling, which sends a request at intervals, regardless
of whether messages are available.
3-WebSocket makes real-time communication much more efficient.
You can always use polling (and sometimes even streaming) over HTTP to receive
notifications over HTTP. However, WebSocket saves bandwidth, CPU power, and latency.
WebSocket is an innovation in performance.
4-WebSocket is an underlying network protocol that enables you to build other standard
protocols on top of it.
5-WebSocket is part of an effort to provide advanced capabilities to HTML5 applications in
order to compete with other platforms.
6-WebSocket is about Simplicity
Here's an article about benefits of websocket over polling at websocket.org
It isn't clear that we do need them. In the scenario of pushing events to the client, a page can make ordinary AJAX GET requests in a loop, and the server can "hang" until events are available. After some timeout passes the server can return a "no events" response so the client will reconnect. During the period where the connection is open and the client is waiting for a response, there is an effective push channel from the server back to the client.
The timeout period can be adjusted to reduce the amount of unnecessary reconnecting, though it cannot typically be infinite because most server frameworks will terminate the server-side process if it appears to have hung for too long.
Given this existing capability, the question is: does a new communication framework really add significant value over what can already be done? It wouldn't really enable something that cannot be done. It would only slightly improve it.