3 mini questions regarding websocket connections
When the client sends data to the server there is latency. When the server sends data to the client is there latency or it is instant?
If the client sends data to the server VERY FAST in a specific row - let's say [1, 2, 3], is there any chance that, due to latency or other reasons, the data to be received by the server in a different row? ( like [2, 1, 3] )
(Same as question #2, but when the server sends the data)
Yes, there is latency. Its still a connection and there is still a chain to navigate. Latency only matters when things are changing and given that it takes X amount of time for the message to reach the client and another X ms for the client to do anything about it, its quite possible the state will change during those ms. In the same way that HTTP requests (WebSockets are just about the same thing) become 'hot', I believe the latency will diminish (all other things being equal) but it will still exist.
No, WebSockets are via TCP, so they'll be in order. UDP transport is fire-and-forget, it doesnt send any notification of receipt and it doesnt regenerate the packets using timing information, so you can send messages faster but can make no assumptions regarding receipt or order or events. Page impressions would be a great example of where you dont care really in what order and you probably dont care too much about when the server receives such a message, WebRTC may bring UDP connections between JS and server but the standard is still emerging. For now, WebSockets connect via an HTTP upgrade, meaning they are TCP, where order information and receipt confirmation is a thing (larger and more messages being sent to and fro).
Same answer! It all happens over TCP so the whole trip is a round-trip, but order is assured.
Related
I am building an API endpoint that has to call multiple external services/DBs and I do not want my users to have to wait for this process to take place, however, the result of this process is essential for my users.
My first thought is to add the request to a queue and return immediately, then at some later time, the user can query a different endpoint for the result.
Is there a better way to go about this? Should there be a webhook response instead of asking users to query the API twice?
Three main ways I've seen:
Client sends the API request and immediately gets back a job number. The client can then send a different API request with that job number every so often (every minute or so depending upon how long the usual result takes to get) so check on the progress. On one of those checks the job will be done and the get the data.
Client makes a webSocket or socket.io connection. Client sends a request over that websocket/socket.io connection. Server starts working on the result. When the result is done, it is immediately sent over the webSocket/socket.io connection back to the client. The client can then keep the websocket/socket.io connection connected for other requests or close the connection.
Use Server-Sent events. Send, the query and then when the result is done, the server can send it back on that same connection.
I don't think there's a "best practice" among these three as each have some advantages and some other uses which may be relevant. The polling option #1 is the lowest common denominator and will work in any situation, but requires a polling strategy by the client and may have some latency (result ready before client polls).
The choices #2 and #3 are both very efficient and their general technology may have other uses also.
I am creating a question answering application using Node.js + Express for my back-end. Front-end sends the question data to the back-end, which in turn makes requests to multiple third-party APIs to get the answer data.
Problem is, some of those third-party APIs take too long to respond, since they have to do some intense processing and calculations. For that reason, i have already implemented a caching system that saves answer data for each different question. Nevertheless, that first request each time might take up to 5 minutes.
Since my back-end server waits and does not respond back to the front-end until data arrives (the connections are being kept open), it can only serve 6 requests concurrently (that's what I have found). This is unacceptable in terms of performance.
What would be a workaround to this problem? Is there a way to not "clog" the server, so it can serve more than 6 users?
Is there a design pattern, in which the servers gives an initial response, and then serves the full data?
Perhaps, something that sets the request to "sleep" and opens up space for new connections?
Your server can serve many thousands of simultaneous requests if things are coded properly and it's not CPU intensive, just waiting for network responses. This is something that node.js is particularly good at.
A single browser, however, will only send a few requests at a time (it varies by browser) to the same endpoint (queuing the others until the earlier ones finish). So, my guess is that you're trying to test this from a single browser. That's not going to test what you really want to test because the browser itself is limiting the number of simultaneous requests. node.js is particularly good at having lots of request in flight at the same time. It can easily do thousands.
But, if you really have an operation that takes up to 5 minutes, that probably won't even work for an http request from a browser because the browser will probably time out an inactive connection still waiting for a result.
I can think of a couple possible solutions:
First, you could make the first http request be to just start the process and have it return immediately with an ID. Then, the client can check every 30 seconds of so after that sending the ID in an http request and your server can respond whether it has the result yet or not for that ID. This would be a client-polling solution.
Second, you could establish a webSocket or socket.io connection from client to server. Then, send a message over that socket to start the request. Then, whenever the server finishes its work, it can just send the result directly to the client over the webSocket or socket.io connection. After receiving the response, the client can either keep the webSocket/socket.io connection open for use again in the future or it can close it.
How Websockets are implemented?
What is the algorithm behind this new tech (in comparison to Long-Polling)?
How can they be better than Long-Polling in term of performance?
I am asking these questions because here we have a sample code of Jetty websocket implementation (server-side).
If we wait long enough, a timeout will occur, resulting in the
following message on the client.
And that is definately the problem I'm facing when using Long-polling. It stops the process to prevent server overload, doesn't it ?
How Websockets are implemented?
webSockets are implemented as follows:
Client makes HTTP request to server with "upgrade" header on the request
If server agrees to the upgrade, then client and server exchange some security credentials and the protocol on the existing TCP socket is switched from HTTP to webSocket.
There is now a lasting open TCP socket connecting client and server.
Either side can send data on this open socket at any time.
All data must be sent in a very specific webSocket packet format.
Because the socket is kept open as long as both sides agree, this gives the server a channel to "push" information to the client whenever there is something new to send. This is generally much more efficient than using client-driven Ajax calls where the client has to regularly poll for new information. And, if the client needs to send lots of messages to the server (perhaps something like a mnulti-player game), then using an already open socket to send a quick message to the server is also more efficient than an Ajax call.
Because of the way webSockets are initiated (starting with an HTTP request and then repurposing that socket), they are 100% compatible with existing web infrastructure and can even run on the same port as your existing web requests (e.g. port 80 or 443). This makes cross-origin security simpler and keeps anyone on either client or server side infrastructure from having to modify any infrastructure to support webSocket connections.
What is the algorithm behind this new tech (in comparison to
Long-Polling)?
There's a very good summary of how the webSocket connection algorithm and webSocket data format works here in this article: Writing WebSocket Servers.
How can they be better than Long-Polling in term of performance?
By its very nature, long-polling is a bit of a hack. It was invented because there was no better alternative for server-initiated data sent to the client. Here are the steps:
The client makes an http request for new data from the client.
If the server has some new data, it returns that data immediately and then the client makes another http request asking for more data. If the server doesn't have new data, then it just hangs onto the connection for awhile without providing a response, leaving the request pending (the socket is open, the client is waiting for a response).
If, at any time while the request is still pending, the server gets some data, then it forms that data into a response and returns a response for the pending request.
If no data comes in for awhile, then eventually the request will timeout. At that point, the client will realize that no new data was returned and it will start a new request.
Rinse, lather, repeat. Each piece of data returned or each timeout of a pending request is then followed by another ajax request from the client.
So, while a webSocket uses one long-lived socket over which either client or server can send data to the other, the long-polling consists of the client asking the server "do you have any more data for me?" over and over and over, each with a new http request.
Long polling works when done right, it's just not as efficient on the server infrastructure, bandwidth usage, mobile battery life, etc...
What I want is explanation about this: the fact Websockets keep an
open connection between C/S isn't quite the same to Long Polling wait
process? In other words, why Websockets don't overload the server?
Maintaining an open webSocket connection between client and server is a very inexpensive thing for the server to do (it's just a TCP socket). An inactive, but open TCP socket takes no server CPU and only a very small amount of memory to keep track of the socket. Properly configured servers can hold hundreds of thousands of open sockets at a time.
On the other hand a client doing long-polling, even one for which there is no new information to be sent to it, will have to regularly re-establish its connection. Each time it re-establishes a new connection, there's a TCP socket teardown and new connection and then an incoming HTTP request to handle.
Here are some useful references on the topic of scaling:
600k concurrent websocket connections on AWS using Node.js
Node.js w/1M concurrent connections!
HTML5 WebSocket: A Quantum Leap in Scalability for the Web
Do HTML WebSockets maintain an open connection for each client? Does this scale?
Very good explanation about web sockets, long polling and other approaches:
In what situations would AJAX long/short polling be preferred over HTML5 WebSockets?
Long poll - request → wait → response. Creates connection to server like AJAX does, but keep-alive connection open for some time (not long though), during connection open client can receive data from server. Client have to reconnect periodically after connection is closed due to timeouts or data eof. On server side it is still treated like HTTP request same as AJAX, except the answer on request will happen now or some time in the future defined by application logic. Supported in all major browsers.
WebSockets - client ↔ server. Create TCP connection to server, and keep it as long as needed. Server or client can easily close it. Client goes through HTTP compatible handshake process, if it succeeds, then server and client can exchange data both directions at any time. It is very efficient if application requires frequent data exchange in both ways. WebSockets do have data framing that includes masking for each message sent from client to server so data is simply encrypted. support chart (very good)
Overall, sockets have much better performance than long polling and you should use them instead of long polling.
How twitter page queries/receives notifications, information about new tweets?
I'd like to implement something like this mechanism for my html+js client->webservice
I don't know what Twitter uses exactly but there are few techniques to handle server notifications.
You can use long-polling (your client issues the same ajax request every few seconds to get new information):
http://techoctave.com/c7/posts/60-simple-long-polling-example-with-javascript-and-jquery/
Or there is the "new" standard called Websocket. A good start to how to write a websocket client is this mozilla tutorial.
There are multiple ways to implement real-time notifications:
HTTP Long Polling : The client initiates a request. The server checks if it has any new notifications. Irrespective of whether or not it has new notifications appropriate response is send and connection is closed. After time X client initiates another request (+ Very easy to implement - notifications are not real time. They depend on X since data retrieval is client initiated. As X decreases overhead on server increases )
HTTP Streaming: This is very similar to HTTP Long Polling however the connection is not closed. The server sends chunked response. So as soon as server receives new notification that it wants to push it can simply write to the socket. ( + lower latency than long polling and almost real time behaviour / overhead of closing connection and re opening reduced - memory usage client side keeps on piling up / ugly hacks etc )
WebSocket: TCP based protocol provides true two way communication. The server can push data to client any time. ( + ve: true real time - some older browsers dont support it ). Read more about it WebSocket.org | About WebSocket
I'm planning to write a real time co-op multiplayer game. Currently I'm in the research phase. I've already written a turn-based game which used websockets and it was working fine.
I haven't tried writing a real time game using this technology however. My questions is about websockets. Is there an alternative way to handle communications between (browser) clients? My idea is to have the game state in each client and only send the deltas to the clients using the server as a mediator/synchronization tool.
My main concern is network speed. I want clients to be able to receive each other's actions as fast as possible so my game can stay real time. I have about 20-30 frames per second with less than a KByte of data per frame (which means a maximum of 20-30 KBytes of data per second per client).
I know that things like "network speed" depend on the connection but I'm interested in the "if all else equals" case.
From a standard browser, a webSocket is going to be your best bet. The only two alternatives are webSocket and Ajax. Both are TCP under the covers so once the connection is established they pretty much offer the same transport. But, a webSocket is a persistent connection so you save connection overhead everytime you want to send something. Plus the persistent connection of the webSocket allows you to send directly from server to client which you will want.
In a typical gaming design, the underlying gaming engine needs to adapt to the speed of the transport between the server and any given client. If the client has a slower connection, then you have to drop the number of packets you send to something that can keep up (perhaps fewer frame updates in your case). The connection speed is what it is so you have to make your app deliver the best experience you can at the speed that there is.
Some other things you can do to optimize the use of the transport:
Collect all data you need to send at one time and send it in one larger send operation rather than lots of small sends. In a webSocket, don't send three separate messages each with their own data. Instead, create one message that contains the info from all three messages.
Whenever possible, don't rely on the latency of the connection by sending, waiting for a response, sending again, waiting for response, etc... Instead, try to parallelize operations so you send, send, send and then process responses as they come in.
Tweak settings for outgoing packets from your server so their is no Nagle delay waiting to see if there is other data to go in the same packet. See Nagle's Algorithm. I don't think you have the ability in a browser to tweak this from the client.
Make sure your data is encoded as efficiently as possible for smallest packet size.