I'm developing a Node.js back-end which communicates with some desktop clients via websockets, and the communication from the server side is initiated from a web front-end. Everything works ok since I am storing the SockJS Connection instances in an array. But if I would like to scale out the service, I guess no such thing would work, I need to share the connections or something like that.
Is there any way to do this, or change my architecture in any way to support the scaling one day?
You could scale horizontally by using a load balancer in front of multiple SockJS servers. If you need to share data across multiple SockJS servers, you could use one or more Redis instances (this is what the socket-redis module does).
Related
I am trying to make simple game of Tic-Tac-Toe in JS.
I made almost everything. Now there is just one thing to do.
I would like to make it available to play online with someone.
I want to send data between two games via Internet.
Unfortunately my server does not support Node.JS.
Is there a way to make it happen without any server-side "socket".
I thought I could make it with XMLHttpRequest() for saving/loading data into/from server files and play like that, but I think it would require a lot of code and maybe for ttt it would be fast enough, but for more complicated games it would not be sufficient.
I know this is tough, but how did they do that before node.js?
For a game like Tic-Tac-Toe where players take turns, latency definitely takes second seat to every other factor in communications. For this reason alone, communicating with just the HTTP protocol, typically by utilizing the XMLHttpRequest class or the Fetch API, is a very reasonable approach which will save you a lot of programming effort.
Otherwise, when wanting one or several low-latency and/or RTC channels (for hopefully a good reason), both WebRTC and WebSocket are viable candidates.
WebRTC, for one, can absolutely do peer-to-peer, while WebSocket uses the client-server model. But even WebRTC requires a "signalling" service to exchange peer identifiers initially, before eventually switching to communicating between the peers directly. While peer identifiers are required to set up WebRTC communication, the API deliberately does not cover how peer identifiers are exchanged -- however you want to design your signalling service, is up to you. For all WebRTC cares, you can "POST" a peer ID to a HTTP server and retrieve it with the other peer's Web browser and vice-versa. WebRTC starts with already known peer IDs.
Otherwise, if configured to do so, WebRTC is able to utilize STUN and/or TURN services to maintain peer-to-peer connection, on networks that otherwise prohibit straightforward IP routing between any two clients -- a necessary prerequisite of true peer-to-peer communication.
STUN/TURN services aren't required in all cases, but knowing average network conditions, without using either STUN or TURN or both, your application wouldn't be very reliable for any two clients separated by multiple arbitrary networks. Like in scenarios where both parties are separated by at least one firewall or a stubborn router that functions as one.
A TURN service would then transparently route WebRTC communication, working as a relay.
A STUN service punches holes in the firewalls between clients in such a way that peer-to-peer communication is possible afterwards. Meaning that in contrast with a TURN service, it does not play any active part in communication after latter is established.
WebRTC is a bit complex, especially if you are expecting an API along the lines of send and receive, but a simplified connection example should be understandable to a developer.
You may also not need to use WebRTC API directly, there are libraries that encapsulate WebRTC into a simpler API of one flavour or another, API that simultaneously hides the more fringe or "boilerplate" aspects of WebRTC and which also helps minimize the risk of getting into trouble as different user agents are notorious for implementing different parts of WebRTC a bit differently.
One of these libraries is PeerJS but there are others, without a doubt.
The WebSocket API, unlike WebRTC, requires a WebSocket compliant server, and WebSocket API does not do peer-to-peer. The good news is that 1) a WebSocket compliant service is typically just an advanced relay (often fused with an application back-end logic), albeit working on the application level instead of the session level for TURN and 2) there are plenty of "turn-key" WebSocket server implementations out there.
WebRTC signalling is driving me crazy. My use-case is quite simple: a bidirectional audio intercom between a kiosk and to a control room webapp. Both computers are on the same network. Neither has internet access, all machines have known static IPs.
Everything I read wants me to use STUN/TURN/ICE servers. The acronyms for this is endless, contributing to my migraine but if this were a standard application, I'd just open a port, tell the other client about it (I can do this via the webapp if I need to) and have the other connect.
Can I do this with WebRTC? Without running a dozen signalling servers?
For the sake of examples, how would you connect a browser running on 192.168.0.101 to one running on 192.168.0.102?
STUN/TURN is different from signaling.
STUN/TURN in WebRTC are used to gather ICE candidates. Signaling is used to transmit between these two PCs the session description (offer and answer).
You can use free STUN server (like stun.l.google.com or stun.services.mozilla.org). There are also free TURN servers, but not too many (these are resource expensive). One is numb.vigenie.ca.
Now there's no signaling server, because these are custom and can be done in many ways. Here's an article that I wrote. I ended up using Stomp now on client side and Spring on server side.
I guess you can tamper with SDP and inject the ICE candidates statically, but you'll still need to exchange SDP (and that's dinamycally generated each session) between these two PCs somehow. Even though, taking into account that the configuration will not change, I guess you can exchange it once (through the means of copy-paste :) ), stored it somewhere and use it every time.
If your end-points have static IPs then you can ignore STUN, TURN and ICE, which are just power-tools to drill holes in firewalls. Most people aren't that lucky.
Due to how WebRTC is structured, end-points do need a way to exchange call setup information (SDP) like media ports and key information ahead of time. How you get that information from A to B and back to A, is entirely up to you ("signaling server" is just a fancy word for this), but most people use something like a web socket server, the tic-tac-toe of client-initiated communication.
I think the simplest way to make this work on a private network without an internet connection is to install a basic web socket server on one of the machines.
As an example I recommend the very simple https://github.com/emannion/webrtc-web-socket which worked on my private network without an internet connection.
Follow the instructions to install the web socket server on e.g. 192.168.1.101, then have both end-points connect to 192.168.0.101:1337 with Chrome or Firefox. Share camera on both ends in the basic demo web UI, and hit Connect and you should be good to go.
If you need to do this entirely without any server, then this answer to a related question at least highlights the information you'd need to send across (in a cut'n'paste demo).
We have kerberos authentication at our nginx layer and want to connect to deepstream.io instances as a reverse proxy. From my reading of the docs, it looks like putting a webserver in front of deepstream.io instances will hamper performance. Also, there is the question of who does load balancing - usually it is at nginx layer but deepstream.io does seem to have inbuilt capabilities to ask other instances to handle load (via messaging)
What would be the best way to get deepstream.io instances play well with a web server ? It is non-trivial to re-implement kerberos authentication in Node.
It is possible (and a good idea) to deploy and loadbalance deepstream behind Nginx, HAProxy etc. There are a few things to be aware of though:
engine.io, websockets and sticky sessions.
Deepstream uses engine.io (the transport-layer behind socket.io) to connect to browsers.
engine.io uses a number of different transport mechanisms, most notably long-polling (keeping an http request open until data needs to be send) and WebSockets.
For long-polling it is crucial that all requests from the same client are routed to the same deepstream server, so make sure that sticky/persistent sessions are enabled in your nginx upstream group (just add the ip_hash directive). For websocket it is important that the http update headers are forwarded.
sync data / messages between your deepstream nodes
Make sure that your deepstream nodes are connected with each other, both by a cache and message-bus. You have a choice of caches and messaging systems, but the simplest is to just use Redis for both. Have a look at the section on "What’s the simplest production-ready setup?" at the bottom of https://deepstream.io/tutorials/connectors-and-deployment.html.
TCP Connections from other clients (e.g. NodeJS)
engine.io is mainly used for browser clients, backend processes may directly connect via TCP. Loadbalancing TCP connections is perfectly possible with NginX, but requires an extra stream group in your configuration.
I'm creating an app where the server and the clients will run on the same local network. Is it possible to use web sockets, or rather more specifically, socket.io to have one central server and many clients that are running native apps
? The way I understand socket.io to work is that the clients read the web-pages that are served from the server but what happens when your clients become tablet devices running native apps instead of web pages in a browser?
The scenario I'm working with at the minute will have one central server containing a MEAN app and the clients (iPads) will make GET requests to the data available on the server. However, I'd also like there to be real-time functionality so if someone triggers a POST request on their iPad, the server acknowledges it and displays it in the server's client-side. The iPad apps will (ideally) be running native phonegap applications rather than accessing 192.168.1.1:9000 from their browser.
Is this technically possible to connect to the socket server from the native apps or would the devices have to send POST requests to a central server that's constantly listening for new 'messages'? I'm totally new to the whole real-time stuff so I'm just trying to wrap my head around it all.
Apologies if this isn't totally clear, it's a bit hard to describe with just text but I think you get the idea?
Correct me if I am wrong.
You have multiple iPads running native app. They send a POST request to your node JS server which is running in a computer in the same local network. Whenever the server receives a request from app, you want to display that a request has been received in your computer screen.
If my assumptions about the scenario is correct, then it is fairly easy to do. Here are the steps to do it.
Create a small webpage (front end). Load socket IO in the front end page like this -
<script type="text/javascript" src="YOUR_SERVER_IP/socket.io/socket.io.js"></script>
Then connect to server using var socket = io(). This should trigger connection event in your backend.
Handle all POST request from apps normally. Nothing special. Just add a small snippet in between. socket.emit('new_request', request_data). This sends new_request event to front end.
Handle the new_request in your front end using socket.on('new_request', function(request_data) { ... });. That's it. No need to add anything to your native app for realtime update.
The second step would be a little complicated as it is necessary to make socket variable available inside all POST requests. Since you chose node.js, I don't think you need any help with that.
Not totally clear on your project, but I'll try to give you some pointers.
An effective way to send data between native apps and a server is using a REST server. REST is based on HTTP requests and allows you to modify data on the server, which can connect to your database. The data returned is typically either JSON or XML formatted. See here for a brief intro: http://www.infoq.com/articles/rest-introduction
Android/iOS/etc have built in APIs for making HTTP requests. Your native app would send a request to the server, parse the response, and update your native UI accordingly. The same server can be used from a website using jQuery ajax HTTP requests.
Express.js is more suited to serving web pages and includes things like templating. Look into "restify" (see here: mcavage.me/node-restify/) if you just want to have a REST server that handles requests. Both run on top of node.js (nodejs.org).
As far as real-time communication, if you're developing for iOS look into APNS (Apple Push Notification Service). Apple maintains a persistent connection, and by going through their servers you can easily send messages to your app. The equivalent of this on Android is GCM (Google Cloud Messaging).
You can also do sockets directly if that's easier for you. Be careful with maintaining an open socket on a mobile device though, it can be a huge battery drain. Here's a library for connecting ObjC to Socket.IO using websockets, it may be useful for you: https://github.com/pkyeck/socket.IO-objc
Hope that helps!
To answer your question, it is definitely possible. Socket.io would serve as the central server that can essentially emit messages to all of the client. You can also make Socket.io listen for the messages from any of the clients and serve the emitted message to the rest of the clients.
Here's an example of how socket.io can be used. Simply clone, npm install, and run using 'node app.js'
All you have to do is to provide a valid server address when you connect your socket from the iPad clients:
var socket = io.connect( 'http://my.external.nodejs.server' );
Let us know if you need help with actual sending/receiving of socket events.
It is possible to connect to Websockets from your apps.
If you are using PhoneGap then you need a pluging that gives support to websockets in your app (the client) and then use websocket like normal way using Javascript see this.
If your app is native iOS look into this it could help you.
The primary use of the Sockets in your case is to be a bidirectional "pipe" between an app and server. There is no need of server sending the whole web-page to the native app. All what you need is to send some data from server to the client(app) in response to POST (or GET) request and then using this data on client side to update client's UI in real-time. If you are going to use moderate amount of devices (say tens of them), you may have connected all of them to the server permanently keeping individual socket connection open for every individual link server-to-app. Thus you may deliver data and update client's state in real time.
In fact web browsers also employ sockets to communicate to web servers. However as in general case there is no control on amount of concurrent clients in Internet, for the sake of limited networking resources conservation, servers do not keep sockets open for a long time, closing it just after the web-page was sent to client (or timeout has expired). That's how HTTP protocol works on the low level. The server waiting for the HTTP clients (browsers) by listening the 80 port, responding them by sending the whole web page content, then closing the connection and keep waiting for another requests on the same port.
In your case it's basically a good idea to use socket.io as it's a uniform implementation of sockets (ok WebSockets) on both client and server side. The good starting point is here
Is it possible to send directly message from JavaScript in client browser to 0mq?
If not in JavaScript, then I should use Flash or setup some http proxy server?
0mq is not meant for Internet facing ports. There is a project called nullmq which does what you want though by translating from web protocols to zmq behind the firewall, while retaining zeromq like api on the browser.
I suspect it would be easiest to have your client browser make an XMLHttpRequest() to your web application and then have your web application talk to your 0MQ infrastructure.
There is a javascript/flash binding for 0MQ, but I've never worked with it myself so I can't comment on stability or anything.
If you tell us more about what you're trying to accomplish we might be able to suggest viable alternatives.
You can use websockets on the client-side if you want a persistent connection and use a websocket server like tornado or socket.io to relay the messages to zmq and back.