We have kerberos authentication at our nginx layer and want to connect to deepstream.io instances as a reverse proxy. From my reading of the docs, it looks like putting a webserver in front of deepstream.io instances will hamper performance. Also, there is the question of who does load balancing - usually it is at nginx layer but deepstream.io does seem to have inbuilt capabilities to ask other instances to handle load (via messaging)
What would be the best way to get deepstream.io instances play well with a web server ? It is non-trivial to re-implement kerberos authentication in Node.
It is possible (and a good idea) to deploy and loadbalance deepstream behind Nginx, HAProxy etc. There are a few things to be aware of though:
engine.io, websockets and sticky sessions.
Deepstream uses engine.io (the transport-layer behind socket.io) to connect to browsers.
engine.io uses a number of different transport mechanisms, most notably long-polling (keeping an http request open until data needs to be send) and WebSockets.
For long-polling it is crucial that all requests from the same client are routed to the same deepstream server, so make sure that sticky/persistent sessions are enabled in your nginx upstream group (just add the ip_hash directive). For websocket it is important that the http update headers are forwarded.
sync data / messages between your deepstream nodes
Make sure that your deepstream nodes are connected with each other, both by a cache and message-bus. You have a choice of caches and messaging systems, but the simplest is to just use Redis for both. Have a look at the section on "What’s the simplest production-ready setup?" at the bottom of https://deepstream.io/tutorials/connectors-and-deployment.html.
TCP Connections from other clients (e.g. NodeJS)
engine.io is mainly used for browser clients, backend processes may directly connect via TCP. Loadbalancing TCP connections is perfectly possible with NginX, but requires an extra stream group in your configuration.
Related
I'm trying to make a web socket connection on my webserver. The connection is run from a client in javascript and connects to a php script on the webserver. The javascript is also placed on the webserver, but runned from a clients webbrowser.
The problem is that hostgator doesn't allow inbound socket connections unless you buy a dedicated server, but I'm not aware if this counts as an inbound socket connection.
So does anyone know if this counts as an inbound socket connection or any other web hosting sites that would allow inbound connections?
Thank you in advance :)
A websocket connection starts life as an incoming HTTP connection (usually on the same port as is being used for web requests) with some custom headers on it which is something all web servers have to be configured to accept (or they wouldn't be any use as a web server). After a brief exchange with the client, the client requests an upgrade and a switch to the websocket protocol (the initial connection was the HTTP protocol). That connection which started life as an HTTP connection then becomes a webSocket connection (if the web server agrees to the protocol switch).
So, yes it is an incoming connection to the web server, but it's an incoming HTTP connection which your web server has to already accept. webSockets were designed this way on purpose to make them highly compatible with existing HTTP networking infrastructure, firewalls, etc... so they could be used by only upgrading the HTTP server software (to support the webSocket protocol) and not changing any of the networking infrastructure.
FYI, there are other hosting issues with using webSockets. A webSocket is a continuous, long lasting socket connection. In order to use it, you typically need a continuous, long lasting server process. Many of the lower cost, shared hosting environments do not support that. They tend to accept an incoming HTTP request, dispatch it to whatever script it is supposed to run (e.g. a PHP script), let it run on that request and then the script exits and your server process does not continue to run. This works well for low cost, shared hosting because no server resources are consumed by your app when it is not actively in the middle of serving a page. But, that model won't work for webSockets where you must have a continuous server process for the webSocket to be connected to.
I don't know specifically about hostgator, but this is another issue to look into. On my shared hosting on Dreamhost, I cannot have a long running server process. On Dreamhost this requires a VPS hosting plan and from what I've read this is common for other shared hosting environments too.
I'm developing a Node.js back-end which communicates with some desktop clients via websockets, and the communication from the server side is initiated from a web front-end. Everything works ok since I am storing the SockJS Connection instances in an array. But if I would like to scale out the service, I guess no such thing would work, I need to share the connections or something like that.
Is there any way to do this, or change my architecture in any way to support the scaling one day?
You could scale horizontally by using a load balancer in front of multiple SockJS servers. If you need to share data across multiple SockJS servers, you could use one or more Redis instances (this is what the socket-redis module does).
I'm exploring using the node-http-proxy proxy server so that I can have our proxy server on port 80 forward requests to our app server on port 8000. However, I'm a little confused as to why this is a good idea, and what exactly this set up would protect against security-wise.
The note-http-proxy documentation discusses a lot about using it as a way to forward requests to an app with multiple ports or ip addresses. This obviously would be very useful, particularly with a basic round-robin load balancer strategy. However, we only have one app on one port, so there is no need for us to do this.
If there is an important security reason why we should be using this proxy-server, then I'd love to know what types of attacks it protects against. Also, we're using socket.io, so if there is something that the proxy does to help the websocket server scale up, I'd like to understand that as well. We're having trouble figuring out how to run our app without sudo (since all ports below 1024 require root access), so if there really is no good reason to use a proxy server at this point, we're just going to scrap at. If anyone knows how to run this app with the proxy server on port 80 without root access, that'd be very helpful as well. Thanks!
The reasons for running a reverse proxy are:
You have limited IP ports open and need to run many Node services each of which needs it's own port
Your back-end service does not support HTTPS but you need it (e.g. Derby)
To add some other feature to the request that cannot be easily done with the back end such as adding Basic Authentication or some form of common logging/auditing
To enforce an addition or change to outgoing responses common across several back end services
To provide a load-balancing service
Unless your needs are quite simple, it would be better to use a dedicated proxy such as HAproxy since node-http-proxy is rather simplistic.
Well, if you're only running one instance of server, then theres not really a reason. The node-http-proxy docs mention using a single SSL certificate across multiple apps, which is very possible. You can also load balance across several HTTP and web socket servers (say, run 10 socket.io servers for real time data but only 1 HTTP server to serve out assets and REST APIs). Of course with one instance these don't provide any benefits.
If you want to run node servers without sudo, maybe you could try setting up IP tables port forwarding from port 80 to a port above 1024. See Can I run Node.JS with low privileges?
We use mainly the http-proxy to have multiple back-end server behind a single IP, but we also use it to forward https to http. It strengthens our app.
Security wise, you may have more confidence on the good quality of http-proxy than on your app. The proxy build by nodejitsu is ready for production and it should be harder for attaquants to gain privileges (like reading the private key files) on a http-proxy instead of your own app (of course this depends on your security development skill and your trust in the open source http-proxy project).
I begin with this technology. I want to establish a TCP/IP connection with an electronic card that has an IP address (the server's map).
I wonder if the WebSocket allow me to make this connection, knowing that at present my interface communicates with the card through a socket implanted in an applet.
Does anyone know the syntax to connect with WebSocket as a parameter an IP address: 135.120.138.105
Thank you
WebSockets are not raw TCP sockets. They have many of the same characteristics (low overhead, persistent, bidirectional, full-duplex) as raw TCP sockets, but they have an initial HTTP-like handshake to implement CORS security and allow easier integration with web servers and existing firewall policies. WebSockets are also message based and have a small header on each frame (2 bytes overhead for small payloads).
You have a couple of options. You can use a program that bridges/proxies between WebSockets and raw TCP sockets such websockify (Disclaimer: I made websockify) or you can implement the server side of the WebSocket protocol in your server.
WebSockets won't work in your case. While they use TCP/IP, WebSockets have a different API that's designed for higher-level messages to be passed between server and client and it requires support on the server as well, so unless the card contains a WebSocket-enabled web server I think you're out of luck and you'll have to continue with your applet-based design.
I have been thinking about building a client to client program. But the way I want is to use the broswer to do it, helped by a server that can make that connection.
The troubles comes when I need to have an unnconected socket (or pasive) in a client, waiting for a connection.
I have been thinking about Html5 WebSockets, but it doesn't give to the client the posibility of having a pasive socket without connecting it with a TCP protocol.
I'm learning this and trying to find the way to do this. All ideas are wellcome :D.
You can have a passive socket in Java applets, Flash and other browser plugins, but in general that can be problematic for public web applications.
First of all it will be difficult to get through firewalls, etc, and you'll need to depend and write code for a browser plugin that implements a socket API, and bridge it to JavaScript. If you are interested in some solutions, you may want to check out the following Stack Overflow post:
How can I communicate over TCP sockets from JavaScript?
The traditional approach for peer-to-peer communications between browsers is to have your server acting as a gateway for all the connections. Browsers initiate the connection (either with WebSockets or with XMLHttpRequest) and keep an active connection to the server at all times, re-establishing it if it drops. Since the server application will always find an open TCP connection to all the connected browsers, it can easily route messages to/from all clients.
As Daniel says, you are going to have a very hard time trying to do true peer-to-peer (a la Skype etc) in the browser, and it is certainly not possible without the use of plugins. And even Skype etc rely on falling back to a server acting as a gateway when a direct connection cannot be established (due to firewalls etc).
So you really need to have a gateway server regardless, and that there are a number of options. Try searching here for 'comet'. Some options that I have played with include Orbited (http://orbited.org/) and Hookbox (http://hookbox.org/) but there are many others.