I am building a JSON-RPC server that accepts requests over HTTP. I would like to support bi-directional communication (both client and server can send requests), the specific use case being a publish/subscribe architecture where a client sends a subscribe(X) request and receives changed(X) requests in (almost) real-time. As far as I know, there are several ways to implement this with HTTP:
long polling
WebSockets
polling calls using a cookie-based session model
streaming (keeping the HTTP connection open)
a combination of some of the above
What I'm looking for is a solution that is based on accepted internet standards (if possible), usable from a web browser and easy to work with on the client side. So far, I favour the streaming thing (Twitter, CouchDB do it that way), but I'm not sure about how well this is supported within browsers and JSON-RPC libraries. Also, there may be other ways to do it that I'm not aware of.
Thank you in advance.
I think you should have a look at socket.io to accomplish your task. You could if you wanted to watch this video from the author: "Socket.IO Workshop: Guillermo Rauch". It is easy to work with on both server as client. I have created a simple sample pubsub using redis on top of socket.io.
To my knowledge, Streaming is supported by FF, Chrome (Has bufffering issues that require a datatype of application/octet-stream or a prelude to work) and IE8 (through a little XDomainRequest). I don't know about opera.
I don't really know of any comet industry standards, the Bayeux is probably the closest. It's hard to see how facebook/gmail/twitter do it as all the code is obfuscated, and it's exceedingly difficult to find much info on how all the browsers handle everything.
Even more difficult is that you will need to use a specialized server, keeping this many connections open will require thread pooling etc.. A normal server will blow up pretty fast.
It is a very powerful design if you can get it to work reliably though.
You should take a look at JSONRPC-bidirectional.It supports bidirectional RPC over WebSocket, Worker, WebRTC and HTTP and it is highly extensible.
If anyone is interested in a Java implementation I just wrote a sample app and a blog post about it. It uses Java, Maven, Comet, Bayeux, Spring.
http://jaye.felipera.cloudbees.net/
http://geeks.aretotally.in/thinking-in-reverse-not-taking-orders-from-yo
Related
What is the best technology to use if I want to make instant user notification,
like StackOverflow has?
I consider SSE and WebSockets. What are pros and cons of each solution?
Should I use socket.io or better to use WebSockets directly?
The main difference is that with SSE you can only receive messages from the server. You cannot send messages to the server. Everything doable with SSE is doable with WebSockets. But not vice versa - WebSocket is capable of sending data to the server. So from that point of view WebSockets win. I can't really see any advantage of SSE (perhaps performance?), but then again I don't have much experience with it.
Note that StackOverflow uses WebSockets. They might have some fallback for older browser, I don't know about that.
As for the third question: perhaps you should ask what language you want to use in the first place? I've been working with WebSockets and Python and it worked really well. You could work with WebSockets directly. The advantage of using socket.io is mostly the fallback (assuming it matters - it does not IMHO): if WebSockets are not available it can automatically switch to other ways of communication (like Flash or long polling). The disadvantage is that it is Node.js (in the sense that you have to restrict yourself to one language) plus there are some performance issues, i.e. socket.io does not scale well beyond one machine.
You might consider using a library like SocketIO that abstracts out the transport layer, so you don't have to worry about the mechanics of how the real-time connection is maintained. This will save you a TON of headaches.
I'm working on a project where a page, needs to be able to keep updated according to the state of a server.
I like websockets for this since they offer me a way of pushing messages from the server, but availability is a problem.
I need generic way to do two way communication between a webserver and a browser-client.
I would like to be able to hold a large amount of clients on my server, so busywaiting clients is not a good solution.
I've looked at long pooling, but this seames like busy waiting on the clients part -- is it the only way to go if I need IE support?
This question is only about the clients end of the transactions.
Do you need two-way communication? If not you should use SSE (Server-sent Events). They are also easier to simulate in IE (as SSE actually degrades gracefully to long-polling on older systems).
Yes, you are correct there is a problem with longpolling, it tends to consume loads of resources.
What you need is as i can see solution that has a fallback to HTTP longopolling for elder not Websockets API speaking browsers. SSE is a alternvative,but Websockets feels as a more convinent
If you are running on the .NET plattform XSockets.NET can be a alternative, it supports Websockets (RFC6544 and Hybi00) and fallbacks on HTTP Longpolling when needed (i.e IE )
Have a look at http://xsockets.net
Have you reviewed http://signalr.net/ ? Based on websockets but will gracefully downgrade to the nearest available component to support a socket type connection.
Docs can be found here: https://github.com/SignalR/SignalR/wiki
I hope this is okay to ask here as its more a request for advice than a technical problem.
I have been developing a game using html5+js and my goal was to make it multiplayer. It's a dungeon crawling game and my intent is to have a main village where all players online and in the village can see each other going about and talk and form parties but when a player enters a dungeon its a seperate instance for them and their party. Party size maxium of 4.
My intention was to use websockets and write a server in c#. The problem is I just found out that IE does not support websockets and is still holding around 25% of the browser market share.
my options seem to be to use websockets anyway and cut out the IE crowd or maybe drop multiplayer support. Someone else suggested that I just write world data to a database and have players read from it every frame and update that way.. it sounds horriable.
I found this socket.io thing that seems like it can use websockets OR do the same deal in other ways - but how does this effect me writing a server? If I use the c# implementation of websockets and socket.io will IE users be able to talk to my server?
Or there may be other ways of doing client->server communication that I don't even know about.
To be completely honest i'm tempted to drop the multiplayer idea! But before I do I look to you guys for advice and experienced suggestions on how I could handle this. Thanks for your time and I hope this kind of question is okay here :)
Do not drop the multiplayer idea! It's way cooler with multiplayer. :)
Socket.io is a Server-Side JavaScript library. This means that you need Node JS server for it. IE users will be able to talk with Socket.io server, because it uses other protocols if WebSockets are not available. For example: FlashSocket or XHR long-polling. The last technique is available in every browser that supports XHR, but it is inefficient.
The greatest advantage of socket.io is the fallback. You can set it to start with any protocol (like WebSockets) and if the client does not support it, then it tries the others. It is really great, since you can use WebSockets (which will slowly but surely dominate web apps) and still work with browsers which do not support it, like IE or Opera or Safari. I don't know whether there is any other library with this advantage.
I don't know any library for real-time connections for C# (I'm not a C# developer), but it is unlikely that there are none (look at this question). Also note that real-time connections require a bit different server architecture then normal HTTP requests, so probably you will need additional server for handling them.
Also I think that neither nginx nor Apache handle WebSockets (without some hard core tricks) at the moment (but Node JS does!). I'm not sure though.
There is no reason to lock yourself in platform-wise for something as simple as transport. Support for these things change over time, and you will want to decouple yourself from them as much as possible. You are, after all, making a game, not a network tech demo.
Have a look at Orbited/Orbited2 TCPSocket. You can write your server as standard TCP in whatever manner you like. This also makes life easier if you decide to make a native client.
I have a couple of questions concerning Web Sockets.
The latest Firefox 4.0 nightlies support Web Sockets. So does Webkit (Chrome 4 + Safari 4/5). Internet Explorer 9 is supposed to feature Web Sockets at some point according to Microsoft (before the stable release).
Anyway, my questions are:
I am building a JavaScript admin interface to manage a website. Should I use Web Sockets for the client-server communication instead of XMLHttpRequest if I told you that I do not need to care about browser compatibility?
Would Web Sockets result in faster save, deletion and update calls compared to the usual situation with XMLHttpRequest? Would the requests be more instant?
I am aware of the HTML5's navigator.online and window.addEventListener('offline', ...), but with Web Sockets (upon connection loss), am I able to detect connection issues more accurately and faster? I mean, when I turn off my Internet connection, or block it with my firewall, Firefox still seems to state that navigator.online is true. With Web Sockets, it seems, the connection to the server will be lost instantly, thus, I can detect connection problems more accurately?
Can I support Web Sockets server-side with pure PHP, so, that the code is portable with other web servers (no need to install any Apache modules or do other customization). I would like to distribute the software to a couple of places without having to ask people to install all sorts of modules to their HTTPD or so.
I wish you can answer to as many of those questions as possible. I am really interested in answers.
I am building a JavaScript admin interface to manage a website. Should I use Web Sockets for the client-server communication instead of XMLHttpRequest if I told you that I do not need to care about browser compatibility?
It appears to me that you want to use WebSockets just for the sake of it. The main reason to use WebSockets is when you want to push data from the server to the client. If your application does not need this, you should probably not use WebSockets.
Would Web Sockets result in faster save, deletion and update calls compared to the usual situation with XMLHttpRequest? Would the requests be more instant?
You could probably save some time on both ends (client and server) due to the absence of headers. But the gain is probably pretty small.
with Web Sockets (upon connection loss), am I able to detect connection issues more accurately and faster?
Yes, an event will fire instantly when the WebSocket closes. Alternatives would be long-polling or periodic XHRs. Or event client-side storage.
Can I support Web Sockets server-side with pure PHP, so, that the code is portable with other web servers
First I suggest you read through this. WebSockets don't work very well in a synchronous way. PHP and apache don't work very well in an asynchronous way. Although there are some implementations, many of them are outdated. I personally would use an other language for this, such as ruby, python, java or server-side javascript. Simply because the languages have better support for the asynchronous model and the WebSocket implementations are more sophisticated.
The WebSocket protocol is currently still a draft, it can change. Just like it did a few weeks ago. So your code may very well break.
My advice is: Do not use WebSockets just for the sake of it. If you have a real-time event-driven application, then it is probably the right choice. Make sure you understand what WebSockets are for and what it takes on the server side, also in terms of event-driven applications. Don't use it for anything production, it's way too fragile.
I want to create an application like this:
http://collabedit.com/
What is the most efficient way to create this real time application?
Flash? Long polling? Http Streaming? or something else?
Thanks ;)
For now, long polling is probably the best solution. Many big-name sites have long polling implementations, including Facebook, Google and eBay. Not everyone has Flash installed/enabled in their browsers. In the future Web Sockets might be able to do an easier job of it for us.
Update: As of this writing, the WebSocket API is implemented in the latest WebKit (Chrome/Safari) and Firefox 4 beta. There is also a public snapshot build of Opera available for download with an implementation of the API. This means testing the API is widely available. For more information, see this answer.
All the different methods have different pros and cons, I'm not an specialist on the differences, that's why I'll recomend you to avoid making the choice, avoid the development and tuning that each approach involves, avoid the future changes in available technologies (ie. as HTML5 web sockets arrival), using a library that abstracts the transport method used, and chooses the best approach based on client capabilities:
http://socket.io/
this wonderful library makes creating realtime apps amazingly simple! and there are various server-side implementations: Python (Tornado), Java, Google GO, Rack (Ruby), besides the mainstream implementation in Node.js (server-side JavaScript)
I don't think long-polling is most efficient way to do Comet. Anyway, it sends new HTTP request after response is got. It cost more extra HTTP requests than HTTP streaming.
But, long-polling might be more reliable and easier to implement than HTTP streaming. According to this article in Google Code, HTTP streaming might not be functional if intermediate HTTP proxy buffers content.
It is interesting that GMail doesn't use long-polling. With the help of Http sniffer, it is clear that it uses HTTP streaming for Comet.