We Keep Coding

SSE or WebSockets for Stackoverflow-like instant notification

What is the best technology to use if I want to make instant user notification,
like StackOverflow has?
I consider SSE and WebSockets. What are pros and cons of each solution?
Should I use socket.io or better to use WebSockets directly?

The main difference is that with SSE you can only receive messages from the server. You cannot send messages to the server. Everything doable with SSE is doable with WebSockets. But not vice versa - WebSocket is capable of sending data to the server. So from that point of view WebSockets win. I can't really see any advantage of SSE (perhaps performance?), but then again I don't have much experience with it.
Note that StackOverflow uses WebSockets. They might have some fallback for older browser, I don't know about that.
As for the third question: perhaps you should ask what language you want to use in the first place? I've been working with WebSockets and Python and it worked really well. You could work with WebSockets directly. The advantage of using socket.io is mostly the fallback (assuming it matters - it does not IMHO): if WebSockets are not available it can automatically switch to other ways of communication (like Flash or long polling). The disadvantage is that it is Node.js (in the sense that you have to restrict yourself to one language) plus there are some performance issues, i.e. socket.io does not scale well beyond one machine.

You might consider using a library like SocketIO that abstracts out the transport layer, so you don't have to worry about the mechanics of how the real-time connection is maintained. This will save you a TON of headaches.

Socket Server in Javascript (in browsers)?

I wanted to be allow users to play p2p in a multiplayer game that I'm developing, but to be able to do that, javascript needs to be able to create a socket server in the browser. Is that even possible? I don't know of any API that let clients connect to other clients in javascript. Is there any other way? Like using a hidden flash element?
I am asking for something that doesn't require a server at all. The packets need to travel from client to client directly

In short no, p2p in a browser is not possible.
The closest you can get is using NodeJS (for potentially p2p JS) or a centralised server (or several servers) and websockets (for sockets in a browser)

This question is old, but I can now give an answer: YES, there is finally a way to do p2p communication between browsers!
Thanks to the new standard WebRTC, modern browsers got support for Data Channels, something much more powerful than WebSockets.
Take a look here:
WebRTC Data Channels
Online Example: Banana Bread 3D is a First Person Shooter game compiled to JS+WebGL, using WebRTC data channels in multiplayer mode:
BananaBread 3D Multiplayer online fps game

Interesting question, but probably a duplicate:
What techniques are available to do P2P in the browser?
i know for sure this can not be done using only javascript(in every browser). According to another answer on Stackoverflow in above topic you might be able do this using rtmfp-api.
This project expose Rtmfp protocol (provided by Flash version 10) to
javascript application throught a hidden flash applet. The protocol
allow multiple clients to communicate directly. See the references for
more details about the protocol.
Looking quickly at the site you still need a rtmfpUrl-server in the middle, which i totally understand because the clients need to be be able to find each other(IPs). But I assume after that it will be p2p. Doing a quick search I also found open-source rtmfp-server(s).
I haven't tried this out myself, but I maybe this will help you achieve your goal.
Some other links:
https://stackoverflow.com/search?q=browser+p2p
https://stackoverflow.com/a/7933140/11926
https://stackoverflow.com/a/5211895/11926
https://stackoverflow.com/a/5023048/11926

While this is a shopping question, i'd look into APE
http://www.ape-project.org/
At the very least you could check out how they've structured it.

In order to implement such a game, your JavaScript client must communicate with the server. The server then runs the game logic, and sends the result back to the client.
JavaScript receives user input and sends it to the server
Server ensures that the input is valid (to prevent cheating) and updates the game with the new input
Server periodically sends the game state to JavaScript (either by long polling or by having JS request it at an interval).
Basically, never trust anything coming from JavaScript as it is extremely easy to modify. Everything should be done server-side.

Here's a solution with mobl (but I haven't tried it yet).
http://zef.me/3391/moving-the-server-to-the-browser

It is possible to go serverless with Flash. This is doable with Adobe Flash's Peer to Peer capabilities. I once wrote a peer to peer chat with it. The drawback is Actionscript is a dying language and may not be supported much in the future.
Here is the raw class.
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/NetGroup.html
Here is resources if you don't want to write your own.
http://www.as3gamegears.com/category/multiplayer/
If you want a Server option that is light on the server side. Try this node.js extension.
http://socket.io/
I recommend using a java socket server of some sort. Electroserver used to be one of the leaders in the field, it had Unity support and was scalable to hundreds of thousands. Although I think they have fallen on hard times. The Electroserver site has not been accessible for sometime. I know there are others out there but Electroserver is the only one I have used.

Best way to make a webpage live and multiuser

I'm building a web app for 'brainstorming.' Here's how it works: essentially, a user can come onto the app, and submit a challenge, or click on one that's already there, then think up ideas to resolve that challenge and post them up. I hacked together a basic example here on couchdb: http://wamoyo.iriscouch.com/ideageneration/_design/IdeaGeneration/attachments%2findex.html
I'm going to rebuild it from scratch and all, and I'm hitting up against a challenge that's very unfamiliar to me. I'd like for multiple users to be able to generate ideas for the same challenge at the same time. Kinda like the way google docs allows multiple people to edit a shared document. I have some preliminary thoughts on how to go about this, but I thought I'd ask the expert network here.
I'm fairly comfortable with AJAX, is there a pure AJAX way to make it live and multiuser? Would there be an enormous benefit to going with node.js? What might be some other options?
Thanks soo much!

There are several approaches in making such web pages, using plain ajax polling, using long polling and using web sockets.
Ajax polling - easy to implement, essentially connecting to server recurrently via javascript timer, retrieve data from server and send it back via regular Ajax.
Advantages: easy to implement, works everywhere
Disadvantages: the updates are not in real-time, the data is exchanged only when the timer ticks.
Long polling - the idea is that the connection stays open until it times out, then the connection is reestablished. Can be tricky to implement because of different settings for request timeouts for different web servers, routers, etc.
Web sockets - part of HTML5 umbrella, works only in fairly modern browsers, the protocol changes often which may cause incompatibilities during development and production. Can be used natively with modern browsers and via a Flash plugin with older ones. This technology is most lightweight, because it doesn't incur all the HTTP overhead. Think of it as bi-directional, full-duplex communication channel between a browser and a web server via TCP.
For a detailed discussion, I recommend reading this good post by Scott Hanselman. It tells the story about SignalR, but is applicable to other server-side frameworks.
There is also a podcast by same author, the guest goes fairly deeply into explaining these technologies. Worth listening, IMO.
To answer your question about node.js, please share us your current server technology, so we could get more insight into your stack.

Websocket API to replace REST API? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I have an application whose primary function works in real time, through websockets or long polling.
However, most of the site is written in a RESTful fashion, which is nice for application s and other clients in the future. However, I'm thinking about transitioning to a websocket API for all site functions, away from REST. That would make it easier for me to integrate real time features into all parts of the site. Would this make it more difficult to build applications or mobile clients?
I found that some people are already doing stuff like this: SocketStream

Not to say that the other answers here don't have merit, they make some good points. But I'm going to go against the general consensus and agree with you that moving to websockets for more than just realtime features is very appealing.
I am seriously considering moving my app from a RESTful architecture to more of an RPC style via websockets. This is not a "toy app", and I'm not talking about only realtime features, so I do have reservations. But I see many benefits in going this route and feel it could turn out to be an exceptional solution.
My plan is to use DNode, SocketIO, and Backbone. With these tools, my Backbone models and collections can be passed around from/to client and server by simply calling a functions RPC-style. No more managing REST endpoints, serializing/deserializing objects, and so forth. I haven't worked with socketstream yet, but it looks worth checking out.
I still have a long way to go before I can definitively say this is a good solution, and I'm sure it isn't the best solution for every application, but I'm convinced that this combination would be exceptionally powerful. I admit that there are some drawbacks, such as losing the ability to cache resources. But I have a feeling the advantages will outweigh them.
I'd be interested in following your progress exploring this type of solution. If you have any github experiments, please point me at them. I don't have any yet, but hope to soon.
Below is a list of to-read-later links that I've been collecting. I can't vouch that they are all worthwhile, as I've only skimmed many of them. But hopefully some will help.
Great tutorial on using Socket.IO with Express. It exposes express sessions to socket.io and discusses how to have different rooms for each authenticated user.
http://www.danielbaulig.de/socket-ioexpress/
Tutorial on node.js/socket.io/backbone.js/express/connect/jade/redis with authentication, Joyent hosting, etc:
http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/
http://fzysqr.com/2011/03/27/nodechat-js-continued-authentication-profiles-ponies-and-a-meaner-socket-io/
Tutorial on using Pusher with Backbone.js (using Rails):
http://blog.pusher.com/2011/6/21/backbone-js-now-realtime-with-pusher
Build application with backbone.js on the client and node.js with express, socket.io, dnode on the server.
http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-server-with-node/
http://addyosmani.com/blog/building-spas-jquerys-best-friends/
http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/
http://fzysqr.com/2011/03/27/nodechat-js-continued-authentication-profiles-ponies-and-a-meaner-socket-io/
Using Backbone with DNode:
http://quickleft.com/blog/backbone-without-ajax-part-ii
http://quickleft.com/blog/backbone-without-ajax-part-1
http://sorensen.posterous.com/introducing-backbone-redis
https://github.com/cowboyrushforth/minespotter
http://amir.unoc.net/how-to-share-backbonejs-models-with-nodejs
http://hackerne.ws/item?id=2222935
http://substack.net/posts/24ab8c

HTTP REST and WebSockets are very different. HTTP is stateless, so the web server doesn't need to know anything, and you get caching in the web browser and in proxies. If you use WebSockets, your server is becoming stateful and you need to have a connection to the client on the server.
Request-Reply communication vs Push
Use WebSockets only if you need to PUSH data from the server to the client, that communication pattern is not included in HTTP (only by workarounds). PUSH is helpful if events created by other clients needs to be available to other connected clients e.g. in games where users should act on other clients behaviour. Or if your website is monitoring something, where the server pushes data to the client all the time e.g. stock markets (live).
If you don't need to PUSH data from the server, it's usually easier to use a stateless HTTP REST server. HTTP uses a simple Request-Reply communication pattern.

I'm thinking about transitioning to a WebSocket api for all site functions
No. You should not do it. There is no harm if you support both models. Use REST for one way communication/simple requests & WebSocket for two way communication especially when server want to send real time notification.
WebSocket is a more efficient protocol than RESTful HTTP but still RESTful HTTP scores over WebSocket in below areas.
Create/Update/Delete resources have been defined well for HTTP. You have to implement these operations at low level for WebSockets.
WebSocket connections scale vertically on a single server where as HTTP connections scale horizontally. There are some proprietary non standards-based solutions for WebSocket horizontal scaling .
HTTP comes with a lot of good features such as caching, routing, multiplexing, gzipping etc. These have to built on top of Websocket if you chose Websocket.
Search engine optimizations works well for HTTP URLs.
All Proxy, DNS, firewalls are not yet fully aware of WebSocket traffic. They allow port 80 but might restrict traffic by snooping on it first.
Security with WebSocket is all-or-nothing approach.
Have a look at this article for more details.

The only problem I can using TCP (WebSockets) as your main web content delivery strategy is that there is very little reading material out there about how to design your website architecture and infrastructure using TCP.
So you can't learn from other people's mistakes and development is going to be slower. It's also not a "tried and tested" strategy.
Of course your also going to lose all the advantages of HTTP (Being stateless, and caching are the bigger advantages).
Remember that HTTP is an abstraction for TCP designed for serving web content.
And let's not forget that SEO and search engines don't do websockets. So you can forget about SEO.
Personally I would recommend against this as there's too much risk.
Don't use WS for serving websites, use it for serving web applications
However if you have a toy or a personal websites by all means go for it. Try it, be cutting-edge. For a business or company you cannot justify the risk of doing this.

I learned a little lesson (the hard way). I made a number crunching application that runs on Ubuntu AWS EC2 cloud services (uses powerful GPUs), and I wanted to make a front-end for it just to watch its progress in realtime. Due to the fact that it needed realtime data, it was obvious that I needed websockets to push the updates.
It started with a proof of concept, and worked great. But then when we wanted to make it available to the public, we had to add user session, so we needed login features. And no matter how you look at it, the websocket has to know which user it deals with, so we took the shortcut of using the websockets to authenticate the users. It seemed obvious, and it was convenient.
We actually had to spend quiet some time to make the connections reliable. We started out with some cheap websocket tutorials, but discovered that our implementation was not able to automatically reconnect when the connection was broken. That all improved when we switched to socket-io. Socket-io is a must !
Having said all that, to be honest, I think we missed out on some great socket-io features. Socket-io has a lot more to offer, and I am sure, if you take it in account in your initial design, you can get more out of it. In contrast, we just replaced the old websockets with the websocket functionality of socket-io, and that was it. (no rooms, no channels, ...) A redesign could have made everything more powerful. But we didn't have time for that. That's something to remember for our next project.
Next we started to store more and more data (user history, invoices, transactions, ...). We stored all of it in an AWS dynamodb database, and AGAIN, we used socket-io to communicate the CRUD operations from the front-end to the backend. I think we took a wrong turn there. It was a mistake.
Because shortly after we found out that Amazon's cloud services (AWS) offer some great load-balancing/scaling tools for RESTful applications.
We have the impression now that we need to write a lot of code to perform the handshakes of the CRUD operations.
Recently we implemented Paypal integration. We managed to get it to work. But again, all tutorials are doing it with RESTful APIs. We had to rewrite/rethink their examples to implement them with websockets. We got it to work fairly fast though. But it does feel like we are going against the flow.
Having said all that, we are going live next week. We got there in time, everything works. And it's fast, but will it scale ?

I would consider using both. Each technology has their merit and there is no one-size fits all solution.
The separation of work goes this way:
WebSockets would be the primary method of an application to communicate with the server where a session is required. This eliminates many hacks that are needed for the older browsers (the problem is support for the older browsers which will eliminate this)
RESTful API is used for GET calls that are not session oriented (i.e. not authentication needed) that benefit from browser caching. A good example of this would be reference data for drop downs used by a web application. However. can change a bit more often than...
HTML and Javascript. These comprise the UI of the webapp. These would generally benefit being placed on a CDN.
Web Services using WSDL are still the best way of enterprise level and cross-enterprise communication as it provides a well defined standard for message and data passing. Primarily you'd offload this to a Datapower device to proxy to your web service handler.
All of this happen on the HTTP protocol which gives use secure sockets via SSL already.
For the mobile application though, websockets cannot reconnect back to a disconnected session (How to reconnect to websocket after close connection) and managing that isn't trivial. So for mobile apps, I would still recommend REST API and polling.
Another thing to watch out for when using WebSockets vs REST is scalability. WebSocket sessions are still managed by the server. RESTful API when done properly are stateless (which mean there is no server state that needs to be managed), thus scalability can grow horizontally (which is cheaper) than vertically.

Do I want updates from the server?
Yes: Socket.io
No: REST
The downsides to Socket.io are:
Scalability: WebSockets require open connections and a much different Ops setup to web scale.
Learnin: I don't have unlimited time for my learnin. Things have to get done!
I'll still use Socket.io in my project, but not for basic web forms that REST will do nicely.

WebSockets (or long polling) based transports mostly serve for (near) real-time communication between the server and client. Although there are numerous scenarios where these kinds of transports are required, such as chat or some kind of real-time feeds or other stuff, not all parts of some web application need to be necessarily connected bidirectionally with the server.
REST is resource based architecture which is well understood and offers it's own benefits over other architectures. WebSockets incline more to streams/feeds of data in real-time which would require you to create some kind of server based logic in order to prioritize or differentiate between resources and feeds (in case you don't want to use REST).
I assume that eventually there would be more WebSockets centric frameworks like socketstream in the future when this transport would be more widespread and better understood/documented in the form of data type/form agnostic delivery. However, I think, this doesn't mean that it would/should replace the REST just because it offers functionality which isn't necessarily required in numerous use cases and scenarios.

I'd like to point out this blog post that is up to me, the best answer to this question.
In short, YES
The post contains all the best practices for such kind of API.

That's not a good idea. The standard isn't even finalized yet, support varies across browsers, etc. If you want to do this now you'll end up needing to fallback to flash or long polling, etc. In the future it probably still won't make a lot of sense, since the server has to support leaving connections open to every single user. Most web servers are designed instead to excel at quickly responding to requests and closing them as quickly as possibly. Heck even your operating system would have to be tuned to deal with a high number of simultaneous connections (each connection using up more ephemeral ports and memory). Stick to using REST for as much of the site as you can.

Real-time bi-directional JSON-RPC communication over HTTP

I am building a JSON-RPC server that accepts requests over HTTP. I would like to support bi-directional communication (both client and server can send requests), the specific use case being a publish/subscribe architecture where a client sends a subscribe(X) request and receives changed(X) requests in (almost) real-time. As far as I know, there are several ways to implement this with HTTP:
long polling
WebSockets
polling calls using a cookie-based session model
streaming (keeping the HTTP connection open)
a combination of some of the above
What I'm looking for is a solution that is based on accepted internet standards (if possible), usable from a web browser and easy to work with on the client side. So far, I favour the streaming thing (Twitter, CouchDB do it that way), but I'm not sure about how well this is supported within browsers and JSON-RPC libraries. Also, there may be other ways to do it that I'm not aware of.
Thank you in advance.

I think you should have a look at socket.io to accomplish your task. You could if you wanted to watch this video from the author: "Socket.IO Workshop: Guillermo Rauch". It is easy to work with on both server as client. I have created a simple sample pubsub using redis on top of socket.io.

To my knowledge, Streaming is supported by FF, Chrome (Has bufffering issues that require a datatype of application/octet-stream or a prelude to work) and IE8 (through a little XDomainRequest). I don't know about opera.
I don't really know of any comet industry standards, the Bayeux is probably the closest. It's hard to see how facebook/gmail/twitter do it as all the code is obfuscated, and it's exceedingly difficult to find much info on how all the browsers handle everything.
Even more difficult is that you will need to use a specialized server, keeping this many connections open will require thread pooling etc.. A normal server will blow up pretty fast.
It is a very powerful design if you can get it to work reliably though.

You should take a look at JSONRPC-bidirectional.It supports bidirectional RPC over WebSocket, Worker, WebRTC and HTTP and it is highly extensible.

If anyone is interested in a Java implementation I just wrote a sample app and a blog post about it. It uses Java, Maven, Comet, Bayeux, Spring.
http://jaye.felipera.cloudbees.net/
http://geeks.aretotally.in/thinking-in-reverse-not-taking-orders-from-yo