I'm in the middle of a very large project to migrate my application from Java to JavaScript, and am trying to decide what the messaging protocol should be for client/server. Some notes on what I'm looking for:
There is a very large amount of data streamed from the server to client, with infrequent small requests made from the client to server (responses may be large). Network performance is a very big concern.
We are going to deploy the new JS-based client primarily in an Electron container, but with support for "modern" browsers in the future (no ETA for that requirement, but I also don't want to be scrambling when it becomes mandatory).
The server side is and will remain in Java.
Need good backward compatibility options, as we will not always be able to control the client versions.
The research we've done so far has indicated gRPC (with ProtoBuf) as a strong candidate that ticks all the boxes. The grpc package seems to work great in Electron, and the server coding in Java is very easy. The biggest downside is that, since it uses HTTP/2, we have to jump through hoops to run it in a browser (grpc-web combined with a data proxy).
Are there any good alternatives that use websockets, or other suggestions entirely? Ideally we would like to use the same code when executing in Electron vs. browser.
You could look at WAMP protocol. The lead implementation is Autobhan. It's an RPC & PUB/SUB client-server framework that works by exchanging JSON (or msgpack) messages over websocket. So fits in well with Javascript and browser side.
I was wondering if it's good idea to intercept all internal links and load the target page with ajax.
The new history api from Html5 makes it possible to also change the url in the address bar.
are there any disadvantages between this way and the old traditional way that let's the browser load a new page (besides the increased complexity of the code)?
Many frameworks use the HTML5 History API to have client side routing. I don't think you mean that you should load the target page with ajax, but rather change the DOM without requesting the page from the server. There's a whole debate going on (and has been for the past several years) about which architecture is better and honestly it's all down to what you're trying to achieve.
Angular and Ember are client side frameworks that help build rich javascript applications (rather than the traditional website). Since JavaScript has gotten faster and more powerful in all the browsers, it has been possible to build more and more complex applications in the browser (opposed to desktop applications written in C++ or .NET/Java). The advantages of using this way of routing is that you get nice clean URLs, and you don't waste time going to the server for each request. However, you lose authentication (so if you want to protect content you'll need to go to the server anyway), and not all browsers are up and running with the History API yet (look at IE7,8,9). Consider your target audience and ask yourself if they will use these browsers or not. The frameworks I mentioned use fallback methods and implement the hashbang system instead. This has arguable problems with SEO however.
On the other side you have your server side frameworks like Rails (for Ruby) and Express (for NodeJS) that will serve the pages to the client using clean URLs. If we go further back you get into the realms of ASP.NET, PHP and plain old HTML that use the 'unclean' URL way of routing. The advantages of using the server to give you your content should be obvious - if you have protected content and the user should be authenticated then you can easily check this.
One final thing to note is the question of JavaScript. Ask yourself if your users are going to have JavaScript enabled and what will happen to your application if they have it disabled. Does this matter? How complicated is your application going to be? Do you need to use a big framework for your app, or will simply using a modern technology that gives you clean URLs be enough? Loading every bit of JavaScript takes time, and that time can be eliminated if the server is only giving the client the bare minimum of what it needs.
I am working on app where I need to pass messages between a C++ application and a Javascript web app.
Certainly I could write sockets code myself in either language and I have done this in the past when necessary.
What I would really like is a higher-level message posting or message queueing API that does a lot of the work for me. Does anyone know of such an API?
I have looked at ICE, and it doesn't appear to have Javascript bindings. I have also looked at Boost message queue, but it only caters for the C++ side of things. If necessary I might roll my own Javascript bindings for either of these technologies.
UPDATE: Sorry should have mentioned this before, I want to run this in a browser.
To give a more complete story what I want is a simple browser-based app that is used to configure and display logging for a C++ application.
I know there are other ways of doing this, but I am specifically interested in a high-level library in both C++ and browser-based Javascript that builds a message queue ontop of the sockets API (if there isn't one then I might consider implementing it myself and writing up a code project article).
ALSO: I'm not bothered about portability in terms of the web browser. Eg if there is a high-level IPC Javascript library that only works in Chrome, I'll be happy with that.
With JavaScript I assume that you are running it in a browser? In this case your C++ application needs to provide a webserver and some kind of JSON based webservice that you can call. On the JavaScript side you just use AJAX to communicate with that webservice.
An alternative would be websockets which might be a little harder to implement on the C++ side though.
To simply answer your question: No, there is no IPC implemented in ECMAscript out of the box.
But you actually answered you question already. If you try to communicate with Javascript that runs in a browser, you indeed should use (web-)sockets connections to pipe date in either direction. Of course you could write a simple HTTP server in C++, but I guess that is overkill and does not have the capabilitys of bi-directional sockets.
It's still some work to implement a web-socket connection in C++ from the scratch (the specs were in flux for a long time), but I guess there are some librarys out already.
If you're trying to communicate with node.js, this is an almost trivial task using real sockets/pipes.
I have found a solution that meets my needs. It isn't exactly perfect but I think it works well enough.
Some people suggested using HTTP and ajax. That turned out to be a useful idea and after some prototyping I think it solves my rather basic needs.
To be more specific I am using the Mongoose HTTP server embedded in my C++ application and I am using the jQuery ajax function to pull data from the server. The jQuery client polls the server continously for new data, not particularly efficient but I think it will do the job good enough for me.
Once my implementation is complete I'll write an article explaining how to do this in detail and then I'll update this answer.
You could try DBus, it has very simple mechanism to define, query and use interfaces, and there are some components for XPCOM and webkit based browsers (for example http://sandbox.movial.com/wiki/index.php/Browser_DBus_Bridge and http://code.google.com/p/v8-dbus/). Also DBus is opensource and cross platform.
For a server side or non-browser implementation how about named pipes?
Yes it's vintage technology and the usage depends which OS you use, but as long as your server side js environment has ability to read and write files it may work, and it fits the description 'high-level' inter-process communication.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I have an application whose primary function works in real time, through websockets or long polling.
However, most of the site is written in a RESTful fashion, which is nice for application s and other clients in the future. However, I'm thinking about transitioning to a websocket API for all site functions, away from REST. That would make it easier for me to integrate real time features into all parts of the site. Would this make it more difficult to build applications or mobile clients?
I found that some people are already doing stuff like this: SocketStream
Not to say that the other answers here don't have merit, they make some good points. But I'm going to go against the general consensus and agree with you that moving to websockets for more than just realtime features is very appealing.
I am seriously considering moving my app from a RESTful architecture to more of an RPC style via websockets. This is not a "toy app", and I'm not talking about only realtime features, so I do have reservations. But I see many benefits in going this route and feel it could turn out to be an exceptional solution.
My plan is to use DNode, SocketIO, and Backbone. With these tools, my Backbone models and collections can be passed around from/to client and server by simply calling a functions RPC-style. No more managing REST endpoints, serializing/deserializing objects, and so forth. I haven't worked with socketstream yet, but it looks worth checking out.
I still have a long way to go before I can definitively say this is a good solution, and I'm sure it isn't the best solution for every application, but I'm convinced that this combination would be exceptionally powerful. I admit that there are some drawbacks, such as losing the ability to cache resources. But I have a feeling the advantages will outweigh them.
I'd be interested in following your progress exploring this type of solution. If you have any github experiments, please point me at them. I don't have any yet, but hope to soon.
Below is a list of to-read-later links that I've been collecting. I can't vouch that they are all worthwhile, as I've only skimmed many of them. But hopefully some will help.
Great tutorial on using Socket.IO with Express. It exposes express sessions to socket.io and discusses how to have different rooms for each authenticated user.
http://www.danielbaulig.de/socket-ioexpress/
Tutorial on node.js/socket.io/backbone.js/express/connect/jade/redis with authentication, Joyent hosting, etc:
http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/
http://fzysqr.com/2011/03/27/nodechat-js-continued-authentication-profiles-ponies-and-a-meaner-socket-io/
Tutorial on using Pusher with Backbone.js (using Rails):
http://blog.pusher.com/2011/6/21/backbone-js-now-realtime-with-pusher
Build application with backbone.js on the client and node.js with express, socket.io, dnode on the server.
http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-server-with-node/
http://addyosmani.com/blog/building-spas-jquerys-best-friends/
http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/
http://fzysqr.com/2011/03/27/nodechat-js-continued-authentication-profiles-ponies-and-a-meaner-socket-io/
Using Backbone with DNode:
http://quickleft.com/blog/backbone-without-ajax-part-ii
http://quickleft.com/blog/backbone-without-ajax-part-1
http://sorensen.posterous.com/introducing-backbone-redis
https://github.com/cowboyrushforth/minespotter
http://amir.unoc.net/how-to-share-backbonejs-models-with-nodejs
http://hackerne.ws/item?id=2222935
http://substack.net/posts/24ab8c
HTTP REST and WebSockets are very different. HTTP is stateless, so the web server doesn't need to know anything, and you get caching in the web browser and in proxies. If you use WebSockets, your server is becoming stateful and you need to have a connection to the client on the server.
Request-Reply communication vs Push
Use WebSockets only if you need to PUSH data from the server to the client, that communication pattern is not included in HTTP (only by workarounds). PUSH is helpful if events created by other clients needs to be available to other connected clients e.g. in games where users should act on other clients behaviour. Or if your website is monitoring something, where the server pushes data to the client all the time e.g. stock markets (live).
If you don't need to PUSH data from the server, it's usually easier to use a stateless HTTP REST server. HTTP uses a simple Request-Reply communication pattern.
I'm thinking about transitioning to a WebSocket api for all site functions
No. You should not do it. There is no harm if you support both models. Use REST for one way communication/simple requests & WebSocket for two way communication especially when server want to send real time notification.
WebSocket is a more efficient protocol than RESTful HTTP but still RESTful HTTP scores over WebSocket in below areas.
Create/Update/Delete resources have been defined well for HTTP. You have to implement these operations at low level for WebSockets.
WebSocket connections scale vertically on a single server where as HTTP connections scale horizontally. There are some proprietary non standards-based solutions for WebSocket horizontal scaling .
HTTP comes with a lot of good features such as caching, routing, multiplexing, gzipping etc. These have to built on top of Websocket if you chose Websocket.
Search engine optimizations works well for HTTP URLs.
All Proxy, DNS, firewalls are not yet fully aware of WebSocket traffic. They allow port 80 but might restrict traffic by snooping on it first.
Security with WebSocket is all-or-nothing approach.
Have a look at this article for more details.
The only problem I can using TCP (WebSockets) as your main web content delivery strategy is that there is very little reading material out there about how to design your website architecture and infrastructure using TCP.
So you can't learn from other people's mistakes and development is going to be slower. It's also not a "tried and tested" strategy.
Of course your also going to lose all the advantages of HTTP (Being stateless, and caching are the bigger advantages).
Remember that HTTP is an abstraction for TCP designed for serving web content.
And let's not forget that SEO and search engines don't do websockets. So you can forget about SEO.
Personally I would recommend against this as there's too much risk.
Don't use WS for serving websites, use it for serving web applications
However if you have a toy or a personal websites by all means go for it. Try it, be cutting-edge. For a business or company you cannot justify the risk of doing this.
I learned a little lesson (the hard way). I made a number crunching application that runs on Ubuntu AWS EC2 cloud services (uses powerful GPUs), and I wanted to make a front-end for it just to watch its progress in realtime. Due to the fact that it needed realtime data, it was obvious that I needed websockets to push the updates.
It started with a proof of concept, and worked great. But then when we wanted to make it available to the public, we had to add user session, so we needed login features. And no matter how you look at it, the websocket has to know which user it deals with, so we took the shortcut of using the websockets to authenticate the users. It seemed obvious, and it was convenient.
We actually had to spend quiet some time to make the connections reliable. We started out with some cheap websocket tutorials, but discovered that our implementation was not able to automatically reconnect when the connection was broken. That all improved when we switched to socket-io. Socket-io is a must !
Having said all that, to be honest, I think we missed out on some great socket-io features. Socket-io has a lot more to offer, and I am sure, if you take it in account in your initial design, you can get more out of it. In contrast, we just replaced the old websockets with the websocket functionality of socket-io, and that was it. (no rooms, no channels, ...) A redesign could have made everything more powerful. But we didn't have time for that. That's something to remember for our next project.
Next we started to store more and more data (user history, invoices, transactions, ...). We stored all of it in an AWS dynamodb database, and AGAIN, we used socket-io to communicate the CRUD operations from the front-end to the backend. I think we took a wrong turn there. It was a mistake.
Because shortly after we found out that Amazon's cloud services (AWS) offer some great load-balancing/scaling tools for RESTful applications.
We have the impression now that we need to write a lot of code to perform the handshakes of the CRUD operations.
Recently we implemented Paypal integration. We managed to get it to work. But again, all tutorials are doing it with RESTful APIs. We had to rewrite/rethink their examples to implement them with websockets. We got it to work fairly fast though. But it does feel like we are going against the flow.
Having said all that, we are going live next week. We got there in time, everything works. And it's fast, but will it scale ?
I would consider using both. Each technology has their merit and there is no one-size fits all solution.
The separation of work goes this way:
WebSockets would be the primary method of an application to communicate with the server where a session is required. This eliminates many hacks that are needed for the older browsers (the problem is support for the older browsers which will eliminate this)
RESTful API is used for GET calls that are not session oriented (i.e. not authentication needed) that benefit from browser caching. A good example of this would be reference data for drop downs used by a web application. However. can change a bit more often than...
HTML and Javascript. These comprise the UI of the webapp. These would generally benefit being placed on a CDN.
Web Services using WSDL are still the best way of enterprise level and cross-enterprise communication as it provides a well defined standard for message and data passing. Primarily you'd offload this to a Datapower device to proxy to your web service handler.
All of this happen on the HTTP protocol which gives use secure sockets via SSL already.
For the mobile application though, websockets cannot reconnect back to a disconnected session (How to reconnect to websocket after close connection) and managing that isn't trivial. So for mobile apps, I would still recommend REST API and polling.
Another thing to watch out for when using WebSockets vs REST is scalability. WebSocket sessions are still managed by the server. RESTful API when done properly are stateless (which mean there is no server state that needs to be managed), thus scalability can grow horizontally (which is cheaper) than vertically.
Do I want updates from the server?
Yes: Socket.io
No: REST
The downsides to Socket.io are:
Scalability: WebSockets require open connections and a much different Ops setup to web scale.
Learnin: I don't have unlimited time for my learnin. Things have to get done!
I'll still use Socket.io in my project, but not for basic web forms that REST will do nicely.
WebSockets (or long polling) based transports mostly serve for (near) real-time communication between the server and client. Although there are numerous scenarios where these kinds of transports are required, such as chat or some kind of real-time feeds or other stuff, not all parts of some web application need to be necessarily connected bidirectionally with the server.
REST is resource based architecture which is well understood and offers it's own benefits over other architectures. WebSockets incline more to streams/feeds of data in real-time which would require you to create some kind of server based logic in order to prioritize or differentiate between resources and feeds (in case you don't want to use REST).
I assume that eventually there would be more WebSockets centric frameworks like socketstream in the future when this transport would be more widespread and better understood/documented in the form of data type/form agnostic delivery. However, I think, this doesn't mean that it would/should replace the REST just because it offers functionality which isn't necessarily required in numerous use cases and scenarios.
I'd like to point out this blog post that is up to me, the best answer to this question.
In short, YES
The post contains all the best practices for such kind of API.
That's not a good idea. The standard isn't even finalized yet, support varies across browsers, etc. If you want to do this now you'll end up needing to fallback to flash or long polling, etc. In the future it probably still won't make a lot of sense, since the server has to support leaving connections open to every single user. Most web servers are designed instead to excel at quickly responding to requests and closing them as quickly as possibly. Heck even your operating system would have to be tuned to deal with a high number of simultaneous connections (each connection using up more ephemeral ports and memory). Stick to using REST for as much of the site as you can.
I've been trying to learn a little about SproutCore, following the "Todos" tutorial, and I have a couple of questions that haven't been able to find online.
SproutCore is supposed to move all of the business logic to the client. How is that not insecure? A malicious user could easily tamper with the code (since it's all on the client) and change the way the app behaves. How am I wrong here?
SproutCore uses "DataStores", and some of them can be remote. How can I avoid that a malicious user does not interact with the backend on his own? Using some sort of API key wouldn't work since the code is on the client side. Is there some sort of convention here? Any ideas? This really bugs me.
Thanks in advance!
PS: Anyone thinks Cappuccino is a better alternative? I decided to go with SproutCore because the documentation on Cappuccino seemed pretty bad, although SproutCore's doesn't get any better.
Ian
your concerns are valid. The thing is, they apply to all client side code, no matter what framework. So:
Web applications are complicated things. Moving processing to the client is a good thing, because it speeds up the responsiveness of the application. However, it is imperative that the server validate all data inputs, just like in any other web application.
Additionally, all web applications should use the well known authentication/authorization paradigms that are prevalent in system security. Authentication means you must verify that the user is who they say they are, and they can use the system, with Authorization means that the server must verify that the user can do what they are trying e.g. can they create a new data entry, or edit an existing one. It is good design to not present users with UI options that they are not allowed to perform, but you should not rely on that.
All web applications must do those things.
With respect to the 'interacting with the back end' concern: Again, all web applications have this concern. You can open up firebug/webkit, and look at all the the xhr requests that RIAs use in their operations, and mimic them to try to do something on that system. Again, this concern is dealt with by the authentication/authorization checks that you must implement. Anybody can use any webclient to send a request to the server. It is up to the developer to validate that request.
The DataSources in SproutCore are just an abstraction around how SC apps interact with the server. At the end of the day, however, all SC is doing is making XHR requests to the server, like any other RIA.