I have searched everywhere for a solution to this problem but for whatever reason I cannot find a clear answer as to how I can carry out this task.
I have built a very simple server with node.js that accesses two numbers from a website API and outputs them onto a localhost port on my computer as shown below:
My question is how can I take my server and make it accessible to applications without having to go into the command line and run the server file? Is there a way I can host it online instead of locally so that I can distribute the application and anyone with the application can pull from this server? What would be the best way to go about accomplishing this task?
Heroku is probably one of the easiest ways to get started with deploying the application: https://devcenter.heroku.com/articles/getting-started-with-nodejs#introduction
However, if you are not familiar with git, Microsoft Azure Webapps is also another great option: https://tryappservice.azure.com/
They both offer free plans which should get you up and running fast!
Question 1: how can I take my server and make it accessible to applications without having to go into the command line and run the server file?
To start a node server you will always need to use the console. If you are not used to,it's time to start :) .
You will be using it not only for node servers but for administrating (almost) every server in the world.
Question 2: Is there a way I can host it online instead of locally ?
There are a lot of nodejs hosting platforms, you can choose between PaaS solutions or IaaS solutions ( AWS EC2, Digital Ocean, etc.) . Probability the easiest way to start ar PaaS services, in this blogbost you will find a good list of PaaS hosting providers. Some of them have free plans.
Take a look at Heroku. They have a free tier (with limitations) and a hobby tier for $7/month.
I have implemented that with Amazon AWS, they offer free option for a limited time, but still great because you have full control on the console, you should be careful while installing the machine on cloud, you have many guides online that you can follow for steps, this one for example has step by step info, try to follow the steps and you will be able to run your app and access it from any public network.
So I am generally quite new to the MEAN stack and had some questions regarding compression with the NodeJS / express combo.
After digging around I noticed it is not feature that is widely used and it had some issues in the past (zlib) and also has now been separated out in Express 4.0
Having also spotted the documentation for the compression module: https://github.com/expressjs/compression
I have to say, the 'Server-Sent Events' code example raises an eyebrow with the setInterval method. I guess that is mainly for streaming the data rather than giving it one go but still seemed a little strange at first.
So for those who are using this set-up and stack with the above module, can anybody notify me of any gotcha's or issues to be aware of as searching around hasn't given me anything recent.
FYI, I am primarily looking to use it for large amount of JSON transfers and maybe smaller HTML static as well later.
I would also be interested to know what else people are doing for compression if they are not using the above.
I wouldn't put zlib compression on within my application. I would leave it at the web server level where it is highly optimized.
A lot of folks make the mistake of using their Node.js server's HTTP server as their primary serving method. This obviously works, but you can make things more efficient by putting a proper web server, such as Nginx, out in front. Why tie up your application with spoon-feeding slow connections, compression, caching, authentication, etc.
Think of the HTTP server in Node.js as a replacement for CGI/FastCGI. While it can be used as a general purpose HTTP server, you're better off using it as the communication protocol between your application and your web server for most web applications.
(Obviously there are exceptions to this... not all Node.js apps even have anything to do with web serving. Use your best judgement when architecting your app.)
I found node zlib to be quite slow for my intended use.
This is what I tried to use it for:
take a multipart form post with attached files
pipe it through zLib and crypto (aes-128-ctr) to Amazon S3.
After some timing experiments I found out that the encryption part is quite speedy. It adds only about 5-10% to the total transfer time from client to S3.
ZLib on the other hand adds about 50% to the transfer time. I ended up not compressing the files. Storage on S3 is cheap.
I hope this is okay to ask here as its more a request for advice than a technical problem.
I have been developing a game using html5+js and my goal was to make it multiplayer. It's a dungeon crawling game and my intent is to have a main village where all players online and in the village can see each other going about and talk and form parties but when a player enters a dungeon its a seperate instance for them and their party. Party size maxium of 4.
My intention was to use websockets and write a server in c#. The problem is I just found out that IE does not support websockets and is still holding around 25% of the browser market share.
my options seem to be to use websockets anyway and cut out the IE crowd or maybe drop multiplayer support. Someone else suggested that I just write world data to a database and have players read from it every frame and update that way.. it sounds horriable.
I found this socket.io thing that seems like it can use websockets OR do the same deal in other ways - but how does this effect me writing a server? If I use the c# implementation of websockets and socket.io will IE users be able to talk to my server?
Or there may be other ways of doing client->server communication that I don't even know about.
To be completely honest i'm tempted to drop the multiplayer idea! But before I do I look to you guys for advice and experienced suggestions on how I could handle this. Thanks for your time and I hope this kind of question is okay here :)
Do not drop the multiplayer idea! It's way cooler with multiplayer. :)
Socket.io is a Server-Side JavaScript library. This means that you need Node JS server for it. IE users will be able to talk with Socket.io server, because it uses other protocols if WebSockets are not available. For example: FlashSocket or XHR long-polling. The last technique is available in every browser that supports XHR, but it is inefficient.
The greatest advantage of socket.io is the fallback. You can set it to start with any protocol (like WebSockets) and if the client does not support it, then it tries the others. It is really great, since you can use WebSockets (which will slowly but surely dominate web apps) and still work with browsers which do not support it, like IE or Opera or Safari. I don't know whether there is any other library with this advantage.
I don't know any library for real-time connections for C# (I'm not a C# developer), but it is unlikely that there are none (look at this question). Also note that real-time connections require a bit different server architecture then normal HTTP requests, so probably you will need additional server for handling them.
Also I think that neither nginx nor Apache handle WebSockets (without some hard core tricks) at the moment (but Node JS does!). I'm not sure though.
There is no reason to lock yourself in platform-wise for something as simple as transport. Support for these things change over time, and you will want to decouple yourself from them as much as possible. You are, after all, making a game, not a network tech demo.
Have a look at Orbited/Orbited2 TCPSocket. You can write your server as standard TCP in whatever manner you like. This also makes life easier if you decide to make a native client.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I have an application whose primary function works in real time, through websockets or long polling.
However, most of the site is written in a RESTful fashion, which is nice for application s and other clients in the future. However, I'm thinking about transitioning to a websocket API for all site functions, away from REST. That would make it easier for me to integrate real time features into all parts of the site. Would this make it more difficult to build applications or mobile clients?
I found that some people are already doing stuff like this: SocketStream
Not to say that the other answers here don't have merit, they make some good points. But I'm going to go against the general consensus and agree with you that moving to websockets for more than just realtime features is very appealing.
I am seriously considering moving my app from a RESTful architecture to more of an RPC style via websockets. This is not a "toy app", and I'm not talking about only realtime features, so I do have reservations. But I see many benefits in going this route and feel it could turn out to be an exceptional solution.
My plan is to use DNode, SocketIO, and Backbone. With these tools, my Backbone models and collections can be passed around from/to client and server by simply calling a functions RPC-style. No more managing REST endpoints, serializing/deserializing objects, and so forth. I haven't worked with socketstream yet, but it looks worth checking out.
I still have a long way to go before I can definitively say this is a good solution, and I'm sure it isn't the best solution for every application, but I'm convinced that this combination would be exceptionally powerful. I admit that there are some drawbacks, such as losing the ability to cache resources. But I have a feeling the advantages will outweigh them.
I'd be interested in following your progress exploring this type of solution. If you have any github experiments, please point me at them. I don't have any yet, but hope to soon.
Below is a list of to-read-later links that I've been collecting. I can't vouch that they are all worthwhile, as I've only skimmed many of them. But hopefully some will help.
Great tutorial on using Socket.IO with Express. It exposes express sessions to socket.io and discusses how to have different rooms for each authenticated user.
http://www.danielbaulig.de/socket-ioexpress/
Tutorial on node.js/socket.io/backbone.js/express/connect/jade/redis with authentication, Joyent hosting, etc:
http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/
http://fzysqr.com/2011/03/27/nodechat-js-continued-authentication-profiles-ponies-and-a-meaner-socket-io/
Tutorial on using Pusher with Backbone.js (using Rails):
http://blog.pusher.com/2011/6/21/backbone-js-now-realtime-with-pusher
Build application with backbone.js on the client and node.js with express, socket.io, dnode on the server.
http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-server-with-node/
http://addyosmani.com/blog/building-spas-jquerys-best-friends/
http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/
http://fzysqr.com/2011/03/27/nodechat-js-continued-authentication-profiles-ponies-and-a-meaner-socket-io/
Using Backbone with DNode:
http://quickleft.com/blog/backbone-without-ajax-part-ii
http://quickleft.com/blog/backbone-without-ajax-part-1
http://sorensen.posterous.com/introducing-backbone-redis
https://github.com/cowboyrushforth/minespotter
http://amir.unoc.net/how-to-share-backbonejs-models-with-nodejs
http://hackerne.ws/item?id=2222935
http://substack.net/posts/24ab8c
HTTP REST and WebSockets are very different. HTTP is stateless, so the web server doesn't need to know anything, and you get caching in the web browser and in proxies. If you use WebSockets, your server is becoming stateful and you need to have a connection to the client on the server.
Request-Reply communication vs Push
Use WebSockets only if you need to PUSH data from the server to the client, that communication pattern is not included in HTTP (only by workarounds). PUSH is helpful if events created by other clients needs to be available to other connected clients e.g. in games where users should act on other clients behaviour. Or if your website is monitoring something, where the server pushes data to the client all the time e.g. stock markets (live).
If you don't need to PUSH data from the server, it's usually easier to use a stateless HTTP REST server. HTTP uses a simple Request-Reply communication pattern.
I'm thinking about transitioning to a WebSocket api for all site functions
No. You should not do it. There is no harm if you support both models. Use REST for one way communication/simple requests & WebSocket for two way communication especially when server want to send real time notification.
WebSocket is a more efficient protocol than RESTful HTTP but still RESTful HTTP scores over WebSocket in below areas.
Create/Update/Delete resources have been defined well for HTTP. You have to implement these operations at low level for WebSockets.
WebSocket connections scale vertically on a single server where as HTTP connections scale horizontally. There are some proprietary non standards-based solutions for WebSocket horizontal scaling .
HTTP comes with a lot of good features such as caching, routing, multiplexing, gzipping etc. These have to built on top of Websocket if you chose Websocket.
Search engine optimizations works well for HTTP URLs.
All Proxy, DNS, firewalls are not yet fully aware of WebSocket traffic. They allow port 80 but might restrict traffic by snooping on it first.
Security with WebSocket is all-or-nothing approach.
Have a look at this article for more details.
The only problem I can using TCP (WebSockets) as your main web content delivery strategy is that there is very little reading material out there about how to design your website architecture and infrastructure using TCP.
So you can't learn from other people's mistakes and development is going to be slower. It's also not a "tried and tested" strategy.
Of course your also going to lose all the advantages of HTTP (Being stateless, and caching are the bigger advantages).
Remember that HTTP is an abstraction for TCP designed for serving web content.
And let's not forget that SEO and search engines don't do websockets. So you can forget about SEO.
Personally I would recommend against this as there's too much risk.
Don't use WS for serving websites, use it for serving web applications
However if you have a toy or a personal websites by all means go for it. Try it, be cutting-edge. For a business or company you cannot justify the risk of doing this.
I learned a little lesson (the hard way). I made a number crunching application that runs on Ubuntu AWS EC2 cloud services (uses powerful GPUs), and I wanted to make a front-end for it just to watch its progress in realtime. Due to the fact that it needed realtime data, it was obvious that I needed websockets to push the updates.
It started with a proof of concept, and worked great. But then when we wanted to make it available to the public, we had to add user session, so we needed login features. And no matter how you look at it, the websocket has to know which user it deals with, so we took the shortcut of using the websockets to authenticate the users. It seemed obvious, and it was convenient.
We actually had to spend quiet some time to make the connections reliable. We started out with some cheap websocket tutorials, but discovered that our implementation was not able to automatically reconnect when the connection was broken. That all improved when we switched to socket-io. Socket-io is a must !
Having said all that, to be honest, I think we missed out on some great socket-io features. Socket-io has a lot more to offer, and I am sure, if you take it in account in your initial design, you can get more out of it. In contrast, we just replaced the old websockets with the websocket functionality of socket-io, and that was it. (no rooms, no channels, ...) A redesign could have made everything more powerful. But we didn't have time for that. That's something to remember for our next project.
Next we started to store more and more data (user history, invoices, transactions, ...). We stored all of it in an AWS dynamodb database, and AGAIN, we used socket-io to communicate the CRUD operations from the front-end to the backend. I think we took a wrong turn there. It was a mistake.
Because shortly after we found out that Amazon's cloud services (AWS) offer some great load-balancing/scaling tools for RESTful applications.
We have the impression now that we need to write a lot of code to perform the handshakes of the CRUD operations.
Recently we implemented Paypal integration. We managed to get it to work. But again, all tutorials are doing it with RESTful APIs. We had to rewrite/rethink their examples to implement them with websockets. We got it to work fairly fast though. But it does feel like we are going against the flow.
Having said all that, we are going live next week. We got there in time, everything works. And it's fast, but will it scale ?
I would consider using both. Each technology has their merit and there is no one-size fits all solution.
The separation of work goes this way:
WebSockets would be the primary method of an application to communicate with the server where a session is required. This eliminates many hacks that are needed for the older browsers (the problem is support for the older browsers which will eliminate this)
RESTful API is used for GET calls that are not session oriented (i.e. not authentication needed) that benefit from browser caching. A good example of this would be reference data for drop downs used by a web application. However. can change a bit more often than...
HTML and Javascript. These comprise the UI of the webapp. These would generally benefit being placed on a CDN.
Web Services using WSDL are still the best way of enterprise level and cross-enterprise communication as it provides a well defined standard for message and data passing. Primarily you'd offload this to a Datapower device to proxy to your web service handler.
All of this happen on the HTTP protocol which gives use secure sockets via SSL already.
For the mobile application though, websockets cannot reconnect back to a disconnected session (How to reconnect to websocket after close connection) and managing that isn't trivial. So for mobile apps, I would still recommend REST API and polling.
Another thing to watch out for when using WebSockets vs REST is scalability. WebSocket sessions are still managed by the server. RESTful API when done properly are stateless (which mean there is no server state that needs to be managed), thus scalability can grow horizontally (which is cheaper) than vertically.
Do I want updates from the server?
Yes: Socket.io
No: REST
The downsides to Socket.io are:
Scalability: WebSockets require open connections and a much different Ops setup to web scale.
Learnin: I don't have unlimited time for my learnin. Things have to get done!
I'll still use Socket.io in my project, but not for basic web forms that REST will do nicely.
WebSockets (or long polling) based transports mostly serve for (near) real-time communication between the server and client. Although there are numerous scenarios where these kinds of transports are required, such as chat or some kind of real-time feeds or other stuff, not all parts of some web application need to be necessarily connected bidirectionally with the server.
REST is resource based architecture which is well understood and offers it's own benefits over other architectures. WebSockets incline more to streams/feeds of data in real-time which would require you to create some kind of server based logic in order to prioritize or differentiate between resources and feeds (in case you don't want to use REST).
I assume that eventually there would be more WebSockets centric frameworks like socketstream in the future when this transport would be more widespread and better understood/documented in the form of data type/form agnostic delivery. However, I think, this doesn't mean that it would/should replace the REST just because it offers functionality which isn't necessarily required in numerous use cases and scenarios.
I'd like to point out this blog post that is up to me, the best answer to this question.
In short, YES
The post contains all the best practices for such kind of API.
That's not a good idea. The standard isn't even finalized yet, support varies across browsers, etc. If you want to do this now you'll end up needing to fallback to flash or long polling, etc. In the future it probably still won't make a lot of sense, since the server has to support leaving connections open to every single user. Most web servers are designed instead to excel at quickly responding to requests and closing them as quickly as possibly. Heck even your operating system would have to be tuned to deal with a high number of simultaneous connections (each connection using up more ephemeral ports and memory). Stick to using REST for as much of the site as you can.
Could anyone that has used both share his/her experience? What are the main differences and which one do you prefer? Thank you.
Different socket.io vs APE:
socket.io is coded in javascript(node.js) while APE is coded in C. I believe that is a big difference when you want to contribute. Maybe because you like the project or maybe because you want some more functionality. I think it will be easier to contribute Socket.io because you program Javascript, which is easier to grasp according to a lot of people(I Agree, although C is also very cool language).
I believe socket.io supports a lot more browsers/transport compared to APE, but I am not sure. Socket.io information vs information from APE page:
APE Server is an Comet server
implementing the POST and GET methods
of the HTTP protocol. It does not
replace a regular Web Server (such as
Apache, Lighttpd or Nginx), however,
the APE Server is only used for AJAX
Push.
So I guess APE supports less transports then socket.io.
Like Raynos said it is difficult to compare those two products and I believe you should play with them both and then decide which one you like more.
Same Socket.io/APE:
You can both code in Javascript to communicate with the server. I think you will have more freedom using socket.io because everything is exposed via Javascript.
Experience:
I only have experience with socket.io and I like it a lot.
Having spent some time developing with Node.js, I can't say that APE appears to be better than node. Based on sheer popularity, it seems that Node.js is probably the developer's choice -- and node.js appears to be more versatile as well.
Node essentially is you making a full on HTTP or TCP/IP. So, all of the mime type handling, data buffering, response headers, and server side routing are all things you'll have to do with your code. Node does streaming as well. I'm not sure about whether this is considered less problematic than normal ajax long polling at this point.
After googling around, I've found that people consider APE to be more of just a plain ol' push server, in which comet functionality is already there to be consumed, rather than Node, which would have you write your own. Don't be afraid of the prospect of writing things out with node though, they've got a very thorough documentation, and their methods are very easy to learn. I had some serious functionality written out in minutes.
Check this out also: http://groups.google.com/group/nodejs/browse_thread/thread/9d9b301479851b1f?pli=1
I played a little bit with node.js, tried out socket.io - but in the end did a big project with APE.
I think, as always, there is the question of what you want to achieve.
Only comparing the server parts: With node.js you get a machine that won't do anything on it's own, you need to write it yourself (or use libraries)
With APE, the handling of channels and connections is already built in (compiled C). Still you need to build parts of your own logic on top with JS - or use the examples.
On the client side, socket.io provides a client framework with three commands - and APE has it's APE_JSF that handles the connections (which brings more functionality than socket.io regarding channels)
Personally, I prefer APE, even though there is a lack of documentation for beginners.
However, keep in mind that APE won't deliver files/images, it's not a full Web-Server but optimized for real time push where it can handle ~10K concurrent users