I'm currently building a web chat with node.js for the backend. It uses web sockets for the communication between server and client. I haven't worked with node before and I always used PHP/MySQL and Ajax to store and retrieve data. The chat can be compared to MSN etc. with user accounts and contact lists.
I was looking for some tips and frameworks, which could help me with developing a website which heavily relies on JavaScript and which does all its communication via node.js.
What would you recommend to store the data? Mongo db (mongoose? mongo-db native?), SQLite? An easy solution would be appreciated.
Would you send everything via the websocket and would you establish a new connection for every conversation?
Is there a way to get around PHP completely?
Would you recommend the usage Backbone.js or Ember.js?
Thanks for you help!
What would you recommend to store the data? Mongo db (mongoose?
mongo-db native?), SQLite? An easy solution would be appreciated.
I would suggest you to use redis, because it is insanely fast.
> Example of benchmark result The test was done with 50 simultaneous
> clients performing 100000 requests. The value SET and GET is a 256
> bytes string. The Linux box is running Linux 2.6, it's Xeon X3320 2.5
> GHz. Text executed using the loopback interface (127.0.0.1). Results:
> about 110000 SETs per second, about 81000 GETs per second.
As client I would use node_redis
Would you send everything via the websocket and would you establish a
new connection for every conversation?
websockets are not yet support by every browser(pointing especially to Internet Explorer ;)). I think you should use socket.io which supports multiple transports so that it will work in every major browser.
Is there a way to get around PHP completely?
You could get around PHP completely. Use for example express as your web framework instead of PHP
Would you recommend the usage Backbone.js or Ember.js?
Also need to have a better look at both of them.
Use mongo-db or couchDB to store the data.
Establishing a new connection is better than sending everything through websockets and parsing them.
Yes. Use node.js on the server side.
You can try either one of them. Both have their pros & cons.
Related
I am preparing to build a WebSockets system to have a bidirectional communication from my server to my clients through WebSockets.
I know that Laravel now very well supports 3rd parties like Pusher Channels and Ably.
There are also some other ways that are also very convenient and simple to use such as Laravel Websockets, soketi, laravel-echo-server.
But my problem is client-side cannot include 3rd party Client Library. My client-side is a Cross-platform Game based on Javascript and HTML5. I can only connect client-side to server-side through WebSocket.
As far as I know, when using 3rd party for server-side, I should use their JS Client Library like pusher/pusher-js, laravel/echo ... Therefore, I can't find any documents if I want to use them with WebSocket.
My goal is to find the best and most suitable solution for both client and server side.
What I have tried?
IDEA 1: Convert pusher.js to my own js
I spent 2 weeks on this. I see this as possible and some functions might work. However, it will take a long time if I want to convert the whole thing.
IDEA 2: Using Pusher and connect the client via Pusher Channels Protocol
I have tested them as follows
var socket = new WebSocket('ws://ws-[cluster_name].pusher.com:[port]/app/[key]');
It worked. I can see them connected on the Pusher Debug Console. However, I can't find any documentation regarding the basics like Subcribe channel, Listening events, Send message to other client, Ping, Pong, ...
IDEA 3: Using Laravel Websockets
Like idea 2, I couldn't find any documentation or examples.
IDEA 4: Using NodeJS instead of Laravel. Choose ws packages
I tried it and it worked as expected. WS is really simple but it is exactly what I want. However I don't have many years of NodeJS experience. Therefore, I feel there are many risks if I choose it to build a new project.
Could you tell me a best way to deal with this problem?
Thanks
The Pusher and Laravel Echo JavaScript clients work with any Pusher-compatible websockets host. (Pusher uses websockets, too.)
All it takes is a little configuration; see, for example, Laravel Websockets' guide on configuring Laravel and then configuring Echo. Both steps point your Pusher requests at your chosen Pusher replacement instead.
No need to write your own library or manually consume websocket data.
We have an application which consumes a large amount of data. Currently a desktop app, but we would like to deliver it via the browser.
It doesn't make sense to me to create a web app where we need to transfer a ll the data used for the visualizations.
We're looking at RDP and some products out there that provide RDP access via a fully javascript client. They seem to work well with our app, but I've been thinking about what it would take to move off Windows.
Switching the front end so that it could run under Linux would not be trivial, but not impossible, so the main stumbling block would be delivery.
I was wondering if there are any X11 javascript servers out there, but have not found any leads.
Use xpra's builtin html5 client, it supports any application you can run on an X11 desktop.
You can use an HTML5 VNC viewer like https://github.com/kanaka/noVNC coupled with a VNC server like RealVNC
AFAIK, recent GTK has been ported to HTML5+Javascript in Gtk Broadway
And you could make your application a web application, for instance by using Wt, or by making it an HTTP server thru specialized HTTP server libraries like libonion, libmicrohttpd etc.
By using AJAX techniques (e.g. thru jquery) your application won't transmit all the display data to the browser at once (but only incrementally and only the actually shown data).
You might also consider fastcgi as a way to connect your application to some web server.
I know two, both at very infancy:
https://github.com/GothAck/javascript-x-server
and
https://github.com/ttaubert/x-server-js
Both need simple tcp-to-websockets proxy in front, but all X11 logic happen inside web page and all x11 objects exist and interact within browser (so it's not just remote framebuffer but real server)
You can ever run full Linux distribution in Web Browser, but that's require to run x86/ARM emulator and GNU/Linux inside it. It provides X server with possible web connection too.
For very simple applications you can use libgreattao toolkit and tao-network-client to connect to it. I'm the author of both project. The API isn't yet frozen, but it rather behaves stable. You can read about it here:
https://nintyfan.wordpress.com/2015/04/30/server-buildin-into-libgreattao-and-tao-network-client/
It can provide some problems with applications with a lot of data, because all elements must be send to client, when it were created, but instead we don't send full graphics(only icons is send) and user interface could be changed quickly. It also don't support mouse enter/leave/move events.
I must tell: do not download tarbar, but download version from svn.
Sounds like the easiest approach for you is to get xrdp, which is an RDP-server for X. Then you would use your RDP client to connect to it. I think Nomachine NX supports html directly now, but I'm not sure. There was talk of an html X2go-client, but I don't know anything about that either.
Of course I am aware of Ajax, but the problem with Ajax is that the browser should poll the server frequently to find whether there is new data. This increases server load.
Is there any better method (even using Ajax) other than polling the server frequently?
Yes, what you're looking for is COMET http://en.wikipedia.org/wiki/Comet_(programming). Other good Google terms to search for are AJAX-push and reverse-ajax.
Yes, it's called Reverse Ajax or Comet. Comet is basically an umbrella term for different ways of opening long-lived HTTP requests in order to push data in real-time to a web browser. I'd recommend StreamHub Push Server, they have some cool demos and it's much easier to get started with than any of the other servers. Check out the Getting Started with Comet and StreamHub Tutorial for a quick intro. You can use the Community Edition which is available to download for free but is limited to 20 concurrent users. The commercial version is well worth it for the support alone plus you get SSL and Desktop .NET & Java client adapters. Help is available via the Google Group, there's a good bunch of tutorials on the net and there's a GWT Comet adapter too.
Nowadays you should use WebSockets.
This is 2011 standard that allows to initiate connections with HTTP and then upgrade them to two-directional client-server message-based communication.
You can easily initiate the connection from javascript:
var ws = new WebSocket("ws://your.domain.com/somePathIfYouNeed?args=any");
ws.onmessage = function (evt)
{
var message = evt.data;
//decode message (with JSON or something) and do the needed
};
The sever-side handling depend on your tenchnology stack.
Look into Comet (a spoof on the fact that Ajax is a cleaning agent and so is Comet) which is basically "reverse Ajax." Be aware that this requires a long-lived server connection for each user to receive notifications so be aware of the performance implications when writing your app.
http://en.wikipedia.org/wiki/Comet_(programming)
Comet is definitely what you want. Depending on your language/framework requirements, there are different server libraries available. For example, WebSync is an IIS-integrated comet server for ASP.NET/C#/IIS developers, and there are a bunch of other standalone servers as well if you need tighter integration with other languages.
I would strongly suggest to invest some time on Comet, but I dont know an actual implementation or library you could use.
For an sort of "callcenter control panel" of a web app that involved updating agent and call-queue status for a live Callcenter we developed an in-house solution that works, but is far away from a library you could use.
What we did was to implement a small service on the server that talks to the phone-system, waits for new events and maintains a photograph of the situation. This service provides a small webserver.
Our web-clients connects over HTTP to this webserver and ask for the last photo (coded in XML), displays it and then goes again, asking for the new photo. The webserver at this point can:
Return the new photo, if there is one
Block the client for some seconds (30 in our setup) waiting for some event to ocurr and change the photograph. If no event was generated at that point, it returns the same photo, only to allow the connection to stay alive and not timeout the client.
This way, when clients polls, it get a response in 0 to 30 seconds max. If a new event was already generated it gets it immediately), otherwise it blocks until new event is generated.
It's basically polling, but it somewhat smart polling to not overheat the webserver. If Comet is not your answer, I'm sure this could be implemented using the same idea but using more extensively AJAX or coding in JSON for better results. This was designed pre-AJAX era, so there are lots of room for improvement.
If someone can provide a actual lightweight implementation of this, great!
An interesting alternative to Comet is to use sockets in Flash.
Yet another, standard, way is SSE (Server-Sent Events, also known as EventSource, after the JavaScript object).
Comet was actually coined by Alex Russell from Dojo Toolkit ( http://www.dojotoolkit.org ). Here is a link to more infomration http://cometdproject.dojotoolkit.org/
There are other methods. Not sure if they are "better" in your situation. You could have a Java applet that connects to the server on page load and waits for stuff to be sent by the server. It would be a quite a bit slower on start-up, but would allow the browser to receive data from the server on an infrequent basis, without polling.
You can use a Flash/Flex application on the client with BlazeDS or LiveCycle on the server side. Data can be pushed to the client using an RTMP connection. Be aware that RTMP uses a non standard port. But you can easily fall back to polling if the port is blocked.
It's possible to achive what you're aiming at through the use of persistent http connections.
Check out the Comet article over at wikipedia, that's a good place to start.
You're not providing much info but if you're looking at building some kind of event-driven site (a'la digg spy) or something along the lines of that you'll probably be looking at implementing a hidden IFRAME that connects to a url where the connection never closes and then you'll push script-tags from the server to the client in order to perform the updates.
Might be worth checking out Meteor Server which is a web server designed for COMET. Nice demo and it also is used by twitterfall.
Once a connection is opened to the server it can be kept open and the server can Push content a long while ago I did with using multipart/x-mixed-replace but this didn't work in IE.
I think you can do clever stuff with polling that makes it work more like push by not sending content unchanged headers but leaving the connection open but I've never done this.
You could try out our Comet Component - though it's extremely experimental...!
please check this library https://github.com/SignalR/SignalR to know how to push data to clients dynamically as it becomes available
You can also look into Java Pushlets if you are using jsp pages.
Might want to look at ReverseHTTP also.
There is a server I need to talk to that publishes a protocol over TCP/IP for querying data from a database and listening on a socket to receive notifications when data is updated. The sever guys provide a Java API which uses this TCP protocol. This means I could easily write a Swing App to talk to this server.
I would like a browser based solution. As the protocol is known to me, could I do this in JavaScript? My app will have to display the data in a table. I have heard of Web Sockets but I'm not sure if it will allow this two way communication. Is it feasible? Is there a better way that is cross platform and will work in most browsers? Should I be considering a Java Swing based solution that runs inside a browser?
EDIT: What about changing the code in my C++ server to add an additional interface that my Javascript code can communicate directly with it?
The WebSocket protocol differs from TCP/IP sockets. You will have to write something to link them together.
You can do this perfectly well in JavaScript: use Node.js. There's enough tutorials to be found on the subject. The best way to link it to your in-browser JS is through Socket.IO.
Create a Node.js server that connects to the api
Make the server talk to your web app
Use it :)
This will work cross-platform and cross-browser (Socket.IO can use/emulate websockets even on IE6(!!)). You'll have to run a server-app (the Node.js app) though.
My personal opinion is that if you want a web/browser based solution, you should use native technology, and not Java.
Hope this helps :)
Wanted to get some feedback on this implementation.
I'm developing an application on the PC to send and receive data to the serial port.
Some of the data received by the application will be solicited, while other data unsolicited.
Controlling the serial port and processing messages would be handled by a Python application that would reside between the serial port and the MySQL database. This would be a threaded application with one thread handling sending/receiving using the Queue library and other threads handling logic and the database chores.
They MySQL database would contain tables for storing data received from the serial port, as well as tables of outgoing commands that need to be sent to the serial port. A command sent out may or not be received, so some means of handling retries would be required.
The webapp using HTML, PHP, and javascript would provide the UI. Users can query data and send commands to change parameters, etc. All commands sent out would be written into an outgoing table in the database and picked up by the python app.
My question: Is this a reasonable implementation? Any ideas or thoughts would be appreciated. Thanks.
It seems there's a lot of places for things to go wrong.
Why not just cut out PHP all together and use python?
e.g. Use a python web framework & let your JavaScript communicate with that and while also reading the serial port and logging to MySQL.
That's just me though. I'd try and cut out as many points where it could fail as possible and keep it super simple.
You might also want to check out pySerial (http://pyserial.sourceforge.net/). You might also want to think about you sampling rates, i.e. how much data are you going to be generating and at what frequency. in other words how much data are you planning to store. Will give you some idea of system sizing.