In my nearest future I will have to make a system with C++ backend and web frontend (requirements). At the moment, I don't know much more about it. I think that Frontend will be triggering data delivery, not backend - so no need for Comet-like things.
Because of possibly little experience in this field, I'd really appreciate your comments about design decisions I made.
First of all, I don't like the option of generating HTML from C++.
So, C++ backend will have to communicate with Javascript frontend. Simplest option I see here is Ajax. I think it should be ok, so far.
Commucating through Ajax with C++ backend means that backend should be capable of handling HTTP. It'd be nice to separate backend which provides actual data from HTTP handling functionality.
Here I see the place for Node.js. I got an overview of it and this's the place where all my doubts lie.
To have a HTTP handling server on Node.js, which will have the 'data backend' as a Node.js module? I think, it should be ok - but I'm not sure that I really need all this asynchronization, so there may be some simpler options I'm not aware of? How would you make such a system?
Thanks in advance.
"All this asynchronization" is not something that Node.js works very hard to provide as an extra. It is a different view of Web serving that is easy as breathing once you understand how Node.js works.
For example, my colleagues needed a way to wrap a C++ program as a web service, but the program had a significant start-up cost, so they wanted to run just one instance of the program, running in a loop, serving all the web requests. The whole thing in Node.js took less than two screenfuls.
Wrapping a single program that is called for each request can be done in less than ten lines of Node.js. Don't think of asynchronicity as a chore - if you embrace it, Node.js is awesome.
That said, you could go the CGI route, and do it in a bit more standard way, and the end result would be pretty much the same. This may or may not come in handy.
Did you consider CGI/FCGI module option with nginx, Apache, etc. ?
If not then I think it makes sense to start from it. Your module will handle data/json request and the rest will be handled by HTTP server.
Related
I am a beginner to web development - I am building a website that requires some entries from the user, does some complicated mathematical processing on those entries, and returns the result to the client. I was thinking of implementing the mathematical stuff in another separate application which is better suited for such work (like in Java or C++ where there are good math libraries and the implementation would be more robust and faster). I was wondering what is the best way to do this, architecture-wise.
The "dummy" approach would be spawning a process from the Node.js application and waiting on its output from stdout, parsing it (probably in JSON) and then processing it before sending the result to the client. I have trouble believing that this is the best way to do this (it seems too error-prone, no proper error handling, dependent on the output, and just plain bad practice). A slightly better approach would be to have the Java or C++ application listening on a specific port and waiting for requests from the Node.js application. However this requires more thinking in terms of load-balancing (how would it scale with the number of requests?). Finally, the last approach I found online was to use a queuing system such as RabbitMQ as a way to communicate between the Node.js application and the Java application.
Typically (in a "traditional" software), implementing a separate library that holds all the math magic to which we can make calls would be a good way to go.
What is the best approach to achieve this with a Node.js/web application? There must be good practices/models/architectures/designs for such a problem. Thanks!
The best approach I can think of is writing a native nodejs module, you can use C++ and any library you have to write the code and export some APIs to be called from node's javascript.
This also allows you to package your native application as a module for nodejs, which can then be distributed and compiled/installed with npm.
Here's the official tutorial from node's website.
It isn't the easiest approach, but certainly the more elegant and mantainable one.
IF your problem can be structured in a stateless way -- i.e. you send a request out with the data and the associated task, and then later receive the result as a response, I would probably create one angular app for the front end, and multiple node servers for the back end (IF you needed to handle many requests simultaneously).
As pointed out in the first answer below, you can put whatever parts of the problem that you want to into native code, BUT what might be much easier is to use sockets to connect to a server to do the computation, in any language, etc... In one project I use a socket.service to handle all the communication back and forth to the servers. (The angular-fullstack generator in the next paragraph has such a skeleton socket.service piece.)
In terms of the frontend in combination with a node backend, The Yeoman generator (http://yeoman.io/generators/) for creating MEAN stack applications, using MongoDB, Express, AngularJS, and Node - lets you quickly set up a project following best practices. And the angular-fullstack generator will give you a framework that is light years ahead in so many ways. You can use yeoman to generate a sample application frontend (angular, express) and backend (NodeJs, MongoDB) with support for sockets, etc...
This too is not very simple, but in the end, with a few months of work, you would have a structured problem and solution that you would never outgrow. When I first wanted to work into this space, that was the path I took. And in truth the front-end was/is the harder part and the backend, connected with sockets, http, or whatever has gotten to be much simpler. In terms of servers, AWS, or even just nginx on Linux through DigitalOcean or many others is inexpensive and flexible.
I hope this helps.
The company I'm working at uses Perl for all "backend related" stuff. However, we would like to use some real-time communication between server-processes and connected clients via browser.
We are also using Apache as Webserver with mod.perl. So that is my first question, I don't see any practical way to combine a WebSocket-Server in that constelation. Maybe there is one I didn't found yet?
The only thing which really works serious about that topic, is Mojolicious. However I'm not that experienced with that yet, so I'd be happy if someone could state if I can use that in my current mod-perl environment. I think I also would have to let this run as standalone webserver process, No?
Which brings me to my second question. What is the best practice, if you have multiple perl files, which do certain things running on Apache/modperl, but you want to keep all your connected users informed about things. What I mean is, all these scripts are accessed via XHR, but some actions require other users to get informed. Currently, we do a classic ajax polling.
The problem I'm struggling around with is, that if there is a dedicated websocket server, which runs independently, all those scripts would need to somehow communicate with this process as well right ? How would one do that? Pipes? Sockets? Shared memory ?
Theoretically, if I would choose to go with such an independent ws server solution, I could write it in any language right? Could even be Ruby or Node. I'm just wondering if that is the best way or if there is a good solution which is more integrated in existing perl/modperl constructs.
TL;DR
Is it best practice to have a standalone, independent web-socket server which communicates with the rest of your Apache/modperl scripts aswell as with its connected clients ?
You could look on AnyEvent CPAN module:
http://metacpan.org/pod/AnyEvent
With it you can write your own standalone event-driven WebSocket-server, also you could find a lot of examples in google or in AnyEvent's perldoc.
I am working on app where I need to pass messages between a C++ application and a Javascript web app.
Certainly I could write sockets code myself in either language and I have done this in the past when necessary.
What I would really like is a higher-level message posting or message queueing API that does a lot of the work for me. Does anyone know of such an API?
I have looked at ICE, and it doesn't appear to have Javascript bindings. I have also looked at Boost message queue, but it only caters for the C++ side of things. If necessary I might roll my own Javascript bindings for either of these technologies.
UPDATE: Sorry should have mentioned this before, I want to run this in a browser.
To give a more complete story what I want is a simple browser-based app that is used to configure and display logging for a C++ application.
I know there are other ways of doing this, but I am specifically interested in a high-level library in both C++ and browser-based Javascript that builds a message queue ontop of the sockets API (if there isn't one then I might consider implementing it myself and writing up a code project article).
ALSO: I'm not bothered about portability in terms of the web browser. Eg if there is a high-level IPC Javascript library that only works in Chrome, I'll be happy with that.
With JavaScript I assume that you are running it in a browser? In this case your C++ application needs to provide a webserver and some kind of JSON based webservice that you can call. On the JavaScript side you just use AJAX to communicate with that webservice.
An alternative would be websockets which might be a little harder to implement on the C++ side though.
To simply answer your question: No, there is no IPC implemented in ECMAscript out of the box.
But you actually answered you question already. If you try to communicate with Javascript that runs in a browser, you indeed should use (web-)sockets connections to pipe date in either direction. Of course you could write a simple HTTP server in C++, but I guess that is overkill and does not have the capabilitys of bi-directional sockets.
It's still some work to implement a web-socket connection in C++ from the scratch (the specs were in flux for a long time), but I guess there are some librarys out already.
If you're trying to communicate with node.js, this is an almost trivial task using real sockets/pipes.
I have found a solution that meets my needs. It isn't exactly perfect but I think it works well enough.
Some people suggested using HTTP and ajax. That turned out to be a useful idea and after some prototyping I think it solves my rather basic needs.
To be more specific I am using the Mongoose HTTP server embedded in my C++ application and I am using the jQuery ajax function to pull data from the server. The jQuery client polls the server continously for new data, not particularly efficient but I think it will do the job good enough for me.
Once my implementation is complete I'll write an article explaining how to do this in detail and then I'll update this answer.
You could try DBus, it has very simple mechanism to define, query and use interfaces, and there are some components for XPCOM and webkit based browsers (for example http://sandbox.movial.com/wiki/index.php/Browser_DBus_Bridge and http://code.google.com/p/v8-dbus/). Also DBus is opensource and cross platform.
For a server side or non-browser implementation how about named pipes?
Yes it's vintage technology and the usage depends which OS you use, but as long as your server side js environment has ability to read and write files it may work, and it fits the description 'high-level' inter-process communication.
Could anyone that has used both share his/her experience? What are the main differences and which one do you prefer? Thank you.
Different socket.io vs APE:
socket.io is coded in javascript(node.js) while APE is coded in C. I believe that is a big difference when you want to contribute. Maybe because you like the project or maybe because you want some more functionality. I think it will be easier to contribute Socket.io because you program Javascript, which is easier to grasp according to a lot of people(I Agree, although C is also very cool language).
I believe socket.io supports a lot more browsers/transport compared to APE, but I am not sure. Socket.io information vs information from APE page:
APE Server is an Comet server
implementing the POST and GET methods
of the HTTP protocol. It does not
replace a regular Web Server (such as
Apache, Lighttpd or Nginx), however,
the APE Server is only used for AJAX
Push.
So I guess APE supports less transports then socket.io.
Like Raynos said it is difficult to compare those two products and I believe you should play with them both and then decide which one you like more.
Same Socket.io/APE:
You can both code in Javascript to communicate with the server. I think you will have more freedom using socket.io because everything is exposed via Javascript.
Experience:
I only have experience with socket.io and I like it a lot.
Having spent some time developing with Node.js, I can't say that APE appears to be better than node. Based on sheer popularity, it seems that Node.js is probably the developer's choice -- and node.js appears to be more versatile as well.
Node essentially is you making a full on HTTP or TCP/IP. So, all of the mime type handling, data buffering, response headers, and server side routing are all things you'll have to do with your code. Node does streaming as well. I'm not sure about whether this is considered less problematic than normal ajax long polling at this point.
After googling around, I've found that people consider APE to be more of just a plain ol' push server, in which comet functionality is already there to be consumed, rather than Node, which would have you write your own. Don't be afraid of the prospect of writing things out with node though, they've got a very thorough documentation, and their methods are very easy to learn. I had some serious functionality written out in minutes.
Check this out also: http://groups.google.com/group/nodejs/browse_thread/thread/9d9b301479851b1f?pli=1
I played a little bit with node.js, tried out socket.io - but in the end did a big project with APE.
I think, as always, there is the question of what you want to achieve.
Only comparing the server parts: With node.js you get a machine that won't do anything on it's own, you need to write it yourself (or use libraries)
With APE, the handling of channels and connections is already built in (compiled C). Still you need to build parts of your own logic on top with JS - or use the examples.
On the client side, socket.io provides a client framework with three commands - and APE has it's APE_JSF that handles the connections (which brings more functionality than socket.io regarding channels)
Personally, I prefer APE, even though there is a lack of documentation for beginners.
However, keep in mind that APE won't deliver files/images, it's not a full Web-Server but optimized for real time push where it can handle ~10K concurrent users
The most common theme I read about why to use node.js is for high scalability due to it's evented, non-blocking I/O model. I'm trying to understand other non-scalability uses cases (and aside from being used as a general server-side javascript engine).
Does node.js have other use cases if scalability isn't a concern of mine?
If yes to #1, what are they?
Is node.js usage appropriate for any particular type of application architectures? E.g. similar to how some key/value (nosql - ugh I hate that term) databases are useful other than for scalability reasons.
My reason for trying out node is the fact that it is incredibly easy to send JSON data between the server and the client for ajax requests.
If you use something like MongoDB, which stores data as JSON object too, you never have to worry about translating or parsing your data.
If your site uses a lot of ajax, and you're sending your data as JSON objects (rather than XML or plain text) then node.js will save you a fair bit of effort.
I think this blog posts sums it up quite well:
http://debuggable.com/posts/understanding-node-js:4bd98440-45e4-4a9a-8ef7-0f7ecbdd56cb
In short (pro node.js):
Speed
Javascript (especially if you know it already)
Efficiency
node.js is really great. Give it a try! :)
I can think of one reason, but its not very deep. Basically, If you are developing an RIA, your entire stack can be javascript. That might have some value.
But Ill question my own answer, namely the thought is, even if it makes the server side code more accessible to client side developers, they still need to understand how their server stack works. So there is still some learning to.
To be accurate, I think the general server-side JavaScript engine in your question would be V8, while according to its creator Node was built for "scripting network programs."
Based on many of his comments I don't believe he views it as broadly as many of us do, but recognizes where it can go. [I can't speak for anyone else--that's just my interpretation based on the writings and presentations I've seen.]
So it approaches things from a somewhat lower level, makes HTTP a first-class citizen and just happens to be really cool, which I think makes it enough of a "use case" for most of us. ;)
It does have a learning curve and isn't the most stable platform to build on due to its rapid development. I believe time will tell where it's most useful.
For now people are using it for "real-time" apps due to its lightweight, asynchronous nature, as well as general Web development, though IMO its sweet spot remains with its originally stated purpose.
What I like about node.js aside from non-blocking I/O model, scalability and all that "primary reasons" stuff:
Lightweight nature of it's framework. Basics are easy to learn.
Developer community building tons of useful modules and libraries on github which are expanding node.js lightweight core and it's capabilities.
It's really easy and fast to build server side and real-time systems (for example http or socket based) without the knowledge of complex libraries.
I like to use NodeJS to write testing harness because you can write a stub/server/client really quickly. And you can drive your application, with ease. I can easily script a third party back-end server to do performance testing on my application. I also use it to drive my application. I can perform complex client server scenarios using setTimout to cause multiple events to be trigger based on any logic I want and test them at scale.