Related
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I am new to this kind of stuff, but lately I've been hearing a lot about how good Node.js is. Considering how much I love working with jQuery and JavaScript in general, I can't help but wonder how to decide when to use Node.js. The web application I have in mind is something like Bitly - takes some content, archives it.
From all the homework I have been doing in the last few days, I obtained the following information. Node.js
is a command-line tool that can be run as a regular web server and lets one run JavaScript programs
utilizes the great V8 JavaScript engine
is very good when you need to do several things at the same time
is event-based so all the wonderful Ajax-like stuff can be done on the server side
lets us share code between the browser and the backend
lets us talk with MySQL
Some of the sources that I have come across are:
Diving into Node.js – Introduction and Installation
Understanding NodeJS
Node by Example (Archive.is)
Let’s Make a Web App: NodePad
Considering that Node.js can be run almost out-of-the-box on Amazon's EC2 instances, I am trying to understand what type of problems require Node.js as opposed to any of the mighty kings out there like PHP, Python and Ruby. I understand that it really depends on the expertise one has on a language, but my question falls more into the general category of: When to use a particular framework and what type of problems is it particularly suited for?
You did a great job of summarizing what's awesome about Node.js. My feeling is that Node.js is especially suited for applications where you'd like to maintain a persistent connection from the browser back to the server. Using a technique known as "long-polling", you can write an application that sends updates to the user in real time. Doing long polling on many of the web's giants, like Ruby on Rails or Django, would create immense load on the server, because each active client eats up one server process. This situation amounts to a tarpit attack. When you use something like Node.js, the server has no need of maintaining separate threads for each open connection.
This means you can create a browser-based chat application in Node.js that takes almost no system resources to serve a great many clients. Any time you want to do this sort of long-polling, Node.js is a great option.
It's worth mentioning that Ruby and Python both have tools to do this sort of thing (eventmachine and twisted, respectively), but that Node.js does it exceptionally well, and from the ground up. JavaScript is exceptionally well situated to a callback-based concurrency model, and it excels here. Also, being able to serialize and deserialize with JSON native to both the client and the server is pretty nifty.
I look forward to reading other answers here, this is a fantastic question.
It's worth pointing out that Node.js is also great for situations in which you'll be reusing a lot of code across the client/server gap. The Meteor framework makes this really easy, and a lot of folks are suggesting this might be the future of web development. I can say from experience that it's a whole lot of fun to write code in Meteor, and a big part of this is spending less time thinking about how you're going to restructure your data, so the code that runs in the browser can easily manipulate it and pass it back.
Here's an article on Pyramid and long-polling, which turns out to be very easy to set up with a little help from gevent: TicTacToe and Long Polling with Pyramid.
I believe Node.js is best suited for real-time applications: online games, collaboration tools, chat rooms, or anything where what one user (or robot? or sensor?) does with the application needs to be seen by other users immediately, without a page refresh.
I should also mention that Socket.IO in combination with Node.js will reduce your real-time latency even further than what is possible with long polling. Socket.IO will fall back to long polling as a worst case scenario, and instead use web sockets or even Flash if they are available.
But I should also mention that just about any situation where the code might block due to threads can be better addressed with Node.js. Or any situation where you need the application to be event-driven.
Also, Ryan Dahl said in a talk that I once attended that the Node.js benchmarks closely rival Nginx for regular old HTTP requests. So if we build with Node.js, we can serve our normal resources quite effectively, and when we need the event-driven stuff, it's ready to handle it.
Plus it's all JavaScript all the time. Lingua Franca on the whole stack.
Reasons to use NodeJS:
It runs Javascript, so you can use the same language on server and client, and even share some code between them (e.g. for form validation, or to render views at either end.)
The single-threaded event-driven system is fast even when handling lots of requests at once, and also simple, compared to traditional multi-threaded Java or ROR frameworks.
The ever-growing pool of packages accessible through NPM, including client and server-side libraries/modules, as well as command-line tools for web development. Most of these are conveniently hosted on github, where sometimes you can report an issue and find it fixed within hours! It's nice to have everything under one roof, with standardized issue reporting and easy forking.
It has become the defacto standard environment in which to run Javascript-related tools and other web-related tools, including task runners, minifiers, beautifiers, linters, preprocessors, bundlers and analytics processors.
It seems quite suitable for prototyping, agile development and rapid product iteration.
Reasons not to use NodeJS:
It runs Javascript, which has no compile-time type checking. For large, complex safety-critical systems, or projects including collaboration between different organizations, a language which encourages contractual interfaces and provides static type checking may save you some debugging time (and explosions) in the long run. (Although the JVM is stuck with null, so please use Haskell for your nuclear reactors.)
Added to that, many of the packages in NPM are a little raw, and still under rapid development. Some libraries for older frameworks have undergone a decade of testing and bugfixing, and are very stable by now. Npmjs.org has no mechanism to rate packages, which has lead to a proliferation of packages doing more or less the same thing, out of which a large percentage are no longer maintained.
Nested callback hell. (Of course there are 20 different solutions to this...)
The ever-growing pool of packages can make one NodeJS project appear radically different from the next. There is a large diversity in implementations due to the huge number of options available (e.g. Express/Sails.js/Meteor/Derby). This can sometimes make it harder for a new developer to jump in on a Node project. Contrast that with a Rails developer joining an existing project: he should be able to get familiar with the app pretty quickly, because all Rails apps are encouraged to use a similar structure.
Dealing with files can be a bit of a pain. Things that are trivial in other languages, like reading a line from a text file, are weird enough to do with Node.js that there's a StackOverflow question on that with 80+ upvotes. There's no simple way to read one record at a time from a CSV file. Etc.
I love NodeJS, it is fast and wild and fun, but I am concerned it has little interest in provable-correctness. Let's hope we can eventually merge the best of both worlds. I am eager to see what will replace Node in the future... :)
To make it short:
Node.js is well suited for applications that have a lot of concurrent connections and each request only needs very few CPU cycles, because the event loop (with all the other clients) is blocked during execution of a function.
A good article about the event loop in Node.js is Mixu's tech blog: Understanding the node.js event loop.
I have one real-world example where I have used Node.js. The company where I work got one client who wanted to have a simple static HTML website. This website is for selling one item using PayPal and the client also wanted to have a counter which shows the amount of sold items. Client expected to have huge amount of visitors to this website. I decided to make the counter using Node.js and the Express.js framework.
The Node.js application was simple. Get the sold items amount from a Redis database, increase the counter when item is sold and serve the counter value to users via the API.
Some reasons why I chose to use Node.js in this case
It is very lightweight and fast. There has been over 200000 visits on this website in three weeks and minimal server resources has been able to handle it all.
The counter is really easy to make to be real time.
Node.js was easy to configure.
There are lots of modules available for free. For example, I found a Node.js module for PayPal.
In this case, Node.js was an awesome choice.
The most important reasons to start your next project using Node ...
All the coolest dudes are into it ... so it must be fun.
You can hangout at the cooler and have lots of Node adventures to brag about.
You're a penny pincher when it comes to cloud hosting costs.
Been there done that with Rails
You hate IIS deployments
Your old IT job is getting rather dull and you wish you were in a shiny new Start Up.
What to expect ...
You'll feel safe and secure with Express without all the server bloatware you never needed.
Runs like a rocket and scales well.
You dream it. You installed it. The node package repo npmjs.org is the largest ecosystem of open source libraries in the world.
Your brain will get time warped in the land of nested callbacks ...
... until you learn to keep your Promises.
Sequelize and Passport are your new API friends.
Debugging mostly async code will get umm ... interesting .
Time for all Noders to master Typescript.
Who uses it?
PayPal, Netflix, Walmart, LinkedIn, Groupon, Uber, GoDaddy, Dow Jones
Here's why they switched to Node.
There is nothing like Silver Bullet. Everything comes with some cost associated with it. It is like if you eat oily food, you will compromise your health and healthy food does not come with spices like oily food. It is individual choice whether they want health or spices as in their food.
Same way Node.js consider to be used in specific scenario. If your app does not fit into that scenario you should not consider it for your app development. I am just putting my thought on the same:
When to use Node.JS
If your server side code requires very few cpu cycles. In other world you are doing non blocking operation and does not have heavy algorithm/Job which consumes lots of CPU cycles.
If you are from Javascript back ground and comfortable in writing Single Threaded code just like client side JS.
When NOT to use Node.JS
Your server request is dependent on heavy CPU consuming algorithm/Job.
Scalability Consideration with Node.JS
Node.JS itself does not utilize all core of underlying system and it is single threaded by default, you have to write logic by your own to utilize multi core processor and make it multi threaded.
Node.JS Alternatives
There are other option to use in place of Node.JS however Vert.x seems to be pretty promising and has lots of additional features like polygot and better scalability considerations.
Another great thing that I think no one has mentioned about Node.js is the amazing community, the package management system (npm) and the amount of modules that exist that you can include by simply including them in your package.json file.
My piece: nodejs is great for making real time systems like analytics, chat-apps, apis, ad servers, etc.
Hell, I made my first chat app using nodejs and socket.io under 2 hours and that too during exam
week!
Edit
Its been several years since I have started using nodejs and I have used it in making many different things including static file servers, simple analytics, chat apps and much more.
This is my take on when to use nodejs
When to use
When making system which put emphasis on concurrency and speed.
Sockets only servers like chat apps, irc apps, etc.
Social networks which put emphasis on realtime resources like geolocation, video stream, audio stream, etc.
Handling small chunks of data really fast like an analytics webapp.
As exposing a REST only api.
When not to use
Its a very versatile webserver so you can use it wherever you want but probably not these places.
Simple blogs and static sites.
Just as a static file server.
Keep in mind that I am just nitpicking. For static file servers, apache is better mainly because it is widely available. The nodejs community has grown larger and more mature over the years and it is safe to say nodejs can be used just about everywhere if you have your own choice of hosting.
It can be used where
Applications that are highly event driven & are heavily I/O bound
Applications handling a large number of connections to other systems
Real-time applications (Node.js was designed from the ground up for real time and to be easy
to use.)
Applications that juggle scads of information streaming to and from other sources
High traffic, Scalable applications
Mobile apps that have to talk to platform API & database, without having to do a lot of data
analytics
Build out networked applications
Applications that need to talk to the back end very often
On Mobile front, prime-time companies have relied on Node.js for their mobile solutions. Check out why?
LinkedIn is a prominent user. Their entire mobile stack is built on Node.js. They went from running 15 servers with 15 instances on each physical machine, to just 4 instances – that can handle double the traffic!
eBay launched ql.io, a web query language for HTTP APIs, which uses Node.js as the runtime stack. They were able to tune a regular developer-quality Ubuntu workstation to handle more than 120,000 active connections per node.js process, with each connection consuming about 2kB memory!
Walmart re-engineered its mobile app to use Node.js and pushed its JavaScript processing to the server.
Read more at: http://www.pixelatingbits.com/a-closer-look-at-mobile-app-development-with-node-js/
Node best for concurrent request handling -
So, Let’s start with a story. From last 2 years I am working on JavaScript and developing web front end and I am enjoying it. Back end guys provide’s us some API’s written in Java,python (we don’t care) and we simply write a AJAX call, get our data and guess what ! we are done. But in real it is not that easy, If data we are getting is not correct or there is some server error then we stuck and we have to contact our back end guys over the mail or chat(sometimes on whatsApp too :).) This is not cool. What if we wrote our API’s in JavaScript and call those API’s from our front end ? Yes that’s pretty cool because if we face any problem in API we can look into it. Guess what ! you can do this now , How ? – Node is there for you.
Ok agreed that you can write your API in JavaScript but what if I am ok with above problem. Do you have any other reason to use node for rest API ?
so here is the magic begins. Yes I do have other reasons to use node for our API’s.
Let’s go back to our traditional rest API system which is based on either blocking operation or threading. Suppose two concurrent request occurs( r1 and r2) , each of them require database operation. So In traditional system what will happens :
1. Waiting Way : Our server starts serving r1 request and waits for query response. after completion of r1 , server starts to serve r2 and does it in same way. So waiting is not a good idea because we don’t have that much time.
2. Threading Way : Our server will creates two threads for both requests r1 and r2 and serve their purpose after querying database so cool its fast.But it is memory consuming because you can see we started two threads also problem increases when both request is querying same data then you have to deal with deadlock kind of issues . So its better than waiting way but still issues are there.
Now here is , how node will do it:
3. Nodeway : When same concurrent request comes in node then it will register an event with its callback and move ahead it will not wait for query response for a particular request.So when r1 request comes then node’s event loop (yes there is an event loop in node which serves this purpose.) register an event with its callback function and move ahead for serving r2 request and similarly register its event with its callback. Whenever any query finishes it triggers its corresponding event and execute its callback to completion without being interrupted.
So no waiting, no threading , no memory consumption – yes this is nodeway for serving rest API.
My one more reason to choose Node.js for a new project is:
Be able to do pure cloud based development
I have used Cloud9 IDE for a while and now I can't imagine without it, it covers all the development lifecycles. All you need is a browser and you can code anytime anywhere on any devices. You don't need to check in code in one Computer(like at home), then checkout in another computer(like at work place).
Of course, there maybe cloud based IDE for other languages or platforms (Cloud 9 IDE is adding supports for other languages as well), but using Cloud 9 to do Node.js developement is really a great experience for me.
One more thing node provides is the ability to create multiple v8 instanes of node using node's child process( childProcess.fork() each requiring 10mb memory as per docs) on the fly, thus not affecting the main process running the server. So offloading a background job that requires huge server load becomes a child's play and we can easily kill them as and when needed.
I've been using node a lot and in most of the apps we build, require server connections at the same time thus a heavy network traffic. Frameworks like Express.js and the new Koajs (which removed callback hell) have made working on node even more easier.
Donning asbestos longjohns...
Yesterday my title with Packt Publications, Reactive Programming with JavaScript. It isn't really a Node.js-centric title; early chapters are intended to cover theory, and later code-heavy chapters cover practice. Because I didn't really think it would be appropriate to fail to give readers a webserver, Node.js seemed by far the obvious choice. The case was closed before it was even opened.
I could have given a very rosy view of my experience with Node.js. Instead I was honest about good points and bad points I encountered.
Let me include a few quotes that are relevant here:
Warning: Node.js and its ecosystem are hot--hot enough to burn you badly!
When I was a teacher’s assistant in math, one of the non-obvious suggestions I was told was not to tell a student that something was “easy.” The reason was somewhat obvious in retrospect: if you tell people something is easy, someone who doesn’t see a solution may end up feeling (even more) stupid, because not only do they not get how to solve the problem, but the problem they are too stupid to understand is an easy one!
There are gotchas that don’t just annoy people coming from Python / Django, which immediately reloads the source if you change anything. With Node.js, the default behavior is that if you make one change, the old version continues to be active until the end of time or until you manually stop and restart the server. This inappropriate behavior doesn’t just annoy Pythonistas; it also irritates native Node.js users who provide various workarounds. The StackOverflow question “Auto-reload of files in Node.js” has, at the time of this writing, over 200 upvotes and 19 answers; an edit directs the user to a nanny script, node-supervisor, with homepage at http://tinyurl.com/reactjs-node-supervisor. This problem affords new users with great opportunity to feel stupid because they thought they had fixed the problem, but the old, buggy behavior is completely unchanged. And it is easy to forget to bounce the server; I have done so multiple times. And the message I would like to give is, “No, you’re not stupid because this behavior of Node.js bit your back; it’s just that the designers of Node.js saw no reason to provide appropriate behavior here. Do try to cope with it, perhaps taking a little help from node-supervisor or another solution, but please don’t walk away feeling that you’re stupid. You’re not the one with the problem; the problem is in Node.js’s default behavior.”
This section, after some debate, was left in, precisely because I don't want to give an impression of “It’s easy.” I cut my hands repeatedly while getting things to work, and I don’t want to smooth over difficulties and set you up to believe that getting Node.js and its ecosystem to function well is a straightforward matter and if it’s not straightforward for you too, you don’t know what you’re doing. If you don’t run into obnoxious difficulties using Node.js, that’s wonderful. If you do, I would hope that you don’t walk away feeling, “I’m stupid—there must be something wrong with me.” You’re not stupid if you experience nasty surprises dealing with Node.js. It’s not you! It’s Node.js and its ecosystem!
The Appendix, which I did not really want after the rising crescendo in the last chapters and the conclusion, talks about what I was able to find in the ecosystem, and provided a workaround for moronic literalism:
Another database that seemed like a perfect fit, and may yet be redeemable, is a server-side implementation of the HTML5 key-value store. This approach has the cardinal advantage of an API that most good front-end developers understand well enough. For that matter, it’s also an API that most not-so-good front-end developers understand well enough. But with the node-localstorage package, while dictionary-syntax access is not offered (you want to use localStorage.setItem(key, value) or localStorage.getItem(key), not localStorage[key]), the full localStorage semantics are implemented, including a default 5MB quota—WHY? Do server-side JavaScript developers need to be protected from themselves?
For client-side database capabilities, a 5MB quota per website is really a generous and useful amount of breathing room to let developers work with it. You could set a much lower quota and still offer developers an immeasurable improvement over limping along with cookie management. A 5MB limit doesn’t lend itself very quickly to Big Data client-side processing, but there is a really quite generous allowance that resourceful developers can use to do a lot. But on the other hand, 5MB is not a particularly large portion of most disks purchased any time recently, meaning that if you and a website disagree about what is reasonable use of disk space, or some site is simply hoggish, it does not really cost you much and you are in no danger of a swamped hard drive unless your hard drive was already too full. Maybe we would be better off if the balance were a little less or a little more, but overall it’s a decent solution to address the intrinsic tension for a client-side context.
However, it might gently be pointed out that when you are the one writing code for your server, you don’t need any additional protection from making your database more than a tolerable 5MB in size. Most developers will neither need nor want tools acting as a nanny and protecting them from storing more than 5MB of server-side data. And the 5MB quota that is a golden balancing act on the client-side is rather a bit silly on a Node.js server. (And, for a database for multiple users such as is covered in this Appendix, it might be pointed out, slightly painfully, that that’s not 5MB per user account unless you create a separate database on disk for each user account; that’s 5MB shared between all user accounts together. That could get painful if you go viral!) The documentation states that the quota is customizable, but an email a week ago to the developer asking how to change the quota is unanswered, as was the StackOverflow question asking the same. The only answer I have been able to find is in the Github CoffeeScript source, where it is listed as an optional second integer argument to a constructor. So that’s easy enough, and you could specify a quota equal to a disk or partition size. But besides porting a feature that does not make sense, the tool’s author has failed completely to follow a very standard convention of interpreting 0 as meaning “unlimited” for a variable or function where an integer is to specify a maximum limit for some resource use. The best thing to do with this misfeature is probably to specify that the quota is Infinity:
if (typeof localStorage === 'undefined' || localStorage === null)
{
var LocalStorage = require('node-localstorage').LocalStorage;
localStorage = new LocalStorage(__dirname + '/localStorage',
Infinity);
}
Swapping two comments in order:
People needlessly shot themselves in the foot constantly using JavaScript as a whole, and part of JavaScript being made respectable language was a Douglas Crockford saying in essence, “JavaScript as a language has some really good parts and some really bad parts. Here are the good parts. Just forget that anything else is there.” Perhaps the hot Node.js ecosystem will grow its own “Douglas Crockford,” who will say, “The Node.js ecosystem is a coding Wild West, but there are some real gems to be found. Here’s a roadmap. Here are the areas to avoid at almost any cost. Here are the areas with some of the richest paydirt to be found in ANY language or environment.”
Perhaps someone else can take those words as a challenge, and follow Crockford’s lead and write up “the good parts” and / or “the better parts” for Node.js and its ecosystem. I’d buy a copy!
And given the degree of enthusiasm and sheer work-hours on all projects, it may be warranted in a year, or two, or three, to sharply temper any remarks about an immature ecosystem made at the time of this writing. It really may make sense in five years to say, “The 2015 Node.js ecosystem had several minefields. The 2020 Node.js ecosystem has multiple paradises.”
If your application mainly tethers web apis, or other io channels, give or take a user interface, node.js may be a fair pick for you, especially if you want to squeeze out the most scalability, or, if your main language in life is javascript (or javascript transpilers of sorts). If you build microservices, node.js is also okay. Node.js is also suitable for any project that is small or simple.
Its main selling point is it allows front-enders take responsibility for back-end stuff rather than the typical divide. Another justifiable selling point is if your workforce is javascript oriented to begin with.
Beyond a certain point however, you cannot scale your code without terrible hacks for forcing modularity, readability and flow control. Some people like those hacks though, especially coming from an event-driven javascript background, they seem familiar or forgivable.
In particular, when your application needs to perform synchronous flows, you start bleeding over half-baked solutions that slow you down considerably in terms of your development process. If you have computation intensive parts in your application, tread with caution picking (only) node.js. Maybe http://koajs.com/ or other novelties alleviate those originally thorny aspects, compared to when I originally used node.js or wrote this.
I can share few points where&why to use node js.
For realtime applications like chat,collaborative editing better we go with nodejs as it is event base where fire event and data to clients from server.
Simple and easy to understand as it is javascript base where most of people have idea.
Most of current web applications going towards angular js&backbone, with node it is easy to interact with client side code as both will use json data.
Lot of plugins available.
Drawbacks:-
Node will support most of databases but best is mongodb which won't support complex joins and others.
Compilation Errors...developer should handle each and every exceptions other wise if any error accord application will stop working where again we need to go and start it manually or using any automation tool.
Conclusion:-
Nodejs best to use for simple and real time applications..if you have very big business logic and complex functionality better should not use nodejs.
If you want to build an application along with chat and any collaborative functionality.. node can be used in specific parts and remain should go with your convenience technology.
Node is great for quick prototypes but I'd never use it again for anything complex.
I spent 20 years developing a relationship with a compiler and I sure miss it.
Node is especially painful for maintaining code that you haven't visited for awhile. Type info and compile time error detection are GOOD THINGS. Why throw all that out? For what? And dang, when something does go south the stack traces quite often completely useless.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I am new to this kind of stuff, but lately I've been hearing a lot about how good Node.js is. Considering how much I love working with jQuery and JavaScript in general, I can't help but wonder how to decide when to use Node.js. The web application I have in mind is something like Bitly - takes some content, archives it.
From all the homework I have been doing in the last few days, I obtained the following information. Node.js
is a command-line tool that can be run as a regular web server and lets one run JavaScript programs
utilizes the great V8 JavaScript engine
is very good when you need to do several things at the same time
is event-based so all the wonderful Ajax-like stuff can be done on the server side
lets us share code between the browser and the backend
lets us talk with MySQL
Some of the sources that I have come across are:
Diving into Node.js – Introduction and Installation
Understanding NodeJS
Node by Example (Archive.is)
Let’s Make a Web App: NodePad
Considering that Node.js can be run almost out-of-the-box on Amazon's EC2 instances, I am trying to understand what type of problems require Node.js as opposed to any of the mighty kings out there like PHP, Python and Ruby. I understand that it really depends on the expertise one has on a language, but my question falls more into the general category of: When to use a particular framework and what type of problems is it particularly suited for?
You did a great job of summarizing what's awesome about Node.js. My feeling is that Node.js is especially suited for applications where you'd like to maintain a persistent connection from the browser back to the server. Using a technique known as "long-polling", you can write an application that sends updates to the user in real time. Doing long polling on many of the web's giants, like Ruby on Rails or Django, would create immense load on the server, because each active client eats up one server process. This situation amounts to a tarpit attack. When you use something like Node.js, the server has no need of maintaining separate threads for each open connection.
This means you can create a browser-based chat application in Node.js that takes almost no system resources to serve a great many clients. Any time you want to do this sort of long-polling, Node.js is a great option.
It's worth mentioning that Ruby and Python both have tools to do this sort of thing (eventmachine and twisted, respectively), but that Node.js does it exceptionally well, and from the ground up. JavaScript is exceptionally well situated to a callback-based concurrency model, and it excels here. Also, being able to serialize and deserialize with JSON native to both the client and the server is pretty nifty.
I look forward to reading other answers here, this is a fantastic question.
It's worth pointing out that Node.js is also great for situations in which you'll be reusing a lot of code across the client/server gap. The Meteor framework makes this really easy, and a lot of folks are suggesting this might be the future of web development. I can say from experience that it's a whole lot of fun to write code in Meteor, and a big part of this is spending less time thinking about how you're going to restructure your data, so the code that runs in the browser can easily manipulate it and pass it back.
Here's an article on Pyramid and long-polling, which turns out to be very easy to set up with a little help from gevent: TicTacToe and Long Polling with Pyramid.
I believe Node.js is best suited for real-time applications: online games, collaboration tools, chat rooms, or anything where what one user (or robot? or sensor?) does with the application needs to be seen by other users immediately, without a page refresh.
I should also mention that Socket.IO in combination with Node.js will reduce your real-time latency even further than what is possible with long polling. Socket.IO will fall back to long polling as a worst case scenario, and instead use web sockets or even Flash if they are available.
But I should also mention that just about any situation where the code might block due to threads can be better addressed with Node.js. Or any situation where you need the application to be event-driven.
Also, Ryan Dahl said in a talk that I once attended that the Node.js benchmarks closely rival Nginx for regular old HTTP requests. So if we build with Node.js, we can serve our normal resources quite effectively, and when we need the event-driven stuff, it's ready to handle it.
Plus it's all JavaScript all the time. Lingua Franca on the whole stack.
Reasons to use NodeJS:
It runs Javascript, so you can use the same language on server and client, and even share some code between them (e.g. for form validation, or to render views at either end.)
The single-threaded event-driven system is fast even when handling lots of requests at once, and also simple, compared to traditional multi-threaded Java or ROR frameworks.
The ever-growing pool of packages accessible through NPM, including client and server-side libraries/modules, as well as command-line tools for web development. Most of these are conveniently hosted on github, where sometimes you can report an issue and find it fixed within hours! It's nice to have everything under one roof, with standardized issue reporting and easy forking.
It has become the defacto standard environment in which to run Javascript-related tools and other web-related tools, including task runners, minifiers, beautifiers, linters, preprocessors, bundlers and analytics processors.
It seems quite suitable for prototyping, agile development and rapid product iteration.
Reasons not to use NodeJS:
It runs Javascript, which has no compile-time type checking. For large, complex safety-critical systems, or projects including collaboration between different organizations, a language which encourages contractual interfaces and provides static type checking may save you some debugging time (and explosions) in the long run. (Although the JVM is stuck with null, so please use Haskell for your nuclear reactors.)
Added to that, many of the packages in NPM are a little raw, and still under rapid development. Some libraries for older frameworks have undergone a decade of testing and bugfixing, and are very stable by now. Npmjs.org has no mechanism to rate packages, which has lead to a proliferation of packages doing more or less the same thing, out of which a large percentage are no longer maintained.
Nested callback hell. (Of course there are 20 different solutions to this...)
The ever-growing pool of packages can make one NodeJS project appear radically different from the next. There is a large diversity in implementations due to the huge number of options available (e.g. Express/Sails.js/Meteor/Derby). This can sometimes make it harder for a new developer to jump in on a Node project. Contrast that with a Rails developer joining an existing project: he should be able to get familiar with the app pretty quickly, because all Rails apps are encouraged to use a similar structure.
Dealing with files can be a bit of a pain. Things that are trivial in other languages, like reading a line from a text file, are weird enough to do with Node.js that there's a StackOverflow question on that with 80+ upvotes. There's no simple way to read one record at a time from a CSV file. Etc.
I love NodeJS, it is fast and wild and fun, but I am concerned it has little interest in provable-correctness. Let's hope we can eventually merge the best of both worlds. I am eager to see what will replace Node in the future... :)
To make it short:
Node.js is well suited for applications that have a lot of concurrent connections and each request only needs very few CPU cycles, because the event loop (with all the other clients) is blocked during execution of a function.
A good article about the event loop in Node.js is Mixu's tech blog: Understanding the node.js event loop.
I have one real-world example where I have used Node.js. The company where I work got one client who wanted to have a simple static HTML website. This website is for selling one item using PayPal and the client also wanted to have a counter which shows the amount of sold items. Client expected to have huge amount of visitors to this website. I decided to make the counter using Node.js and the Express.js framework.
The Node.js application was simple. Get the sold items amount from a Redis database, increase the counter when item is sold and serve the counter value to users via the API.
Some reasons why I chose to use Node.js in this case
It is very lightweight and fast. There has been over 200000 visits on this website in three weeks and minimal server resources has been able to handle it all.
The counter is really easy to make to be real time.
Node.js was easy to configure.
There are lots of modules available for free. For example, I found a Node.js module for PayPal.
In this case, Node.js was an awesome choice.
The most important reasons to start your next project using Node ...
All the coolest dudes are into it ... so it must be fun.
You can hangout at the cooler and have lots of Node adventures to brag about.
You're a penny pincher when it comes to cloud hosting costs.
Been there done that with Rails
You hate IIS deployments
Your old IT job is getting rather dull and you wish you were in a shiny new Start Up.
What to expect ...
You'll feel safe and secure with Express without all the server bloatware you never needed.
Runs like a rocket and scales well.
You dream it. You installed it. The node package repo npmjs.org is the largest ecosystem of open source libraries in the world.
Your brain will get time warped in the land of nested callbacks ...
... until you learn to keep your Promises.
Sequelize and Passport are your new API friends.
Debugging mostly async code will get umm ... interesting .
Time for all Noders to master Typescript.
Who uses it?
PayPal, Netflix, Walmart, LinkedIn, Groupon, Uber, GoDaddy, Dow Jones
Here's why they switched to Node.
There is nothing like Silver Bullet. Everything comes with some cost associated with it. It is like if you eat oily food, you will compromise your health and healthy food does not come with spices like oily food. It is individual choice whether they want health or spices as in their food.
Same way Node.js consider to be used in specific scenario. If your app does not fit into that scenario you should not consider it for your app development. I am just putting my thought on the same:
When to use Node.JS
If your server side code requires very few cpu cycles. In other world you are doing non blocking operation and does not have heavy algorithm/Job which consumes lots of CPU cycles.
If you are from Javascript back ground and comfortable in writing Single Threaded code just like client side JS.
When NOT to use Node.JS
Your server request is dependent on heavy CPU consuming algorithm/Job.
Scalability Consideration with Node.JS
Node.JS itself does not utilize all core of underlying system and it is single threaded by default, you have to write logic by your own to utilize multi core processor and make it multi threaded.
Node.JS Alternatives
There are other option to use in place of Node.JS however Vert.x seems to be pretty promising and has lots of additional features like polygot and better scalability considerations.
Another great thing that I think no one has mentioned about Node.js is the amazing community, the package management system (npm) and the amount of modules that exist that you can include by simply including them in your package.json file.
My piece: nodejs is great for making real time systems like analytics, chat-apps, apis, ad servers, etc.
Hell, I made my first chat app using nodejs and socket.io under 2 hours and that too during exam
week!
Edit
Its been several years since I have started using nodejs and I have used it in making many different things including static file servers, simple analytics, chat apps and much more.
This is my take on when to use nodejs
When to use
When making system which put emphasis on concurrency and speed.
Sockets only servers like chat apps, irc apps, etc.
Social networks which put emphasis on realtime resources like geolocation, video stream, audio stream, etc.
Handling small chunks of data really fast like an analytics webapp.
As exposing a REST only api.
When not to use
Its a very versatile webserver so you can use it wherever you want but probably not these places.
Simple blogs and static sites.
Just as a static file server.
Keep in mind that I am just nitpicking. For static file servers, apache is better mainly because it is widely available. The nodejs community has grown larger and more mature over the years and it is safe to say nodejs can be used just about everywhere if you have your own choice of hosting.
It can be used where
Applications that are highly event driven & are heavily I/O bound
Applications handling a large number of connections to other systems
Real-time applications (Node.js was designed from the ground up for real time and to be easy
to use.)
Applications that juggle scads of information streaming to and from other sources
High traffic, Scalable applications
Mobile apps that have to talk to platform API & database, without having to do a lot of data
analytics
Build out networked applications
Applications that need to talk to the back end very often
On Mobile front, prime-time companies have relied on Node.js for their mobile solutions. Check out why?
LinkedIn is a prominent user. Their entire mobile stack is built on Node.js. They went from running 15 servers with 15 instances on each physical machine, to just 4 instances – that can handle double the traffic!
eBay launched ql.io, a web query language for HTTP APIs, which uses Node.js as the runtime stack. They were able to tune a regular developer-quality Ubuntu workstation to handle more than 120,000 active connections per node.js process, with each connection consuming about 2kB memory!
Walmart re-engineered its mobile app to use Node.js and pushed its JavaScript processing to the server.
Read more at: http://www.pixelatingbits.com/a-closer-look-at-mobile-app-development-with-node-js/
Node best for concurrent request handling -
So, Let’s start with a story. From last 2 years I am working on JavaScript and developing web front end and I am enjoying it. Back end guys provide’s us some API’s written in Java,python (we don’t care) and we simply write a AJAX call, get our data and guess what ! we are done. But in real it is not that easy, If data we are getting is not correct or there is some server error then we stuck and we have to contact our back end guys over the mail or chat(sometimes on whatsApp too :).) This is not cool. What if we wrote our API’s in JavaScript and call those API’s from our front end ? Yes that’s pretty cool because if we face any problem in API we can look into it. Guess what ! you can do this now , How ? – Node is there for you.
Ok agreed that you can write your API in JavaScript but what if I am ok with above problem. Do you have any other reason to use node for rest API ?
so here is the magic begins. Yes I do have other reasons to use node for our API’s.
Let’s go back to our traditional rest API system which is based on either blocking operation or threading. Suppose two concurrent request occurs( r1 and r2) , each of them require database operation. So In traditional system what will happens :
1. Waiting Way : Our server starts serving r1 request and waits for query response. after completion of r1 , server starts to serve r2 and does it in same way. So waiting is not a good idea because we don’t have that much time.
2. Threading Way : Our server will creates two threads for both requests r1 and r2 and serve their purpose after querying database so cool its fast.But it is memory consuming because you can see we started two threads also problem increases when both request is querying same data then you have to deal with deadlock kind of issues . So its better than waiting way but still issues are there.
Now here is , how node will do it:
3. Nodeway : When same concurrent request comes in node then it will register an event with its callback and move ahead it will not wait for query response for a particular request.So when r1 request comes then node’s event loop (yes there is an event loop in node which serves this purpose.) register an event with its callback function and move ahead for serving r2 request and similarly register its event with its callback. Whenever any query finishes it triggers its corresponding event and execute its callback to completion without being interrupted.
So no waiting, no threading , no memory consumption – yes this is nodeway for serving rest API.
My one more reason to choose Node.js for a new project is:
Be able to do pure cloud based development
I have used Cloud9 IDE for a while and now I can't imagine without it, it covers all the development lifecycles. All you need is a browser and you can code anytime anywhere on any devices. You don't need to check in code in one Computer(like at home), then checkout in another computer(like at work place).
Of course, there maybe cloud based IDE for other languages or platforms (Cloud 9 IDE is adding supports for other languages as well), but using Cloud 9 to do Node.js developement is really a great experience for me.
One more thing node provides is the ability to create multiple v8 instanes of node using node's child process( childProcess.fork() each requiring 10mb memory as per docs) on the fly, thus not affecting the main process running the server. So offloading a background job that requires huge server load becomes a child's play and we can easily kill them as and when needed.
I've been using node a lot and in most of the apps we build, require server connections at the same time thus a heavy network traffic. Frameworks like Express.js and the new Koajs (which removed callback hell) have made working on node even more easier.
Donning asbestos longjohns...
Yesterday my title with Packt Publications, Reactive Programming with JavaScript. It isn't really a Node.js-centric title; early chapters are intended to cover theory, and later code-heavy chapters cover practice. Because I didn't really think it would be appropriate to fail to give readers a webserver, Node.js seemed by far the obvious choice. The case was closed before it was even opened.
I could have given a very rosy view of my experience with Node.js. Instead I was honest about good points and bad points I encountered.
Let me include a few quotes that are relevant here:
Warning: Node.js and its ecosystem are hot--hot enough to burn you badly!
When I was a teacher’s assistant in math, one of the non-obvious suggestions I was told was not to tell a student that something was “easy.” The reason was somewhat obvious in retrospect: if you tell people something is easy, someone who doesn’t see a solution may end up feeling (even more) stupid, because not only do they not get how to solve the problem, but the problem they are too stupid to understand is an easy one!
There are gotchas that don’t just annoy people coming from Python / Django, which immediately reloads the source if you change anything. With Node.js, the default behavior is that if you make one change, the old version continues to be active until the end of time or until you manually stop and restart the server. This inappropriate behavior doesn’t just annoy Pythonistas; it also irritates native Node.js users who provide various workarounds. The StackOverflow question “Auto-reload of files in Node.js” has, at the time of this writing, over 200 upvotes and 19 answers; an edit directs the user to a nanny script, node-supervisor, with homepage at http://tinyurl.com/reactjs-node-supervisor. This problem affords new users with great opportunity to feel stupid because they thought they had fixed the problem, but the old, buggy behavior is completely unchanged. And it is easy to forget to bounce the server; I have done so multiple times. And the message I would like to give is, “No, you’re not stupid because this behavior of Node.js bit your back; it’s just that the designers of Node.js saw no reason to provide appropriate behavior here. Do try to cope with it, perhaps taking a little help from node-supervisor or another solution, but please don’t walk away feeling that you’re stupid. You’re not the one with the problem; the problem is in Node.js’s default behavior.”
This section, after some debate, was left in, precisely because I don't want to give an impression of “It’s easy.” I cut my hands repeatedly while getting things to work, and I don’t want to smooth over difficulties and set you up to believe that getting Node.js and its ecosystem to function well is a straightforward matter and if it’s not straightforward for you too, you don’t know what you’re doing. If you don’t run into obnoxious difficulties using Node.js, that’s wonderful. If you do, I would hope that you don’t walk away feeling, “I’m stupid—there must be something wrong with me.” You’re not stupid if you experience nasty surprises dealing with Node.js. It’s not you! It’s Node.js and its ecosystem!
The Appendix, which I did not really want after the rising crescendo in the last chapters and the conclusion, talks about what I was able to find in the ecosystem, and provided a workaround for moronic literalism:
Another database that seemed like a perfect fit, and may yet be redeemable, is a server-side implementation of the HTML5 key-value store. This approach has the cardinal advantage of an API that most good front-end developers understand well enough. For that matter, it’s also an API that most not-so-good front-end developers understand well enough. But with the node-localstorage package, while dictionary-syntax access is not offered (you want to use localStorage.setItem(key, value) or localStorage.getItem(key), not localStorage[key]), the full localStorage semantics are implemented, including a default 5MB quota—WHY? Do server-side JavaScript developers need to be protected from themselves?
For client-side database capabilities, a 5MB quota per website is really a generous and useful amount of breathing room to let developers work with it. You could set a much lower quota and still offer developers an immeasurable improvement over limping along with cookie management. A 5MB limit doesn’t lend itself very quickly to Big Data client-side processing, but there is a really quite generous allowance that resourceful developers can use to do a lot. But on the other hand, 5MB is not a particularly large portion of most disks purchased any time recently, meaning that if you and a website disagree about what is reasonable use of disk space, or some site is simply hoggish, it does not really cost you much and you are in no danger of a swamped hard drive unless your hard drive was already too full. Maybe we would be better off if the balance were a little less or a little more, but overall it’s a decent solution to address the intrinsic tension for a client-side context.
However, it might gently be pointed out that when you are the one writing code for your server, you don’t need any additional protection from making your database more than a tolerable 5MB in size. Most developers will neither need nor want tools acting as a nanny and protecting them from storing more than 5MB of server-side data. And the 5MB quota that is a golden balancing act on the client-side is rather a bit silly on a Node.js server. (And, for a database for multiple users such as is covered in this Appendix, it might be pointed out, slightly painfully, that that’s not 5MB per user account unless you create a separate database on disk for each user account; that’s 5MB shared between all user accounts together. That could get painful if you go viral!) The documentation states that the quota is customizable, but an email a week ago to the developer asking how to change the quota is unanswered, as was the StackOverflow question asking the same. The only answer I have been able to find is in the Github CoffeeScript source, where it is listed as an optional second integer argument to a constructor. So that’s easy enough, and you could specify a quota equal to a disk or partition size. But besides porting a feature that does not make sense, the tool’s author has failed completely to follow a very standard convention of interpreting 0 as meaning “unlimited” for a variable or function where an integer is to specify a maximum limit for some resource use. The best thing to do with this misfeature is probably to specify that the quota is Infinity:
if (typeof localStorage === 'undefined' || localStorage === null)
{
var LocalStorage = require('node-localstorage').LocalStorage;
localStorage = new LocalStorage(__dirname + '/localStorage',
Infinity);
}
Swapping two comments in order:
People needlessly shot themselves in the foot constantly using JavaScript as a whole, and part of JavaScript being made respectable language was a Douglas Crockford saying in essence, “JavaScript as a language has some really good parts and some really bad parts. Here are the good parts. Just forget that anything else is there.” Perhaps the hot Node.js ecosystem will grow its own “Douglas Crockford,” who will say, “The Node.js ecosystem is a coding Wild West, but there are some real gems to be found. Here’s a roadmap. Here are the areas to avoid at almost any cost. Here are the areas with some of the richest paydirt to be found in ANY language or environment.”
Perhaps someone else can take those words as a challenge, and follow Crockford’s lead and write up “the good parts” and / or “the better parts” for Node.js and its ecosystem. I’d buy a copy!
And given the degree of enthusiasm and sheer work-hours on all projects, it may be warranted in a year, or two, or three, to sharply temper any remarks about an immature ecosystem made at the time of this writing. It really may make sense in five years to say, “The 2015 Node.js ecosystem had several minefields. The 2020 Node.js ecosystem has multiple paradises.”
If your application mainly tethers web apis, or other io channels, give or take a user interface, node.js may be a fair pick for you, especially if you want to squeeze out the most scalability, or, if your main language in life is javascript (or javascript transpilers of sorts). If you build microservices, node.js is also okay. Node.js is also suitable for any project that is small or simple.
Its main selling point is it allows front-enders take responsibility for back-end stuff rather than the typical divide. Another justifiable selling point is if your workforce is javascript oriented to begin with.
Beyond a certain point however, you cannot scale your code without terrible hacks for forcing modularity, readability and flow control. Some people like those hacks though, especially coming from an event-driven javascript background, they seem familiar or forgivable.
In particular, when your application needs to perform synchronous flows, you start bleeding over half-baked solutions that slow you down considerably in terms of your development process. If you have computation intensive parts in your application, tread with caution picking (only) node.js. Maybe http://koajs.com/ or other novelties alleviate those originally thorny aspects, compared to when I originally used node.js or wrote this.
I can share few points where&why to use node js.
For realtime applications like chat,collaborative editing better we go with nodejs as it is event base where fire event and data to clients from server.
Simple and easy to understand as it is javascript base where most of people have idea.
Most of current web applications going towards angular js&backbone, with node it is easy to interact with client side code as both will use json data.
Lot of plugins available.
Drawbacks:-
Node will support most of databases but best is mongodb which won't support complex joins and others.
Compilation Errors...developer should handle each and every exceptions other wise if any error accord application will stop working where again we need to go and start it manually or using any automation tool.
Conclusion:-
Nodejs best to use for simple and real time applications..if you have very big business logic and complex functionality better should not use nodejs.
If you want to build an application along with chat and any collaborative functionality.. node can be used in specific parts and remain should go with your convenience technology.
Node is great for quick prototypes but I'd never use it again for anything complex.
I spent 20 years developing a relationship with a compiler and I sure miss it.
Node is especially painful for maintaining code that you haven't visited for awhile. Type info and compile time error detection are GOOD THINGS. Why throw all that out? For what? And dang, when something does go south the stack traces quite often completely useless.
The most common theme I read about why to use node.js is for high scalability due to it's evented, non-blocking I/O model. I'm trying to understand other non-scalability uses cases (and aside from being used as a general server-side javascript engine).
Does node.js have other use cases if scalability isn't a concern of mine?
If yes to #1, what are they?
Is node.js usage appropriate for any particular type of application architectures? E.g. similar to how some key/value (nosql - ugh I hate that term) databases are useful other than for scalability reasons.
My reason for trying out node is the fact that it is incredibly easy to send JSON data between the server and the client for ajax requests.
If you use something like MongoDB, which stores data as JSON object too, you never have to worry about translating or parsing your data.
If your site uses a lot of ajax, and you're sending your data as JSON objects (rather than XML or plain text) then node.js will save you a fair bit of effort.
I think this blog posts sums it up quite well:
http://debuggable.com/posts/understanding-node-js:4bd98440-45e4-4a9a-8ef7-0f7ecbdd56cb
In short (pro node.js):
Speed
Javascript (especially if you know it already)
Efficiency
node.js is really great. Give it a try! :)
I can think of one reason, but its not very deep. Basically, If you are developing an RIA, your entire stack can be javascript. That might have some value.
But Ill question my own answer, namely the thought is, even if it makes the server side code more accessible to client side developers, they still need to understand how their server stack works. So there is still some learning to.
To be accurate, I think the general server-side JavaScript engine in your question would be V8, while according to its creator Node was built for "scripting network programs."
Based on many of his comments I don't believe he views it as broadly as many of us do, but recognizes where it can go. [I can't speak for anyone else--that's just my interpretation based on the writings and presentations I've seen.]
So it approaches things from a somewhat lower level, makes HTTP a first-class citizen and just happens to be really cool, which I think makes it enough of a "use case" for most of us. ;)
It does have a learning curve and isn't the most stable platform to build on due to its rapid development. I believe time will tell where it's most useful.
For now people are using it for "real-time" apps due to its lightweight, asynchronous nature, as well as general Web development, though IMO its sweet spot remains with its originally stated purpose.
What I like about node.js aside from non-blocking I/O model, scalability and all that "primary reasons" stuff:
Lightweight nature of it's framework. Basics are easy to learn.
Developer community building tons of useful modules and libraries on github which are expanding node.js lightweight core and it's capabilities.
It's really easy and fast to build server side and real-time systems (for example http or socket based) without the knowledge of complex libraries.
I like to use NodeJS to write testing harness because you can write a stub/server/client really quickly. And you can drive your application, with ease. I can easily script a third party back-end server to do performance testing on my application. I also use it to drive my application. I can perform complex client server scenarios using setTimout to cause multiple events to be trigger based on any logic I want and test them at scale.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I don't fully get what Node.js is all about. Maybe it's because I am mainly a web based business application developer. What is it and what is the use of it?
My understanding so far is that:
The programming model is event driven, especially the way it handles I/O.
It uses JavaScript and the parser is V8.
It can be easily used to create concurrent server applications.
Are my understandings correct? If yes, then what are the benefits of evented I/O, is it just more for the concurrency stuff? Also, is the direction of Node.js to become a framework like, JavaScript based (V8 based) programming model?
I use Node.js at work, and find it to be very powerful. Forced to choose one word to describe Node.js, I'd say "interesting" (which is not a purely positive adjective). The community is vibrant and growing. JavaScript, despite its oddities can be a great language to code in. And you will daily rethink your own understanding of "best practice" and the patterns of well-structured code. There's an enormous energy of ideas flowing into Node.js right now, and working in it exposes you to all this thinking - great mental weightlifting.
Node.js in production is definitely possible, but far from the "turn-key" deployment seemingly promised by the documentation. With Node.js v0.6.x, "cluster" has been integrated into the platform, providing one of the essential building blocks, but my "production.js" script is still ~150 lines of logic to handle stuff like creating the log directory, recycling dead workers, etc. For a "serious" production service, you also need to be prepared to throttle incoming connections and do all the stuff that Apache does for PHP. To be fair, Ruby on Rails has this exact problem. It is solved via two complementary mechanisms: 1) Putting Ruby on Rails/Node.js behind a dedicated webserver (written in C and tested to hell and back) like Nginx (or Apache / Lighttd). The webserver can efficiently serve static content, access logging, rewrite URLs, terminate SSL, enforce access rules, and manage multiple sub-services. For requests that hit the actual node service, the webserver proxies the request through. 2) Using a framework like Unicorn that will manage the worker processes, recycle them periodically, etc. I've yet to find a Node.js serving framework that seems fully baked; it may exist, but I haven't found it yet and still use ~150 lines in my hand-rolled "production.js".
Reading frameworks like Express makes it seem like the standard practice is to just serve everything through one jack-of-all-trades Node.js service ... "app.use(express.static(__dirname + '/public'))". For lower-load services and development, that's probably fine. But as soon as you try to put big time load on your service and have it run 24/7, you'll quickly discover the motivations that push big sites to have well baked, hardened C-code like Nginx fronting their site and handling all of the static content requests (...until you set up a CDN, like Amazon CloudFront)). For a somewhat humorous and unabashedly negative take on this, see this guy.
Node.js is also finding more and more non-service uses. Even if you are using something else to serve web content, you might still use Node.js as a build tool, using npm modules to organize your code, Browserify to stitch it into a single asset, and uglify-js to minify it for deployment. For dealing with the web, JavaScript is a perfect impedance match and frequently that makes it the easiest route of attack. For example, if you want to grovel through a bunch of JSON response payloads, you should use my underscore-CLI module, the utility-belt of structured data.
Pros / Cons:
Pro: For a server guy, writing JavaScript on the backend has been a "gateway drug" to learning modern UI patterns. I no longer dread writing client code.
Pro: Tends to encourage proper error checking (err is returned by virtually all callbacks, nagging the programmer to handle it; also, async.js and other libraries handle the "fail if any of these subtasks fails" paradigm much better than typical synchronous code)
Pro: Some interesting and normally hard tasks become trivial - like getting status on tasks in flight, communicating between workers, or sharing cache state
Pro: Huge community and tons of great libraries based on a solid package manager (npm)
Con: JavaScript has no standard library. You get so used to importing functionality that it feels weird when you use JSON.parse or some other build in method that doesn't require adding an npm module. This means that there are five versions of everything. Even the modules included in the Node.js "core" have five more variants should you be unhappy with the default implementation. This leads to rapid evolution, but also some level of confusion.
Versus a simple one-process-per-request model (LAMP):
Pro: Scalable to thousands of active connections. Very fast and very efficient. For a web fleet, this could mean a 10X reduction in the number of boxes required versus PHP or Ruby
Pro: Writing parallel patterns is easy. Imagine that you need to fetch three (or N) blobs from Memcached. Do this in PHP ... did you just write code the fetches the first blob, then the second, then the third? Wow, that's slow. There's a special PECL module to fix that specific problem for Memcached, but what if you want to fetch some Memcached data in parallel with your database query? In Node.js, because the paradigm is asynchronous, having a web request do multiple things in parallel is very natural.
Con: Asynchronous code is fundamentally more complex than synchronous code, and the up-front learning curve can be hard for developers without a solid understanding of what concurrent execution actually means. Still, it's vastly less difficult than writing any kind of multithreaded code with locking.
Con: If a compute-intensive request runs for, for example, 100 ms, it will stall processing of other requests that are being handled in the same Node.js process ... AKA, cooperative-multitasking. This can be mitigated with the Web Workers pattern (spinning off a subprocess to deal with the expensive task). Alternatively, you could use a large number of Node.js workers and only let each one handle a single request concurrently (still fairly efficient because there is no process recycle).
Con: Running a production system is MUCH more complicated than a CGI model like Apache + PHP, Perl, Ruby, etc. Unhandled exceptions will bring down the entire process, necessitating logic to restart failed workers (see cluster). Modules with buggy native code can hard-crash the process. Whenever a worker dies, any requests it was handling are dropped, so one buggy API can easily degrade service for other cohosted APIs.
Versus writing a "real" service in Java / C# / C (C? really?)
Pro: Doing asynchronous in Node.js is easier than doing thread-safety anywhere else and arguably provides greater benefit. Node.js is by far the least painful asynchronous paradigm I've ever worked in. With good libraries, it is only slightly harder than writing synchronous code.
Pro: No multithreading / locking bugs. True, you invest up front in writing more verbose code that expresses a proper asynchronous workflow with no blocking operations. And you need to write some tests and get the thing to work (it is a scripting language and fat fingering variable names is only caught at unit-test time). BUT, once you get it to work, the surface area for heisenbugs -- strange problems that only manifest once in a million runs -- that surface area is just much much lower. The taxes writing Node.js code are heavily front-loaded into the coding phase. Then you tend to end up with stable code.
Pro: JavaScript is much more lightweight for expressing functionality. It's hard to prove this with words, but JSON, dynamic typing, lambda notation, prototypal inheritance, lightweight modules, whatever ... it just tends to take less code to express the same ideas.
Con: Maybe you really, really like coding services in Java?
For another perspective on JavaScript and Node.js, check out From Java to Node.js, a blog post on a Java developer's impressions and experiences learning Node.js.
Modules
When considering node, keep in mind that your choice of JavaScript libraries will DEFINE your experience. Most people use at least two, an asynchronous pattern helper (Step, Futures, Async), and a JavaScript sugar module (Underscore.js).
Helper / JavaScript Sugar:
Underscore.js - use this. Just do it. It makes your code nice and readable with stuff like _.isString(), and _.isArray(). I'm not really sure how you could write safe code otherwise. Also, for enhanced command-line-fu, check out my own Underscore-CLI.
Asynchronous Pattern Modules:
Step - a very elegant way to express combinations of serial and parallel actions. My personal reccomendation. See my post on what Step code looks like.
Futures - much more flexible (is that really a good thing?) way to express ordering through requirements. Can express things like "start a, b, c in parallel. When A, and B finish, start AB. When A, and C finish, start AC." Such flexibility requires more care to avoid bugs in your workflow (like never calling the callback, or calling it multiple times). See Raynos's post on using futures (this is the post that made me "get" futures).
Async - more traditional library with one method for each pattern. I started with this before my religious conversion to step and subsequent realization that all patterns in Async could be expressed in Step with a single more readable paradigm.
TameJS - Written by OKCupid, it's a precompiler that adds a new language primative "await" for elegantly writing serial and parallel workflows. The pattern looks amazing, but it does require pre-compilation. I'm still making up my mind on this one.
StreamlineJS - competitor to TameJS. I'm leaning toward Tame, but you can make up your own mind.
Or to read all about the asynchronous libraries, see this panel-interview with the authors.
Web Framework:
Express Great Ruby on Rails-esk framework for organizing web sites. It uses JADE as a XML/HTML templating engine, which makes building HTML far less painful, almost elegant even.
jQuery While not technically a node module, jQuery is quickly becoming a de-facto standard for client-side user interface. jQuery provides CSS-like selectors to 'query' for sets of DOM elements that can then be operated on (set handlers, properties, styles, etc). Along the same vein, Twitter's Bootstrap CSS framework, Backbone.js for an MVC pattern, and Browserify.js to stitch all your JavaScript files into a single file. These modules are all becoming de-facto standards so you should at least check them out if you haven't heard of them.
Testing:
JSHint - Must use; I didn't use this at first which now seems incomprehensible. JSLint adds back a bunch of the basic verifications you get with a compiled language like Java. Mismatched parenthesis, undeclared variables, typeos of many shapes and sizes. You can also turn on various forms of what I call "anal mode" where you verify style of whitespace and whatnot, which is OK if that's your cup of tea -- but the real value comes from getting instant feedback on the exact line number where you forgot a closing ")" ... without having to run your code and hit the offending line. "JSHint" is a more-configurable variant of Douglas Crockford's JSLint.
Mocha competitor to Vows which I'm starting to prefer. Both frameworks handle the basics well enough, but complex patterns tend to be easier to express in Mocha.
Vows Vows is really quite elegant. And it prints out a lovely report (--spec) showing you which test cases passed / failed. Spend 30 minutes learning it, and you can create basic tests for your modules with minimal effort.
Zombie - Headless testing for HTML and JavaScript using JSDom as a virtual "browser". Very powerful stuff. Combine it with Replay to get lightning fast deterministic tests of in-browser code.
A comment on how to "think about" testing:
Testing is non-optional. With a dynamic language like JavaScript, there are very few static checks. For example, passing two parameters to a method that expects 4 won't break until the code is executed. Pretty low bar for creating bugs in JavaScript. Basic tests are essential to making up the verification gap with compiled languages.
Forget validation, just make your code execute. For every method, my first validation case is "nothing breaks", and that's the case that fires most often. Proving that your code runs without throwing catches 80% of the bugs and will do so much to improve your code confidence that you'll find yourself going back and adding the nuanced validation cases you skipped.
Start small and break the inertial barrier. We are all lazy, and pressed for time, and it's easy to see testing as "extra work". So start small. Write test case 0 - load your module and report success. If you force yourself to do just this much, then the inertial barrier to testing is broken. That's <30 min to do it your first time, including reading the documentation. Now write test case 1 - call one of your methods and verify "nothing breaks", that is, that you don't get an error back. Test case 1 should take you less than one minute. With the inertia gone, it becomes easy to incrementally expand your test coverage.
Now evolve your tests with your code. Don't get intimidated by what the "correct" end-to-end test would look like with mock servers and all that. Code starts simple and evolves to handle new cases; tests should too. As you add new cases and new complexity to your code, add test cases to exercise the new code. As you find bugs, add verifications and / or new cases to cover the flawed code. When you are debugging and lose confidence in a piece of code, go back and add tests to prove that it is doing what you think it is. Capture strings of example data (from other services you call, websites you scrape, whatever) and feed them to your parsing code. A few cases here, improved validation there, and you will end up with highly reliable code.
Also, check out the official list of recommended Node.js modules. However, GitHub's Node Modules Wiki is much more complete and a good resource.
To understand Node, it's helpful to consider a few of the key design choices:
Node.js is EVENT BASED and ASYNCHRONOUS / NON-BLOCKING. Events, like an incoming HTTP connection will fire off a JavaScript function that does a little bit of work and kicks off other asynchronous tasks like connecting to a database or pulling content from another server. Once these tasks have been kicked off, the event function finishes and Node.js goes back to sleep. As soon as something else happens, like the database connection being established or the external server responding with content, the callback functions fire, and more JavaScript code executes, potentially kicking off even more asynchronous tasks (like a database query). In this way, Node.js will happily interleave activities for multiple parallel workflows, running whatever activities are unblocked at any point in time. This is why Node.js does such a great job managing thousands of simultaneous connections.
Why not just use one process/thread per connection like everyone else? In Node.js, a new connection is just a very small heap allocation. Spinning up a new process takes significantly more memory, a megabyte on some platforms. But the real cost is the overhead associated with context-switching. When you have 10^6 kernel threads, the kernel has to do a lot of work figuring out who should execute next. A bunch of work has gone into building an O(1) scheduler for Linux, but in the end, it's just way way more efficient to have a single event-driven process than 10^6 processes competing for CPU time. Also, under overload conditions, the multi-process model behaves very poorly, starving critical administration and management services, especially SSHD (meaning you can't even log into the box to figure out how screwed it really is).
Node.js is SINGLE THREADED and LOCK FREE. Node.js, as a very deliberate design choice only has a single thread per process. Because of this, it's fundamentally impossible for multiple threads to access data simultaneously. Thus, no locks are needed. Threads are hard. Really really hard. If you don't believe that, you haven't done enough threaded programming. Getting locking right is hard and results in bugs that are really hard to track down. Eliminating locks and multi-threading makes one of the nastiest classes of bugs just go away. This might be the single biggest advantage of node.
But how do I take advantage of my 16 core box?
Two ways:
For big heavy compute tasks like image encoding, Node.js can fire up child processes or send messages to additional worker processes. In this design, you'd have one thread managing the flow of events and N processes doing heavy compute tasks and chewing up the other 15 CPUs.
For scaling throughput on a webservice, you should run multiple Node.js servers on one box, one per core, using cluster (With Node.js v0.6.x, the official "cluster" module linked here replaces the learnboost version which has a different API). These local Node.js servers can then compete on a socket to accept new connections, balancing load across them. Once a connection is accepted, it becomes tightly bound to a single one of these shared processes. In theory, this sounds bad, but in practice it works quite well and allows you to avoid the headache of writing thread-safe code. Also, this means that Node.js gets excellent CPU cache affinity, more effectively using memory bandwidth.
Node.js lets you do some really powerful things without breaking a sweat. Suppose you have a Node.js program that does a variety of tasks, listens on a TCP port for commands, encodes some images, whatever. With five lines of code, you can add in an HTTP based web management portal that shows the current status of active tasks. This is EASY to do:
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end(myJavascriptObject.getSomeStatusInfo());
}).listen(1337, "127.0.0.1");
Now you can hit a URL and check the status of your running process. Add a few buttons, and you have a "management portal". If you have a running Perl / Python / Ruby script, just "throwing in a management portal" isn't exactly simple.
But isn't JavaScript slow / bad / evil / spawn-of-the-devil? JavaScript has some weird oddities, but with "the good parts" there's a very powerful language there, and in any case, JavaScript is THE language on the client (browser). JavaScript is here to stay; other languages are targeting it as an IL, and world class talent is competing to produce the most advanced JavaScript engines. Because of JavaScript's role in the browser, an enormous amount of engineering effort is being thrown at making JavaScript blazing fast. V8 is the latest and greatest javascript engine, at least for this month. It blows away the other scripting languages in both efficiency AND stability (looking at you, Ruby). And it's only going to get better with huge teams working on the problem at Microsoft, Google, and Mozilla, competing to build the best JavaScript engine (It's no longer a JavaScript "interpreter" as all the modern engines do tons of JIT compiling under the hood with interpretation only as a fallback for execute-once code). Yeah, we all wish we could fix a few of the odder JavaScript language choices, but it's really not that bad. And the language is so darn flexible that you really aren't coding JavaScript, you are coding Step or jQuery -- more than any other language, in JavaScript, the libraries define the experience. To build web applications, you pretty much have to know JavaScript anyway, so coding with it on the server has a sort of skill-set synergy. It has made me not dread writing client code.
Besides, if you REALLY hate JavaScript, you can use syntactic sugar like CoffeeScript. Or anything else that creates JavaScript code, like Google Web Toolkit (GWT).
Speaking of JavaScript, what's a "closure"? - Pretty much a fancy way of saying that you retain lexically scoped variables across call chains. ;) Like this:
var myData = "foo";
database.connect( 'user:pass', function myCallback( result ) {
database.query("SELECT * from Foo where id = " + myData);
} );
// Note that doSomethingElse() executes _BEFORE_ "database.query" which is inside a callback
doSomethingElse();
See how you can just use "myData" without doing anything awkward like stashing it into an object? And unlike in Java, the "myData" variable doesn't have to be read-only. This powerful language feature makes asynchronous-programming much less verbose and less painful.
Writing asynchronous code is always going to be more complex than writing a simple single-threaded script, but with Node.js, it's not that much harder and you get a lot of benefits in addition to the efficiency and scalability to thousands of concurrent connections...
Locked. There are disputes about this answer’s content being resolved at this time. It is not currently accepting new interactions.
I think the advantages are:
Web development in a dynamic language (JavaScript) on a VM that is incredibly fast (V8). It is much faster than Ruby, Python, or Perl.
Ability to handle thousands of concurrent connections with minimal overhead on a single process.
JavaScript is perfect for event loops with first class function objects and closures. People already know how to use it this way having used it in the browser to respond to user initiated events.
A lot of people already know JavaScript, even people who do not claim to be programmers. It is arguably the most popular programming language.
Using JavaScript on a web server as well as the browser reduces the impedance mismatch between the two programming environments which can communicate data structures via JSON that work the same on both sides of the equation. Duplicate form validation code can be shared between server and client, etc.
V8 is an implementation of JavaScript. It lets you run standalone JavaScript applications (among other things).
Node.js is simply a library written for V8 which does evented I/O. This concept is a bit trickier to explain, and I'm sure someone will answer with a better explanation than I... The gist is that rather than doing some input or output and waiting for it to happen, you just don't wait for it to finish. So for example, ask for the last edited time of a file:
// Pseudo code
stat( 'somefile' )
That might take a couple of milliseconds, or it might take seconds. With evented I/O you simply fire off the request and instead of waiting around you attach a callback that gets run when the request finishes:
// Pseudo code
stat( 'somefile', function( result ) {
// Use the result here
} );
// ...more code here
This makes it a lot like JavaScript code in the browser (for example, with Ajax style functionality).
For more information, you should check out the article Node.js is genuinely exciting which was my introduction to the library/platform... I found it quite good.
Node.js is an open source command line tool built for the server side JavaScript code. You can download a tarball, compile and install the source. It lets you run JavaScript programs.
The JavaScript is executed by the V8, a JavaScript engine developed by Google which is used in Chrome browser. It uses a JavaScript API to access the network and file system.
It is popular for its performance and the ability to perform parallel operations.
Understanding node.js is the best explanation of node.js I have found so far.
Following are some good articles on the topic.
Learning Server-Side JavaScript with Node.js
This Time, You’ll Learn Node.js
The closures are a way to execute code in the context it was created in.
What this means for concurency is that you can define variables, then initiate a nonblocking I/O function, and send it an anonymous function for its callback.
When the task is complete, the callback function will execute in the context with the variables, this is the closure.
The reason closures are so good for writing applications with nonblocking I/O is that it's very easy to manage the context of functions executing asynchronously.
Two good examples are regarding how you manage templates and use progressive enhancements with it. You just need a few lightweight pieces of JavaScript code to make it work perfectly.
I strongly recommend that you watch and read these articles:
Video glass node
Node.js YUI DOM manipulation
Pick up any language and try to remember how you would manage your HTML file templates and what you had to do to update a single CSS class name in your DOM structure (for instance, a user clicked on a menu item and you want that marked as "selected" and update the content of the page).
With Node.js it is as simple as doing it in client-side JavaScript code. Get your DOM node and apply your CSS class to that. Get your DOM node and innerHTML your content (you will need some additional JavaScript code to do this. Read the article to know more).
Another good example, is that you can make your web page compatible both with JavaScript turned on or off with the same piece of code. Imagine you have a date selection made in JavaScript that would allow your users to pick up any date using a calendar. You can write (or use) the same piece of JavaScript code to make it work with your JavaScript turned ON or OFF.
There is a very good fast food place analogy that best explains the event driven model of Node.js, see the full article, Node.js, Doctor’s Offices and Fast Food Restaurants – Understanding Event-driven Programming
Here is a summary:
If the fast food joint followed a traditional thread-based model, you'd order your food and wait in line until you received it. The person behind you wouldn't be able to order until your order was done. In an event-driven model, you order your food and then get out of line to wait. Everyone else is then free to order.
Node.js is event-driven, but most web servers are thread-based.York explains how Node.js works:
You use your web browser to make a request for "/about.html" on a
Node.js web server.
The Node.js server accepts your request and calls a function to retrieve
that file from disk.
While the Node.js server is waiting for the file to be retrieved, it
services the next web request.
When the file is retrieved, there is a callback function that is
inserted in the Node.js servers queue.
The Node.js server executes that function which in this case would
render the "/about.html" page and send it back to your web browser."
Well, I understand that
Node's goal is to provide an easy way
to build scalable network programs.
Node is similar in design to and influenced by systems like Ruby's Event Machine or Python's Twisted.
Evented I/O for V8 javascript.
For me that means that you were correct in all three assumptions. The library sure looks promising!
Also, do not forget to mention that Google's V8 is VERY fast. It actually converts the JavaScript code to machine code with the matched performance of compiled binary. So along with all the other great things, it's INSANELY fast.
Q: The programming model is event driven, especially the way it handles I/O.
Correct. It uses call-backs, so any request to access the file system would cause a request to be sent to the file system and then Node.js would start processing its next request. It would only worry about the I/O request once it gets a response back from the file system, at which time it will run the callback code. However, it is possible to make synchronous I/O requests (that is, blocking requests). It is up to the developer to choose between asynchronous (callbacks) or synchronous (waiting).
Q: It uses JavaScript and the parser is V8.
Yes
Q: It can be easily used to create concurrent server applications.
Yes, although you'd need to hand-code quite a lot of JavaScript. It might be better to look at a framework, such as http://www.easynodejs.com/ - which comes with full online documentation and a sample application.
Is the use of server side javascript prevalent? Why would one use it as opposed the any other server side scripting? Is there a specific use case(s) that makes it better than other server side languages?
Also, confused on how to get started experimenting with it, I'm on freeBSD, what would I need installed in order to run server side javascript?
It goes like this:
Servers are expensive, but users will give you processing time in their browsers for free. Therefore, server-side code is relatively expensive compared to client-side code on any site big enough to need to run more than one server. However, there are some things you can't leave to the client, like data validation and retrieval. You'd like to do them on the client, because it means faster response times for the users and less server infrastructure for yourself, but security and accessibility concerns mean server-side code is required.
What typically happens is you do both. You write server-side logic because you have to, but you also write the same logic in javascript in hopes of providing faster responses to the user and saving your servers a little extra work in some situations. This is especially effective for validation code; a failed validation check in a browser can save an entire http request/response pair on the server.
Since we're all (mostly) programmers here we should immediately spot the new problem. There's not only the extra work involved in developing two sets of the same logic, but also the work involved in maintaining it, the inevitable bugs resulting from platforms don't match up well, and the bugs introduced as the implementations drift apart over time.
Enter server-side javascript. The idea is you can write code once, so the same code runs on both server and client. This would appear to solve most of the issue: you get the full set of both server and client logic done all at once, there's no drifting, and no double maintenance. It's also nice when your developers only need to know one language for both server and client work.
Unfortunately, in the real world it doesn't work out so well. The problem is four-fold:
The server view of a page is still very different from the client view of a page. The server needs to be able to do things like talk directly to a database that just shouldn't be done from the browser. The browser needs to do things like manipulate a DOM that doesn't match up with the server.
You don't control the javascript engine of the client, meaning there will still be important language differences between your server code and your client code.
The database is normally a bigger bottleneck than the web server, so savings and performance benefit ends up less than expected.
While just about everyone knows a little javascript, not many developers really know and understand javascript well.
These aren't completely unassailable technical problems: you constrain the server-supported language to a sub-set of javascript that's well supported across most browsers, provide an IDE that knows this subset and the server-side extensions, make some rules about page structure to minimize DOM issues, and provide some boiler-plate javascript for inclusion on the client to make the platform a little nicer to use. The result is something like Aptana Studio/Jaxer, or more recently Node.js, which can be pretty nice.
But not perfect. In my opinion, there are just too many pitfalls and little compatibility issues to make this really shine. Ultimately, additional servers are still cheap compared to developer time, and most programmers are able to be much more productive using something other than javascript.
What I'd really like to see is partial server-side javascript. When a page is requested or a form submitted the server platform does request validation in javascript, perhaps as a plugin to the web server that's completely independent from the rest of it, but the response is built using the platform of your choice.
I think a really cool use of server-side Javascript that isn't used nearly often enough is for data validation. With it, you can write one javascript file to validate a form, check it on the client side, then check it again on the server side because we shouldn't trust anything on the client side. It lets you keep your validation rules DRY. Quite handy.
Also see:
Will server-side Javascript take off? Which implementation is most stable?
When and how do you use server side JavaScript?
Javascript is just a language. Because it is just a language, you can use it anywhere you want... in your browser, on the server, embedded in other applications, stand-alone applications, etc.
That being said, I don't know that there is a lot of new development happening with "Server-Side Javascript"
Javascript is a perfectly good language with a self / scheme prototype style base and a C style syntax. There are some problems, see Javascript the Good Parts, but in general it's a first rate language. The problem is that most javascript programmers are terrible programmers because it's very accessible to get started.
One team at google built out Rhino on Rails, which is an MVC framework like Ruby on Rails which is written in javascript and runs on Rhino a javascript interpreter for the Java VM. In this case they had a requirement to use the Java VM, but wanted to get a language which was fast (javascript is fast), supported duck typing, and was flexible.
Another example is something like CouchDB, a document oriented database which uses json as it's transport format and javascript as it's query & index language. They wanted the database to be as web native as possible.
Javascript is good at string and dom (xml) manipulation, being sandboxed, networking, extending itself, etc... Those kind of features are the thing you often do when developing web applications.
All that said, i don't actually develop server side javascript. It's not a bad idea, but definitely less common.
We use javascript on the client because it is there, not because from a list of languages it was our choice. I wouldn't choose it for any chore on the server.
You can run any language you like on the server, in fact, as many as you like.
javascript is reliable and easy to use, but it is just too labor intensive for common tasks on the server.
I have used both Javascript (NodeJS) and compiled languages (such as Java or C#.NET). There are huge discussions on the internet about which is preferable. This question is very old (2009). Since then the Javascript world has changed considerably, whereas honestly the Java world has not changed much (relatively).
There have been huge advancements in the Javascript world with Typescript, amazing frameworks (such as Next.JS), reactive programming, functional programming, GraphQL, SSR etc.
When I look at compiled code, especially Java code it all still seems to be the same old tools - Spring (maybe SpringBoot) and Jackson. .NET has advanced server side, but not to the extent of the JS world.
Sure my list there can be used with several languages, but I believe Javascript has improved the software engineering world considerably.
Server side development with Javascript, Typescript and NodeJs is engaging, fun and productive. Use it and enjoy it. Like millions are today.