I have a web service handling http requests to redirect to specific URLs. Right the CPU is hammered at about 5 million hits per day, but I need to scale it up to handle 20 million plus. This is a production environment so I am a little apprehensive about the new Node Cluster method b/c it is still listed as experimental. I need suggestions on how to cluster Node on handle the traffic on a linux server. Any thoughts?
5 million per day is equivalent to 57.87 per second, and 25 million is 289.4 per second. These numbers are not too much for a single server for your case. If you only want to redirect specif urls, you can go with another alternatives such as nginx that is more fit for that job. However, if you still want to use NodeJS, I think a modern server can handle that load. Look at my blog post as an example of how to use clustering: NodeJS: Simple Clustering Benchmark. If you want to use all of your cores, you should use clustering.
Related
We have built a webapp using Meteor.JS and we would like to find out
how will it perform in real production mode when thousands/millions of users can log request/response.
How efficient will be the framework when it comes to volume and response time.
We would like to know if there is any tools or best practices which we can use.
Thank you for your time in advance.
What you need to do is look at load testing your app. There are many ways to do this, but the premise is that you script what a user interaction consists of and you run that script multiple times, ramping up the volume until you get to the concurrency numbers you are looking for.
As mentioned, there are LOTS of tools to do this. Some I have personally used:
Loadrunner
Apache jMeter
Rational Performance Tester
Simple shell scripts with curl
There is a list on Wikipedia that would give you a good start on what other tools may be available.
The next point I'd like to make is that load testing is not just ramping up the concurrent users until the system breaks. A good load test will attempt to mimic the type of traffic you see or expect to see. To say it another way, two users do not do the exact same thing each time they hit your system, and not all requests to your server produce the same load on your system. Your users may even use the system differently at different times of the day.
The ultimate goal would be to have several "types" of users and transactions scripted, and have your load tool of choice weight the requests such that they match the expected percentages for each type. This way you can better see how your application will perform over time with loads that match what you really expect to see.
Finally, you need to consider your infrastructure. Typically, in the Enterprise environments I have worked in, we try to make our QA environment as close to Production as possible. We would have the exact same server hardware, exact same configuration (both physical and software), and attempt everything possible to make it mirror. Where we sometimes deviate is the size. For instance, I worked in on environment where we had 4 app servers in Prod, each with 4 nodes, for a total of 16 nodes in a cluster. In QA, we had the same bare metal hardware, but everything was cut in half, so we had 2 physical servers, each with 4 nodes totally 8 nodes in the QA cluster. When we tested for load, we would then half the expected total, so if we expected 200 users concurrently in Prod, a successful QA test would be 100 users concurrently. We would also copy and obfuscate data from the Production databases to the QA databases to make sure we are operating with the same sized data.
The process for load testing is this:
Grab a "baseline" result
Make whatever changes you need to make
Rerun the EXACT same tests as #1, only adding any new functionality to your tests
The baseline is important when making changes over time to ensure that your change doesn't effect performance. I can't tell you how many times I have seen SQL statements that ran perfectly fine in Dev that completely took down a system because of differences in the amount of data.
I am in the middle of creating a node.js project however I am concerned as to the performance of the website once it is up and running.
I am expecting it to have a surge of maybe 2000 users for 4-5 hours over a period of one night per week.
The issue is that each user could be receiving a very small message once every second. i.e a timer adjustment or a price change.
2000*60 = 120000 messages in total per minute.
Would this be possible it would be extremely important that there was minimum lag, less than 1 second if possible?
Thanks for the help
You can certainly scale the users with socket.io but the question is how you want to do that. It is unclear whether you have considered the cluster module as that will significantly take the load of the single node process for that amount of users and reduce the latency time. Of course when you do this you need to stop using the in-memory store that socket.io uses by default and use something like redis instead so that you don't end up with duplicate authentication handshakes. Socket.io even has a document to explain how to do this.
What you really need to do is test the performance of your application by creating 2000 clients and simulating 2000 users. The socket.io client can let you set this up in a node application (and then perhaps sign up for the free tier of an EC2 machine to run them all).
It's worth noting that I haven't actually seen v1.0 benchmarks and really the socket.io developers should have a page on it dedicated to benchmarks as this is always a common question with developers.
I am creating a web application using Node.js, socket.io & express modules. and I would like to find out concerning peak loads on server. Every user make 5-7 request per second.
Server:
RAM 2GB
CPU 2x2 GHz
How much connections can this server process?
Is it necessary to use Web Workers?
What recommendations can give concerning high load or maybe you know some statistic information.
This is almost impossible to answer question since no one knows what/how you are going to develop. In case you need a general number that you can compare to your system and case, you can check this link for single node process, cluster and multithreaded benchmarks for node and jxcore distro.
After having learnt node, javascript and all the rest the hard way, I am finally about to release my first web app.
So I subscribed to Amazon Web Services and created a micro instance, planning on the first year free tier to allow me to make the app available to the world.
My concern is more about hidden costs. I know that with the free tier comes 1 million I/O requests per month for the Amazon EC2 EBS.
Thing is, I started testing my app one the ec2 instance to check that everything was running fine; and I am already at more than 100, 000 I/O requests. And I have basically been the only one using it so far (37 hours that the instance runs).
So I am quite afraid of what could happen if my app gets some traffic, and I don't want to end up with a huge unexpected bill at the end of the month.
I find it quite surprising, because I mainly serve static stuff, and my server side code consists in :
Receving a search request from a client
1 http request to a website
1 https request to the youtube api
saving the data to a mongoDB
Sending the results to the client
Do you have any advice on how to dramatically reduce my IO?
I do not use any other Amazon services so far, maybe am I missing something?
Or maybe Amazon free tier in not enough in my case, but then what can it be enough for? I mean, my app is really simple after all.
I'd be really glad for any help you could provide me
Thanks!
You did not mention total number of visits to your app. So I am assuming you have fairly less visits.
What are I/O requests ?
A single I/O request is a read/write instruction that reaches the EBS volumes. Beware! Execution of large read/writes is broken into multiple smaller pieces, which is the block size of the volume.
Possible reasons of high I/O:
Your app uses lot of RAM. After you hit the limit, the OS starts swapping memory to and fro from swap area in your disk, constantly.
This is most likely the problem, the mongoDB search. mongoDB searches can be long complex queries internally. From one of the answers to this question, the person was using mySQL and it caused him 1 billion I/O requests in 24 days. So 1 database search can be many I/O requests.
Cache is disabled, or you write/modify lot of files. You mentioned you were testing. Free-teir is just not suitable for developing stuff.
You should read this, in case you want to know what happens after free-tier expires.
I recently ran into a similar situation recording very high I/O request rates for a website with little to no traffic. The culprit seems to be a variation of what #prajwalkman discovered testing Chef deployments on a micro instance.
I am not using Chef, but I have been using boto3, Docker and Git to automatically 'build' test images inside a micro instance. Each time I go through my test script, a new image is built and I was not careful to read the fine print regarding default settings on the VolumeType argument on the boto3 run_instance command. Each test image was being built with the 'standard' volume type which, according to current EBS pricing, bills at the rate of $0.05/million I/Os. Whereas, the 'gp2' general purpose memory has a flat cost of $0.10 per GB per month with no extra charge for I/O.
With a handful of lean docker containers taking up a total of 2GB, on top of the 1.3GB for the amazon-ecs-optimized-ami, my storage is well within the free-tier usage. So, once I fixed the volumetype attribute in the blockdevicemappings settings in my scripts to 'gp2', I no longer had an I/O problem on my server.
Prior to that point, the constant downloading of docker images and git repos produced nearly 10m I/Os in less than a week.
The micro instance and the free tier is meant for testing their offerings, not a free way for you to host your site/web application.
You may have to pay money at the end of the month, but I really doubt if you can get away with paying less by using some other company for hosting. AFAIK AWS really is the rock bottom of the price charts.
As for the IO requests themselves, it's hard to give generic advice. I once was in a situation where my micro instance racked up ridiculous number of IO requests. Turns out testing Chef deployments on EC2 is a bad idea.
I/O Requests have to do with reading and writing blocks to EBS volumes. You can reduce this by using as much in memory caching as possible. Micro instances only have about 613 MB of memory available, so you may not be able to do much here.
Ok, so it seems like I/O requests are related to the EBS volume, and that caching may reduce it.
Something I had not considered though is all the operations I made to get my app running.
I updated the linux image, installed node and npm, several modules, mongodb, ....
This is likely to be the main cause of the I/O.
The number requests hasn't grown much in the last days, where the server stayed mostly idle.
We need to develop a client application that has to update some values (about 10-20 integers) each second, over HTTP protocol (however, the HTTP server is running the same machine, so requesting 'localhost' is fast).
Since the UI must be easily modified, the decision was made to develop simply a HTML website and update the values using xmlHttpRequest (actually, using jQuery).
The problem is that the client is supposed to work continuously a whole year with no restart... This gives 3600 requests per hour, 86 400 per day, finally 30 758 400 requests per year. I'm very afraid how the browser will deal with such amount of requests... Has anyone any experience with "stresstest" of AJAX requests?
Would reloading the page once a day help the browser with cleaning up memory?
First issue: Javascript runtime performance varies from browser to browser. You'd be better off finding a fast, stable browser than worrying about jQuery's AJAX performance. I'd be much more worried about keeping an instance of a browser rendering and executing for a whole year.
Second issue: javascript in the browser isn't a timing-precise language. setInterval and setTimeout are unreliable and do not guarantee timing. jQuery's author wrote this article on Javascript time issues, so Javascript might lead to some problems if you need that really precise timing.
Third issue: if your client needs to run for 365 days without restarting, then aren't the sacrifices you're making by choosing to build an HTML/JS frontend somewhat silly for the loft goal of "easily editable?"
Make sure they're not using IE and you'll be off to a flying start.
xmlHttpRequest is able to handle that much queries (short-polling fw like comet does this), but ... lots of queries may have an impact on the browser's responding capabilities depending on the browser. You can easily scale down the number of queries by a 10-20 factor just by requesting all integers together (using json data structure for example). You may have a look at short-polling as well as it is designed for that kind of purpose.