I am in the middle of creating a node.js project however I am concerned as to the performance of the website once it is up and running.
I am expecting it to have a surge of maybe 2000 users for 4-5 hours over a period of one night per week.
The issue is that each user could be receiving a very small message once every second. i.e a timer adjustment or a price change.
2000*60 = 120000 messages in total per minute.
Would this be possible it would be extremely important that there was minimum lag, less than 1 second if possible?
Thanks for the help
You can certainly scale the users with socket.io but the question is how you want to do that. It is unclear whether you have considered the cluster module as that will significantly take the load of the single node process for that amount of users and reduce the latency time. Of course when you do this you need to stop using the in-memory store that socket.io uses by default and use something like redis instead so that you don't end up with duplicate authentication handshakes. Socket.io even has a document to explain how to do this.
What you really need to do is test the performance of your application by creating 2000 clients and simulating 2000 users. The socket.io client can let you set this up in a node application (and then perhaps sign up for the free tier of an EC2 machine to run them all).
It's worth noting that I haven't actually seen v1.0 benchmarks and really the socket.io developers should have a page on it dedicated to benchmarks as this is always a common question with developers.
Related
So, I have a NodeJS MySql system, which pings the database every 10 Minutes (might set that to one Minute) to not lose the connection, but that is not the code quality I like to write.
So, now I am asking myself, if there is any way to avoid that.
I have no plan on what Pools are, or what they do. I've seen them in a few other posts but not looked in to them, is that the only option?Thanks in advance.
Yes, you should learn about connection pools - they are a good tool for managing a connection (or a bunch of them to support concurrency).
Connections - whether to databases or to any other remote systems - are unreliable. They can come and go, and you must not assume that a connection will live forever. Hosts can sometimes be restarted, network links can go down, or a firewall may choose to terminate your TCP session.
Of course, you could implement a "connection recycler" that maintains a single connection and replaces it with a fresh one whenever it's closed. However, that would be unproductive since a pool already does that - only it typically manages more than 1 instance under the hood. It's a good excercise for learning purposes in any case.
A pool has another advantage - it can scale according to load, creating and destroying connections as needed. This lets you put less load on the database when the application load is low.
As a closing remark, if you use query builders (knex.js), object-relational mappers (sequelize, TypeORM), or other types of tools that abstract database access, they'll typically use a pool under the hood, anyway - so understanding this important layer of infrastructure is beneficial in the long run.
We have built a webapp using Meteor.JS and we would like to find out
how will it perform in real production mode when thousands/millions of users can log request/response.
How efficient will be the framework when it comes to volume and response time.
We would like to know if there is any tools or best practices which we can use.
Thank you for your time in advance.
What you need to do is look at load testing your app. There are many ways to do this, but the premise is that you script what a user interaction consists of and you run that script multiple times, ramping up the volume until you get to the concurrency numbers you are looking for.
As mentioned, there are LOTS of tools to do this. Some I have personally used:
Loadrunner
Apache jMeter
Rational Performance Tester
Simple shell scripts with curl
There is a list on Wikipedia that would give you a good start on what other tools may be available.
The next point I'd like to make is that load testing is not just ramping up the concurrent users until the system breaks. A good load test will attempt to mimic the type of traffic you see or expect to see. To say it another way, two users do not do the exact same thing each time they hit your system, and not all requests to your server produce the same load on your system. Your users may even use the system differently at different times of the day.
The ultimate goal would be to have several "types" of users and transactions scripted, and have your load tool of choice weight the requests such that they match the expected percentages for each type. This way you can better see how your application will perform over time with loads that match what you really expect to see.
Finally, you need to consider your infrastructure. Typically, in the Enterprise environments I have worked in, we try to make our QA environment as close to Production as possible. We would have the exact same server hardware, exact same configuration (both physical and software), and attempt everything possible to make it mirror. Where we sometimes deviate is the size. For instance, I worked in on environment where we had 4 app servers in Prod, each with 4 nodes, for a total of 16 nodes in a cluster. In QA, we had the same bare metal hardware, but everything was cut in half, so we had 2 physical servers, each with 4 nodes totally 8 nodes in the QA cluster. When we tested for load, we would then half the expected total, so if we expected 200 users concurrently in Prod, a successful QA test would be 100 users concurrently. We would also copy and obfuscate data from the Production databases to the QA databases to make sure we are operating with the same sized data.
The process for load testing is this:
Grab a "baseline" result
Make whatever changes you need to make
Rerun the EXACT same tests as #1, only adding any new functionality to your tests
The baseline is important when making changes over time to ensure that your change doesn't effect performance. I can't tell you how many times I have seen SQL statements that ran perfectly fine in Dev that completely took down a system because of differences in the amount of data.
I want to use cron module (later, node-cron etc.) in my node server to schedule jobs. The jobs are notifications (eg. email) sent to a user to update his profile picture if he hasen't done so after 1 hour of signup. I am using later module to schedule tasks when user signs up, to be executed 1 hour after to check whether he has uploaded profile picture, and send notification if not. My questions are:
1. Will having a large number of scheduled jobs affect server performance?
2. What is the best way to schedule large number of jobs without affecting the system performance?
It's hard to say whether a large number will affect system performance, since it depends an aweful lot on what they're doing and how they're doing it, as well as how many end up hitting at one time, etc.
That said, my opinion is that using this sort of thing for managing the sorts of things you're describing will severely limit your scalability, as each one is scoped to the application instance (so if you want to scale horizontally, each manages they're own; if one crashes w/o sending notifications out, what happens? etc).
I think you're far better off using some kind of distributed task manager, based perhaps on a Redis queue or similar. Your web app posts jobs to that, and a separate process or processes can handle running tasks when they expire. There's a few modules for this sort of thing, kue is the one I've played with.
The basics
Right now a few of my friends and I are trying to develope a browser game made in nodejs. It's a multiplayer top-down shooter, and most of both the client-side and server-side code is in javascript. We have a good general direction that we'd like to go in, and we're having a lot of fun developing the game. One of our goals when making this game was to make it as hard as possible to cheat. Do do that, we have all of the game logic handled server-side. The client only sends their input the the server via web socket, and the server updates the client (also web socket) with what is happening in the game. Here's the start of our problem.
All of the server side math is getting pretty hefty, and we're finding that we need to scale in some way to handle anything more than 10 players (we want to be able to host many more). At first we had figured that we could just scale vertically as we needed to, but since nodejs is single threaded, is can only take advantage of one core. This means that getting a beefier server won't help that problem. Our only solution is to scale horizontally.
Why we're asking here
We haven't been able to find any good examples of how to scale out a nodejs game. Our use case is pretty particular, and while we've done our best to do this by ourselves, we could really benefit from outside opinions and advice
Details
We've already put a LOT of thought into how to solve this problem. We've been working on it for over a week. Here's what we have put together so far:
Four types of servers
We're splitting tasks into 4 different 'types' of servers. Each one will have a specific task it completes.
The proxy server
The proxy server would sit at the front of the entire stack, and be the only server directly accessible from the internet (there could potentially be more of these). It would have haproxy on it, and it would route all connections to the web servers. We chose haproxy because of its rich feature set, reliability, and nearly unbeatable speed.
The web server
The web server would receive the web-requests, and serve all web-pages. They would also handle lobby creation/management and game creation/management. To do this, they would tell the game servers what lobbies it has, what users are in that lobby, and info about the game they're going to play. The web servers would then update the game servers about user input, and the game server would update the web servers (who would then update the clients) of what's happening in the game. The web servers would use TCP sockets to communicate with the game servers about any type of management, and they would use UDP sockets when communicating about game updates. This would all be done with nodejs.
The game server
The game server would handle all the game math and variable updates about the game. The game servers also communicate with the db servers to record cool stats about players in game. This would be done with nodejs.
The db server
The db server would host the database. This part actually turned out to be the easiest since we found rethinkdb, the coolest db ever. This scales easily, and oddly enough, turned out to be the easiest part of scaling our application.
Some other details
If you're having trouble getting your head around our whole getup, look at this, it's a semi-accurate chart of how we think we'll scale.
If you're just curious, or think it might be helpful to look at our game, it's currently hosted in it's un-scaled state here.
Some things we don't want
We don't want to use the cluster module of nodejs. It isn't stable (said here), and it doesn't scale to other servers, only other processors. We'd like to just take the leap to horizontal scaling.
Our question, summed up
We hope we're going in the right direction, and we've done our homework, but we're not certain. We could certainly take a few tips on how to do this the right way.
Thanks
I realize that this is a pretty long question, and making a well thought out answer will not be easy, but I would really appreciate it.
Thanks!!
Following my spontaneous thoughts on your case:
Multicore usage
node.js can scale with multiple cores as well. How, you can read for example here (or just think about it: You have one thread/process running on one core, what do you need to use multiple cores? Multiple threads or multiple processes. Push work from main thread to other threads or processes and you are done).
I personally would say it is childish to develop an application, which does not make use of multiple cores. If you make use of some background processes, ok, but if you until now only do work in the node.js main event loop, you should definitely invest some time to make the app scalable over cores.
Implementing something like IPC is not that easy by the way. You can do, but if your case is complicated maybe you are good to go with the cluster module. This is obviously not your favorite, but just because something is called "experimental" it does not mean it's trashy. Just give it a try, maybe you can even fix some bugs of the module on the way. It's most likely better to use some broadly used software for complex problems, than invent a new wheel.
You should also (if you do not already) think about (wise) usage of nextTick functionality. This allows the main event loop to pause some cpu intensive task and perform other work in the meanwhile. You can read about it for example here.
General thoughts on computations
You should definitely take a very close look at your algorithms of the game engine. You already noticed that this is your bottleneck right now and actually computations are the most critical part of mostly every game. Scaling does solve this problem in one way, but scaling introduces other problems. Also you cannot throw "scaling" as problem solver on everything and expect every problem to disappear.
Your best bet is to make your game code elegant and fast. Think about how to solve problems efficiently. If you cannot solve something in Javascript efficiently, but the problem can easily be extracted, why not write a little C component instead? This counts as a separate process as well, which reduces load on your main node.js event loop.
Proxy?
Personally I do not see the advantage of the proxy level right now. You do not seem to expect large amount of users, you therefore won't need to solve problems like CDN solves or whatever... it's okay to think about it, but I would not invest much time there right now.
Technically there is a high chance your webserver software provides proxy functionality anyway. So it is ok to have it on the paper, but I would not plan with dedicated hardware right now.
Epilogue
The rest seems more or less fine to me.
Little late to the game, but take a look here: http://goldfirestudios.com/blog/136/Horizontally-Scaling-Node.js-and-WebSockets-with-Redis
You did not mention anything to do with memory management. As you know, nodejs doesn't share its memory with other processes, so an in-memory database is a must if you want to scale. (Redis, Memcache, etc). You need to setup a publisher & subscriber event on each node to accept incoming requests from redis. This way, you can scale up x nilo amount of servers (infront of your HAProxy) and utilize the data piped from redis.
There is also this node addon: http://blog.varunajayasiri.com/shared-memory-with-nodejs That lets you share memory between processes, but only works under Linux. This will help if you don't want to send data across local processes all the time or have to deal with nodes ipc api.
You can also fork child processes within node for a new v8 isolate to help with expensive cpu bound tasks. For example, players can kill monsters and obtain quite a bit of loot within my action rpg game. I have a child process called LootGenerater, and basically whenever a player kills a monster it sends the game id, mob_id, and user_id to the process via the default IPC api .send. Once the child process receives it, it iterates over the large loot table and manages the items (stores to redis, or whatever) and pipes it back.
This helps free up the event loop greatly, and just one idea I can think of to help you scale. But most importantly you will want to use an in-memory database system and make sure your game code architecture is designed around whatever database system you use. Don't make the mistake I did by now having to re-write everything :)
Hope this helps!
Note: If you do decide to go with Memcache, you will need to utilize another pub/sub system.
After having learnt node, javascript and all the rest the hard way, I am finally about to release my first web app.
So I subscribed to Amazon Web Services and created a micro instance, planning on the first year free tier to allow me to make the app available to the world.
My concern is more about hidden costs. I know that with the free tier comes 1 million I/O requests per month for the Amazon EC2 EBS.
Thing is, I started testing my app one the ec2 instance to check that everything was running fine; and I am already at more than 100, 000 I/O requests. And I have basically been the only one using it so far (37 hours that the instance runs).
So I am quite afraid of what could happen if my app gets some traffic, and I don't want to end up with a huge unexpected bill at the end of the month.
I find it quite surprising, because I mainly serve static stuff, and my server side code consists in :
Receving a search request from a client
1 http request to a website
1 https request to the youtube api
saving the data to a mongoDB
Sending the results to the client
Do you have any advice on how to dramatically reduce my IO?
I do not use any other Amazon services so far, maybe am I missing something?
Or maybe Amazon free tier in not enough in my case, but then what can it be enough for? I mean, my app is really simple after all.
I'd be really glad for any help you could provide me
Thanks!
You did not mention total number of visits to your app. So I am assuming you have fairly less visits.
What are I/O requests ?
A single I/O request is a read/write instruction that reaches the EBS volumes. Beware! Execution of large read/writes is broken into multiple smaller pieces, which is the block size of the volume.
Possible reasons of high I/O:
Your app uses lot of RAM. After you hit the limit, the OS starts swapping memory to and fro from swap area in your disk, constantly.
This is most likely the problem, the mongoDB search. mongoDB searches can be long complex queries internally. From one of the answers to this question, the person was using mySQL and it caused him 1 billion I/O requests in 24 days. So 1 database search can be many I/O requests.
Cache is disabled, or you write/modify lot of files. You mentioned you were testing. Free-teir is just not suitable for developing stuff.
You should read this, in case you want to know what happens after free-tier expires.
I recently ran into a similar situation recording very high I/O request rates for a website with little to no traffic. The culprit seems to be a variation of what #prajwalkman discovered testing Chef deployments on a micro instance.
I am not using Chef, but I have been using boto3, Docker and Git to automatically 'build' test images inside a micro instance. Each time I go through my test script, a new image is built and I was not careful to read the fine print regarding default settings on the VolumeType argument on the boto3 run_instance command. Each test image was being built with the 'standard' volume type which, according to current EBS pricing, bills at the rate of $0.05/million I/Os. Whereas, the 'gp2' general purpose memory has a flat cost of $0.10 per GB per month with no extra charge for I/O.
With a handful of lean docker containers taking up a total of 2GB, on top of the 1.3GB for the amazon-ecs-optimized-ami, my storage is well within the free-tier usage. So, once I fixed the volumetype attribute in the blockdevicemappings settings in my scripts to 'gp2', I no longer had an I/O problem on my server.
Prior to that point, the constant downloading of docker images and git repos produced nearly 10m I/Os in less than a week.
The micro instance and the free tier is meant for testing their offerings, not a free way for you to host your site/web application.
You may have to pay money at the end of the month, but I really doubt if you can get away with paying less by using some other company for hosting. AFAIK AWS really is the rock bottom of the price charts.
As for the IO requests themselves, it's hard to give generic advice. I once was in a situation where my micro instance racked up ridiculous number of IO requests. Turns out testing Chef deployments on EC2 is a bad idea.
I/O Requests have to do with reading and writing blocks to EBS volumes. You can reduce this by using as much in memory caching as possible. Micro instances only have about 613 MB of memory available, so you may not be able to do much here.
Ok, so it seems like I/O requests are related to the EBS volume, and that caching may reduce it.
Something I had not considered though is all the operations I made to get my app running.
I updated the linux image, installed node and npm, several modules, mongodb, ....
This is likely to be the main cause of the I/O.
The number requests hasn't grown much in the last days, where the server stayed mostly idle.