I currently have ReactJS + NodeJS/ExpressJS + Webpack onto EC2, Amazon Web Services (AWS) under one project and would like to get it deployed together at once, in one project.
What are some suggestions on how to go about doing so? Done the research, and I've only seen tutorials on deploying one in specific, whether it be just ReactJS or just NodeJS. Any insights or leads would be greatly appreciated.
Will accept/upvote answer. Thank you in advance
You don't "deploy" ReactJS, it's just a static file or files like any other JS libraries in your applications. You also don't deploy Webpack. Webpack should run on a developer machine (or in CI/CD stack or build system).
As for the NodeJS part just use Elastic Beanstalk.
I do not commit builds to source control. I see that a lot and it can make things easier, but you can also forget to rebuild as you have to do it manually, and it adds a lot of bloat to your repo.
I believe builds should be run as part of the deployment process. Assuming you are using git, you can add script hooks/post-receive in a remote repo there. When you push to that remote, the script will run. This is where I do my webpack build.
You may want to look into https://github.com/git-deploy/git-deploy for context, but I do this manually.
In my projects, on the deployment machine I do git --init --bare /var/git/myproject.git then add the script in /var/git/hooks/post-receive. The hook checks out the code into /var/www/myproject, runs the build, which fills in the /var/www/myproject/build. Then it removes the old /var/www/myproject/public and renames build to public. And done.
I'm coming from more of an operations background and would say that if your goal includes keeping that site up as much as possible then use Packer to generate AMI's and CloudFormation to build an Application Load Balancer (the newer, cheaper brother of ELB) in front of an AutoScalingGroup which keeps the EC2 instances up and running.
I'm currently working on a large scale project doing exactly what you describe. First off, there are so many different ways to do this, so what you really need is some general guidelines to get started, then we can dig a little deeper into details when some initial decisions are made, if you'd like. If you've already got the app deploying and running in two separate steps, but are just looking to combine those, I can definitely help. I'd just need to know how you're currently building/deploying. If you're just getting started on building your pipeline and need to set up the process from scratch, then read on:
First off you'll want to set up some kind of build server that will install your npm dependancies and run your webpack build. Most likely you'll want a separate webpack config that's just for your build server, this'll give you a build optimized for production or qa/staging environments. This config should split out vendor files that you won't update all the time, pull out seperate css files with extract text plugin and uglify the files. If you have an isomorphic React app, or are using es6 features not supported in your version of node, then you'll need a webpack build for your server code as well. This is really different from the hot reloading build you'll want to have on your local machine while you're actually coding the app. I'll be happy to show some examples if you'd like of our webpack config files for both local development and our CI build. You may also need a build.sh or makefile to do something with the compiled .js files that your webpack build creates, but that'll depend on your deployment which I'll cover later. You can run your production build locally as your getting your config just right and fire up the app from those files to test it's all working. Additionally, since you'll likely want to be able to automate all of this, you probably want to run your tests and linting right before you build your app, we run eslint and mocha/jdom to run our enzyme/expect specs as part of our build. Once that's all working nicely, you'll most likely want to set up a build server that can run your builds automatically. My team is using Jenkins for this, which is a little more work to set up, but it's free (aside from the ec2 box we run it on). There are also a ton of subscription based build/continuous integration servers, such as Travis and CodeShip. There's plenty of articles on the pros and cons of these different products and how to set them up. The bottom line is you'll want to have a build server that can pull down your code from source control, install npm deps, lint, test and build your app. If anything fails it should fail your build and if your build succeeds you'll have some sort of archive that you'll later deploy to an ec2 instance. In our shop we use a build.sh file to tarball up our build archive (basically a folder with our node server files as well as our minified client files, css files and any fonts or images needed to run the app) and upload it to an S3 bucket that we deploy from. We like this fairly old school method because the tarball will never change, so we have ultra reliable roll backs.
What you do with your build archive will depend on how you want to do deployments. We have a custom deployment system using puppet, but there are plenty of products that do this such as elastic beanstalk, that would be much easier to set up. You'll want some kind of process supervisor to actually run your node app, so unless you have a dev ops team that wants to build custom pipelines, using AWS built in features will probably be the easiest way to get started. As usual, there are so many ways to do this, but the basic principal is that you need something to download your build archive and run/supervise your node process. You also may want to be able to create and configure ec2 boxes on the fly (Puppet, chef, etc.), or even use containers (Docker) which allow you to move complete stacks around as single units. Using automation to create and configure servers is crucial if you need to scale your app, but it is complicated and may not be necessary for smaller projects. This is definitely an area where you can start simply and add complexity later on, as long as you have good long term goals and make sure to take the necessary step to prepare for future complexity.
All of this can get you pretty far in the weeds, so it's best to find the simplest thing that will serve your needs as you get started and then add complexity as real life situations demand so. I'll be happy to elaborate on any of these details if you provide a little more context about how big and well funded of a project your working on. If it's a little side project to learn the tech, I'd have very different advice then if you're trying to build an app that'll have a lot of traffic and/or complex features.
This could get 100 different answers and they could all end up being good ideas. First, you mention react + nodejs - keep in mind that these solve different tasks. React is going to be frontend and served out via static files. Nodejs is focused more around the server-side and would be the code that serves data. They can easily work together. You might use Express for the webserver (nodejs) to serve the HTML/React pages.
Unfortunately, I saw that you mentioned webpack, so you are going to have to 'build' your application with something - either via webpack, gulp, grunt, etc. This is where source control and build servers are great - but if you're new to it, it might be more complex than you need.
If you have just basic EC2 images as webservers and only 1-2, then the biggest hurdle is just pushing up your code. Something like https://deploybot.com/ could work as it can push your git repo down to multiple hosts via ftp, etc. If you wanted to get a bit fancier, you could look at something like Jenkins or some of the other items.
Docker is a great choice and if you are going to be dealing with multiple developers, server environments, deployments - it's worth the time. Otherwise, keep it simple and just get your code on the EC2 instance ;).
Related
I am developing an application which comprises a SPA front end and a Rest back-end.
To implement the Rest back-end I am using Node and Express.
Considering that both front-end and back-end are written in JavaScript (and TypeScript), I would like to share some code between these 2 parts (namely Interfaces and simple utils).
So basically my project is composed of three parts: Client, Server, Shared. Therefore I am inclined to have a project directory structure similar to this:
ProjecFolder
ClientFolder
.....
ServerFolder
.....
SharedFolder
.....
I am looking for suggestions on how best organize my project. I have done some research and I have found this interesting article which suggests to use a mechanism based on Gulp tasks that copy all files from SharedFolder into both ClientFolder and ServerFolder and then runs transpling.
I am wondering whether there can be an alternative approach or tools that perform what otherwise has to be configures as Gulp workflow.
My recommendation is to use a package manager tool. When you have dependencies, and the requirements of the server changed, you have to change the module. You don't want the SPA (frontend), to break, when you need to make changes to the server.
This is why package managers give you versions. Each module that depends on your shared code, can use a different version of it. You can use NPM for that. Build a module, publish it, and install it on your frontend and backend.
Many times, in production you split the frontend and backend. The frontend may exist in a file storage system (S3, Google Cloud Storage and similar), and the backend executed on your servers. Then it will be harder to use the same files on both systems.
My team is building a large React application. I am wanting to know if what we are trying to accomplish in regards to build and deployment are possible with Webpack.
Let’s say our team is building Google Admin. There are 4 modules/app within the admin that 4 different teams are focused on. There is then a console application that is the entry point to these 4 modules/apps. We want to be able to work on each of the modules independently and be able to deploy them independently.
How we have it setup right now is there would be 4 separate applications that are dev harnesses to build these modules. We build them and copy the distribution .js and .js.map files to the console's ./modules folder. We would reference these modules lazily using System.import.
Is it possible, while the console app is built and in production, to “swap out” the module-one.js and module-one.js.map files that the console depends on without having to rebuild and redeploy the entire console app?
Goals:
Do not package these apps for npm. This would definitely require the console app to update and rebuild.
Build any module and deploy just that specific module to production without having to rebuild the console application.
Do not redirect to separate SPAs.
I tried my best to explain the goal. Any input would be much appreciated. I have found nothing in my search.
webpack loads the modules into memory and watches the filesystem for changes, as long as webpack is running you shouldn't have an issue replacing any given module. However webpack will attempt to build the entire in memory bundle with each module change (as it has no way of knowing that your module is truly independent). The only thing I can thin of would be to write a shim between the console app and the modules that watches the files (like webpack) but only replaces the in memory version of the local file that was changed. Reading this I'm not even sure if it makes sense to me...
I am writing an application with a Node.js backend and a single-page web app front-end.
I am keeping the client and server logic in the same project for simplicity and speed of development.
I am considering how best to organise the artifacts.
The Node.js part is straightforward because it doesn't need to go through a battery of pre-processors (transpilation, minification, concatenation etc).
The front-end needs to be transformed per the above, and I guess placed in a dist folder.
The current hierarchy of files is like so:
my-app
- src
- client
- server
Should I put the dist folder for the client artifacts under src/client?
Has anyone tried this and found problems with this approach?
I am using Heroku (a deployment system that uses git).
Committing the built artifacts for the client feels wrong, but if I want to deploy it by pushing to Heroku I think I need to commit them. Is this correct?
This question, as is, invites opinionated answers, so I'll start by saying this is by no means the only way to go, but in my opinion, it is the easiest to work with and makes the most sense.
The production client code, after pre-processing, should be located in my-app/dist or my-app/dst, which could either mean distribution or destination, depending on how you look at it. Either way, my recommendation is to commit this folder, as it saves you a lot of hassle debugging remotely.
For example, if your code works locally but not remotely, using something like the postinstall hook to generate your dist folder adds yet another suspect to check when trying to determine what the issue is with your program.
Another advantage of committing the dist folder is it allows you to specify all the packages you use for your build process as devDependencies rather than dependencies. This is a huge plus, and makes deployment a lot faster, as well as less memory usage on your heroku process.
That being said, I still recommend (as you already probably plan to do) using an automated watch task to build your dist folder for ease of development, even if you decide you don't want to use that same build process remotely and opt for committing the dist directory instead. You could add that as a custom npm command, e.g. npm run build and have that invoke your gulp task.
One last thing. For those of you using templating languages like pug or dust or ejs instead of a framework like react or angular, I recommend determining whether you can run any of your templates to build static HTML files that will be served in production.
If not, you should at least compile your templates (not to be confused with running them) by following the recommendations provided by your particular templating language. Typically, they'll suggest using their command line utility to generate the compiled templates, so that they don't have to be compiled every time they're invoked in production. This will make your node.js server respond faster to requests at the expense of using more memory to cache the compiled templates.
If you're planning to go this route, I would edit nodejs app.js/index.js to serve static file and point the directory to dist/.
Also, you would need to tell express to forward all non-api requests to the frontend.
I got a question regarding build controllers of Visual Studio.
I got a project where I run multiple C# unit test. I currently implemented Javascript Unit tests to the project and I want to let the unit test be part of the build.
Several tutorials are available on the internet. One of them I used as guideline towards running JS intergrated within my TFS which is running on a buildserver.
The problem that I have is that the tutorial is saying that I should check in the files (of Chutzpah) and add the source file to the build controller. Here is my problem. Due to the fact that I do not want to affect all the other unit tests and build processes I can not modify the build controller. I can change any build definition but I can not change the "Version Control Path to custom Assemblies". So I was wondering is there alternative method where I still can make sure that JS unit tests are part of the build but not changing the version control path for the whole project?
I hope I stated my situation clear enough.
You can enable your build process to leverage binaries that you have
uploaded to your Team Foundation Server, for example:
Assemblies that contain your custom workflow activities.
Third-party unit test frameworks. See Run tests in your build process.
Custom MSBuild tasks
To enable your build processes to leverage these kinds of code, upload
the binaries to the folder (or any of its descendant folders) that you
specify in the Version control path to custom assemblies box. MSDN
So, if you haven't configured this path of your build control. It's easy, you just need to set a sever path. This will not affect other unit tests and build processes cuase they didn't even use this path before.
If there is aleady a server path, you just need to add the files mentioned in the tutorials into the source control with the same path. Just like a share folder, when the build definitions need the file, the build control will automatically find and call it in this path. When you set or modify the value in this box, the build server automatically restarts to load the assemblies.
I'm trying to get into Grunt, which I am new to, but I do not understand its utility.
I understand that it is a taskrunner. I understand that it can be used to do things like bundle, uglify, jshint, minify, etc etc etc, anything that can be turned into a scripted task.
But I don't see what advantage this gives. Nearly all of these can be run from the command line anyway, which is to say you could just combine them using a simple shell script. It seems to me that setting up grunt + gruntfiles and writing tasks is more work than writing a shell script, rather than less.
What am I missing about this?
Grunt is basically a build / task manager written on top of NodeJS. I would call it the NodeJS stack equivalent of ANT for Java. Here are some common scenarios you would want to use grunt under:
You have a project with javascript files requiring minification, and generally generating a front end build seperately (in case you're using say JAVA for your backend). (grunt-contrib-uglify)
When you save code on your machine during development, you want the browser to reload your page automatically (might seem like a small thing, but believe me this has saved me lots of time). (Live reload)
When a developer saves code on his machine, he wants a comprehensive list of JS errors / general best practice violations to be shown. (grunt-contrib-jshint)
You have a project with SASS/ LESS files which need to be compiled to CSS files on the developers machine during development, For example whenever he saves a SASS file, you want it to be compiled to a CSS file automatically, for inclusion in your page. (grunt-contrib-sass)
You have a team of front end developers who're working on the UI, and a team of backend developers working on the backend, you want the front end devs to use the backend REST API's without having to compile & deploy code everytime on their own machines. In case you were wondering, this isn't possible with a typical web server setup because XHR isn't allowed to be cross-domain by browser. Grunt can setup a proxy for you redirecting XHR requests on your own system within the grunt connect server to another system! (grunt-contrib-proxy, grunt-contrib-connect)
I do not think your shell script can do ALL of these. To summarize, yes, setting up a Gruntfile.js is tedious for someone who has had little exposure to javascript / is new to nodeJS, I went through the same pains as a learner, but Grunt is an amazing piece of software. DO invest the time to setup a proper Gruntfile.js for your front end project, and you'll thank god for making your life a lot easier :)
The Advantage vs shell script:
If you write shell script for every one of these tasks, it is tedious to maintain and then customize for every one of your needs. Gruntfile.js is actually pretty easy. there is a config that you init it with, specifying what tasks you want to perform, the sources and targets for each.
The integration with project seed generators on Yeoman, Gulp is another major factor to consider. Yeoman and Gulp come with Gruntfile.js' with intelligent defaults. For someone who is the sole UI contributor on his team, this is priceless to me!
For someone who is working on frontend technologies, if you have more than one person working with you, its rather easy for them to get to know Grunt, which is already well documented with a lot of answers on SO, than to get to know your shell scripts. This might be a factor in large teams.
The numerous plugins for Grunt extend base functionality. Unless your shell script is VERY popular, and VERY modular, I dont see plugins being built for it. This also extends to inclusion of new front end technologies in your project. Say, if you want to use typescript in your project tomorrow, your shell script will need to incorporate this and account for it with your own effort. With Grunt, its just as simple as "npm install " and adding a config.
Even though I agree with most advantages pointed in Accepted Answer, I still have to consider the disadvantages that are highlighted by Keith Cirkel in Why we should stop using Grunt & Gulp
Thus, some advantages are rebut by Grunt overheads and at least you should consider all this in your final decision of using Grunt, or not.