I've been using grunt task runner in all of the angular projects I was involved in.
Currently, I found a gulp-protractor-qa plugin for gulp that watches all my element selectors in the tests on the fly. It is good at what it does, but now I have to have a separate gulpfile.js config for the another build system (I cannot find an appropriate alternative among grunt plugins).
Is it okay to use both grunt and gulp build systems in a single project? What are the generally accepted actions in this case?
gulp-protractor-qa is just an example. I can imagine this would hit me again when I would need different suitable plugins in both build systems and would have to make a choice: try to sit on two chairs?
Like you said, if you don't have any other alternative with grunt, I think that the need of using both task runners is real until an equivalent solution is found or created.
You should of course take care of the possible overrides and conflicts that comes with each task you add in gulp, and to me, never mix your watchers. Always choose to use them in one or the other runner cause of the infinite loops. Even if in this case with gulp-protractor-qa you are watching files, there is no dependent task which runs and nothing is written so you should be fine.
Also, since the new dependencies will mainly be the dev ones, the performance of your deployed app will be very slightly affected or not, except if you run your tests on each deploy, that may take a little longer to install them.
It looks like something is going wrong with running two build systems in one frontend project. You have an option to run grunt tasks from gulp (https://www.npmjs.org/package/gulp-grunt) or vice versa, gulp tasks from grunt (https://www.npmjs.org/package/grunt-gulp). First way looks more efficient, because gulp is much faster than grunt. Consider moving to gulp entirely; many grunt plugins have "native" gulp counterparts.
If moving to gulp seems like an option, but you don't like the idea of spending all that time to rewrite buil config for new tool, try to use some yeoman-genereators. In particular, I'd recommend use of awesome boom generator (npmjs.org/package/generator-boom). It's the best generator for angular with gulp build from the box on the table today.
Related
I bought an HTML template recently, which contains many plugins placed inside a bower_components directory and a package.js file inside. I wanted to install another package I liked, but decided to use npm for this purpose.
When I typed:
npc install pnotify
the node_modules directory was created and contained about 900 directories with other packages.
What are those? Why did they get installed along with my package? I did some research and it turned out that those were needed, but do I really need to deliver my template in production with hundreds of unnecessary packages?
This is a very good question. There are a few things I want to point out.
The V8 engine, Node Modules (dependencies) and "requiring" them:
Node.js is built on V8 engine, which is written in C++. This means that Node.js' dependencies are fundamentally written in C++.
Now when you require a dependency, you really require code/functions from a C++ program or js library, because that's how new libraries/dependencies are made.
Libraries have so many functions that you will not use
For example, take a look at the express-validator module, which contains so many functions. When you require the module, do you use all the functions it provides? The answer is no. People most often require packages like this just to use one single benefit of it, although all of the functions end up getting downloaded, which takes up unnecessary space.
Think of the node dependencies that are made from other node dependencies as Interpreted Languages
For example, JavaScript is written in C/C++, whose functions and compilers are in turn originally written in assembly. Think of it like a tree. You create new branches each time for more convenient usage and, most importantly, to save time . It makes things faster. Similarly, when people create new dependencies, they use/require ones that already exist, instead of rewriting a whole C++ program or js script, because that makes everything easier.
Problem arises when requiring other NPMs for creating a new one
When the authors of the dependencies require other dependencies from here and there just to use a few (small amount) benefits from them, they end up downloading them all, (which they don't really care about because they mostly do not worry about the size or they'd rather do this than explicitly writing a new dependency or a C++ addon) and this takes extra space. For example you can see the dependencies that the express-validator module uses by accessing this link.
So, when you have big projects that use lots of dependencies you end up taking so much space for them.
Ways to solve this
Number 1
This requires some expert people on Node.js. To reduce the amount of the downloaded packages, a professional Node.js developer could go to the directories that modules are saved in, open the javascript files, take a look at their source code, and delete the functions that they will not use without changing the structure of the package.
Number 2 (Most likely not worth your time)
You could also create your own personal dependencies that are written in C++, or more preferably js, which would literally take up the least space possible, depending on the programmer, but would take/waste the most time, in order to reduce size instead of doing work. (Note: Most dependencies are written in js.)
Number 3 (Common)
Instead of Using option number 2, you could implement WebPack.
Conclusion & Note
So, basically, there is no running away from downloading all the node packages, but you could use solution number 1 if you believe you can do it, which also has the possibility of screwing up the whole intention of a dependency. (So make it personal and use it for specific purposes.) Or just make use of a module like WebPack.
Also, ask this question to yourself: Do those packages really cause you a problem?
No, there is no point to add about 900 packages dependencies in your project just because you want to add some template. But it is up to you!
The heavyness of a template is not challenging the node.js ecosystem nor his main package system npm.
It is a fact that javascript community tend to make smallest possible module to be responsible for one task, and just one.
It is not a bad thing I guess. But it could result of a situation where you have a lot of dependencies in your project.
Nowadays hard drive memory is cheap and nobody cares any more about making efficient/small apps.
As always, it's only a matter of choice.
What is the point of delivering hundreds of packages weighing hundreds of MB for a few kB project.
There isn't..
If you intend to provide it to other developers, just gitignore (or remove from shared package) node_modules or bower_components directories. Developers simply install dependencies again as required ;)
If it is something as simple as an HTML templates or similar stuff, node would most likely be there just for making your life as a developer easier providing live reload, compiling/transpiling typescript/babel/SCSS/SASS/LESS/Coffee... ( list goes on ;P ) etc.
And in that case dependencies would most likely only be dev_dependencies and won't be required at all in production environment ;)
Also many packages come with separate production and dev dependencies, So you just need to install production dependencies...
npm install --only=prod
If your project does need many projects in production, and you really really wanna avoid that stuff, just spend some time and include css/js files your your project needs(this can be a laborious task).
Update
Production vs default install
Most projects have different dev and production dependencies,
Dev dependencies may include stuff like SASS, typescript etc. compilers, uglifiers (minification), maybe stuff like live reload etc.
Where as production version will not have those things reducing the size node_modules directory.
** No node_modules**
In some html template kind of projects, you may not need any node_modules in production, so you skip doing an npm install.
No access to node_modules
Or in some cases, when server that serves exists in node_modules itself, access to it may be blocked (coz there is no need to access these from frontend).
What are those? Why did they get installed along with my package?
Dependencies exists to facilitate code reuse through modularity.
... do I need to deliver my template in production with hundreds of unnecessary packages?
One shouldn't be so quick to dismiss this modularity. If you inline your requires and eliminate dead code, you'll lose the benefit of maintenance patches for the dependencies automatically being applied to your code. You should see this as a form of compilation, because... well... it is compilation.
Nonetheless, if you're licensed to redistribute all of your dependencies in this compiled form, you'll be happy to learn those optimisations are performed by a compiler which compile Javascript to Javascript. The Closure Compiler, as the first example I stumbled across, appears to perform advanced compilation, which means you get dead code removal and function inlining... That seems promising!
This does however have another side effect when you are required to justify the licensing of all npm modules..so when you have hundreds of npm modules due to dependencies this effort also becomes a more cumbersome task
Very old question but I happened to come across very similar situation just as RA pointed out.
I tried to work with node.js framework using vscode and the moment when I tried to install start npm using npm init -y, it generated so many different dependencies. In my case, it was vscode extension ESlint that I added to prior to running npm init -y
Uninstalling ESlint
Restarted vscode to apply that uninstallation
removed previously generated package.json and node-modules folder
do npm init -y again
This solved my problem of starting out with so many dependencies.
Update: I was able to solve the issue. One of the third part scripts I was including was missing a semicolon, thereby breaking the closure. When the minified version was being used, the generator didn't know where the block ended and new blocks began. The grunt build process works fine as is now, without modification.
I've got an angular app that I built with the generator-angular yeoman generator and it works wonderfully in the local "grunt serve" state. When I run "grunt build," however, the built version doesn't work properly. I'm using the standard Gruntfile that comes with the generator.
The only different between the grunt serve and the grunt build tasks are the minification, concatenation and uglification of files. You'd think it would simply concat and minify the files in the same order it loads them in the development build, but it looks like things are being loaded in a different order—throwing a app.init() is not a function error—which tells me the app module isn't loaded before that section.
Is there something that needs to be done to the default build task to fix this issue?
I'm not sure what the problem is, but I think that can be the minification process, there are several ways of writing a controller and some of them have problems at the time of the minification.
If the problem is the minification you could check these pages.
AnularJs - A note on minification
Best Practice of Minification StackOverflow
Example about minifying AngularJs Controllers
I am working in some development team. We have unit tests in our programm. There are many tests in it. Each test covers one module. In common, one developer is working only on one or some modules (but not many) per one task. To improve unit tests speed I use focused specs (I use Jasmine and our app is AngularJS app).
The problem: sometimes I forget to remove focused specs. I commit and PR with fit/fdescribe calls in programm. It can cause problems for me and other developers in the future. Obvious example: developer is developing :), he made a mistake, he started tests and only focused specs worked, since he worked on another file - it is a problem - there will be mistake but he wouldn't see it.
Is there any methods to automatically find focused specs in my tests and warn developer about it? Our team is using CI with grunt running on it. Grunt has tasks for jscs and jshint. Maybe it is somehow possible to create own rules to prevent this mistake?
I would setup a git hook and utilise grunt-ddescribe-iit.
For now, let's use grunt-githooks to setup the git hook.
grunt.initConfig({
githooks: {
options: {
'pre-push': 'grunt ddescribe-iit'
}
},
ddescribe-iit: {
files: [
'test/**/*.js',
'app/**/*.spec.js'
]
}
});
That's a very contrived example of how I imagine it would work, I haven't utilised ddescribe-iit myself (though, like you - I should be!) nor have I used grunt-githooks.
Looking through how they're both put together however, my established guess is that this would be a fairly painless process to set up.
Like you, we are also utilising a CI system and it has always been a pain when a build passes, but it only ran a couple of chosen tests. Using a git hook that screams in your face before pushing, kills the problem at the root I reckon (without introducing watchers and additional processes during the development step).
All at the amazing cost of development overhead of 0.
edit: this answer assumes that you are utilising grunt as a task runner. there are equivalents for gulp.
What's the best practice for minifying and bundling js/css in a pure front end app, and how do the tools work?
I know how this can be done with server side apps like .NET/Java/LAMP/etc. But what about pure front end projects, SPA projects or backendless projects that are built with say, ember or angular these days? Say your entire project consists of HTML/css/js, which interfaces with a RESTful service elsewhere.
What kind of process or tool do you use to minify and bundle the resources for that?
I've seen grunt plugins that exist for this, but I find the documentation to be pretty magical and it's still unclear to me how they work.
Specifically, does the tool:
1) Replace src="/js/a.js",src="/js/b.js" with src="/js/bundle-a+b.min.js"? (and likewise with css?) in the source html files?
2) have different modes for dev and release, or is the tool only run when the project is released?
Or are the resource requests entirely managed by a js tool and js/css files have to be requested via a library function? Wouldn't the lag be noticeable in this case?
Thanks.
Through the use of Build tools front end devs can have minified javascript, css, or even images and html files automatically minified as they develop. The most common is grunt, with gulp close behind.
You configure grunt tasks, like grunt-contrib-uglify and grunt-contrib-copy, and put those tasks under a grunt-contrib-watch task. Have the grunt watch task watch the files you modify, and every time a change is detected those .min files are automatically generated.
These build tools have no impact on your application, they are run before the files are servered. You were correct to assume there was an easy way to do this. I suggest you look at grunt getting started, a sample gruntfile, or a project that uses grunt - here's mine, it does minification like you requested. Clone my repo, run sudo npm install, then sudo grunt. I don't have watch set up in my project but grunt is very well documented.
I'm trying to get into Grunt, which I am new to, but I do not understand its utility.
I understand that it is a taskrunner. I understand that it can be used to do things like bundle, uglify, jshint, minify, etc etc etc, anything that can be turned into a scripted task.
But I don't see what advantage this gives. Nearly all of these can be run from the command line anyway, which is to say you could just combine them using a simple shell script. It seems to me that setting up grunt + gruntfiles and writing tasks is more work than writing a shell script, rather than less.
What am I missing about this?
Grunt is basically a build / task manager written on top of NodeJS. I would call it the NodeJS stack equivalent of ANT for Java. Here are some common scenarios you would want to use grunt under:
You have a project with javascript files requiring minification, and generally generating a front end build seperately (in case you're using say JAVA for your backend). (grunt-contrib-uglify)
When you save code on your machine during development, you want the browser to reload your page automatically (might seem like a small thing, but believe me this has saved me lots of time). (Live reload)
When a developer saves code on his machine, he wants a comprehensive list of JS errors / general best practice violations to be shown. (grunt-contrib-jshint)
You have a project with SASS/ LESS files which need to be compiled to CSS files on the developers machine during development, For example whenever he saves a SASS file, you want it to be compiled to a CSS file automatically, for inclusion in your page. (grunt-contrib-sass)
You have a team of front end developers who're working on the UI, and a team of backend developers working on the backend, you want the front end devs to use the backend REST API's without having to compile & deploy code everytime on their own machines. In case you were wondering, this isn't possible with a typical web server setup because XHR isn't allowed to be cross-domain by browser. Grunt can setup a proxy for you redirecting XHR requests on your own system within the grunt connect server to another system! (grunt-contrib-proxy, grunt-contrib-connect)
I do not think your shell script can do ALL of these. To summarize, yes, setting up a Gruntfile.js is tedious for someone who has had little exposure to javascript / is new to nodeJS, I went through the same pains as a learner, but Grunt is an amazing piece of software. DO invest the time to setup a proper Gruntfile.js for your front end project, and you'll thank god for making your life a lot easier :)
The Advantage vs shell script:
If you write shell script for every one of these tasks, it is tedious to maintain and then customize for every one of your needs. Gruntfile.js is actually pretty easy. there is a config that you init it with, specifying what tasks you want to perform, the sources and targets for each.
The integration with project seed generators on Yeoman, Gulp is another major factor to consider. Yeoman and Gulp come with Gruntfile.js' with intelligent defaults. For someone who is the sole UI contributor on his team, this is priceless to me!
For someone who is working on frontend technologies, if you have more than one person working with you, its rather easy for them to get to know Grunt, which is already well documented with a lot of answers on SO, than to get to know your shell scripts. This might be a factor in large teams.
The numerous plugins for Grunt extend base functionality. Unless your shell script is VERY popular, and VERY modular, I dont see plugins being built for it. This also extends to inclusion of new front end technologies in your project. Say, if you want to use typescript in your project tomorrow, your shell script will need to incorporate this and account for it with your own effort. With Grunt, its just as simple as "npm install " and adding a config.
Even though I agree with most advantages pointed in Accepted Answer, I still have to consider the disadvantages that are highlighted by Keith Cirkel in Why we should stop using Grunt & Gulp
Thus, some advantages are rebut by Grunt overheads and at least you should consider all this in your final decision of using Grunt, or not.