What is Babel, Webpack and Browserify? [duplicate] - javascript

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I'm trying to summarize my knowledge about the most popular JavaScript package managers, bundlers, and task runners. Please correct me if I'm wrong:
npm & bower are package managers. They just download the dependencies and don't know how to build projects on their own. What they know is to call webpack/gulp/grunt after fetching all the dependencies.
bower is like npm, but builds a flattened dependency trees (unlike npm which does it recursively). Meaning npm fetches the dependencies for each dependency (may fetch the same a few times), while bower expects you to manually include sub-dependencies. Sometimes bower and npm are used together for front-end and back-end respectively (since each megabyte might matter in front-end).
grunt and gulp are task runners to automate everything that can be automated (i.e. compile CSS/Sass, optimize images, make a bundle and minify/transpile it).
grunt vs. gulp (is like maven vs. gradle or configuration vs. code). Grunt is based on configuring separate independent tasks, each task opens/handles/closes file. Gulp requires less amount of code and is based on Node streams, which allows it to build pipe chains (w/o reopening the same file) and makes it faster.
webpack (webpack-dev-server) - for me it's a task runner with hot reloading of changes which allows you to forget about all JS/CSS watchers.
npm/bower + plugins may replace task runners. Their abilities often intersect so there are different implications if you need to use gulp/grunt over npm + plugins. But task runners are definitely better for complex tasks (e.g. "on each build create bundle, transpile from ES6 to ES5, run it at all browsers emulators, make screenshots and deploy to dropbox through ftp").
browserify allows packaging node modules for browsers. browserify vs node's require is actually AMD vs CommonJS.
Questions:
What is webpack & webpack-dev-server? Official documentation says it's a module bundler but for me it's just a task runner. What's the difference?
Where would you use browserify? Can't we do the same with node/ES6 imports?
When would you use gulp/grunt over npm + plugins?
Please provide examples when you need to use a combination

Webpack and Browserify
Webpack and Browserify do pretty much the same job, which is processing your code to be used in a target environment (mainly browser, though you can target other environments like Node). Result of such processing is one or more bundles - assembled scripts suitable for targeted environment.
For example, let's say you wrote ES6 code divided into modules and want to be able to run it in a browser. If those modules are Node modules, the browser won't understand them since they exist only in the Node environment. ES6 modules also won't work in older browsers like IE11. Moreover, you might have used experimental language features (ES next proposals) that browsers don't implement yet so running such script would just throw errors. Tools like Webpack and Browserify solve these problems by translating such code to a form a browser is able to execute. On top of that, they make it possible to apply a huge variety of optimisations on those bundles.
However, Webpack and Browserify differ in many ways, Webpack offers many tools by default (e.g. code splitting), while Browserify can do this only after downloading plugins but using both leads to very similar results. It comes down to personal preference (Webpack is trendier). Btw, Webpack is not a task runner, it is just processor of your files (it processes them by so called loaders and plugins) and it can be run (among other ways) by a task runner.
Webpack Dev Server
Webpack Dev Server provides a similar solution to Browsersync - a development server where you can deploy your app rapidly as you are working on it, and verify your development progress immediately, with the dev server automatically refreshing the browser on code changes or even propagating changed code to browser without reloading with so called hot module replacement.
Task runners vs NPM scripts
I've been using Gulp for its conciseness and easy task writing, but have later found out I need neither Gulp nor Grunt at all. Everything I have ever needed could have been done using NPM scripts to run 3rd-party tools through their API. Choosing between Gulp, Grunt or NPM scripts depends on taste and experience of your team.
While tasks in Gulp or Grunt are easy to read even for people not so familiar with JS, it is yet another tool to require and learn and I personally prefer to narrow my dependencies and make things simple. On the other hand, replacing these tasks with the combination of NPM scripts and (propably JS) scripts which run those 3rd party tools (eg. Node script configuring and running rimraf for cleaning purposes) might be more challenging. But in the majority of cases, those three are equal in terms of their results.
Examples
As for the examples, I suggest you have a look at this React starter project, which shows you a nice combination of NPM and JS scripts covering the whole build and deploy process. You can find those NPM scripts in package.json in the root folder, in a property named scripts. There you will mostly encounter commands like babel-node tools/run start. Babel-node is a CLI tool (not meant for production use), which at first compiles ES6 file tools/run (run.js file located in tools) - basically a runner utility. This runner takes a function as an argument and executes it, which in this case is start - another utility (start.js) responsible for bundling source files (both client and server) and starting the application and development server (the dev server will be probably either Webpack Dev Server or Browsersync).
Speaking more precisely, start.js creates both client and server side bundles, starts an express server and after a successful launch initializes Browser-sync, which at the time of writing looked like this (please refer to react starter project for the newest code).
const bs = Browsersync.create();
bs.init({
...(DEBUG ? {} : { notify: false, ui: false }),
proxy: {
target: host,
middleware: [wpMiddleware, ...hotMiddlewares],
},
// no need to watch '*.js' here, webpack will take care of it for us,
// including full page reloads if HMR won't work
files: ['build/content/**/*.*'],
}, resolve)
The important part is proxy.target, where they set server address they want to proxy, which could be http://localhost:3000, and Browsersync starts a server listening on http://localhost:3001, where the generated assets are served with automatic change detection and hot module replacement. As you can see, there is another configuration property files with individual files or patterns Browser-sync watches for changes and reloads the browser if some occur, but as the comment says, Webpack takes care of watching js sources by itself with HMR, so they cooperate there.
Now I don't have any equivalent example of such Grunt or Gulp configuration, but with Gulp (and somewhat similarly with Grunt) you would write individual tasks in gulpfile.js like
gulp.task('bundle', function() {
// bundling source files with some gulp plugins like gulp-webpack maybe
});
gulp.task('start', function() {
// starting server and stuff
});
where you would be doing essentially pretty much the same things as in the starter-kit, this time with task runner, which solves some problems for you, but presents its own issues and some difficulties during learning the usage, and as I say, the more dependencies you have, the more can go wrong. And that is the reason I like to get rid of such tools.

Update October 2018
If you are still uncertain about Front-end dev, you can take a quick look into an excellent resource here.
https://github.com/kamranahmedse/developer-roadmap
Update June 2018
Learning modern JavaScript is tough if you haven’t been there since the beginning. If you are the newcomer, remember to check this excellent written to have a better overview.
https://medium.com/the-node-js-collection/modern-javascript-explained-for-dinosaurs-f695e9747b70
Update July 2017
Recently I found a comprehensive guide from Grab team about how to approach front-end development in 2017. You can check it out as below.
https://github.com/grab/front-end-guide
I've been also searching for this quite some time since there are a lot of tools out there and each of them benefits us in a different aspect. The community is divided across tools like Browserify, Webpack, jspm, Grunt and Gulp. You might also hear about Yeoman or Slush. That’s not a problem, it’s just confusing for everyone trying to understand a clear path forward.
Anyway, I would like to contribute something.
Table Of Contents
Table Of Content
1. Package Manager
NPM
Bower
Difference between Bower and NPM
Yarn
jspm
2. Module Loader/Bundling
RequireJS
Browserify
Webpack
SystemJS
3. Task runner
Grunt
Gulp
4. Scaffolding tools
Slush and Yeoman
1. Package Manager
Package managers simplify installing and updating project dependencies, which are libraries such as: jQuery, Bootstrap, etc - everything that is used on your site and isn't written by you.
Browsing all the library websites, downloading and unpacking the archives, copying files into the projects — all of this is replaced with a few commands in the terminal.
NPM
It stands for: Node JS package manager helps you to manage all the libraries your software relies on. You would define your needs in a file called package.json and run npm install in the command line... then BANG, your packages are downloaded and ready to use. It could be used both for front-end and back-end libraries.
Bower
For front-end package management, the concept is the same with NPM. All your libraries are stored in a file named bower.json and then run bower install in the command line.
Bower is recommended their user to migrate over to npm or yarn. Please be careful
Difference between Bower and NPM
The biggest difference between Bower and NPM is that NPM does nested
dependency tree while Bower requires a flat dependency tree as below.
Quoting from What is the difference between Bower and npm?
NPM
project root
[node_modules] // default directory for dependencies
-> dependency A
-> dependency B
[node_modules]
-> dependency A
-> dependency C
[node_modules]
-> dependency B
[node_modules]
-> dependency A
-> dependency D
Bower
project root
[bower_components] // default directory for dependencies
-> dependency A
-> dependency B // needs A
-> dependency C // needs B and D
-> dependency D
There are some updates on npm 3 Duplication and Deduplication,
please open the doc for more detail.
Yarn
A new package manager for JavaScript published by Facebook recently with some more advantages compared to NPM. And with Yarn, you still can use both NPMand Bower registry to fetch the package. If you've installed a package before, yarn creates a cached copy which facilitates offline package installs.
jspm
JSPM is a package manager for the SystemJS universal module loader, built on top of the dynamic ES6 module loader. It is not an entirely new package manager with its own set of rules, rather it works on top of existing package sources. Out of the box, it works with GitHub and npm. As most of the Bower based packages are based on GitHub, we can install those packages using jspm as well. It has a registry that lists most of the commonly used front-end packages for easier installation.
See the different between Bower and jspm:
Package Manager: Bower vs jspm
2. Module Loader/Bundling
Most projects of any scale will have their code split between several files. You can just include each file with an individual <script> tag, however, <script> establishes a new HTTP connection, and for small files – which is a goal of modularity – the time to set up the connection can take significantly longer than transferring the data. While the scripts are downloading, no content can be changed on the page.
The problem of download time can largely be solved by concatenating a group of simple modules into a single file and minifying it.
E.g
<head>
<title>Wagon</title>
<script src=“build/wagon-bundle.js”></script>
</head>
The performance comes at the expense of flexibility though. If your modules have inter-dependency, this lack of flexibility may be a showstopper.
E.g
<head>
<title>Skateboard</title>
<script src=“connectors/axle.js”></script>
<script src=“frames/board.js”></script>
<!-- skateboard-wheel and ball-bearing both depend on abstract-rolling-thing -->
<script src=“rolling-things/abstract-rolling-thing.js”></script>
<script src=“rolling-things/wheels/skateboard-wheel.js”></script>
<!-- but if skateboard-wheel also depends on ball-bearing -->
<!-- then having this script tag here could cause a problem -->
<script src=“rolling-things/ball-bearing.js”></script>
<!-- connect wheels to axle and axle to frame -->
<script src=“vehicles/skateboard/our-sk8bd-init.js”></script>
</head>
Computers can do that better than you can, and that is why you should use a tool to automatically bundle everything into a single file.
Then we heard about RequireJS, Browserify, Webpack and SystemJS
RequireJS
It is a JavaScript file and module loader. It is optimized for in-browser use, but it can be used in other JavaScript environments, like Node.
E.g: myModule.js
// package/lib is a dependency we require
define(["package/lib"], function (lib) {
// behavior for our module
function foo() {
lib.log("hello world!");
}
// export (expose) foo to other modules as foobar
return {
foobar: foo,
};
});
In main.js, we can import myModule.js as a dependency and use it.
require(["package/myModule"], function(myModule) {
myModule.foobar();
});
And then in our HTML, we can refer to use with RequireJS.
<script src=“app/require.js” data-main=“main.js” ></script>
Read more about CommonJS and AMD to get understanding easily.
Relation between CommonJS, AMD and RequireJS?
Browserify
Set out to allow the use of CommonJS formatted modules in the browser. Consequently, Browserify isn’t as much a module loader as a module bundler: Browserify is entirely a build-time tool, producing a bundle of code that can then be loaded client-side.
Start with a build machine that has node & npm installed, and get the package:
npm install -g –save-dev browserify
Write your modules in CommonJS format
//entry-point.js
var foo = require("../foo.js");
console.log(foo(4));
And when happy, issue the command to bundle:
browserify entry-point.js -o bundle-name.js
Browserify recursively finds all dependencies of entry-point and assembles them into a single file:
<script src="”bundle-name.js”"></script>
Webpack
It bundles all of your static assets, including JavaScript, images, CSS, and more, into a single file. It also enables you to process the files through different types of loaders. You could write your JavaScript with CommonJS or AMD modules syntax. It attacks the build problem in a fundamentally more integrated and opinionated manner. In Browserify you use Gulp/Grunt and a long list of transforms and plugins to get the job done. Webpack offers enough power out of the box that you typically don’t need Grunt or Gulp at all.
Basic usage is beyond simple. Install Webpack like Browserify:
npm install -g –save-dev webpack
And pass the command an entry point and an output file:
webpack ./entry-point.js bundle-name.js
SystemJS
It is a module loader that can import modules at run time in any of the popular formats used today (CommonJS, UMD, AMD, ES6). It is built on top of the ES6 module loader polyfill and is smart enough to detect the format being used and handle it appropriately. SystemJS can also transpile ES6 code (with Babel or Traceur) or other languages such as TypeScript and CoffeeScript using plugins.
Want to know what is the node module and why it is not well adapted to in-browser.
More useful article:
https://medium.com/#housecor/browserify-vs-webpack-b3d7ca08a0a9#.c1q7ao3h4
http://jamesknelson.com/which-build-system-should-i-use-for-my-javascript-app/
https://appendto.com/2016/06/the-short-history-of-javascript-module-loaders/
Why jspm and SystemJS?
One of the main goals of ES6 modularity is to make it really simple
to install and use any Javascript library from anywhere on the
Internet (Github, npm, etc.). Only two things are needed:
A single command to install the library
One single line of code to import the library and use it
So with jspm, you can do it.
Install the library with a command: jspm install jquery
Import the library with a single line of code, no need to external reference inside your HTML file.
display.js
var $ = require('jquery');
$('body').append("I've imported jQuery!");
Then you configure these things within System.config({ ... }) before
importing your module. Normally when run jspm init, there will be a file
named config.js for this purpose.
To make these scripts run, we need to load system.js and config.js on the HTML page. After that, we will load the display.js file using
the SystemJS module loader.
index.html
<script src="jspm_packages/system.js"></script>
<script src="config.js"></script>
<script>
System.import("scripts/display.js");
</script>
Noted: You can also use npm with Webpack as Angular 2 has applied it. Since jspm was developed to integrate with SystemJS and it works on top of the existing npm source, so your answer is up to you.
3. Task runner
Task runners and build tools are primarily command-line tools. Why we need to use them: In one word: automation. The less work you have to do when performing repetitive tasks like minification, compilation, unit testing, linting which previously cost us a lot of times to do with command line or even manually.
Grunt
You can create automation for your development environment to pre-process codes or create build scripts with a config file and it seems very difficult to handle a complex task. Popular in the last few years.
Every task in Grunt is an array of different plugin configurations, that simply get executed one after another, in a strictly independent, and sequential fashion.
grunt.initConfig({
clean: {
src: ['build/app.js', 'build/vendor.js']
},
copy: {
files: [{
src: 'build/app.js',
dest: 'build/dist/app.js'
}]
}
concat: {
'build/app.js': ['build/vendors.js', 'build/app.js']
}
// ... other task configurations ...
});
grunt.registerTask('build', ['clean', 'bower', 'browserify', 'concat', 'copy']);
Gulp
Automation just like Grunt but instead of configurations, you can write JavaScript with streams like it's a node application. Prefer these days.
This is a Gulp sample task declaration.
//import the necessary gulp plugins
var gulp = require("gulp");
var sass = require("gulp-sass");
var minifyCss = require("gulp-minify-css");
var rename = require("gulp-rename");
//declare the task
gulp.task("sass", function (done) {
gulp
.src("./scss/ionic.app.scss")
.pipe(sass())
.pipe(gulp.dest("./www/css/"))
.pipe(
minifyCss({
keepSpecialComments: 0,
})
)
.pipe(rename({ extname: ".min.css" }))
.pipe(gulp.dest("./www/css/"))
.on("end", done);
});
See more: https://preslav.me/2015/01/06/gulp-vs-grunt-why-one-why-the-other/
4. Scaffolding tools
Slush and Yeoman
You can create starter projects with them. For example, you are planning to build a prototype with HTML and SCSS, then instead of manually create some folder like scss, css, img, fonts. You can just install yeoman and run a simple script. Then everything here for you.
Find more here.
npm install -g yo
npm install --global generator-h5bp
yo h5bp
See more: https://www.quora.com/What-are-the-differences-between-NPM-Bower-Grunt-Gulp-Webpack-Browserify-Slush-Yeoman-and-Express
My answer is not matched with the content of the question but when I'm searching for this knowledge on Google, I always see the question on top so that I decided to answer it in summary. I hope you guys found it helpful.
If you like this post, you can read more on my blog at trungk18.com. Thanks for visiting :)

OK, they all have got some similarities, they do the same things for you in different and similar ways, I divide them in 3 main groups as below:
1) Module bundlers
webpack and browserify as popular ones, work like task runners but with more flexibility, aslo it will bundle everything together as your setting, so you can point to the result as bundle.js for example in one single file including the CSS and Javascript, for more details of each, look at the details below:
webpack
webpack is a module bundler for modern JavaScript applications. When
webpack processes your application, it recursively builds a dependency
graph that includes every module your application needs, then packages
all of those modules into a small number of bundles - often only one -
to be loaded by the browser.
It is incredibly configurable, but to get started you only need to
understand Four Core Concepts: entry, output, loaders, and plugins.
This document is intended to give a high-level overview of these
concepts, while providing links to detailed concept specific
use-cases.
more here
browserify
Browserify is a development tool that allows us to write node.js-style
modules that compile for use in the browser. Just like node, we write
our modules in separate files, exporting external methods and
properties using the module.exports and exports variables. We can even
require other modules using the require function, and if we omit the
relative path it’ll resolve to the module in the node_modules
directory.
more here
2) Task runners
gulp and grunt are task runners, basically what they do, creating tasks and run them whenever you want, for example you install a plugin to minify your CSS and then run it each time to do minifying, more details about each:
gulp
gulp.js is an open-source JavaScript toolkit by Fractal Innovations
and the open source community at GitHub, used as a streaming build
system in front-end web development. It is a task runner built on
Node.js and Node Package Manager (npm), used for automation of
time-consuming and repetitive tasks involved in web development like
minification, concatenation, cache busting, unit testing, linting,
optimization etc. gulp uses a code-over-configuration approach to
define its tasks and relies on its small, single-purposed plugins to
carry them out. gulp ecosystem has 1000+ such plugins made available
to choose from.
more here
grunt
Grunt is a JavaScript task runner, a tool used to automatically
perform frequently used tasks such as minification, compilation, unit
testing, linting, etc. It uses a command-line interface to run custom
tasks defined in a file (known as a Gruntfile). Grunt was created by
Ben Alman and is written in Node.js. It is distributed via npm.
Presently, there are more than five thousand plugins available in the
Grunt ecosystem.
more here
3) Package managers
package managers, what they do is managing plugins you need in your application and install them for you through github etc using package.json, very handy to update you modules, install them and sharing your app across, more details for each:
npm
npm is a package manager for the JavaScript programming language. It
is the default package manager for the JavaScript runtime environment
Node.js. It consists of a command line client, also called npm, and an
online database of public packages, called the npm registry. The
registry is accessed via the client, and the available packages can be
browsed and searched via the npm website.
more here
bower
Bower can manage components that contain HTML, CSS, JavaScript, fonts
or even image files. Bower doesn’t concatenate or minify code or do
anything else - it just installs the right versions of the packages
you need and their dependencies.
To get started, Bower works by fetching and installing packages from
all over, taking care of hunting, finding, downloading, and saving the
stuff you’re looking for. Bower keeps track of these packages in a
manifest file, bower.json.
more here
and the most recent package manager that shouldn't be missed, it's young and fast in real work environment compare to npm which I was mostly using before, for reinstalling modules, it do double checks the node_modules folder to check the existence of the module, also seems installing the modules takes less time:
yarn
Yarn is a package manager for your code. It allows you to use and
share code with other developers from around the world. Yarn does this
quickly, securely, and reliably so you don’t ever have to worry.
Yarn allows you to use other developers’ solutions to different
problems, making it easier for you to develop your software. If you
have problems, you can report issues or contribute back, and when the
problem is fixed, you can use Yarn to keep it all up to date.
Code is shared through something called a package (sometimes referred
to as a module). A package contains all the code being shared as well
as a package.json file which describes the package.
more here

You can find some technical comparison on npmcompare
Comparing browserify vs. grunt vs. gulp vs. webpack
As you can see webpack is very well maintained with a new version coming out every 4 days on average.
But Gulp seems to have the biggest community of them all (with over 20K stars on Github)
Grunt seems a bit neglected (compared to the others)
So if need to choose one over the other i would go with Gulp

What is webpack & webpack-dev-server? Official documentation says it's a module bundler but for me it's just a task runner. What's the difference?
webpack-dev-server is a live reloading web server that Webpack developers use to get immediate feedback what they do. It should only be used during development.
This project is heavily inspired by the nof5 unit test tool.
Webpack as the name implies will create a SINGLE package for the web. The package will be minimized, and combined into a single file (we still live in HTTP 1.1 age). Webpack does the magic of combining the resources (JavaScript, CSS, images) and injecting them like this: <script src="assets/bundle.js"></script>.
It can also be called module bundler because it must understand module dependencies, and how to grab the dependencies and to bundle them together.
Where would you use browserify? Can't we do the same with node/ES6 imports?
You could use Browserify on the exact same tasks where you would use Webpack. – Webpack is more compact, though.
Note that the ES6 module loader features in Webpack2 are using System.import, which not a single browser supports natively.
When would you use gulp/grunt over npm + plugins?
You can forget Gulp, Grunt, Brokoli, Brunch and Bower. Directly use npm command line scripts instead and you can eliminate extra packages like these here for Gulp:
var gulp = require('gulp'),
minifyCSS = require('gulp-minify-css'),
sass = require('gulp-sass'),
browserify = require('gulp-browserify'),
uglify = require('gulp-uglify'),
rename = require('gulp-rename'),
jshint = require('gulp-jshint'),
jshintStyle = require('jshint-stylish'),
replace = require('gulp-replace'),
notify = require('gulp-notify'),
You can probably use Gulp and Grunt config file generators when creating config files for your project. This way you don't need to install Yeoman or similar tools.

Webpack is a bundler. Like Browserfy it looks in the codebase for module requests (require or import) and resolves them recursively. What is more, you can configure Webpack to resolve not just JavaScript-like modules, but CSS, images, HTML, literally everything. What especially makes me excited about Webpack, you can combine both compiled and dynamically loaded modules in the same app. Thus one get a real performance boost, especially over HTTP/1.x. How exactly you you do it I described with examples here http://dsheiko.com/weblog/state-of-javascript-modules-2017/
As an alternative for bundler one can think of Rollup.js (https://rollupjs.org/), which optimizes the code during compilation, but stripping all the found unused chunks.
For AMD, instead of RequireJS one can go with native ES2016 module system, but loaded with System.js (https://github.com/systemjs/systemjs)
Besides, I would point that npm is often used as an automating tool like grunt or gulp. Check out https://docs.npmjs.com/misc/scripts. I personally go now with npm scripts only avoiding other automation tools, though in past I was very much into grunt. With other tools you have to rely on countless plugins for packages, that often are not good written and not being actively maintained. npm knows its packages, so you call to any of locally installed packages by name like:
{
"scripts": {
"start": "npm http-server"
},
"devDependencies": {
"http-server": "^0.10.0"
}
}
Actually you as a rule do not need any plugin if the package supports CLI.

Yarn is a recent package manager that probably deserves to be mentioned.
So, here it is: https://yarnpkg.com/
As far as I know it can fetch both npm and bower dependencies and has other appreciated features.

Related

Publish NPM package with only non-bundled dependencies

Lets say we're developing a small javascript library L.
The code is in ES6. To use some utility function, like debounce, we install lodash as a dependency.
At build the webpack transpiles the code, bundling the tree shaked lodash code, and we end up with a nice little javascript file we want to publish and share as a npm package.
Now, the package.json file lists lodash as a dependency. But that is only true at build time, it is not really needed in production.
What's the proper way to handle this kind of situation?
Does it make sense to consider lodash a devDependency? As such, only webpack's externals would be "real" dependencies?
Or should we somehow tamper the package.json file before publishing it?
Do you know any real examples of projects handling this question?
Webpack "merges" the code of the project with that of the used code of the non-external dependencies into some bundle.js file. This file is then published to NPM, along with a package.json file, which lists all of the dependencies, independently of these being external or embedded.
All of the code of packages referenced in dependencies (or optionalDependencies, or peerDependencies) is expected to be "production" code.
While code in devDependencies is expected to be used only at "development" time, and is thus "development" code. Under this principle, I believe that it would be incorrect to declare non-external dependencies as development dependencies.
However, if all dependencies, embedded or external, are declared equally in the published package.json file, there is no way for a runtime environment to know which are the real dependencies that need to be made available to the package — the ones that the package will import at runtime and which better be available.
For the Node.js environment, bundles and Webpack are not normally used, so this never came to be an issue — all dependencies are always installed (never merged/bundled).
However, if you use the package.json file to drive some web-packages runtime environment, the way that dependencies are currently included in published package.json is not suitable.
You might want to take a look at Pika Web's devDependencies package.json property, which aims to solve a comparable problem (even though their mojo is "a future without Webpack"). They also introduce the concept of publishing a different package.json file than that which is checked-in (tampering the package.json before publishing, as you say).
Interestingly, it looks like a related tool, Pika Pack, caught the attention of NPM folks and is being considered to become part of NPM. So, maybe, NPM has become aware of the specific package format needs of web projects' workflows.

Javascript modules, loaders, bundlers and package managers confusion

I don't get all the connections. Where do they resides and what is the impact of their usage. Are any of following statments correct?
the loader will reside in browser(after loaded)? So when the main app javascript needs some module it will load it behind the scenes
if I use bundler(command line tool) it will create one file from all of my modules(and all common ones used), so no async loading, as there is only the big bundled.js?
why do I need for example bower.json and package.json in the same project(is npm used just to download build utilities)
from where do they take the modules (npm repository or all have its own?)
package managers are just to download the modules to your project for further processing
npm modules holds only .js files
you always needs way to orchastrate the build(gulp, grunt, npm scripts)
So far I have not found resource with information like : packagers do that, loaders do that.
the loader will reside in browser(after loaded)? So when the main app javascript needs some module it will load it behind the scenes
Loaders is a webpack feature, these loaders are used to process code that webpack does not understand by default (css, sass, images, etc.). They are just used when your code is compiled, when you publish your code it does not contains loaders with it. Loaders just run during build time.
--> Edit: this does not apply to the question since the question refeers to require.js for example, but still relevant overall.
if I use bundler(command line tool) it will create one file from all of my modules(and all common ones used), so no async loading, as there is only the big bundled.js?
You can have prefetch, prelight, you can load chunks (pieces of your bundle) only when required (look for code splitting using dynamic import import(...). This makes the code be loadede/requested only when it is going to be loaded.
why do I need for example bower.json and package.json in the same project(is npm used just to download build utilities)
Nowadays you don't need bower, most (if not all) the things that bower has you can find on the npm repositories.
from where do they take the modules (npm repository or all have its own?)
Every package you download was uploaded to npm, you can look at npm as a "github" but for libraries.
packagers are just to download the modules to your project for further processing
Yes
npm modules holds only .js files
False, look at bootstrap.
you always needs way to orchastrate the build(gulp, grunt, npm scripts)
Yes, but if you want to transpile content, you can use only babel-cli.

Why does Electron need to be saved as a developer dependency?

As per the official website, the correct way to save electron files is:
npm install electron --save-dev
Electron is actually required for running the app (quite literally: require()) and this goes against the top voted answer here. So why do we make this exception, if this is even one?
The fact that you require a package is irrelevant to whether it should be considered a dependency or a devDependency (in the npm sense). E.g. many projects use webpack API (i.e. const webpack = require('webpack')) but list it as a devDependency.
The reason is also explained in the post you link to: when you publish your package, if the consumer project needs other packages to use yours, then these must be listed as dependencies.
If your package uses some modules only for build, test, or bundles them into a dist file (i.e. what will be used by the consumer project), then those modules should not be mentioned in dependencies. We still list them in devDependencies for development.
Now in the case of an electron app, there is little chance you will consume your app as a node module of a consumer project, therefore the above convention is not really relevant.
Furthermore, we fall in the case where the electron package is bundled as part of the built output. There is no need for your user to get electron from npm to use your built app. Therefore it matches well the definition of a devDependency.
That being said, IIRC some electron packagers bundle your dependencies into the built app, so you still need some rigour in filling this list.
Cause those binary won't being used when you actually packaging into installer. Most of installer / packager for electron will build package with electron binaries, instead of using dependencies.

What is node_modules directory in AngularJS?

I am exploring AngularJS tutorial project and found it has node_modules directory inside, which size if 60 megabytes.
Does simple clientside javascript project really need so huge corpus of unknown data?
I tried to delete this directory and project still works. I suspect it somehow relates with node.js and it's npm but how? Suppose I need to run my project on some conventional web server (not node.js), then how to know, which files/directories are unneeded?
Many javascript libraries require to use bower to install them. If I use bower, does this mean I need to keep node_modules?
The node_modules directory is only for build tools.
The package.json file in the app root defines what libraries will be installed into node_modules when you run npm install.
Very often with an angular app, on your dev machine or on a build server, you use other Javascript libraries from npm (a node.js package manager) to build your angular app. Tasks can be concatenating resources, using CSS preprocessors like LESS or SASS, minification, replacing of values, etc. etc. The most common tools for managing and running these tasks are called grunt and gulp, which are installed through npm as well.
When you deploy your app, you only distribute the resulting build, not any of the source files or build tools.
It is of course possible to write an AngularJS app without building anything.
edit from comments: When you dive into Angular more, there are more advanced techniques of using libraries installed by npm even in the client app, you then selectively choose the ones you need, not the whole 50MB+ thing. I'd recommend staying with the basic approaches until you get a good grasp on them though.
NPM is the node package manager, which installs packages locally into a project, specifically, into the node_modules folder. From there the package code can be included into a project, yes, can is the important word there.
The browser has no way to include modules in code (yet), so you need to use a library that can expose node's commonJS style modules. Browserify and Webpack are two popular methods of doing so.
Angular complicates this by introducing its own module system, which more closely resembles AMD-style modules. There are ways around this, so that you can use node-style modules, maybe your project uses those.
Using npm for managing dependencies is a great idea, its a fantastic package manager. It is likely in your case though that the project is only built using node and that the node_modules folder contains dependencies only related to the build.

How to create an AngularJs directive in one node package, and then include it in another node package for the app?

If I have a directive, and I wish to package it in its own node package; and then include it from another another node package containing the main angularjs app, how would I do this?
My rough idea about how to go about this is:
put the html, javascript, and css for the directive in the package folder
enable compilation of these assets - preprocessing, minification (how?)
configure as bower package
in the app folder install the bower package
how to do this locally, without publishing?
in the angular.module() statement that creates the main app, add the name of the module containing the directive
Is this correct?
Have I missed out on anything?
Your idea of how to go about this looks good to me. To answer your questions in the list:
Look at Grunt or Gulp for your preprocessing / minification needs. These are both excellent build tools. Grunt is more widely used, but Gulp is newer and gaining a lot of ground. I'd look at both and use the one that suits you.
How to use a local bower dependency w/o publishing:
In your main app's bower.json file, instead of putting a version number for your module, put the folder where it can be found on your local system, like so:
{
"dependencies": {
"my-module": "/home/me/modules/my-module"
}
}
To clarify, you refer to it as a "node package" in your question, but in reality, you are creating a Bower package. Node packages (published to npmjs.org) are for node, and unless processed with something like Browserify, won't run in the browser (and even then, the node package can't do anything the browser doesn't support, like file access.) Bower packages (published on bower.io) are specifically for the browser. You will however find packages that publish to both NPM & Bower, such as jQuery or underscore, but you can't use the npm jquery package in the browser, it's built to run in node, and vice-a-versa.

Categories