I am building Angular Apps, using 1.x and have been for some time now. I use Bower to install Angular and the various packages that go with it and some other bits and pieces like Shivs, JQuery, ChartJs etc etc. I love using Bower as it's nice and quick and keeps everything in a consistent place for me to reference. I use Grunt as well as my task-runner and so I'd also like to be able to automate this process for silky smooth development.
Now as my Angular knowledge has increased and the scale of the apps that I'm building are increasing, I'm finding myself including dozens of calls to files within the index.html and I really want to tidy all this up, ideally into a nice app.js making it all much more manageable, not only for myself but for anyone else coming in and working on these apps.
I've seen maaany tools like requirejs, browserify and commonjs to name but a few, which all provide the kind of functionality I'm after, but when reading various tutorials or watching conference talks on the process, they all seem to conflict with one another on which is the best. I know to some degree (as with all these competing technologies) it's personal preference, and I was leaning towards browserify, but apparently this removes bower from the flow and uses NPM instead. I'd like to stick with Bower if possible. I've really enjoyed using it.
Does anyone have any suggestions or best practices they can offer that might clear this up for me? Would a simple concat with grunt/gulp just be the way to go?
Any useful comments/answers would be much appreciated.
Many thanks.
Use ES6 Modules along with a module bundler (my recommendation would be Webpack).
As you have correctly identified RequireJS and commonjs evolved around different (and slightly conflicting) goals and are incompatible. ES6 modules is a standardized effort towards modular javascript that is already well supported by transpilers (eg. Babel).
This article provides a great introduction to this new feature:
Even though JavaScript never had built-in modules, the community has
converged on a simple style of modules, which is supported by
libraries in ES5 and earlier. This style has also been adopted by ES6:
Each module is a piece of code that is executed once it is loaded.
In
that code, there may be declarations (variable declarations, function
declarations, etc.).
By default, these declarations stay local to the
module.
You can mark some of them as exports, then other modules can
import them.
A module can import things from other modules. It refers
to those modules via module specifiers, strings that are either:
Relative paths ('../model/user'): these paths are interpreted
relatively to the location of the importing module. The file extension
.js can usually be omitted.
Absolute paths ('/lib/js/helpers'): point
directly to the file of the module to be imported.
Names ('util'):
What modules names refer to has to be configured. Modules are
singletons. Even if a module is imported multiple times, only a single
“instance” of it exists. This approach to modules avoids global
variables, the only things that are global are module specifiers.
Example of use of Javascript modules in practice:
//------ lib.js ------
export const sqrt = Math.sqrt;
export function square(x) {
return x * x;
}
export function diag(x, y) {
return sqrt(square(x) + square(y));
}
//------ main.js ------
import { square, diag } from 'lib';
console.log(square(11)); // 121
console.log(diag(4, 3)); // 5
Related
Current Javascript adopts import from ES6 as a standard way to import modules.
However, I sometimes see codes using CommonJS require instead of import.
I first wondered whether two can be used together, but it seems like two are not interchangeable. (releated stackoverflow question)
So, is CommonJS 'require' something that is still used in projects? Or is it slowly dying and needed only for maintaining legacy codes?
CommonJS will likely be supported in nodejs for a long time as there are still millions of lines of code written using it. It is the original module loading mechanism in nodejs. It is not deprecated.
ESM modules (ECMAScript modules) that use import and export are the new Javascript standard for modules and we can expect that nodejs will support these for as long as they are the Javascript standard (probably forever).
These two modules standards are not entirely compatible so mixing and matching within the same project can lead to complications that you have to learn how to deal with.
New Projects
If I were starting a new project today, I'd choose to write my code using ESM modules as it is the future trajectory of the language and nodejs. And, you can still load CommonJS modules into ESM modules if you require backward compatibility with other modules, but you do have to know how to do it properly - it's not always seamless to mix and match module types.
When ESM modules were first supported in nodejs, the interoperability with CommonJS modules was not very full featured and created some difficulties. As of the more recent versions of nodejs, you can load a CommonJS module from an ESM module or vice versa- you just have to use the right techniques to do it. This makes working from an ESM project a lot more feasible now than it was when ESM support in nodejs first came out as you can still access libraries in NPM that only support CommonJS.
Note also that more and more libraries in NPM are supporting direct loading from either module type so they are equally easy to use whether your project is CommonJS or ESM.
Over time, I would expect that any actively developed, shared module on NPM will eventually support direct loading as an ESM module. But, we're in this transition period of time where many have not yet implemented that or there are particular challenges in implementing the new loading scheme (which comes with it's own and different set of rules). In the meantime, you can still load CommonJS modules into ESM projects.
Existing CommonJS Projects
If I had an existing project that was CommonJS, I would not be spending any time thinking about converting it to ESM because there is still widespread support for CommonJS and my developer time is probably better spent on adding features, testing, fixing bugs, etc... than converting module formats.
Interoperability
One of the important interoperability things to know is that you can load a CommonJS module from an ESM module in several different ways:
With a static import: import { callMeCommon } from '../otherProject/common.js';
With a dynamic import import(someFile).then(...)
By using module.createRequire(import.meta.url) and then using that require() function to load your CommonJS module.
You can also use the dynamic import('someModule').then(...) to load an ESM module from a CommonJS module, but it's not synchronous like require() was so it has to be dealt with differently.
It's also useful to know that ESM modules do not have some nodejs niceties that CommonJS modules had. There's no automatic __dirname or __filename to tell you exactly where this file was loaded from. Instead, you have to parse those values out of import.meta.url and compute them. So, it's still possible to get that information, it's just not as convenient as it used to be.
On the other hand, ESM modules have some features like top level await that CommonJS do not which can be very useful.
Your Specific Questions
So, is CommonJS 'require' something that is still used in projects?
Yes, it is still used a lot.
Or is it slowly dying and needed only for maintaining legacy codes?
[Comments as of early 2022] I wouldn't so much say that it is dying as there are very few projects on NPM that don't still support CommonJS. In fact, when a project releases a new version that no longer supports CommonJS, it creates quite a problem for their user base (we see some of these issues here on stackoverflow) because of the significant prevalence of CommonJS projects still and people not familiar with how to load different module types.
So, I'd say that we're still in a very early stages of a transition from CommonJS to ESM and it's more like ESM is getting more and more adoption rather than CommonJS is dying. I'd hazard a guess that the majority of modules on NPM will, over the next few years, move to support direct loading from both module formats.
Transpiling
Lastly, developers often use transpiling to allow them to write code using the newest syntax and features, but using a transpiler to convert the code back to a lower, common denominator that runs anywhere. That can also be done for module architecture (write code in ESM, transpile to CommonJS).
A Few Interesting References
What does it take to support Node.js ESM?
Hybrid npm packages (ESM and CommonJS)
Node Modules at War: Why CommonJS and ES Modules Can’t Get Along
All you need to know to move from CommonJS to ECMAScript Modules (ESM) in Node.js
In Nodejs it doesn't yet appear to be deprecated given the LTS v20 precursor (v19) documentation describes the feature:
https://nodejs.org/api/esm.html#interoperability-with-commonjs
https://github.com/nodejs/Release
If locked into Nodejs the above affords a length of time into 2025 for using LTS Nodejs runtimes, with legacy code, and possibly polyfill the same whenever the Nodejs product changes status (the module system or the whole runtime).
CommonJS is the legacy module system specific to Nodejs
see https://wiki.commonjs.org/wiki/Modules/Meta
Generally outside of Nodejs Commonjs/require has been displaced in JS/TS runtimes by ESModules:
https://developer.mozilla.org/docs/Web/JavaScript/Guide/Modules
As much as possible I've been moving away from Nodejs to Deno and haven't looked at the specifics and details of its npm and Nodejs compatibility. For those who're looking to implement software using contemporary web platform features it's the best forward-looking with current working solutions that I'm aware of (more at https://deno.land/).
import { createRequire } from "https://deno.land/std/node/module.ts";
// import.meta.url...like `__filename` ...
const require = createRequire(import.meta.url);
const path = require("path");
I've been using Modules that are presented with ES6 for a while. While I (instinctively) know that it is better to use them, I struggle to explain why.
With modules;
I can do dependency declaration (not injection). I do not have to use a script tag for each javascript file I load.
Before modules, objects served as namespaces. Now that we have modules, we have namespaces which is a better paradigm for code organization.
Else than this, why should I use modules ?
What advantage they provide over just using objects as namespaces ?
EDIT:
Thanks to the comments of Bergi and Randy Casburn, I am now able to point out two more things.
With Module Pattern(which i been calling object-as-namespaces) we have same functionality. But we get it through closures. In Modules, we have separate files. Features of a separate file is harder to be violated.
Modules hide their internal features. It drives us towards thinking about better software design.
Answering my own question. Thanks for the comments.
Modularity is a must for software. Before ES Modules, we achieved modules through Module Pattern(which is simply putting related things in an object). By ES6, Modules are native. And native is always the better option.
In Module Pattern, we made modules with closures. In ES Modules we code them in seperate files.
Seperate files adds another layer of protection against code violation.
Modules drive us to design better software.
With ES Modules we have export keyword which gives us freedom to choose what to expose.
We also have import keyword to pick whatever we want from a module.
With ES6 Modules we can now include Javascript modules which are in seperate files into our web app without the need to include them in script tags.
TL;DR; Is it possible to implement package inheritance in Node? Or is there a recommended tool to help with this?
I am working in an enterprise environment containing 60+ (and growing) web apps. We’ve also modularized components such as header/footer/sign-in/etc. so the web apps don’t need to repeat the code, and they can just pull them in as dependencies. We also have library modules to provide things like common logging, modeling, and error handling. We also have one main library that provides a foundation for maintenance like testing, and linting.
What I am trying to do is get access to the dependencies this main library uses in upper level modules.
lib-a
|
—> lib-b
|
—> babel, chai, mocha, etc.
I would like to have lib-a “inherit” babel, chai, mocha, etc. from lib-b rather than having to specifically add the dependencies. That way all my libraries, and eventually web apps will have the same version, and I won’t have to repeat the same dependencies in every package.json. Nor will I need to go through the headache of having N number of teams update the 60-100 apps/libs/whatnot, and having to deal with them complaining about maintenance.
I do understand this goes against the core of npm, but on the level we are using this it’s becoming a maintenance headache. Going more DRY would certainly have it’s benefits at this point.
So to repeat the original question at the top - Is it possible to implement package inheritance in Node? Or are there any recommended tools to help with this?
I have seen the following tools. Has anyone ever used them? or have thoughts on them. Are there others?
https://github.com/FormidableLabs/builder
https://github.com/Cosium/dry-dry
It's a bad idea. You should assume that you don't have control over the dependencies. How else would anybody be able to make changes to the dependencies?
Suppose lib a from your example uses mocha. Since it depends on lib b which also depends on mocha, you could decide to not list mocha in lib a's package.json.
If somebody refactors lib b to not use mocha anymore, lib a will fall all a sudden. That's not good.
We work with equally many projects. We use Greenkeeper, RenovateBot, and some tools that apply changes to all our repos at once. In the long run that's probably a better strategy for you than going against Node's dependency model.
Can someone provide some information about Module Loaders and Module Bundlers in JavaScript?
What are the differences?
When should I use a Module Loader and when a Module Bundler?
Why do we need them at all?
Module loaders and bundlers both make it more actionable to write modular JavaScript applications. Let me give you some background:
Module loaders
A module loader is typically some library that can load, interpret and execute JavaScript modules you defined using a certain module format/syntax, such as AMD or CommonJS.
When you write modular JavaScript applications, you usually end up having one file per module. So when writing an application that consist of hundreds of modules it could get quite painful to make sure all files are included and in the correct order. So basically a loader will take care of the dependency management for you, by making sure all modules are loaded when the application is executed. Checkout some popular module loaders such as RequireJS and SystemJS to get an idea.
Module bundlers
Module bundlers are an alternative to module loaders. Basically they do the same thing (manage and load interdependent modules), but do it as part of the application build rather than at runtime. So instead of loading dependencies as they appear when your code is executed, a bundler stitches together all modules into a single file (a bundle) before the execution. Take a look at Webpack and Browserify as two popular options.
When to use what?
Which one is better simply depends on your application's structure and size.
The primary advantage of a bundler is that it leaves you with far fewer files that the browser has to download. This can give your application a performance advantage, as it may decrease the amount of time it takes to load.
However, depending on the number of modules your application has, this doesn't always have to be the case. Especially for big apps a module loader can sometimes provide the better performance, as loading one huge monolithic file can also block starting your app at the beginning. So that is something you have to simply test and find out.
ES6/ES2015 Update
Note that ECMAScript 2015 (or ES6) comes with it's own, native implementation of modules. You can get a quick intro here and here.
So I've been building a single page web app for a while now and I've been writing all my code in an extremely modular approach. I've been using the javascript module pattern for all modules and my main API uses the revealing module pattern to expose a small API for plugins and other modules.
So even if I've been writing my code like this someone mentioned I should be using require.js as it gives a better modular approach.
I decided that require.js doesn't really make it more modular so my next thought was how require.js separates out dependencies. Require.js forces you to name dependencies in each module file. But this to me seems to be a task I have to do for every module and I have a lot of modules. At the moment all my files are concatenated into a single javascript file in my grunt build process so really all my files are loaded at the start. I need most of my modules loaded at the start so it made sense to do this.
So my question is... Should I use require.js even though my code is modular and all my files are concatenated together and loaded at the start? Do i need to worry about dependencies or loading order? Any help/advise or previous experience when dealing with this situation would really help. Thanks
Have you had annoyances putting <script> tags in the correct order to handle dependencies right?
Having a requirejs config file where you declare the third-party code and your own code dependency tree is a much more organised approach than declaring <script> tags by yourself.
When you are testing your app in dev environment, wouldn't be helpful to have all those modules in separated files that are easier to debug instead of all of them concatenated?
With requirejs you can switch between optimised/concatenated/minified code used in production environment and a list of independent files representing each module in the development environment.
Are you creating a global variable for each module in your app?
Requirejs avoids creating a global variable for each module so your global scope doesn't get cluttered.
It's usually a good practice to follow conventions in the way you declare modules. Requirejs implements AMD specification that is a well thought way of loading modules in client javascript.
http://requirejs.org/docs/whyamd.html
It's way easier to follow conventions if they're already implemented in a mature framework. And sometimes we don't notice but between apps we do the same thing differently and that affects in the maintenance phase.
Take a look at the requirejs optimizer. Look all the options it give you. It allows you to exclude files, to change versions of a given module, to change the minification tool, to integrate the optimization process with other tools like grunt or bower, etc.
http://requirejs.org/docs/optimization.html
If you don't need any of this, then just continue doing it in your way.