From time to time I hear that CommonJS http://www.commonjs.org/ is an effort to create a set of modular javascript components but frankly I have never understood anything of it.
Where are the these modular components I can use? I don't see much on their homepage.
CommonJS is only a standard that specifies a way to modularize JavaScript, so CommonJS itself does not provide any JavaScript libraries.
CommonJS specifies a require() function which lets one import the modules and then use them, the modules have a special global variable named exports which is an object which holds the things that will get exported.
// foo.js ---------------- Example Foo module
function Foo() {
this.bla = function() {
console.log('Hello World');
}
}
exports.foo = Foo;
// myawesomeprogram.js ----------------------
var foo = require('./foo'); // './' will require the module relative
// in this case foo.js is in the same directory as this .js file
var test = new foo.Foo();
test.bla(); // logs 'Hello World'
The Node.js standard library and all 3rd party libraries use CommonJS to modularize their code.
One more example:
// require the http module from the standard library
var http = require('http'); // no './' will look up the require paths to find the module
var express = require('express'); // require the express.js framework (needs to be installed)
The idea, it seems (I wasn't aware of this), is to provide javascript to more than just web browsers. For example, CouchDB supports javascript for querying.
Related
I am trying to answer,
when to use import/export and when to use require()/module.exports? But as I try to dig, it seems to get complicated.
Here's my understanding
require()/module.exports: this is nodejs implementation of the module system. This loads the modules syncronously.
with es6, we can use import/export. the docs says
The import statement is used to import bindings which are exported by another module. Imported modules are in strict mode whether you declare them as such or not. The import statement cannot be used in embedded scripts unless such script has a type="module".
Ques1: How does this work with babel or webpack or browsers in general?
As I was exploring I came across stuff like CommonJs, requireJs, Asynchronous Module Definition (AMD)
Ques2: I am more interested in knowing the timeline as how these things evolved in javascript ?
How does this work with babel or webpack or browsers in general?
Babel and Webpack follow the ES spec and transpile the import / export statement to one single file. As they also support the require syntax, they usually transpile the import statements to require() calls and the export statements to module exports, and then ship with a custom loader for modules., If you got for example:
// A.js
export default function() { }
// B.js
import A from "./A";
A();
Then it gets transpiled to the following require syntax:
//A.js
exports.default = function() {};
//B.js
var A = require("./A").default;
A();
That could then get wrapped to something like:
(function() { // prevent leaking to global scope
// emulated loader:
var modules = {};
function require(name) { return modules[name]; }
function define(name, fn) {
var module = modules[name] = { exports: {} };
fn(module, module.exports, require);
}
// The code:
define("A", function(module, exports, require) {
// A.js
exports.default = function() { };
});
define("B", function(module, exports, require) {
// B.js
var A = require("A").default;
A();
});
})();
how these things evolved in javascript ?
A few years ago, writing JS was restricted to browsers, and the only way to load multiple js sources was to use multiple <script> tags and use the global object to exchange functionality. That was ugly.
Then Nodejs was invented and they needed a better way to work with modules and invented the require() thing.
The writers of the spec saw a need for a native syntax for that, so import / export were introduced.
Babel and others then wrote transpilers.
What webpack the bundler does is the following:
You specify an input file in the config
You specify an output file the config
Webpack will look at all the files which the input file requires (commomJS module system) or imports (ES6 module system). It then funnels the code based on file name extention through loaders. Loaders can transpile the individual files to code the browser can understand. An example of a loader is babel or the sass/scss compiler.
After the different files are transpiled with loaders, the plugins can work at the
transform the bundle of generated code into something else. The bundle is just a bunch of code which together forms piece of functionality
In won't go into detail in the internals of webpack too deeply, but the most important thing to understand is:
You use webpack so you can use split up your code in multiple files, which makes them more maintainable and easier to work with. However then requesting all these files by the client would be horrible for performance (many HTTP requests overhead). Therefore, we bundle the files into one file, or a couple so this overhead is reduced.
Generally, you should write all modern code with import/export syntax if you are using a bundler like webpack, or translating with Babel... npm modules may favor require/module syntax but you can still import them.
Also worth noting is the import() method which returns a promise that should resolve to the root module being imported asynchronously. Webpack may bundle these as asynchronous modules if you have it configured to do so.
In practice the resolution through tooling like babel and webpack will fallback to node-style behaviors against the node_modules lookup, where the standard is transport paths, favoring relative paths. Additional support is per environment.
You can experiment with esm support in modern browsers and in current node (behind a flag as of this answer). The behaviors are somewhat inconsistent, but well defined. When in doubt, experiment and try.
I don't like the whole export/require stuff in node, it takes too long. Let's say I have a file server.js and I want to use functions in whatever.js. in html I just add this to the header:
<script src='whatever.js'></script>
and then I can just use all the functions of whatever.js in my body's script.
But in node, in the server.js file I'd do:
var myobject = require('./whatever.js');
but then I need to set it to myobject, and further I need to go to whatever.js and manually decide what functions I want to export. not to mention that typing myobject.someFunction() is alot longer to write than someFunction() and I need to remember what I exposed/didn't expose.
I wanted something where I could just go:
require('./whatever.js');
and it puts it ALL in global, no bs. like in good old html/javascript. Is there a way to do this in node?
This will do the trick,
var fs = require('fs');
eval(fs.readFileSync('whatever.js')+'');
// here call functions from whatever.js file
(I realize this is an old thread but wanted to leave a note here for posterity.)
Here in 2022 there are several approaches for executing code from different files with Node.js:
ESM: Use standard ECMAScript modules
At the time of this writing, much of the node ecosystem (i.e. packages on npm) is in the process of transitioning to this paradigm, and there are some associated growing pains (e.g. things like __dirname are only available in CJS not ESM, though the workaround is easy).
For most developers, it would be advisable to become comfortable with this standard as it transcends node.js (i.e. is implemented in other runtimes like Deno and web browsers) and has been years in the making.
CJS: Use the original "CommonJS" module mechanism, e.g. require('./some-script.js')
It should be noted, particularly for the OP, that even though the "intended" way to use CJS modules is to export functions, constants, etc. and import them explicitly, it is possible to define everything in global scope using globalThis, though I would not recommend this.
// my-script.js
require('./foo.js');
require('./bar.js');
foo(); // This is foo from <...>foo.js
console.log(`bar = ${bar} (in ${__filename})`); // bar = 123 (in <...>my-script.js)
// foo.js
globalThis.foo = function() {
console.log(`This is foo from ${__filename}`);
}
// bar.js
globalThis.bar = 123;
If you try omitting globalThis. you'll find that foo and bar are no longer defined in the main script because require "wraps them" in "module scope."
Use eval
In my experience, there are very few legitimate use cases for eval (see Never use eval()!). Nevertheless, the functionality requested in this question is precisely what eval provides: "run some code as if it were written right here" and you can feed it from a file, as explained above by Mehul Prajapati
// include.js
// Defines a global function that works like C's "#include" preprocessor directive
const { readFileSync } = require('fs');
globalThis.include = function(scriptFile) {
console.warn('!!! EXTREMELY INSECURE !!!');
eval(readFileSync(scriptFile, 'utf-8'));
};
// main.js
require('./include.js'); // loads global include
// (I sure hope you completely trust these sources)
include('./foo.js');
include('./bar.js');
Note: Something that has contributed to much of my confusion in the past is that there have been competing standards/conventions/APIs that use some of the same identifiers, namely require, which require.js and other bundlers that support AMD (Asynchronous Module Definition)
use with different semantics. So for someone building a web application (using AMD for modules in web browsers) with node.js tooling (using CJS for modules locally) it can be frustrating to keep the functions straight, especially if it's an Electron application, which can expose Node.js APIs to scripts running in the renderer (browser). If you find yourself confused why a module is "not found" in a situation like that, check the stack trace to see which require is being called (and you may have to wrap/rename them on globalThis or something to avoid collisions).
Further reading:
JavaScript Modules: A Brief History [2019]
How the module system, CommonJS & require works [updated 2022]
What is AMD, CommonJS, and UMD? [2014]
I am currently developing a local JavaScript app that will help me to dynamically generate some managerial documents such as invoices and vouchers for unpaid services for a specific client.
I'll be using node_modules, and so far I am using (["jsonfile", "uniqid", "mkdirp"]) these modules need require to be stored in a variable.
The problem is that I am calling jsonfile as mention in its documentation
var jsonfile = require("jsonfile");
When console.log(jsonfile) from javascript file it returns function localRequire() unlike if i wrote that command inside browsers console .. for that it will return
>> console.log(jsonfile)
Object { spaces: null, readFile: readFile(), readFileSync: readFileSync(), writeFile: writeFile(), writeFileSync: writeFileSync() }
undefined
and to make the problem more complicated, when calling readFile() from console or from the script it returns that readFile() is not a defined function rather for jsonfile or fs which is a node module that is required for jsonfile
>>function readFile()
jsonfile.readFile()
TypeError: fs.readFile is not a function
I don't really know what is the difference and how to use that variable from console than the variable of script. Especially that I don't redefine it in console after running the app.
Some code snippets you may need:
I am calling two scripts in index.html:
<!--requirejs -->
<script data-main="app" src="node_modules/requirejs/require.js"></script>
<!-- a script which will use $ which is not defined -->
<script src="services/specifyDocType.js"></script>
<!--the script the call require and have the jsonfile variable -->
<script src="jsonfile-test.js"></script>
app.js code:
requirejs.config({
//"baseUrl": "node_modules",
"paths": {
"app": "app",
"jsonfile": "jsonfile/index",
"graceful-fs": "jsonfile/node_modules/graceful-fs/graceful-fs",
"fs": "/node_modules/jsonfile/node_modules/graceful-fs/fs",
"jquery": "node_modules/jquery/dist/jquery.min"
}
});
requirejs(["dist/js/main"]);
main.js :
define("jquery", function($) {
$('body');
});
For the local script that will create jsonfile variable, here is the initial code I had to do the working test:
var data = 'DOCS/voucher/voucher.json';
var jsonfile = require(["node_modules/jsonfile/index"]);
console.log(jsonfile);
//this console that returns the function localRequire() mentioned above.
jsonfile.readFile(data, function(err, obj) {
console.dir(obj);
});
For this app, I don't need to use a framework such as Angularjs, and I need to use require for each module I am using so far to create unique ids and directories in the root directory.
One more thing, this is my console on the first run:
function localRequire() jsonfile-test.js:4:1
TypeError: jsonfile.readFile is not a function[Learn More] jsonfile-test.js:6:1
ReferenceError: module is not defined[Learn More] index.js:133:1
Error: Module name "fs" has not been loaded yet for context: _. Use require([]) http://requirejs.org/docs/errors.html#notloaded require.js:168:17
Use of getPreventDefault() is deprecated. Use defaultPrevented instead.
You seem to think that RequireJS is a drop-in replacement for Node's require call. It isn't.
Node supports the CommonJS module system. RequireJS supports the Asynchronous Module Definition (AMD) system. (See here for a question about the difference between the two.) It also provides support to facilitate consuming CommonJS modules, but it does not support consuming CommonJS modules as-is. At the very least, the CommonJS module must be wrapped in a define call. If you start with this CommonJS module, for instance:
function foo() {}
exports.foo = foo;
then you need at a minimum to convert it to this:
define(function (require, exports) {
function foo() {}
exports.foo = foo;
});
In addition, while RequireJS can consume CommonJS modules, it is not able to provide support for those modules that depend on the node VM. fs is one such module. It depends on the Node VM. It cannot be run in the browser. The DOM does not expose an API that the fs module could use to access the filesystem. jsonfile depends on fs so you cannot just load it with RequireJS in the browser. You could possibly find a module made to run in browsers that present an API compatible with fs and present a fake filesystem to the client code and this may work, but you have to do the job of finding the replacement module. RequireJS won't do this for you.
I was wondering what the difference between these snippets is.
var $ = require('jquery');
var _ = require('underscore');
var BackBone = require('backbone');
and
require(['jquery','underscore','backbone'],function ($, _, BackBone){
//code goes here
})
Both work fine for me, but not sure is their any purpose behind the above two.
Consider the first snippet, which is in the CommonJS style:
var $ = require('jquery');
var _ = require('underscore');
var BackBone = require('backbone');
These calls are synchronous calls: when require returns, it returns the module you requested. CommonJS require calls are synchronous. There is a proposal for supporting asynchronous forms of require but as far as I can tell it has not progressed beyond the proposal level. Node.js used to have require.async which has been removed. There's a package that implements it though. Using this package looks at lot like using AMD style modules.
Now, consider the second snippet, which is in the AMD style:
require(['jquery','underscore','backbone'],function ($, _, BackBone){
//code goes here
})
Since RequireJS implements the AMD type of module system, the above code works with RequireJS. This require call is --- as suggested by the name Asynchronous Module Definition (AMD) --- asynchronous. You can't rely on the return value of require to get a module value. You have to use a callback instead. The define call works in a similar way but defines a module in addition to requiring modules.
Now, if you use RequireJS, it provides facilities that allow to use either style when you define modules so that you can define a module like this:
define(['jquery','underscore','backbone'],function ($, _, BackBone){
//code goes here
});
Or use something that looks more like the CommonJS idiom like this:
define(function (require) {
var $ = require('jquery');
var _ = require('underscore');
var BackBone = require('backbone');
//code goes here
});
It makes it really easy to convert a CommonJS style module to use with RequireJS: just wrap it with a define call as above. There's a tool to help the conversion.
Behind the scenes, RequireJS reads the code of the callback in the 2nd form and creates a list of dependencies so that in the end it is interpreted like:
define(['require', 'jquery','underscore','backbone'], function (require) {
var $ = require('jquery');
var _ = require('underscore');
var BackBone = require('backbone');
//code goes here
})
It may be surprising (given that AMD is asynchronous) that the require calls in the callback are synchronous. This is part of RequireJS's support for the CommonJS style. RequireJS supports a kind of synchronous require call but with the following caveat: if the module is already defined before the call to the synchronous require, then the synchronous require returns the module's value, but otherwise it fails immediately. That is, it does not try to load a module. Because RequireJS interprets a module definition that uses the CommonJS style as I've shown above --- as if the dependencies were actually listed in the define arguments --- then these modules are guaranteed to be loaded by the time the synchronous calls to require are made.
Besides being able to use CommonJS modules in RequireJS (provided a wrapper is added), it is also possible to use modules designed for RequireJS in a CommonJS environment like Node.js. For instance, I've used node-amd-loader to load modules I've designed as AMD modules in Node.js.
The former snippet adheres to the CommonJS spec, which outlines how modules are to be exported/transported to be used in other modules.
So, you could write code which could look like this
src/ExpressModule.js
var express = require('express');
// do initialization
exports.express = express;
And then you can use it like this
src/UserExpressModule.js
var express = require('./ExpressModule');
// do something else
CommonJS spec was built primarily for server side environments, and by extension, is used in Node.js
On the other hand of the spectrum is AMD, most notably implemented by RequireJS. AMD was built for browser environments, or client-side code, and here you can use and define modules like the latter snippet in your code
A sample use could be like this
src/main.js
requirejs.config({
baseUrl: '../vendor'
path: '../src'
});
require(['jquery', 'underscore', 'backbone'], function($, _, Backbone) {
// do some client side computation here
});
You can then include this main.js file in your index.html like so
<script data-main="src/main.js" src="vendor/require.js"></script>
Essentially, CommonJS and AMD are at their core specifications for modularizing Javascript, but are aimed at different execution environments.
I'm working on a mobile website (it's not a single page app) which has a very small JS footprint (less than 10KB minified and gzipped). There are no libraries or external dependencies, all the code is vanilla javascript written in house. It's logically separated into several files that are concatenated before deplyment in order to reduce the number of HTTP requests. There is no explicit namespacing in the files. That is, they look something like:
// crossbrowser.js
function getScrollOffset() {
// implementation
}
function ...
This is less than ideal, there is no explicit dependency resolution and the scope can get easilly polluted from inside the functions. There is no processing done to check this (lint or compiler). As a first step I thought implementing an explicit module system could safeguard against this and make the code better.
Reading into the CommonJS module format and loaders like RequireJS, Lab.js and others, as far as I understood, when using modules on the browser side, they all expect to load them via XHR. I don't want that, I want to keep the single script format which contains all the modules. My file would look something like:
var define = function () { /* ... */ };
var require = function () { /* ... */ };
define("crossbrowser", function (require, exports, module) {
exports.getScrollOffset = function() {
//
};
// etc.
});
define("foo", function (require, exports, module) {
var crossbrowser = require('crossbrowser');
exports.getNewOffset = function () {
var offset = crossbrowser.getScrollOffset();
// do something
return offset;
}
});
window.addEventListener('DOMContentLoaded', function () {
// really dumb example, but I hope it gets the point across
var crossbrowser = require('crossbrowser'),
foo = require('foo');
crossbrowser.scrollTo(foo.getNewOffset());
});
The question is wether any of the loaders work this way or do I have to write my own implementation of define and require?
One of the benefit with loaders like requirejs is that you can use optimizers to combine all your modules into one minified script during your build process, see RequireJS Optimizer
This would allow you to develop in a modular structure but deploy an optimized version
Have a look at webmake.
Webmake allows to use CommonJS modules in the browser. All js files are merged into one js file. The loader is very lightweight. Webmake also works with CoffeeScript.
If you don't need asynchronous loading, you don't need an AMD loader. For example, if you use r.js to combine modules and you have to load in production code the whole RequireJS library. Why not simply going with a compiler? Just take a look at the slides on CommonJS Compiler http://www.slideshare.net/dsheiko/modular-javascript-with-commonjs-compiler and here the sources/docs http://dsheiko.github.io/cjsc/