I'm working on a mobile website (it's not a single page app) which has a very small JS footprint (less than 10KB minified and gzipped). There are no libraries or external dependencies, all the code is vanilla javascript written in house. It's logically separated into several files that are concatenated before deplyment in order to reduce the number of HTTP requests. There is no explicit namespacing in the files. That is, they look something like:
// crossbrowser.js
function getScrollOffset() {
// implementation
}
function ...
This is less than ideal, there is no explicit dependency resolution and the scope can get easilly polluted from inside the functions. There is no processing done to check this (lint or compiler). As a first step I thought implementing an explicit module system could safeguard against this and make the code better.
Reading into the CommonJS module format and loaders like RequireJS, Lab.js and others, as far as I understood, when using modules on the browser side, they all expect to load them via XHR. I don't want that, I want to keep the single script format which contains all the modules. My file would look something like:
var define = function () { /* ... */ };
var require = function () { /* ... */ };
define("crossbrowser", function (require, exports, module) {
exports.getScrollOffset = function() {
//
};
// etc.
});
define("foo", function (require, exports, module) {
var crossbrowser = require('crossbrowser');
exports.getNewOffset = function () {
var offset = crossbrowser.getScrollOffset();
// do something
return offset;
}
});
window.addEventListener('DOMContentLoaded', function () {
// really dumb example, but I hope it gets the point across
var crossbrowser = require('crossbrowser'),
foo = require('foo');
crossbrowser.scrollTo(foo.getNewOffset());
});
The question is wether any of the loaders work this way or do I have to write my own implementation of define and require?
One of the benefit with loaders like requirejs is that you can use optimizers to combine all your modules into one minified script during your build process, see RequireJS Optimizer
This would allow you to develop in a modular structure but deploy an optimized version
Have a look at webmake.
Webmake allows to use CommonJS modules in the browser. All js files are merged into one js file. The loader is very lightweight. Webmake also works with CoffeeScript.
If you don't need asynchronous loading, you don't need an AMD loader. For example, if you use r.js to combine modules and you have to load in production code the whole RequireJS library. Why not simply going with a compiler? Just take a look at the slides on CommonJS Compiler http://www.slideshare.net/dsheiko/modular-javascript-with-commonjs-compiler and here the sources/docs http://dsheiko.github.io/cjsc/
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
Are there any libraries for in-browser javascript that provide the same flexibility/modularity/ease of use as Node's require?
To provide more detail: the reason require is so good is that it:
Allows code to be dynamically loaded from other locations (which is stylistically better, in my opinion, than linking all your code in the HTML)
It provides a consistent interface for building modules
It is easy for modules to depend on other modules (so I could write, for instance, an API that requires jQuery so I can use jQuery.ajax()
Loaded javascript is scoped, meaning I could load with var dsp = require("dsp.js"); and I would be able to access dsp.FFT, which wouldn't interfere with my local var FFT
I have yet to find a library that does this effectively. The workarounds I tend to use are:
coffeescript-concat -- it's easy enough to require other js, but you have to compile it, which means it is less great for fast development (e.g. building APIs in-test)
RequireJS -- It's popular, straightforward, and solves 1-3, but lack of scoping is a real deal-breaker (I believe head.js is similar in that it lacks scoping, though I've never had any occasion to use it. Similarly, LABjs can load and .wait() does mollify dependency issues, but it still doesn't do scoping)
As far as I can tell, there appear to be many solutions for dynamic and/or async loading of javascript, but they tend to have the same scoping issues as just loading the js from HTML. More than anything else, I would like a way to load javascript that does not pollute the global namespace at all, but still allows me to load and use libraries (just as node's require does).
2020 UPDATE: Modules are now standard in ES6, and as of mid-2020 are natively supported by most browsers. Modules support both synchronous and asynchronous (using Promise) loading. My current recommendation is that most new projects should use ES6 modules, and use a transpiler to fall back to a single JS file for legacy browsers.
As a general principle, bandwidth today is also typically much wider than when I originally asked this question. So in practice, you might reasonably chose to always use a transpiler with ES6 modules, and focus your effort on code efficiency rather than network.
PREVIOUS EDIT (or if you don't like ES6 modules): Since writing this, I have extensively used RequireJS (which now has much clearer documentation). RequireJS really was the right choice in my opinion. I'd like to clarify how the system works for people who are as confused as I was:
You can use require in everyday development. A module can be anything returned by a function (typically an object or a function) and is scoped as a parameter. You can also compile your project into a single file for deployment using r.js (in practice this is almost always faster, even though require can load scripts in parallel).
The primary difference between RequireJS and node-style require like browserify (a cool project suggested by tjameson) uses is the way modules are designed and required:
RequireJS uses AMD (Async Module Definition). In AMD, require takes a list of modules (javascript files) to load and a callback function. When it has loaded each of the modules, it calls the callback with each module as a parameter to the callback. Thus it's truly asynchronous and therefore well-suited to the web.
Node uses CommonJS. In CommonJS, require is a blocking call that loads a module and returns it as an object. This works fine for Node because files are read off the filesystem, which is fast enough, but works poorly on the web because loading files synchronously can take much longer.
In practice, many developers have used Node (and therefore CommonJS) before they ever see AMD. In addition, many libraries/modules are written for CommonJS (by adding things to an exports object) rather than for AMD (by returning the module from the define function). Therefore, lots of Node-turned-web developers want to use CommonJS libraries on the web. This is possible, since loading from a <script> tag is blocking. Solutions like browserify take CommonJS (Node) modules and wrap them up so you can include them with script tags.
Therefore, if you are developing your own multi-file project for the web, I strongly recommend RequireJS, since it is truly a module system for the web (though in fair disclosure, I find AMD much more natural than CommonJS). Recently, the distinction has become less important, since RequireJS now allows you to essentially use CommonJS syntax. Additionally, RequireJS can be used to load AMD modules in Node (though I prefer node-amd-loader).
I realize there may be beginners looking to organize their code. This is 2022, and if you're considering a modular JS app, you should get started with npm and Webpack right now.
Here are a few simple steps to get started:
In your project root, run npm init -y to initialize an npm project
Download the Webpack module bundler: npm install webpack webpack-cli
Create an index.html file:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>App</title>
</head>
<body>
<script src="_bundle.js"></script>
</body>
</html>
Pay special attention to _bundle.js file - this will be a final JS file generated by webpack, you will not modify it directly (keep reading).
Create a <project-root>/app.js in which you will import other modules:
const printHello = require('./print-hello');
printHello();
Create a sample print-hello.js module:
module.exports = function() {
console.log('Hello World!');
}
Create a <project-root>/webpack.config.js and copy-paste the following:
var path = require('path');
module.exports = {
entry: './app.js',
output: {
path: path.resolve(__dirname),
filename: '_bundle.js'
}
};
In the code above, there are 2 points:
entry app.js is where you will write your JS code. It will import other modules as shown above.
output _bundle.js is your final bundle generated by webpack. This is what your html will see at the end.
Open your package.json, and replace scripts with the following command:
"scripts": {
"start": "webpack --mode production -w"
},
And finally run the script watch app.js and generate the _bundle.js file by running: npm start.
Enjoy coding!
Check out ender. It does a lot of this.
Also, browserify is pretty good. I've used require-kiss¹ and it works. There are probably others.
I'm not sure about RequireJS. It's just not the same as node's. You may run into problems with loading from other locations, but it might work. As long as there's a provide method or something that can be called.
TL;DR- I'd recommend browserify or require-kiss.
Update:
1: require-kiss is now dead, and the author has removed it. I've since been using RequireJS without problems. The author of require-kiss wrote pakmanager and pakman. Full disclosure, I work with the developer.
Personally I like RequireJS better. It is much easier to debug (you can have separate files in development, and a single deployed file in production) and is built on a solid "standard".
I wrote a small script which allows asynchronous and synchronous loading of Javascript files, which might be of some use here. It has no dependencies and is compatible to Node.js & CommonJS. The installation is pretty easy:
$ npm install --save #tarp/require
Then just add the following lines to your HTML to load the main-module:
<script src="/node_modules/#tarp/require/require.min.js"></script>
<script>Tarp.require({main: "./scripts/main"});</script>
Inside your main-module (and any sub-module, of course) you can use require() as you know it from CommonJS/NodeJS. The complete docs and the code can be found on GitHub.
A variation of Ilya Kharlamov great answer, with some code to make it play nice with chrome developer tools.
//
///- REQUIRE FN
// equivalent to require from node.js
function require(url){
if (url.toLowerCase().substr(-3)!=='.js') url+='.js'; // to allow loading without js suffix;
if (!require.cache) require.cache=[]; //init cache
var exports=require.cache[url]; //get from cache
if (!exports) { //not cached
try {
exports={};
var X=new XMLHttpRequest();
X.open("GET", url, 0); // sync
X.send();
if (X.status && X.status !== 200) throw new Error(X.statusText);
var source = X.responseText;
// fix (if saved form for Chrome Dev Tools)
if (source.substr(0,10)==="(function("){
var moduleStart = source.indexOf('{');
var moduleEnd = source.lastIndexOf('})');
var CDTcomment = source.indexOf('//# ');
if (CDTcomment>-1 && CDTcomment<moduleStart+6) moduleStart = source.indexOf('\n',CDTcomment);
source = source.slice(moduleStart+1,moduleEnd-1);
}
// fix, add comment to show source on Chrome Dev Tools
source="//# sourceURL="+window.location.origin+url+"\n" + source;
//------
var module = { id: url, uri: url, exports:exports }; //according to node.js modules
var anonFn = new Function("require", "exports", "module", source); //create a Fn with module code, and 3 params: require, exports & module
anonFn(require, exports, module); // call the Fn, Execute the module
require.cache[url] = exports = module.exports; //cache obj exported by module
} catch (err) {
throw new Error("Error loading module "+url+": "+err);
}
}
return exports; //require returns object exported by module
}
///- END REQUIRE FN
(function () {
// c is cache, the rest are the constants
var c = {},s="status",t="Text",e="exports",E="Error",r="require",m="module",S=" ",w=window;
w[r]=function R(url) {
url+=/.js$/i.test(url) ? "" : ".js";// to allow loading without js suffix;
var X=new XMLHttpRequest(),module = { id: url, uri: url }; //according to the modules 1.1 standard
if (!c[url])
try {
X.open("GET", url, 0); // sync
X.send();
if (X[s] && X[s] != 200)
throw X[s+t];
Function(r, e, m, X['response'+t])(R, c[url]={}, module); // Execute the module
module[e] && (c[url]=module[e]);
} catch (x) {
throw w[E](E+" in "+r+": Can't load "+m+S+url+":"+S+x);
}
return c[url];
}
})();
Better not to be used in production because of the blocking. (In node.js, require() is a blocking call is well).
Require-stub — provides node-compliant require in browser, resolves both modules and relative paths. Uses technic similar to TKRequire (XMLHttpRequest).
Resulting code is fully browserifyable, in that require-stub can serve as a replacement for watchify.
Webmake bundles Node-style modules to Browser, give it a try.
I am trying to answer,
when to use import/export and when to use require()/module.exports? But as I try to dig, it seems to get complicated.
Here's my understanding
require()/module.exports: this is nodejs implementation of the module system. This loads the modules syncronously.
with es6, we can use import/export. the docs says
The import statement is used to import bindings which are exported by another module. Imported modules are in strict mode whether you declare them as such or not. The import statement cannot be used in embedded scripts unless such script has a type="module".
Ques1: How does this work with babel or webpack or browsers in general?
As I was exploring I came across stuff like CommonJs, requireJs, Asynchronous Module Definition (AMD)
Ques2: I am more interested in knowing the timeline as how these things evolved in javascript ?
How does this work with babel or webpack or browsers in general?
Babel and Webpack follow the ES spec and transpile the import / export statement to one single file. As they also support the require syntax, they usually transpile the import statements to require() calls and the export statements to module exports, and then ship with a custom loader for modules., If you got for example:
// A.js
export default function() { }
// B.js
import A from "./A";
A();
Then it gets transpiled to the following require syntax:
//A.js
exports.default = function() {};
//B.js
var A = require("./A").default;
A();
That could then get wrapped to something like:
(function() { // prevent leaking to global scope
// emulated loader:
var modules = {};
function require(name) { return modules[name]; }
function define(name, fn) {
var module = modules[name] = { exports: {} };
fn(module, module.exports, require);
}
// The code:
define("A", function(module, exports, require) {
// A.js
exports.default = function() { };
});
define("B", function(module, exports, require) {
// B.js
var A = require("A").default;
A();
});
})();
how these things evolved in javascript ?
A few years ago, writing JS was restricted to browsers, and the only way to load multiple js sources was to use multiple <script> tags and use the global object to exchange functionality. That was ugly.
Then Nodejs was invented and they needed a better way to work with modules and invented the require() thing.
The writers of the spec saw a need for a native syntax for that, so import / export were introduced.
Babel and others then wrote transpilers.
What webpack the bundler does is the following:
You specify an input file in the config
You specify an output file the config
Webpack will look at all the files which the input file requires (commomJS module system) or imports (ES6 module system). It then funnels the code based on file name extention through loaders. Loaders can transpile the individual files to code the browser can understand. An example of a loader is babel or the sass/scss compiler.
After the different files are transpiled with loaders, the plugins can work at the
transform the bundle of generated code into something else. The bundle is just a bunch of code which together forms piece of functionality
In won't go into detail in the internals of webpack too deeply, but the most important thing to understand is:
You use webpack so you can use split up your code in multiple files, which makes them more maintainable and easier to work with. However then requesting all these files by the client would be horrible for performance (many HTTP requests overhead). Therefore, we bundle the files into one file, or a couple so this overhead is reduced.
Generally, you should write all modern code with import/export syntax if you are using a bundler like webpack, or translating with Babel... npm modules may favor require/module syntax but you can still import them.
Also worth noting is the import() method which returns a promise that should resolve to the root module being imported asynchronously. Webpack may bundle these as asynchronous modules if you have it configured to do so.
In practice the resolution through tooling like babel and webpack will fallback to node-style behaviors against the node_modules lookup, where the standard is transport paths, favoring relative paths. Additional support is per environment.
You can experiment with esm support in modern browsers and in current node (behind a flag as of this answer). The behaviors are somewhat inconsistent, but well defined. When in doubt, experiment and try.
I don't like the whole export/require stuff in node, it takes too long. Let's say I have a file server.js and I want to use functions in whatever.js. in html I just add this to the header:
<script src='whatever.js'></script>
and then I can just use all the functions of whatever.js in my body's script.
But in node, in the server.js file I'd do:
var myobject = require('./whatever.js');
but then I need to set it to myobject, and further I need to go to whatever.js and manually decide what functions I want to export. not to mention that typing myobject.someFunction() is alot longer to write than someFunction() and I need to remember what I exposed/didn't expose.
I wanted something where I could just go:
require('./whatever.js');
and it puts it ALL in global, no bs. like in good old html/javascript. Is there a way to do this in node?
This will do the trick,
var fs = require('fs');
eval(fs.readFileSync('whatever.js')+'');
// here call functions from whatever.js file
(I realize this is an old thread but wanted to leave a note here for posterity.)
Here in 2022 there are several approaches for executing code from different files with Node.js:
ESM: Use standard ECMAScript modules
At the time of this writing, much of the node ecosystem (i.e. packages on npm) is in the process of transitioning to this paradigm, and there are some associated growing pains (e.g. things like __dirname are only available in CJS not ESM, though the workaround is easy).
For most developers, it would be advisable to become comfortable with this standard as it transcends node.js (i.e. is implemented in other runtimes like Deno and web browsers) and has been years in the making.
CJS: Use the original "CommonJS" module mechanism, e.g. require('./some-script.js')
It should be noted, particularly for the OP, that even though the "intended" way to use CJS modules is to export functions, constants, etc. and import them explicitly, it is possible to define everything in global scope using globalThis, though I would not recommend this.
// my-script.js
require('./foo.js');
require('./bar.js');
foo(); // This is foo from <...>foo.js
console.log(`bar = ${bar} (in ${__filename})`); // bar = 123 (in <...>my-script.js)
// foo.js
globalThis.foo = function() {
console.log(`This is foo from ${__filename}`);
}
// bar.js
globalThis.bar = 123;
If you try omitting globalThis. you'll find that foo and bar are no longer defined in the main script because require "wraps them" in "module scope."
Use eval
In my experience, there are very few legitimate use cases for eval (see Never use eval()!). Nevertheless, the functionality requested in this question is precisely what eval provides: "run some code as if it were written right here" and you can feed it from a file, as explained above by Mehul Prajapati
// include.js
// Defines a global function that works like C's "#include" preprocessor directive
const { readFileSync } = require('fs');
globalThis.include = function(scriptFile) {
console.warn('!!! EXTREMELY INSECURE !!!');
eval(readFileSync(scriptFile, 'utf-8'));
};
// main.js
require('./include.js'); // loads global include
// (I sure hope you completely trust these sources)
include('./foo.js');
include('./bar.js');
Note: Something that has contributed to much of my confusion in the past is that there have been competing standards/conventions/APIs that use some of the same identifiers, namely require, which require.js and other bundlers that support AMD (Asynchronous Module Definition)
use with different semantics. So for someone building a web application (using AMD for modules in web browsers) with node.js tooling (using CJS for modules locally) it can be frustrating to keep the functions straight, especially if it's an Electron application, which can expose Node.js APIs to scripts running in the renderer (browser). If you find yourself confused why a module is "not found" in a situation like that, check the stack trace to see which require is being called (and you may have to wrap/rename them on globalThis or something to avoid collisions).
Further reading:
JavaScript Modules: A Brief History [2019]
How the module system, CommonJS & require works [updated 2022]
What is AMD, CommonJS, and UMD? [2014]
I want to publish a module to several component manager systems: npmjs, bower, etc... plus I want to create downloadable builds as well, for example one in AMD style for requirejs, one in commonJS style, one for the global namespace in browser, minified for each of them, etc... These are more than 10 builds.
I currently have a single AMD build, and I wrote unit tests for it using karma, jasmine and requirejs in an amd style. What do you suggest, how to generate the other builds and the tests for them?
I mean I cannot decide what should I have as a base of transformations. There is a common part in every output package, and there is a package dependent part either.
AMD - requirejs (I am not sure about using the config options)
define(["module", "dependency"], function (module, dependency) {
var m = {
config: function (options){
//...
},
//...
//do something with the dependency
};
m.config(module.config()); //load config options set by require.config()
return m;
});
commonJS
var dependency = require("dependency");
module.exports = {
config: function (options){
//...
},
//...
//do something with the dependency
};
global
var m = (function (){
return {
config: function (options){
//...
},
//...
//do something with the dependency
};
})(dependency);
I don't know, should I develop the common code and build before every test, or should I develop one of the packages, test it, and write a transformation from that into the other builds?
I intend to use gulp for creating the builds and call unit tests automatically for each of them before automatically publishing them. Ohh and ofc I need an auto version number change as well. Btw. is it necessary to call unit tests after the building procedure, what do you think? I just want to be sure, that not a buggy code is published...
There are transformation libraries for gulp:
https://github.com/phated/gulp-wrap-amd (commonjs to amd)
https://github.com/gfranko/amdclean (amd to standard js)
https://github.com/phated/gulp-wrap-umd (commonjs to umd I guess)
https://github.com/adamayres/gulp-wrap (not sure of its capabilities yet, maybe universal with custom templates, maybe nothing)
So it is easy to transform one package format to the two other formats. This can be done with the tests and the code as well. The commonjs can be tested with node and jasmine-node, the standard js can be tested with karma and jasmine, the amd can be tested with karma, requirejs and jasmine.
It would be a poor choice to create a common descriptor and convert that before every test. I don't want to create another language, like coffeescript, etc... so conversion between packages is okay.
Calling unit tests before publishing is okay. There are no other common package types, so this 3 packages will be enough for every component manager I will use.
I am unsure about the versioning. It is not hard to set it manually, but maybe a system like travis or maven could help more... I'll figure out and edit this answer.
Brief problem description: I'd like to use TypeScript to generate AMD-style named modules, which I then want to load from a bundled and minified file using requires.js.
Questions:
Is that possible?
Or do you suggest another approach?
I want to combine the benefits of having all scripts bundled and minified (so that no additional HTTP requests are necessary) with loading named AMD modules which are generated by TypeScript.
Update: I was thinking along the lines of the following JavaScript solution:
define("greeter", ["require", "exports", "jquery"], function(require, exports, $) {
function greet() {
alert("Hello World!");
}
exports.greet = greet;
});
Then I could bundle and minify the generated JavaScript modules into a file that could easily be cached by browsers. Also, require.js wouldn't have to go out there and grab the modules via several GET requests — they're already defined. Plus, I'd have all the advantages of using TypeScript to generate those modules.
I would tackle this by using RequireJS directly in TypeScript.
There is a definition file for it and I've written an example for using raw RequireJS in TypeScript (rather than import statements) with jQuery, that you might be able to adapt.
Here is the resulting TypeScript code...
///<reference path="require.d.ts" />
///<reference path="jquery.d.ts" />
require(['jquery'], function ($) {
$(document).ready(() => {
alert('Your code executes after jQuery has been loaded.');
});
});
If this doesn't do what you want, show me how you would do it in JavaScript and I'll happily supply the TypeScript equivalent.