How to use CommonJS and AMD side by side in node.js - javascript

I've been researching CommonJs, AMD, module loading, and related issues for over a week. I feel like nothing out there does what I need. My basic need is to share code seamlessly between frontend and backend. There are various issues around this including module formats for the client side, script loading, and module format conversions/wrapping. The piece I've been struggling with recently is how to use both CommonJS and AMD (or something AMD-like) in node.js.
You can't get away from commonJs in node.js, so my thinking is that if I want to use AMD, it has to work alongside commonJs. What tools, libraries, or techniques can I use to get something AMD-like working?
For example, I would like to be able to write a module like this:
var x = require('x')
modules.exports = function(a, callback) {
if(a) {
require(['y','z'], function(y,z) {
callback(x, y.o + z.k)
}
} else {
callback(x, "ok")
}
}
Ideally:
Both node.js and the amd-like modules will have paths interpreted in the node.js way (paying attention to node_modules unless the module path starts with "/", "./", or "../")
doesn't require source conversion for the server side in a build step (ie modules will run in node.js without each one being programmatically converted)
module or require don't need to be explicitly passed into the amd-like require function

uRequire is the perfect tool for this requirement, it's all about interoperability between the module formats and their incompatibilities.
Essentially uRequire converts or translates modules from nodejs to AMD and vise versa, plus the UMD format that runs on both nodejs and the browser or a combined .is that requires no AMD loader on browser.
It will require a build step though, but that is a minor concern in contrast to the offering.

You could check out, http://dojotoolkit.org/documentation/tutorials/1.9/node/
I've only played with it a little, but has worked with what I've tried. I got it working with node-orm and remember that being a pain to get going, but might of just been me making a mess while playing with it.
Essentially you end up with AMD on the server, like:
require(["dojo/node!orm","other/amd/module"], function(orm){
//use third party commonjs module and your own amd modules here
}
It looks like you've already investigated Requirejs's suggestion to wrap commonjs modules in an AMD require (automatically during build most likely using r.js).

Related

VSCODE: Can't get even the most basic program to run in lit with javascript [duplicate]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
Are there any libraries for in-browser javascript that provide the same flexibility/modularity/ease of use as Node's require?
To provide more detail: the reason require is so good is that it:
Allows code to be dynamically loaded from other locations (which is stylistically better, in my opinion, than linking all your code in the HTML)
It provides a consistent interface for building modules
It is easy for modules to depend on other modules (so I could write, for instance, an API that requires jQuery so I can use jQuery.ajax()
Loaded javascript is scoped, meaning I could load with var dsp = require("dsp.js"); and I would be able to access dsp.FFT, which wouldn't interfere with my local var FFT
I have yet to find a library that does this effectively. The workarounds I tend to use are:
coffeescript-concat -- it's easy enough to require other js, but you have to compile it, which means it is less great for fast development (e.g. building APIs in-test)
RequireJS -- It's popular, straightforward, and solves 1-3, but lack of scoping is a real deal-breaker (I believe head.js is similar in that it lacks scoping, though I've never had any occasion to use it. Similarly, LABjs can load and .wait() does mollify dependency issues, but it still doesn't do scoping)
As far as I can tell, there appear to be many solutions for dynamic and/or async loading of javascript, but they tend to have the same scoping issues as just loading the js from HTML. More than anything else, I would like a way to load javascript that does not pollute the global namespace at all, but still allows me to load and use libraries (just as node's require does).
2020 UPDATE: Modules are now standard in ES6, and as of mid-2020 are natively supported by most browsers. Modules support both synchronous and asynchronous (using Promise) loading. My current recommendation is that most new projects should use ES6 modules, and use a transpiler to fall back to a single JS file for legacy browsers.
As a general principle, bandwidth today is also typically much wider than when I originally asked this question. So in practice, you might reasonably chose to always use a transpiler with ES6 modules, and focus your effort on code efficiency rather than network.
PREVIOUS EDIT (or if you don't like ES6 modules): Since writing this, I have extensively used RequireJS (which now has much clearer documentation). RequireJS really was the right choice in my opinion. I'd like to clarify how the system works for people who are as confused as I was:
You can use require in everyday development. A module can be anything returned by a function (typically an object or a function) and is scoped as a parameter. You can also compile your project into a single file for deployment using r.js (in practice this is almost always faster, even though require can load scripts in parallel).
The primary difference between RequireJS and node-style require like browserify (a cool project suggested by tjameson) uses is the way modules are designed and required:
RequireJS uses AMD (Async Module Definition). In AMD, require takes a list of modules (javascript files) to load and a callback function. When it has loaded each of the modules, it calls the callback with each module as a parameter to the callback. Thus it's truly asynchronous and therefore well-suited to the web.
Node uses CommonJS. In CommonJS, require is a blocking call that loads a module and returns it as an object. This works fine for Node because files are read off the filesystem, which is fast enough, but works poorly on the web because loading files synchronously can take much longer.
In practice, many developers have used Node (and therefore CommonJS) before they ever see AMD. In addition, many libraries/modules are written for CommonJS (by adding things to an exports object) rather than for AMD (by returning the module from the define function). Therefore, lots of Node-turned-web developers want to use CommonJS libraries on the web. This is possible, since loading from a <script> tag is blocking. Solutions like browserify take CommonJS (Node) modules and wrap them up so you can include them with script tags.
Therefore, if you are developing your own multi-file project for the web, I strongly recommend RequireJS, since it is truly a module system for the web (though in fair disclosure, I find AMD much more natural than CommonJS). Recently, the distinction has become less important, since RequireJS now allows you to essentially use CommonJS syntax. Additionally, RequireJS can be used to load AMD modules in Node (though I prefer node-amd-loader).
I realize there may be beginners looking to organize their code. This is 2022, and if you're considering a modular JS app, you should get started with npm and Webpack right now.
Here are a few simple steps to get started:
In your project root, run npm init -y to initialize an npm project
Download the Webpack module bundler: npm install webpack webpack-cli
Create an index.html file:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>App</title>
</head>
<body>
<script src="_bundle.js"></script>
</body>
</html>
Pay special attention to _bundle.js file - this will be a final JS file generated by webpack, you will not modify it directly (keep reading).
Create a <project-root>/app.js in which you will import other modules:
const printHello = require('./print-hello');
printHello();
Create a sample print-hello.js module:
module.exports = function() {
console.log('Hello World!');
}
Create a <project-root>/webpack.config.js and copy-paste the following:
var path = require('path');
module.exports = {
entry: './app.js',
output: {
path: path.resolve(__dirname),
filename: '_bundle.js'
}
};
In the code above, there are 2 points:
entry app.js is where you will write your JS code. It will import other modules as shown above.
output _bundle.js is your final bundle generated by webpack. This is what your html will see at the end.
Open your package.json, and replace scripts with the following command:
"scripts": {
"start": "webpack --mode production -w"
},
And finally run the script watch app.js and generate the _bundle.js file by running: npm start.
Enjoy coding!
Check out ender. It does a lot of this.
Also, browserify is pretty good. I've used require-kiss¹ and it works. There are probably others.
I'm not sure about RequireJS. It's just not the same as node's. You may run into problems with loading from other locations, but it might work. As long as there's a provide method or something that can be called.
TL;DR- I'd recommend browserify or require-kiss.
Update:
1: require-kiss is now dead, and the author has removed it. I've since been using RequireJS without problems. The author of require-kiss wrote pakmanager and pakman. Full disclosure, I work with the developer.
Personally I like RequireJS better. It is much easier to debug (you can have separate files in development, and a single deployed file in production) and is built on a solid "standard".
I wrote a small script which allows asynchronous and synchronous loading of Javascript files, which might be of some use here. It has no dependencies and is compatible to Node.js & CommonJS. The installation is pretty easy:
$ npm install --save #tarp/require
Then just add the following lines to your HTML to load the main-module:
<script src="/node_modules/#tarp/require/require.min.js"></script>
<script>Tarp.require({main: "./scripts/main"});</script>
Inside your main-module (and any sub-module, of course) you can use require() as you know it from CommonJS/NodeJS. The complete docs and the code can be found on GitHub.
A variation of Ilya Kharlamov great answer, with some code to make it play nice with chrome developer tools.
//
///- REQUIRE FN
// equivalent to require from node.js
function require(url){
if (url.toLowerCase().substr(-3)!=='.js') url+='.js'; // to allow loading without js suffix;
if (!require.cache) require.cache=[]; //init cache
var exports=require.cache[url]; //get from cache
if (!exports) { //not cached
try {
exports={};
var X=new XMLHttpRequest();
X.open("GET", url, 0); // sync
X.send();
if (X.status && X.status !== 200) throw new Error(X.statusText);
var source = X.responseText;
// fix (if saved form for Chrome Dev Tools)
if (source.substr(0,10)==="(function("){
var moduleStart = source.indexOf('{');
var moduleEnd = source.lastIndexOf('})');
var CDTcomment = source.indexOf('//# ');
if (CDTcomment>-1 && CDTcomment<moduleStart+6) moduleStart = source.indexOf('\n',CDTcomment);
source = source.slice(moduleStart+1,moduleEnd-1);
}
// fix, add comment to show source on Chrome Dev Tools
source="//# sourceURL="+window.location.origin+url+"\n" + source;
//------
var module = { id: url, uri: url, exports:exports }; //according to node.js modules
var anonFn = new Function("require", "exports", "module", source); //create a Fn with module code, and 3 params: require, exports & module
anonFn(require, exports, module); // call the Fn, Execute the module
require.cache[url] = exports = module.exports; //cache obj exported by module
} catch (err) {
throw new Error("Error loading module "+url+": "+err);
}
}
return exports; //require returns object exported by module
}
///- END REQUIRE FN
(function () {
// c is cache, the rest are the constants
var c = {},s="status",t="Text",e="exports",E="Error",r="require",m="module",S=" ",w=window;
w[r]=function R(url) {
url+=/.js$/i.test(url) ? "" : ".js";// to allow loading without js suffix;
var X=new XMLHttpRequest(),module = { id: url, uri: url }; //according to the modules 1.1 standard
if (!c[url])
try {
X.open("GET", url, 0); // sync
X.send();
if (X[s] && X[s] != 200)
throw X[s+t];
Function(r, e, m, X['response'+t])(R, c[url]={}, module); // Execute the module
module[e] && (c[url]=module[e]);
} catch (x) {
throw w[E](E+" in "+r+": Can't load "+m+S+url+":"+S+x);
}
return c[url];
}
})();
Better not to be used in production because of the blocking. (In node.js, require() is a blocking call is well).
Require-stub — provides node-compliant require in browser, resolves both modules and relative paths. Uses technic similar to TKRequire (XMLHttpRequest).
Resulting code is fully browserifyable, in that require-stub can serve as a replacement for watchify.
Webmake bundles Node-style modules to Browser, give it a try.

Migrating from requirejs to webpack

I'm migrating/moving a project based on require.js to webpack v3. Since all my modules are using the following syntax:
define([modules,..], function(mod1,..)
Which declares which modules to use, and assigns the modules to the variables in the anonymous function. This seems to be deprecated since v2 of webpack. I can't find any information about this (except for the documentation for web pack v1).
Should I rewrite all my modules to the commonjs (including dependencies) or are there any smart way to use the AMD modules?
Help much appreciated :-)
Regards
AMD never found much use outside of requirejs so likely you will need to convert. There are tools that will help:
https://github.com/anodynos/uRequire can convert code from AMD -> UMD / CommonJS
There are caveats from (https://github.com/anodynos/uRequire/wiki/nodejs-Template):
Runtime translation of paths like models/PersonModel to ../../models/PersonModel, depending on where it was called from. You 'll still get build-time translated bundleRelative paths, to their nodejs fileRelative equivalent.
For most projects this is not an issue.
Can't use the asynchronous version of require(['dep'], function(dep){...})
You should be able to use the synchronous version of require. If using webpack2 you can use System.import or require.ensure
Can't run requirejs loader plugins, like text!... or json!...
You will find webpack version of all of these plugins
There's no mapping of /, ie webRootMap etc or using the requirejs.config's {baseUrl:"...."} or {paths:"lib":"../../lib"}
This can be replicated with https://www.npmjs.com/package/babel-plugin-module-alias
The CaptEmulation's answer is not valid for newer Webpack versions. Webpack supports AMD natively (neither additional loaders, nor plugins need to be installed). A thorough instruction is available here: https://webpack.js.org/api/module-methods.
This fact may easily go unnoticed when one tries to rewrite a RequireJS-based build to Webpack, as RequireJS uses relative paths without the trailing ./, e.g.
define('app/dep1', function(dep1) { ... });
which will not pass in Webpack without additional configuration (assuming that both require.config.js and webpack.config.js are in the same directory):
{
resolve: {
modules: [ './', ... ] // other entries possible here
}
}

What is the best way to distribute reusable JavaScript modules with dependencies?

There are many ways to format JavaScript modules: AMD, CommonJS, UMD, ES6, global script. I've seen projects that structure their source code in whatever way they want and run a build process to generate a dist directory containing code in all the above formats. This has the advantage that the user of the code can just pick whichever format is most applicable to his environment.
This method works fine as long as the module has no dependencies on other modules. In the case where the modules must import other modules, there are implied complications. For example RequireJS uses a config file that looks like:
requirejs.config({
paths: {
'jquery': 'js/lib/jquery',
'ember': 'js/lib/ember',
'handlebars': 'js/lib/handlebars',
'underscore': 'js/lib/underscore'
}
});
Other loaders have equivalent mechanisms for mapping import paths.
If jQuery is a dependency, should the module import it from the path 'jquery'? What if the system in which it is being incorporated stores jQuery at the path 'libs/jquery'? In this case, is it the responsibility of the author of the system incorporating jQuery to provide aliases in the configuration of the import path?
This questioning strongly suggests that a truly reusable module must provide code formatted in all module formats as well as document clearly upon what libraries (and versions thereof) it depends and document what import paths at which those libraries are assumed to exist.
For example I could author a fancy jQuery plugin that I distribute in AMD, CommonJS, ES6, and global variations. I would document that this plugin depends on jQuery version 2.0 imported through the path 'jquery_on_a_path_that_confuses_you'. The would-be user of this plugin must copy the plugin into his project and then configure his module loader or build tool to export jQuery at the path 'jquery_on_a_path_that_confuses_you'.
As far as I can tell:
There is no standard for what to use for import paths.
There is no standard way to express the dependency, version, and import path requirements to the user of a piece of code.
There is no standard remedy to deal with clashing import paths or load multiple versions of a library.
Does there exist any plan to deal with this strange arrangement? To me it seems a little crazy to have module systems that don't know how to name their modules. Am I wrong?
You may want to check jspm.io + SystemJS which is a relatively new package manager and universal module loader which is increasing in popularity.
Please find below some presentations and article on the subject I found useful:
https://www.youtube.com/watch?v=MXzQP38mdnE,
https://vimeo.com/65042246,
https://www.youtube.com/watch?v=szJjsduHBQQ,
http://javascriptplayground.com/blog/2014/11/js-modules-jspm-systemjs/
Late with the answer, but if you're after writing plain JS code (without jQuery or other frameworks), I've found that there's the deploader.js repo, which you can use to wrap any kind of JS into modules and do dependency loading.
May worth checking out.

How to manage requireJS based packages for Node.js applications

Since coming across RequireJS I have started to adopt it wholeheartedly as it seems a great way of organising dependencies etc.
For my current project I have created a 'package' of requireJS-organised modules, which will provide the needed database API, to many node.js applications.
But I have come across a stumbling block ... how can I allow third party applications to use my package, without needing to faff with requireJS?
My directory structure for my applications and API is currently as follows:
api_package/node_modules
api_package/controllers/*
api_package/views/*
api_package/helpers/*
api_package/models/*
api_package/main.js
application_1/node_modules
application_1/app.js
application_2/node_modules
application_2/app.js
I need my applications to be self-contained - so they can be easily deployed - so my current work around is to copy api_package/* into the 'node_modules' directory of application_1 & 2 and setting the their requirejs config to the following:
(function() {
var requirejs;
requirejs = require('requirejs').config({
baseUrl: __dirname,
nodeRequire: require,
packages: [
{
name: 'api_package',
location: './node_modules/api_package'
}
]
});
This feels a little dirty and wrong!
Is there a better way? Am I missing some packaging feature for requireJS?
Is it even possible to hide the implementation details of my api_package (the detail being that I am using requireJS) and allow applications to use it as they would any other module:
require('api_package')
You can use the amdefine package, which allows you to code to the AMD API and have the module work in node programs without requiring those other programs to use AMD.
In addition to the documentation on the amdefine github page, this is also documented on the RequireJS website.

Node.js require() vs RequireJS?

Hello with RequireJS I can set a base path like this: base : './app/' so when I am in ./app/foo/bar/ for example and I have a script where I use require('foo'); RequireJS then would search for ./app/foo.js and not in node_module folder or in ./app/foo/bar/foo.js this comes handy when you have a kind of structure where it would be much cleaner for you as a developer to see the dependencies instead of having ../../foo.js. I could have ./app/foo.js and ./app/foo/foo.js and ./app/foo/bar/foo.js it would be much more cleaner to have:
require('foo');
require('foo/foo');
require('foo/bar/foo');
rather than:
require('../../foo');
require('../foo');
require('./foo');
Now you could say why not change the name and not have foo everywhere, let's say that we can't for any reason…
Another lack of feature that I see in node's require method against RequireJS is the ability of setting path mapping, if I have a directory named ./app/super-sized-directory-name/ in RequireJS I could simply do 'big-dir' : 'super-sized-directory-name' and then I could simply use require('./app/big-dir/foo') with Node.js's require method this is not possible as far as I know…
--alias, -a Register an alias with a colon separator: "to:from"
Example: --alias 'jquery:jquery-browserify'
You can register aliases with browserify, so that covers your renaming.
As for your rooted absolute paths, that can't really be done. As mentioned modul8 has a namespacing mechanism to solve this.
I would recommend you pong SubStack in #stackvm on freenode and ask him directly.
It may or may not help you, but I believe the Dojo Frameworks AMD Loader is API compatible with RequireJS and providing you are using a new microkernel does not pollute the global namespace.
I believe it only has require() and define() in the global namespace now.
Anyway their method of dealing with this is to do something like:
require(["dojo/node!util"], function(util){
// Module available as util
});
The documentation is at http://dojotoolkit.org/reference-guide/1.8/dojo/node.html
Use uRequire which provides a 'bridge' between nodejs require and AMD define modules, without reinventing the wheel (it is build on top of the two standards). It basically converts modules from AMD or commonJS format to the other format or UMD that runs smoothly on both nodejs & the browser.
It is also translating dependency paths with flexible path conventions, so you can have either '../../foo' or 'bar/foo' depending on which makes more sense at the point you are at.
Your AMD or UMD modules are loaded asynchronously on browser (using AMD/requireJs or other AMD loader) and on node the asynchronous require(['dep1', 'dep2'], function(dep1,dep2){...}) is also simulated.

Categories