Is there a way to test automatically if a JavaScript module is compatible with CommonJS (Node) or AMD module formats? Furthermore, how could one write it as a Grunt task?
Compability is an important feature of a module and should be able to be tested automatically at least in some degree. For example a change in a building configuration could produce a compability error that cannot be noticed by unit tests that use only browser interface of the module, like is the case with QUnit. There might be also possibility that the compability-enabling code of a module (e.g. self-made UMD wrapper) is so messy that it is hard to say if a change elsewhere in the code could have an effect on the compability part of the code.
One could use tools that generate probably correct wrappers of different module formats. For example grunt-umd and uRequire are such tools. But as they too depend on the build configuration, how you can be sure that the compability really is there without any testing?
I don't think there is an out-of-the-box or obvious solution.
In general there are several approaches you could take to solve your problem:
Write your code in one module format and use transpilers to produce the other module formats. Babel (5) is pretty good at doing this, but there are probably others.
Use a parser like esprima and statically analyze the code. The "standard" wrappers like those on https://github.com/umdjs/umd are probably easy to detect patterns. Custom wrappers might be harder and you might need a few revisions before you get it all right.
Trial-and-error: Have your grunt task create files (e.g. generated from .jade templates) that include the supposed module in each of the manners you care about (AMD, browser globals), then "run" those (e.g. using nightwatch or selenium) and see if any errors occur. (This might be very difficult to get right if the module you are trying to load has dependencies)
Related
TL;DR; Is it possible to implement package inheritance in Node? Or is there a recommended tool to help with this?
I am working in an enterprise environment containing 60+ (and growing) web apps. We’ve also modularized components such as header/footer/sign-in/etc. so the web apps don’t need to repeat the code, and they can just pull them in as dependencies. We also have library modules to provide things like common logging, modeling, and error handling. We also have one main library that provides a foundation for maintenance like testing, and linting.
What I am trying to do is get access to the dependencies this main library uses in upper level modules.
lib-a
|
—> lib-b
|
—> babel, chai, mocha, etc.
I would like to have lib-a “inherit” babel, chai, mocha, etc. from lib-b rather than having to specifically add the dependencies. That way all my libraries, and eventually web apps will have the same version, and I won’t have to repeat the same dependencies in every package.json. Nor will I need to go through the headache of having N number of teams update the 60-100 apps/libs/whatnot, and having to deal with them complaining about maintenance.
I do understand this goes against the core of npm, but on the level we are using this it’s becoming a maintenance headache. Going more DRY would certainly have it’s benefits at this point.
So to repeat the original question at the top - Is it possible to implement package inheritance in Node? Or are there any recommended tools to help with this?
I have seen the following tools. Has anyone ever used them? or have thoughts on them. Are there others?
https://github.com/FormidableLabs/builder
https://github.com/Cosium/dry-dry
It's a bad idea. You should assume that you don't have control over the dependencies. How else would anybody be able to make changes to the dependencies?
Suppose lib a from your example uses mocha. Since it depends on lib b which also depends on mocha, you could decide to not list mocha in lib a's package.json.
If somebody refactors lib b to not use mocha anymore, lib a will fall all a sudden. That's not good.
We work with equally many projects. We use Greenkeeper, RenovateBot, and some tools that apply changes to all our repos at once. In the long run that's probably a better strategy for you than going against Node's dependency model.
Due the fact, that ES6-modules (JavaScript-modules) are available for testing:
https://www.chromestatus.com/feature/5365692190687232
https://medium.com/dev-channel/es6-modules-in-chrome-canary-m60-ba588dfb8ab7
I wonder, how should I minify and prepare the project release-file? Earlier, I havde bundled all JavaScript-files into the single and minified file, except situations, where I have to load the JS-file dynamically via XHR or Fetch API.
As I understand, it's rather impossible to prepare a single-minified file with the ES6-modules right now or may be, I'm just misunderstanding some ways of work.
So, are the any ways to prepare my ES6-modules into single file and how I should prepare the modern JavaScript-project in 2017 year, where JavaScript-modules will be available?
I wonder, how should I minify and prepare the project release-file?
That is purpose of this action? Okay, minified files take fewer network traffic, and will be downloaded faster, but most NPM libraries provides minified dist-files already. And main question about bundling in one big file.
Why webpack do it? Of cource, due absence of support for ES-modules in browser by native, What's why webpack resolves import statements and round dependencies in synchronous manner*, and then substitute it to IIFE for scoping. And perform babel translation and polyfilling, yes.
But then native support of ES-modules is started, it's become un-useful. One of main goals when exposing your web-app to production, is minify traffic volume for your server, using CDN. Now you can do it in native way, so just import ES-modules from unpkg.org and be happy
*If not using HMR, of course, But it's not appropriate for production mode.
Live examples here: https://jakearchibald.com/2017/es-modules-in-browsers/
This blog explains how you would use the ES6 module syntax and yet still bundle your code into something that the browser will understand.
The blog explains that using SystemJs as an ES6 module polyfill and Babel along with Gulp will enable you to code you modules in ES6 yet sill be able to use it today.
https://www.barbarianmeetscoding.com/blog/2016/02/21/start-using-es6-es2015-in-your-project-with-babel-and-gulp/
Using this guide will help you write your code in ES6 while still having a normal workflow to building, minifying and bundling your code.
Keep in mind there are a lot of tools out there that will help you achieve this but I've followed this method many times and I can vouch for its validity.
I'm currently working on a server side Node project. Although node has it's own module loader using CommonJS I'm evaluating whether to use RequireJS with it. Whilst there are advantages to using RequireJS with Node if the application that has some client side aspects to it, I can't find any benefits for a project which is entirely server side.
Is it commonly thought that for a 100% server side Node project, there are no real advantages to incorporating RequireJS?
There exist reasons to use RequireJS server-side but they are few. Unless you can state a reason, like:
I must use RequireJS because X
where "X" is a reason which justifies the use of RequireJS, then you should not do it.
Note that just wanting to write modules in the AMD format is not reason enough as there exist loaders (like node-amdl-loader) which allow loading AMD modules in Node. I actually use this when I want to test code which does not depend on a browser. I write the modules in AMD format and specify that if used in Node, an AMD loader like node-amd-loader should be used. This way the library works both in Node and in the browser but I test it in Node.
One reason to use RequireJS server-side is for instance, if you need to run code that needs a DOM implementation. If you use something like jsdom to provide the DOM and the code you want to load is a series of AMD modules, then using RequireJS to load the code into the DOM environment created by jsdom makes sense.
I've never found a good reason to use RequireJS in node. However, RequireJS can be run on node, and the documentation has a brief explanation of why you might want to run RequireJS on a Node server:
By using RequireJS on the server, you can use one format for all your
modules, whether they are running server side or in the browser. That
way you can preserve the speed benefits and easy debugging you get
with RequireJS in the browser, and not have to worry about extra
translation costs for moving between two formats.
Most of that is only useful in a project that also has a client element. If it were me, I'd stick with Node's internal module loader.
I'm creating a javascript library, and i want it to be environment agnostic (It will not use DOM, AJAX, or NodeJS api. It will be vanilla javascript). So, it's supposed to run in any javascript environment (browsers, npm, meteor smart packages, V8 C bindings...).
My currently approach is creating git repo with the library, with all the library inside a single global variable, without thinking about patterns like CommonJS or AMD.
Later, i'll create another git repo, using my library as a git submodule, and create what is needed to release it as a npm module. I'm concerned if it's a good approach, i didn't found anyone doing this way.
Pros: code will be vanilla javascript, without awareness of environment patterns. It will not bind itself to CommonJS. It will be repackable (copy and paste or git submodule) to any javascript environment. It will be as small as needed to be sent to browsers.
Cons: I'll have to maintain as many git as environments i want to support. At least a second git repo to deliver on npm.
Taking jQuery as example, it runs in both browser and nodejs, with just one git repo. There is some code to be aware of the "exports" variable to run on nodejs or other CommonJS compatible enviroment.
Pros: Just one git repo to mantain.
Cons: It will be binded to CommonJS pattern (to achieve npm compatibility)
My question is: Am i following a correct (or acceptable) approach? Or should i follow jquery's path, and try to create a single git repo?
Update 1:
Browserify and other require() libraries are not valid answers. My question is not how to use require() on the browser, instead, it's about the architecture pattern to achieve enviroment agnosticism.
Update 2:
Create a browser/nodejs module is not the question, it's known. The question is: can make a real enviroment agnostic library? This example is binded to CommonJS pattern, used in NodeJS.
If you are looking for design recommendation for your future library work then in my opinion you can think-future and just use usual Object Oriented Practices well proven in other languages, systems and libraries.
Mainly concentrate on the UML view of your design.
Forget the "one variable" requirement.
Use features proposed in the planned next version of JavaScript.
http://wiki.ecmascript.org/doku.php?id=strawman:maximally_minimal_classes
http://wiki.ecmascript.org/doku.php?id=harmony:modules_rationale
There is an experimental compiler available that allows you to write ES6-style code even today (see https://www.npmjs.org/package/es6-module-transpiler-rewrite).
Node.js has a --harmony command line switch that allows for the same (see What does `node --harmony` do?)
So in my opinion correct approach is to follow best practices and "think future"
"Use a build tool" is the answer for this question. With a build tool, you can develop with the best code pratices, without accopling your code to some enviroment standard of today (AMD, commonjs...) and still publish your code to these kind of enviroments.
For example, I'm using Grunt.js to run some tasks, like build, lint, test, etc.
It perform tedious operations (minification, compilation...) like Make, Maven, Gulp.js, and various others.
The build task can handle standards (like commonjs) for the compiled code. So, the library can be totally enviroment agnostic, and the build process handle enviroment adaptations.
Note that i'm not talking about compiling to binaries. It's compiling source to another source, like CoffeScript to JavaScript. In my case, it's compilation of JavaScript without enviroment standard to JavaScript with commonjs standard (to run as a Node.js module).
The final result is that i can compile my project to various standards without messing with my code.
Aditionally, with a build phase i can "think-future", like xmojmr answered and use the EcmaScript 6 features on my JavaScript code, using Grunt plugins like grunt-es6-transpiler or grunt-traceur to compile JavaScript code from ES 6 to 5 (so it can run on enviroments of today)
According to modular your library (if you need modules). Read this question Relation between CommonJS, AMD and RequireJS?
Take bootstrap for example, it uses npm to manage project dependencies and use bower to publish as static content for other web apps.
Take a look at browserify as reference, it's a little heavy because it provides the capability to bundle dependent npm modules as resource for browsers.
I want to ask if it is possible (and generally a good idea) to use npm to handle front-end dependencies (Backbone, jQuery).
I have found that Backbone, jQuery and so on are all available through npm but I would have to set another extraction point (the default is node_modules) or symlink or something else...
Has somebody done this before?
Is it possible?
What do I have to change in package.json?
+1 for using Browserify. We use it here at diy.org and love it. The best introduction and reasoning behind Browserify can be found in the Browserify Handbook. Topics like CommonJS & AMD solutions, build pipelines and testing are covered there.
The main reason Browserify works so well is it transparently works w/ NPM. As long as a module can be required it can be Browserified (though not all modules are made to work in the browser).
Basics:
npm install jquery-browserify
main.js
var $ = require('jquery-browserify');
$("img[attr$='png']").hide();
Then run:
browserify main.js > bundle.js
Then include bundle.js in your HTML doc and the code in main.js will execute.
Short answer: sort of.
It is largely up to the module author to support this, but it isn't common. Socket.io is an example of such a supporting module, as demonstrated on their landing page. There are other solutions however. These are the two I actually know anything about:
http://ender.no.de/ - Ender JS, self-described NPM analogue for client modules. A bit too involved for my tastes.
https://github.com/substack/node-browserify - Browserify, a utility that will walk your dependencies and allow you to output a single script by emulating the node.js module pattern. You can use a jake|cake|rake|make build script to spit out your application.js, and even automate it if you want to get fancy. I used this briefly, but decided it was a bit clunky, and became annoying to debug. Also, not all dual-environment npm modules like to be run through browserify.
Personally, I am currently opting for using RequireJS ( http://requirejs.org/ ) and manually managing my modules, similar to how Mozilla does with their BrowserQuest sample application ( https://github.com/mozilla/BrowserQuest ). Note that this comes with the challenge of having to potentially shim modules like backbone or underscore which removed support for AMD style module loaders. You can find an example of what is involved in shimming here: http://tbranyen.com/post/amdrequirejs-shim-plugin-for-loading-incompatible-javascript
Really it seems like it is going to hurt no matter what, which is why native module support is such a hot topic.
Our team maintains a tool called Lineman for building front-end projects. The tool is node-based, so a project relies on a lot of npm modules that operate server-side to build your assets, but out-of-the-box it expects to find your client-side dependencies in copied and committed to vendor/js.
However, a bunch of folks (myself included) have tried integrating with browserify, and we've run into a lot of complexity and problems, ranging from (a) npm modules being maintained by a third party which are either out of date or add unwanted changes, to (b) actual libraries that start failing when loaded traditionally whenever a top-level function named require is even defined, due to AMD/Require.js baggage.
My short-term recommendation is to hold off and stick with good ol' fashioned script concatenation until the dust settles. Until you have problems big enough or complex enough to warrant it, I suspect you'll spend more time debugging and remediating your build than you otherwise would. And I think most of us agree the best use of your time is focusing on your application code, not its build tools.
You might want to take a look at http://jspm.io/ which is a browser package manager. Has nice ES6 support too.
I personally use webmake for my small projects. It is an alternative to browserify in the way it brings npm dependencies into your browser, and it's apparently lighter.
I didn't have the opportunity to compare in details browserify and webmake, but I noticed webmake doesn't work well with modules internally using global variables such as socket.io (which is full of bloat anyway IMO).
I would be cautious about RequireJS, which has been recommended above. Because it is an AMD loader, your browser will load your JS files asynchronously. It will induces more exchanges between your client and server and may degrade the UX of people browsing from mobile networks / under bad WiFi. Moreover, if you succeed to keep your JS code simple and tiny, asynchronous loading is absolutely not needed !