I'm trying to include bitcore-lib partially into my webpage using tree-shaking that rollup provides out of the box and rollup-plugin-commonjs to load Node.js module.
To better illustrate the problem I make a demo project that available on the github
You can have a look at bundle.js. If I define a module in the following way:
const useful = "3";
const useless = "4";
export {usefull, useless}
Tree shaking works correctly - the final bundle includes only useful dependency.
But if I define a module in the way it defined in bitcore-lib (node-lib.js) in demo project:
module.exports = {
useful: "1",
useless: "2"
};
In that case, the final bundle includes the whole module.
I've expected that useless: 2 dependency shouldn't be included because of tree-shaking. My index.js is here:
import {usefull as usefull1} from "./my-node-lib"
import {usefull as usefull2} from "./my-es-lib"
console.log(`hi! ${usefull1} ${usefull2}`);
My rollup.config.js is available here
Is it a problem of module definition or rollup config?
Tree shaking works only for ES6 modules. At least it's true for Webpack and I suppose for rollup as well. Your first definition is ES6, second is commonjs.
Therefore if a library is not compiled/transpiled to ES6 modules tree shaking will not work.
Another feature which will not work is module concatenation.
Depending on the library you can try to recompile it.
Related
Context
I am creating a library with 2 ways of initialization:
Automatic - I download some stuff for you asynchronously, then initialize.
Manual - You already downloaded the stuff before, then I initialize the library immediately (sync).
I have successfully implemented tree-shakable libraries in the past. What we would normally do, is separate the code into two modules, and let the app developer choose which one to import, thus allowing tree-shaking the other part. Similarly to this:
import { LibraryAsyncModule } from 'my-library'; // 🔁 Or LibrarySyncModule
#NgModule({
imports: [LibraryAsyncModule] // 🔁 Or LibrarySyncModule
})
export class AppModule { }
What I want to accomplish ✔
To reduce the learning curve of using my library, I'm trying to design is a single imported module which includes the relevant module and allows tree shaking the other. The following diagram shows the desired structure.
What I want to avoid 🚫
I could create factory providers that will detect the config passed to forRoot() and load the corresponding module at runtime. However, importing the modules at runtime turns initialization to async, and will also prevent angular from bundling the used module with the app.
I need a build time solution. My prototypes show that simply including both sync and async modules in the core module result in both being bundled.
How would that single module look like? Any ideas / suggestions? 🤔
Your approach should work, but you will need to enable it.
Since Tree-shaking works only with es6 modules you need to publish your lib with it & specify module property in your package.json.
// package.json
{
...
"main": "dist/index.js",
"module": "dist/es/index.js", // <- this file should contain es modules not commonjs
...
}
This setup will tell bundlers (webpack in your case) to use the es modules export and allow them to enable tree-shaking feature.
I recommend for package devs to use tsdx cli tool which does this automatically and supports many other features such as TypeScript support etc.
npx tsdx create mylib
I am trying to answer,
when to use import/export and when to use require()/module.exports? But as I try to dig, it seems to get complicated.
Here's my understanding
require()/module.exports: this is nodejs implementation of the module system. This loads the modules syncronously.
with es6, we can use import/export. the docs says
The import statement is used to import bindings which are exported by another module. Imported modules are in strict mode whether you declare them as such or not. The import statement cannot be used in embedded scripts unless such script has a type="module".
Ques1: How does this work with babel or webpack or browsers in general?
As I was exploring I came across stuff like CommonJs, requireJs, Asynchronous Module Definition (AMD)
Ques2: I am more interested in knowing the timeline as how these things evolved in javascript ?
How does this work with babel or webpack or browsers in general?
Babel and Webpack follow the ES spec and transpile the import / export statement to one single file. As they also support the require syntax, they usually transpile the import statements to require() calls and the export statements to module exports, and then ship with a custom loader for modules., If you got for example:
// A.js
export default function() { }
// B.js
import A from "./A";
A();
Then it gets transpiled to the following require syntax:
//A.js
exports.default = function() {};
//B.js
var A = require("./A").default;
A();
That could then get wrapped to something like:
(function() { // prevent leaking to global scope
// emulated loader:
var modules = {};
function require(name) { return modules[name]; }
function define(name, fn) {
var module = modules[name] = { exports: {} };
fn(module, module.exports, require);
}
// The code:
define("A", function(module, exports, require) {
// A.js
exports.default = function() { };
});
define("B", function(module, exports, require) {
// B.js
var A = require("A").default;
A();
});
})();
how these things evolved in javascript ?
A few years ago, writing JS was restricted to browsers, and the only way to load multiple js sources was to use multiple <script> tags and use the global object to exchange functionality. That was ugly.
Then Nodejs was invented and they needed a better way to work with modules and invented the require() thing.
The writers of the spec saw a need for a native syntax for that, so import / export were introduced.
Babel and others then wrote transpilers.
What webpack the bundler does is the following:
You specify an input file in the config
You specify an output file the config
Webpack will look at all the files which the input file requires (commomJS module system) or imports (ES6 module system). It then funnels the code based on file name extention through loaders. Loaders can transpile the individual files to code the browser can understand. An example of a loader is babel or the sass/scss compiler.
After the different files are transpiled with loaders, the plugins can work at the
transform the bundle of generated code into something else. The bundle is just a bunch of code which together forms piece of functionality
In won't go into detail in the internals of webpack too deeply, but the most important thing to understand is:
You use webpack so you can use split up your code in multiple files, which makes them more maintainable and easier to work with. However then requesting all these files by the client would be horrible for performance (many HTTP requests overhead). Therefore, we bundle the files into one file, or a couple so this overhead is reduced.
Generally, you should write all modern code with import/export syntax if you are using a bundler like webpack, or translating with Babel... npm modules may favor require/module syntax but you can still import them.
Also worth noting is the import() method which returns a promise that should resolve to the root module being imported asynchronously. Webpack may bundle these as asynchronous modules if you have it configured to do so.
In practice the resolution through tooling like babel and webpack will fallback to node-style behaviors against the node_modules lookup, where the standard is transport paths, favoring relative paths. Additional support is per environment.
You can experiment with esm support in modern browsers and in current node (behind a flag as of this answer). The behaviors are somewhat inconsistent, but well defined. When in doubt, experiment and try.
Im working with project that is written in Node CommonJS modules. My point was to make this project accessible from a browser. I decided to use Rollup.js, so when the bundle is created you can include script in browser and use functions from library (thanks to iife format). I needed to install plugins for Rollup to convert CommonJS modules into ES6 modules, so browser can understand it.
Everything is fine, except that this project uses bson library from npm. This bson library is required in one of my modules which uses some of it's functions. After I create a bundle and include it into index.html an error appears in console which says: "require is not defined". When I look inside the created bundle there are some requires.
var Map = require('./map'),
Long = require('./long'),
Double = require('./double'),
Timestamp = require('./timestamp'),
ObjectID = require('./objectid'),
BSONRegExp = require('./regexp'),
Symbol$1 = require('./symbol'),
Int32 = require('./int_32'),
Code = require('./code'),
Decimal128 = require('./decimal128'),
MinKey = require('./min_key'),
MaxKey = require('./max_key'),
DBRef = require('./db_ref'),
Binary = require('./binary');
I have created simple code in Plunker to illustrate you my config and simplified structure.
https://plnkr.co/edit/YuiVJxhwhjUQ0Flw0Mg3?p=preview
In this plunker there are two simple modules, which one requires bson library, and second requires this first module. There is also Rollup config file, where I use plugins (if there is no globals plugin there is an error: Uncaught ReferenceError: Buffer is not defined).
I'm really confused. Am I misunderstanding something? Why isn't it converted into ES6 modules just like other of my code?
Here is link to bson library: https://www.npmjs.com/package/bson
Let's see:
In order for rollup to bundle your application you need to transform commonjs modules into es modules first, this is done with rollup-plugin-commonjs (https://github.com/rollup/rollup-plugin-commonjs) which you are already using, maybe you could explicitly set in the options to include the bson package. I usually use it like this, just in case:
commonjs({
include: ['node_modules/**']
})
If some library uses node global modules you will need to include them in the browser, thats why you need rollup-plugin-node-globals.
Finally, if you take a look at bson github repository there is a folder called browser_build, which contains the UMD definition of the library, so if you require 'bson/browser_build' instead of 'bson' it should work, and you may not need to use globals plugin.
Take a look at js module formats (cjs, umd, iife, es, amd) it is worth it.
Does an NPM package need to be modified to be compatible with Angular 2 (eg. add in typings, make directives for them) or will any existing package work? If they're not all compatible, how do I know what is and what is not compatible?
For example, say I want to import this package (https://github.com/pvorb/node-md5). I'm aware there is a ts-md5 package for angular 2 to do md5 - I'm just using this package as an example.
How would I get this to work?
I've installed it using
npm install md5 --save
npm install #types/md5 --save
but I can't seem to be import it
import {md5} from 'md5';
I get this error message after I try to run
Module
'"/Users/xxx/Source/tempProjects/ngUnderscore/node_modules/#types/md5/index"'
resolves to a non-module entity and cannot be imported using this
construct.
I'm not sure what this message means. Does it mean that in its current state, the package is not compatible or am I trying to use the package incorrectly?
I managed to make it work using declare and require instead of import (import won't work for this non-module lib)
declare const require: any;
const md5 = require('md5');
If you don't want to workaround import like this, you can try using Typescript MD5 implementation called ts-md5. Then import like the one below should work. (referenced from this question)
import { Md5 } from 'ts-md5/dist/md5'
If there is no TS implementation, you can search for the types in DefinitelyTyped and then simply install package by npm i --save-dev #types/{package-name}
If the library works on your project depends on many factors: your Angular version, your TypeScript version, etc.
This said, is obvious that we should check the library's documentation and see which dependencies has and its versions, and of course the library should be the Angular 2 version of itself. Following your example, there are several md5 libraries but if you use TypeScript you should maybe consider this one: https://www.npmjs.com/package/ts-md5
If we have all that covered but still there is something not working because of some type of incompatibility, like for example:
My version of angular is 2, the library I just installed works with Angular 4. I have code full of <template>, library uses <ng-template>... What can I do?
You can fork the library in github and modify whatever you need to asure it is compatible with your project. Steps:
Fork library repository and modify what you need
Subscribe to main library repository in order to be updated with changes
In packages.json use your forked version of the library, for example:
"ng2-datetime": "https://github.com/my_user/ng2-datetime/tarball/my_forked_version_name",
If you think that your modifications could suit other users... Make a Pull request! :D
This is more of a TypeScript question since md5 is not an Angular package.
The key is to get the import correct to be equivalent to a require() function.
import * as md5 from "md5";
Use directly in TypeScript file:
console.log(md5('message'));
To use this on the template, preferably it should be used in method implementation, but can also be exposed directly. Add it as a property on the Component:
md5: any = md5;
Then on the template:
{{md5('message')}}
They usually say which Angular it is meant for, sometimes you have one package for both or for each.
If you are using an Angular 1x package and there is no Angular2 compatibility, then you can just use ngUpgrade.
But If you are using a common plugin then there must be an angular 2 solution.
If you want the other way around then you're probably going the wrong way.
The issue you experienced is not related to Angular. It is an existing issue on TypeScript when importing CommonJS packages.
The rule of thumb (my recommendation) is to stay away from using the interop feature (i.e. import * as X from 'x') when importing CommonJS and use the "old" syntax instead (i.e. import X = require('x')).
When CommonJS is exporting a function in the form of module.exports = function() {...}, import * as X from 'x' does not work.
This includes the case when the package is exporting an ES6 class but transpiling to ES5 with Babel.
You may see some packages do work with this syntax, but that is because the typings have a hack in it:
declare module 'x' {
function foo(): void
namespace foo {} // <-- the hack
exports = foo
}
There are other reasons the interop is not a good idea, including the syntax between TypeScript (import * X from 'x') and Babel (import X from 'x') does not agree.
You can learn more about it here and follow the links:
https://github.com/unional/typescript-guidelines/blob/master/pages/default/modules.md#import-keyword
UPDATE: with TypeScript#2.7 released, you can now do import EditableElement from 'Transformer' directly.
Turn on esModuleInterop in your tsconfig.json
I'm trying to use d3-arrays in a project. The module itself includes both original ES6 modules and a UMD build. I would expect to be able to add this directly as a dependency to my ember-cli project and have it available, but that is not the case.
I've seen suggestions that are over a year old saying to use ember-browserify, and others suggesting making a shim, but AFAIK this would really only be ideal if it were a bower dependency, and bower seems to be on the way out.
For the sake of correctness, how can I just import this module as though it were part of my project and use it as import {mean} from 'd3-arrays without needing to convert it to another package format first?
I've tried making a shim which just exports the imported UMD code:
// index.js
var d3ArraysExports = require('d3-arrays');
d3ArraysExports.name = 'd3-arrays-shim';
module.exports = d3ArraysExports;
Ember finds this module just fine, but it never gets added to the require entries list.
If there is some design decision in Ember CLI as to why this doesn't work, please point me to it.
I figured out an elegant solution to this:
I created a shim library which imports the ES6 sources from d3-arrays and makes them available to broccoli. Here's an example:
var path = require('path');
module.exports = {
name: 'd3-arrays',
treeForAddon: function() {
var packagePath = path.join(this.project.addonPackages['ember-d3-arrays-shim'].path, 'node_modules', 'd3-arrays');
var d3ArraysTree = this.treeGenerator(packagePath);
return this._super.treeForAddon.call(this, d3ArraysTree);
}
};