We Keep Coding

How do you maintain directory structure when creating libraries in TS using bundlers like Rollup or Webpack? [duplicate]

I'm using ES6 imports and transpiling them with Rollup.
The output is a single bundle file.
Can Rollup be configured to generate a file-for-file transpile result?
Here is the current configuration I'm using which obviously spits out one file.
gulp.task('rollup', function() {
const rollup = require('rollup');
const nodeResolve = require('rollup-plugin-node-resolve');
const JS_INDEX_FILE = 'src/index.js';
return rollup.rollup({
input: JS_INDEX_FILE,
plugins: [
nodeResolve({
browser: true
})
]
}).then(bundle => {
bundle.write({
sourcemap: true,
format: 'cjs',
strict: true,
file: 'bundle.js'
});
});
});
Basically, I'd like individual files with require() instead of import.

If you want to use rollup to convert ES6 to CommonJS and preserve the file and directory structure, your rollup config might look something like this:
import { nodeResolve } from "#rollup/plugin-node-resolve";
export default [
{
input: ["src/index.js"],
plugins: [nodeResolve()],
output: [
{
dir: "dist",
format: "cjs",
exports: "named",
preserveModules: true, // Keep directory structure and files
}
]
}
]
See Integrating Rollup with Other Tools to determine if you need the #rollup/plugin-node-resolve plugin and exactly what gets "preserved" from the external packages.

Rollup is a module bundler. What you want is a compiler, such as Babel.

Tailor dependencies to ES or CJS with Rollup

I have a NPM package (private) which works in both a browser and Node environment.
This is done by creating separate bundles via Rollup for ES and CJS, so the output looks like:
dist/ejs/index.js // Import this for your browswer environments
dist/cjs/index.js // Use this for Node environments
Pretty standard. Now I'm adding a dependency to this, which follows the same bundling pattern.
I can import the library like so:
import { externalLibrary } from "#external/ejs/externalLibrary";
All is good in a browser environment. But now this does not work in a Node environment, as what I'm importing is not CJS.
I could change the way I import the library to require and target the cjs bundle:
const { externalLibrary } = require("#external/cjs/externalLibrary");
And while this works in both environments, I don't think it's optimal.
Is there a better way of doing this? Some configuration that I could specify when exporting the CJS bundle?
module.exports = {
input: 'src/main.js',
output: {
file: 'bundle.js',
format: 'cjs'
// Behaviour here for #external/cjs/externalLibrary ?
}
};

I overlooked the package.json config. You can specify different entry files depending on the build here:
{
...
"main": "dist/cjs/index.js",
"module": "dist/ejs/index.js",
...
}
Then I removed the implicit import of the EJS file, and targeted just the package:
// Before:
import { externalLibrary } from "#external/dist/ejs/externalLibrary";
// After:
import { externalLibrary } from "#external";
This then ensures either the CJS or ES build is used, depending on the environment using the package.

Looks like you already found the solution for this, but even with old import style
import { externalLibrary } from "#external/dist/ejs/externalLibrary";
you should be able to target appropriate formats for cjs vs esm. With rollup, you would have to configure the output config to be an array of objects with appropriate format set. For example:
module.exports = {
input: 'src/main.js',
output: [{ file: 'dist/index.cjs.js', format: 'cjs' },
{ file: 'dist/index.esm.js', format: 'es' }],
}
Also, being an author of klap, I would recommend giving it a try as it would bring in lot of other optimizations by default.

Tree-shaking with rollup

I have a project in which I bundle a components library using Rollup (generating a bundle.esm.js file). These components are then used in another project, that generates web pages which use these components - each page is using different components.
The problem is, that the entire components library is always bundled with the different page bundles, regardless of which components I'm using, unnecessarily increasing the bundle size.
This is my Rollup setup:
import resolve from 'rollup-plugin-node-resolve';
import commonjs from 'rollup-plugin-commonjs';
import babel from 'rollup-plugin-babel';
import peerDepsExternal from 'rollup-plugin-peer-deps-external';
import pkg from './package.json';
const extensions = [
'.js', '.jsx', '.ts', '.tsx',
];
export default [
{
input: './src/base/index.ts',
plugins: [
peerDepsExternal(),
resolve({ extensions }),
babel({
exclude: 'node_modules/**',
extensions,
}),
commonjs(),
],
output: [
{ file: pkg.main, format: 'cjs', sourcemap: true },
{ file: pkg.module, format: 'es', sourcemap: true },
],
watch: {
clearScreen: false,
},
},
];
I have "modules" set to false in webpack, as well.

There are things you will need to do to achieve treeshakable code from both sides - the built package and the project using it.
From your code snippet, I see that you have not add flag preserveModules: true in the rollup config file to prevent the build output from bundling. Webpack can not treeshake a bundled file FYI.
export default {
...
preserveModules: true,
...
}
On the side of the project that using it, you have to specify sideEffects in the package.json - read the doc to know how to config them. Beside that, the optimization in webpack has to has sideEffects: true, also read the doc here.
Hope this helps!

As you don't know which components of your Component Library (CL) will be needed by the adopters repositories you need to export everything but in a way
the adopters can execute a tree-shaking on your CL when they do their own build (and just include what they really need).
In a few words, you have to make your CL, tree-shakable. In order to achieve this, on your CL repo you have to:
Use bundlers that support tree-shaking (rollup, webpack, etc..)
Create the build for modules of type es/esm, NOT commonJS/cjs, etc..
Ensure no transpilers/compilers (babel,tsconfig, etc..) usually used as plugins, transform your ES module syntax to another module syntax.
By the default, the behavior of the popular Babel preset #babel/preset-env may break this rule, see the documentation for more details.
// babelrc.json example that worked for me
[
"#babel/preset-env",
{
"targets": ">0.2%, not dead, not op_mini all"
}
],
In the codebase, you always have to use import/export (no require) syntax, and import specifically the things you need only.
import arrayUtils from "array-utils"; //WRONG
import { unique, implode, explode } from "array-utils"; //OK
Configure your sideEffects on the package.json.
"sideEffects": ["**/*.css"], //example 1
"sideEffects": false, //example 2
DO NOT create a single-bundle file but keep the files separated after your build process (official docs don't say this but was the only solution that worked for me)
// rollup.config.js example
const config = [
{
input: 'src/index.ts',
output: [
{
format: 'esm', // set ES modules
dir: 'lib', // indicate not create a single-file
preserveModules: true, // indicate not create a single-file
preserveModulesRoot: 'src', // optional but useful to create a more plain folder structure
sourcemap: true, //optional
},
],
... }]
Additionally, you may need to change your module entry point in order the adopters can directly access to the proper index.js file where you are exporting everthing:
// package.json example
{
...
"module": "lib/index.js", //set the entrypoint file
}
Note: Remember that tree-shaking is executed by an adopter repository that has a build process that supports tree-shaking (eg: a CRA repo) and usually tree-shaking is just executed on prod mode (npm run build), no on dev mode. So be sure to properly test if this is working or not.

Rollup.js unresolved dependencies

I am trying to incorporate rollup.js into a project. Currently I am getting the warnings provided below in the console (unresolved dependencies) and I am not sure why or how to fix it:
'fs' is imported by node_modules\filereader\FileReader.js, but could not be resolved – treating it as an external dependency
'fs' is imported by commonjs-external:fs, but could not be resolved – treating it as an external dependency
preferring built-in module 'punycode' over local alternative at 'C:\Users\Ryan\OneDrive\Projects\Custom Coding\Zapier\Ryan Test\node_modules\punycode\punycode.js', pass 'preferBuiltins: false' to disable this behavior or 'preferBuiltins: true' to disable this warning
preferring built-in module 'punycode' over local alternative at 'C:\Users\Ryan\OneDrive\Projects\Custom Coding\Zapier\Ryan Test\node_modules\punycode\punycode.js', pass 'preferBuiltins: false' to disable this behavior or 'preferBuiltins: true' to disable this warning
Here is the test.js script requiring FileReader and https:
var FileReader = require('filereader');
var https = require('https');
Finally the rollup.config.js file which executes creating the bundle:
var rollup = require('rollup');
var commonjs = require('rollup-plugin-commonjs');
var nodeResolve = require('rollup-plugin-node-resolve');
var globals = require('rollup-plugin-node-globals');
var builtins = require('rollup-plugin-node-builtins');
// build bundle
rollup
.rollup({
entry: 'test.js',
plugins: [
nodeResolve(),
commonjs(),
globals(),
builtins()
]
})
.then(bundle => bundle.write({
dest: 'rollupBundle/bundle.js',
format: 'cjs'
}))
.catch(err => console.log(err.stack));

The CLI will generate more informative warnings — if you update your config file to use the standard form, then you can use rollup -c instead and it will often give you a URL to help diagnose issues.
Here's a config file with the necessary changes to squelch those warnings:
import commonjs from 'rollup-plugin-commonjs';
import nodeResolve from 'rollup-plugin-node-resolve';
import globals from 'rollup-plugin-node-globals';
import builtins from 'rollup-plugin-node-builtins';
export default {
entry: 'test.js',
dest: 'rollupBundle/bundle.js',
format: 'cjs',
external: [ 'fs' ], // tells Rollup 'I know what I'm doing here'
plugins: [
nodeResolve({ preferBuiltins: false }), // or `true`
commonjs(),
globals(),
builtins()
]
};
UPDATE: The "official" Rollup plugins are now under the #rollup namespace on npm, if you install the two versions mentioned above you will get an "npm WARN deprecated" message, so instead install the newer versions instead:
npm install #rollup/plugin-commonjs --save-dev
npm install #rollup/plugin-node-resolve --save-dev
then use them like this:
import commonjs from '#rollup/plugin-commonjs';
import { nodeResolve } from '#rollup/plugin-node-resolve';

How to shim tinymce in webpack?

I'm trying to get tinymce recognized by webpack. It sets a property named tinymce on window, so evidently one option is to require() it using syntax like this (described at the bottom of the EXPORTING section of the webpack docs):
require("imports?window=>{}!exports?window.XModule!./file.js
But in this example, how is ./file.js resolved? I installed tinymce via npm, and I can't figure out how to specify the right path to the tinymce.js file.
Regardless, I'd rather handle this in my configuration and be able to just require('tinymce') if possible, so I've installed exports-loader and added the following to my configuration (based on this discussion):
module: {
loaders: [
{
test: /[\/]tinymce\.js$/,
loader: 'exports?tinymce'
}
]
}
Unfortunately this isn't working. What's wrong with my configuration?

The tinymce module on npm can't be required directly, but contains 4 different distributions of the library. Namely:
tinymce/tinymce.js
tinymce/tinymce.min.js
tinymce/tinymce.jquery.js
tinymce/tinymce.jquery.min.js
To be able to do require('tinymce') in your code, you can add an alias in your webpack config, as well as a custom loader for your distribution of choice.
resolve: {
alias: {
// require('tinymce') will do require('tinymce/tinymce')
tinymce: 'tinymce/tinymce',
},
},
module: {
loaders: [
{
// Only apply on tinymce/tinymce
include: require.resolve('tinymce/tinymce'),
// Export window.tinymce
loader: 'exports?window.tinymce',
},
],
},
Where you can replace tinymce/tinymce with your distribution of choice.

Just like #cchamberlain I ended up using script loader for tinymce, but to load the plugins and other resources that were not required by default I used CopyWebpackPlugin instead of ES6 for more configurable solution.
var copyWebpackPlugin = require('copy-webpack-plugin');
module.exports = {
//...
plugins: [
new copyWebpackPlugin([
{ from: './node_modules/tinymce/plugins', to: './plugins' },
{ from: './node_modules/tinymce/themes', to: './themes' },
{ from: './node_modules/tinymce/skins', to: './skins' }
])
]
};

I was able to integrate tinyMCE in my Angular 2/TypeScript based project by using the imports-loader and exports-loader and the copy-webpack-plugin.
First ensure that the necessary dependencies are available and part of the packages.json file of your project:
npm i tinymce --save
npm i exports-loader --save-dev
npm i imports-loader --save-dev
npm i copy-webpack-plugin --save-dev
Then add the required loader to the loaders-section of your webpack configuration:
loaders: [
{
test: require.resolve('tinymce/tinymce'),
loaders: [
'imports?this=>window',
'exports?window.tinymce'
]
},
{
test: /tinymce\/(themes|plugins)\//,
loaders: [
'imports?this=>window'
]
}]
To make the copyWebpackPlugin available in your webpack configuration, import it in the header part of the webpack configuration file:
var copyWebpackPlugin = require('copy-webpack-plugin');
And, as Petri Ryhänen commented, add the following entry to the plugins-section of your webpack configuration:
plugins: [
new copyWebpackPlugin([
{ from: './node_modules/tinymce/plugins', to: './plugins' },
{ from: './node_modules/tinymce/themes', to: './themes' },
{ from: './node_modules/tinymce/skins', to: './skins' }
])
]
This step ensures that (required) addons of tinyMCE are also available in your webpack.
Finally to import tinyMCE in your Angular 2 component file, add
require('tinymce')
declare var tinymce: any;
to the import section and tinyMCE is ready to use.

I got this to work similar to how I bundle React to ensure I don't get two separate instances in DOM. I had some issues with imports / exports / expose loaders so instead I used script-loader.
In my setup I have a commons chunk that I use strictly for vendors (React / tinymce).
entry: { 'loading': '../src/app/entry/loading'
, 'app': '../src/app/entry/app'
, 'timeout': '../src/app/entry/timeout'
, 'commons': [ 'expose?React!react'
, 'expose?ReactDOM!react-dom'
, 'script!tinymce/tinymce.min.js'
]
}
This is working for me the same way that including the script from CDN would work however I now had errors because it could not find my themes / plugins / skins paths from my node_modules location. It was looking for them at paths /assets/plugins, /assets/themes, /assets/skins (I use webpack public path /assets/).
I resolved the second issue by mapping express to serve these two routes statically like so (es6):
const NODE_MODULES_ROOT = path.resolve(__dirname, 'node_modules')
const TINYMCE_PLUGINS_ROOT = path.join(NODE_MODULES_ROOT, 'tinymce/plugins')
const TINYMCE_THEMES_ROOT = path.join(NODE_MODULES_ROOT, 'tinymce/themes')
const TINYMCE_SKINS_ROOT = path.join(NODE_MODULES_ROOT, 'tinymce/skins')
router.use('/assets/plugins', express.static(TINYMCE_PLUGINS_ROOT))
router.use('/assets/themes', express.static(TINYMCE_THEMES_ROOT))
router.use('/assets/skins', express.static(TINYMCE_SKINS_ROOT))
After doing this window.tinymce / window.tinyMCE are both defined and functions same as CDN.

As an addition to this answer (thanks to Petri Ryhänen), I want to add my copyWebpackPlugin and tinymce.init() configuration adjustments.
new copyWebpackPlugin([{
context: './node_modules/tinymce/skins/lightgray',
from: './**/*',
to: './tinymce/skin',
}]),
With this configuration you will get all skin files in {output}/tinymce/skin folder.
Then you can initialize tinymce like this:
import tinymce from 'tinymce/tinymce';
// A theme is also required
import 'tinymce/themes/modern/theme'; // you may change to 'inlite' theme
// Any plugins you want to use has to be imported
import 'tinymce/plugins/advlist/plugin';
// ... possibly other plugins
// Then anywhere in this file you can:
tinymce.init({
// ... possibly other options
skin_url: '/tinymce/skin', // <-- !!! here we tell tinymce where
// to load skin files from
// ... possibly other options
});
With this I have both development and production builds working normally.

We use TinyMCE jQuery 4.1.6 and the accepted answer did not work for us because window seems to be used in other locations by TinyMCE (e.g. window.setTimeout). Also, document not being shimmed seemed to cause problems.
This works for us:
alias: {
'tinymce': 'tinymce/tinymce.jquery.js'
}
module: {
loaders: [
{
test: /tinymce\/tinymce\.jquery\.js/,
loader: 'imports?document=>window.document,this=>window!exports?window.tinymce'
}
]
}
Load your plugins like this:
{
test: /tinymce\/plugins/,
loader: 'imports?tinymce,this=>{tinymce:tinymce}'
}