I have few questions about barrel files, webpack and jest. I've never really wondered how they work and now I'm struggling (with jest) to write tests on a bigger application that does not have ones yet.
I have barrel files in big folders (like components) and they look like this:
/components/index.js
export { default as ComponentA } from './ComponentA';
export { default as ComponentB } from './ComponentB';
export { default as ComponentC } from './ComponentC';
With a setup like this I can easily import those components like this:
import { ComponentA, Component C } from '/components';
instead of writting
import Component A from '/components/ComponentA';
import Component C from '/components/ComponentC';
My main question: In my webpack bundled files, will the ComponentB file be included just because I have it in the components/index.js (but I do not really use it)?
This came to my mind after I started writting tests with jest and it started to throw me errors about files I haven't writted tests yet for. I've tried to search why, and the only reason I can find is that I have imports from barrel files in bigger files (e.g. I import ComponentA from a barrel file when building a page - that I'm now trying to test).
Barrel imports should be supported, at least if you are using ts-jest transform.
Ref: https://github.com/kulshekhar/ts-jest/issues/1149
but I have encouraged similar problem with barrel type definition src/types/index.d.ts referenced usually as #/types all around in codebase
I ended up with configuring extra moduleNameMapper definition
moduleNameMapper: {
// FIXES Could not locate module #/types mapped as: .../src/types.
'^#/types$': '<rootDir>/src/types/index.d',
},
Hope this will help someone :)
Actually yes, all files imported in a barrel file are currently included in the import on Jest. That happens mainly because Jest uses CommonJS and does not do any tree-shaking (which would actually just make everything even slower in this use case).
When working on a big project, it seems to be quite common to have Jest test suites run increasingly slow when using barrel files and that is related to the fact that the entire dependency tree has to be resolved before running the tests.
The quick solution for that is to not use barrel files for imports within the package, either by avoiding them completely or by mocking them using jest.mock.
I personally recommend avoiding imports completely on your modules by using dependency injection instead. If you really want to go the jest.mock way, all you have to do is use it with a factory, like this (from the Jest docs):
jest.mock('../moduleName', () => {
return jest.fn(() => 42);
});
// This runs the function specified as second argument to `jest.mock`.
const moduleName = require('../moduleName');
moduleName(); // Will return '42';
Yes, this will magically replace the imported file before it is imported, avoiding the barrel file issue altogether.
Again: I don't like magic code and recommend you use dependency injection instead.
Jest
I've had the same issue: Jest was throwing errors related to files that I didn't use, only because they were exported in a barrel file which was used by the modules I was testing.
In my case the issue was with tests on Vuex stores which started to break because of modules using the same stores (probably due to circular dependencies).
prices.test.js
// the test imports a module
import prices from '#/store/modules/prices';
prices.js
// the module imports a module from a barrel file
import { isObject } from '#/common';
index.js
// this is the imported module
export { default as isObject } from './isObject';
// this is the other module that breaks my tests, even if I'm not importing it
export { default as getPageTitle } from './getPageTitle';
getPageTitle.js
// when the module uses the store, an error is thrown
import store from '#/store/store';
I think that in my case the issue was a circular dependency, anyway to answer your question: yes, Jest imports all the modules in the barrel files even if they are not imported directly.
In my case the solution was to move the modules that were using Vuex to a separate folder and to create a separate barrel file only for those files.
Webpack
In a different moment I figured out that Webpack is doing the same thing by default. I didn't notice it on a project where modules were small and weren't importing libraries, but on a separate project I had modules importing very large libraries and I noticed that Webpack wasn't doing tree-shaking and wasn't optimizing chunks as it was supposed to do.
I discovered that Webpack imports all the libraries in the barrel file, in a similar way as Jest does. The upside is that you can stop this behavior by disabling side effects on the specific barrel files.
webpack.config.js
{
module: {
rules: [
// other rules...
{
test: [/src\/common\/index.ts/i],
sideEffects: false,
}
]
}
}
You can see the difference using Webpack Bundle Analyzer: when side effects are not turned off, all imports from one specific folder will have the same size. On the other hand, when side effects are turned off you will see a different size for different imports.
Default behavior (all imports the same size)
Side effects turned off (the size depends on which modules you import)
When files in the barrel files import large libraries, the improvements can be even more noticeable (tree-shaking works, chunks are optimized).
You can find more details here: https://github.com/vercel/next.js/issues/12557 and here: Webpack doesn't split a huge vendor file when modules are exported and imported using index files.
Related
Does any build tool support this loading strategy, such as webpack or rollup.js.
Build every dependency to a single bundle, and when loading these dependencies, firstly search it in window['package'], if exist, use it. Otherwise dynamic load dependencies bundle to use.
Such app dependency is React, ReactDOM and UiLib.
The built result is:
React -> a.js
ReactDOM -> b.js
UiLib -> c.js
my code -> d.js
if window.React exist but window.ReactDOM and window.UiLib does not exist. d.js should dynamically load b.js and c.js and use window.React.
I know I can config React to externals, but this is a microapp used in many different apps, I'm not sure which packages exist in every global.
Nope. It is not possible directly. For a bundler, it is a binary choice between bundle or not to bundle.
Why? When a bundler encounters a library via import statements like - import React from 'react', it needs to know what global object it should substitute whenever it encounters react package across the entire application dependency graph. This must happen at compile-time. Additionally, loading a library with dynamic decision at runtime means you are introducing an asynchronous behavior in your code which your components or application cannot handle readily.
There are two form factors - a library and application. As far as library is considered, this is the only way to teach bundler (either bundle it or leave it via externals).
At an application level, you can write your own code to partially achieve what you seek with help of CDN. For this, you use externals and tell Webpack, for example, that react will be available as global React object on window namespace.
Now before your library is getting consumed, you have to add a dynamic code where to check for presence of React object.
function async initialize() {
if (!window.React) {
const React = await import(/* webpackIgnore: true */ 'https://unpkg.com/react#18/umd/react.development.js')
window.React = React;
initializeMicroapp();
} else {
initializeMicroapp();
return Promise.resolve();
}
}
Your initialize function for microapp is async and returns a promise. This is usually the pattern to go ahead with shell + micro-frontends.
On a side note, you can use module federation approach which is actually meant to solve exactly similar use-case. With module federation, you can teach Webpack that if the host/shell provides a library, then Webpack should simply ignore its bundled copy of that library while serving only other necessary code. However, I advice caution as it is a very specific pattern and neither de-facto nor de-jure at this point. It is recommended when you are having sufficient scale and many independent teams working on same product space.
(Code below is a simple example, real scenario is bigger)
I have two modules, mod1.js and mod2.js, which are bundled together (using esbuild). They share a common dependency, util.js.
Problem is: when code in mod2.js imports util.js (using same alias), there's a conflict with names.
util.js:
export class Util {
...
}
mod1.js:
import Util from "./util.js";
...
mod2.js:
/* this raises an error of variable already declared */
import Util from "./util.js";
...
If I change alias in mod2.js, error goes away, as expected. But changing aliases every time I import util.js is a bit clumsy, and makes me think there has to be another way.
Is there a better approach to point to a common dependency from multiple modules which are bundled together?
Thanks in advance!
With help from #Bergi's comment, I figured out that I was not using esbuild to bundle my files, but rather using Hugo to concatenate them, and passing this to esbuild.
This leads to mentioned error because there are multiple imports in the same file, which esbuild correctly doesn't recognize as valid. Instead, using esbuild to bundle my files gives me correct results.
I'm still using Hugo, but I have a single entry point which it consumes, that imports all my scripts. From example, I have another file, say master.js:
master.js:
import mod1 from "./mod1.js";
import mod2 from "./mod2.js";
And I then pass this master.js to Hugo, using its js.Build function, which internally uses esbuild. This way I can import util.js using same alias, because these imports are in separate files, using ES6 linking from esbuild.
Context
I am creating a library with 2 ways of initialization:
Automatic - I download some stuff for you asynchronously, then initialize.
Manual - You already downloaded the stuff before, then I initialize the library immediately (sync).
I have successfully implemented tree-shakable libraries in the past. What we would normally do, is separate the code into two modules, and let the app developer choose which one to import, thus allowing tree-shaking the other part. Similarly to this:
import { LibraryAsyncModule } from 'my-library'; // 🔁 Or LibrarySyncModule
#NgModule({
imports: [LibraryAsyncModule] // 🔁 Or LibrarySyncModule
})
export class AppModule { }
What I want to accomplish ✔
To reduce the learning curve of using my library, I'm trying to design is a single imported module which includes the relevant module and allows tree shaking the other. The following diagram shows the desired structure.
What I want to avoid 🚫
I could create factory providers that will detect the config passed to forRoot() and load the corresponding module at runtime. However, importing the modules at runtime turns initialization to async, and will also prevent angular from bundling the used module with the app.
I need a build time solution. My prototypes show that simply including both sync and async modules in the core module result in both being bundled.
How would that single module look like? Any ideas / suggestions? 🤔
Your approach should work, but you will need to enable it.
Since Tree-shaking works only with es6 modules you need to publish your lib with it & specify module property in your package.json.
// package.json
{
...
"main": "dist/index.js",
"module": "dist/es/index.js", // <- this file should contain es modules not commonjs
...
}
This setup will tell bundlers (webpack in your case) to use the es modules export and allow them to enable tree-shaking feature.
I recommend for package devs to use tsdx cli tool which does this automatically and supports many other features such as TypeScript support etc.
npx tsdx create mylib
I'm developing a module that doesn't have a build that the user imports. Instead, he imports individual components and then bundles them along with his code. However, those components share utilities and I want to import them without going through relative path hell.
I know that's a pretty common question and I did some research. Suppose I have module/components/foo/bar/baz/index.js that wants to import module/utils/helper.js
Option 1
Just use relative paths and do:
import helper from '../../../../utils/helper'
Option 2
Use the module-alias package and have:
import helper from '#utils/helper'
This would work in Node.js because modules are resolved at runtime. However, let's say the module user has Webpack and imports the module:
import component from 'module/components/foo/bar/baz'
Webpack wouldn't be able to resolve #utils unless the user specifies that alias in his own Webpack configuration. That would be pretty annoying.
Option 3
Use Webpack aliases in webpack.config.js:
module.exports = {
resolve: {
alias: {
'#utils': path.join(__dirname, 'utils')
}
}
}
This would work fine if the module was pre-bundled. But as I've previously mentioned, I want the library to be usable with ES6 imports so that users can bundle only the parts they need.
Option 4
I could use the module name in the module's own source code:
import helper from 'module/utils/helper'
This appears to solve the problem, but I think it's a pretty bad solution. For development, you'd have to create a symlink node_modules/module -> module. I'm sure this hides many potential issues and collaborators would have to manually do it as well.
Is there a way to avoid relative paths while allowing the library to be used with ES6 imports?
I am working on an NPM package written in Typescript, and I am having trouble wrapping my head around module resolution when compiling the library to publish.
Throughout the project, I have been using non-relative imports to avoid the annoyance of ../../../. However, I read in the typescript documentation that relative imports should be used within a project.
A relative import is resolved relative to the importing file and cannot resolve to an ambient module declaration. You should use relative imports for your own modules that are guaranteed to maintain their relative location at runtime.
A non-relative import can be resolved relative to baseUrl, or through path mapping, which we’ll cover below. They can also resolve to ambient module declarations. Use non-relative paths when importing any of your external dependencies.
I would like to not have to sacrifice the nice, neat imports in favor of relative imports, but I am not sure how to set up the compiler settings in order to get this to work. When running tests, I specify NODE_PATH in order to resolve the modules, but this isn't working for post-compilation.
I would like to be able to write files using non-relative imports, but have them transformed in some way so that the dist/ files can resolve the imports.
The project is hosted on github here.
The relevant issue is that I end up with an index.d.ts file in my dist/ folder that looks like this:
import { Emitter } from 'emitter';
import { Schema } from 'migrations';
import { Model, model, relation } from 'model';
import { Builder } from 'query';
export { Builder, Emitter, Model, model, relation, Schema };
But all the modules have errors that the module cannot be resolved. How can I keep these imports in their current form, but transform them in some way when building so that when I publish the npm package, the modules can be correctly resolved.
I would follow the advice in the official Typescript docs:
https://www.typescriptlang.org/docs/handbook/declaration-files/publishing.html
Basically, the suggestion is to build your library just before publishing to npm. You will have two files, in output; let's call them main.js and main.d.ts.
The critical point, here, is that by tsc-ing your source files you resolve the dependencies before npm is involved at all, so you can keep your references as you wish.
In your package.json, include two lines (or change them accordingly, if you have them already):
{
...
"main": "./lib/main.js",
"types": "./lib/main.d.ts"
...
}
In this way, any consuming project doesn't need to know about the internals of your library: they can just use the compiled output, referencing the generated typings file.