We Keep Coding

Using non-fully-specified imports with nextjs and webpack 5

I am trying to use a library (a node module I don't control) that contains this import in the packaged code
import _omit from 'lodash/omit';
and it does not work with nextjs12 and webpack5. I get this error
info - Creating an optimized production build
info - Compiled successfully
info - Collecting page data .Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/dev/project/node_modules/lodash/omit' imported from /Users/dev/project/node_modules/library/es/library.js
Did you mean to import lodash/omit.js?
The library has "type": "module" in it, which I think is the cause of this?
I have seen that you need to add fullySpecifed: false, to the webpack loaders, and I have tried that a few different ways.
In my next config, i have
experimental: { esmExternals: true, fullySpecified: false },
I also tried doing this to the config in the webpack function.
config.module.rules.push({
test: /\.m?js$/,
type: 'javascript/auto',
resolve: {
fullySpecified: false,
},
});
Is there something i am missing to make this work? It seems like it might have to do with the fact that this is during "collecting page data" and not "build"?
Node version: 14.16.0
yarn 1

Webpack resolve alias and compile file under that alias

I have project which uses lerna ( monorepo, multiple packages ). Few of the packages are standalone apps.
What I want to achieve is having aliases on few of the packages to have something like dependency injection. So for example I have alias #package1/backendProvider/useCheckout and in webpack in my standalone app I resolve it as ../../API/REST/useCheckout . So when I change backend provider to something else I would only change it in webpack.
Problem
Problem appears when this alias is used by some other package ( not standalone app ). For example:
Directory structure looks like this:
Project
packageA
ComponentA
packageB
API
REST
useCheckout
standalone app
ComponentA is in packageA
useCheckout is in packageB under /API/REST/useCheckout path
ComponentA uses useCheckout with alias like import useCheckout from '#packageA/backendProvider/useCheckout
Standalone app uses componentA
The error I get is that Module not found: Can't resolve '#packageA/backendProvider/useCheckout
However when same alias is used in standalone app ( that has webpack with config provided below ) it is working. Problem occurs only for dependencies.
Potential solutions
I know that one solution would be to compile each package with webpack - but that doesn't really seem friendly. What I think is doable is to tell webpack to resolve those aliases to directory paths and then to recompile it. First part ( resolving aliases ) is done.
Current code
As I'm using NextJS my webpack config looks like this:
webpack: (config, { buildId, dev, isServer, defaultLoaders }) => {
// Fixes npm packages that depend on `fs` module
config.node = {
fs: "empty"
};
const aliases = {
...
"#package1/backendProvider": "../../API/REST/"
};
Object.keys(aliases).forEach(alias => {
config.module.rules.push({
test: /\.(js|jsx)$/,
include: [path.resolve(__dirname, aliases[alias])],
use: [defaultLoaders.babel]
});
config.resolve.alias[alias] = path.resolve(__dirname, aliases[alias]);
});
return config;
}

You don’t need to use aliases. I have a similar setup, just switch to yarn (v1) workspaces which does a pretty smart trick, it adds sym link to all of your packages in the root node_modules.
This way, each package can import other packages without any issue.
In order to apply yarn workspaces with lerna:
// lerna.json
{
"npmClient": "yarn",
"useWorkspaces": true,
"packages": [
"packages/**"
],
}
// package.json
{
...
"private": true,
"workspaces": [
"packages/*",
]
...
}
This will enable yarn workspace with lerna.
The only think that remains to solve is to make consumer package to transpile the required package (since default configs of babel & webpack is to ignore node_module transpilation).
In Next.js project it is easy, use next-transpile-modules.
// next.config.js
const withTM = require('next-transpile-modules')(['somemodule', 'and-another']); // pass the modules you would like to see transpiled
module.exports = withTM();
In other packages that are using webpack you will need to instruct webpack to transpile your consumed packages (lets assume that they are under npm scope of #somescope/).
So for example, in order to transpile typescript, you can add additional module loader.
// webpack.config.js
{
...
module: {
rules: [
{
test: /\.ts$/,
loader: 'ts-loader',
include: /[\\/]node_modules[\\/]#somescope[\\/]/, // <-- instruct to transpile ts files from this path
options: {
allowTsInNodeModules: true, // <- this a specific option of ts-loader
transpileOnly: isDevelopment,
compilerOptions: {
module: 'commonjs',
noEmit: false,
},
},
}
]
}
...
resolve: {
symlinks: false, // <-- important
}
}
If you have css, you will need add a section for css as well.
Hope this helps.
Bonus advantage, yarn workspaces will reduce your node_modules size since it will install duplicate packages (with the same semver version) once!

Tree-shaking with rollup

I have a project in which I bundle a components library using Rollup (generating a bundle.esm.js file). These components are then used in another project, that generates web pages which use these components - each page is using different components.
The problem is, that the entire components library is always bundled with the different page bundles, regardless of which components I'm using, unnecessarily increasing the bundle size.
This is my Rollup setup:
import resolve from 'rollup-plugin-node-resolve';
import commonjs from 'rollup-plugin-commonjs';
import babel from 'rollup-plugin-babel';
import peerDepsExternal from 'rollup-plugin-peer-deps-external';
import pkg from './package.json';
const extensions = [
'.js', '.jsx', '.ts', '.tsx',
];
export default [
{
input: './src/base/index.ts',
plugins: [
peerDepsExternal(),
resolve({ extensions }),
babel({
exclude: 'node_modules/**',
extensions,
}),
commonjs(),
],
output: [
{ file: pkg.main, format: 'cjs', sourcemap: true },
{ file: pkg.module, format: 'es', sourcemap: true },
],
watch: {
clearScreen: false,
},
},
];
I have "modules" set to false in webpack, as well.

There are things you will need to do to achieve treeshakable code from both sides - the built package and the project using it.
From your code snippet, I see that you have not add flag preserveModules: true in the rollup config file to prevent the build output from bundling. Webpack can not treeshake a bundled file FYI.
export default {
...
preserveModules: true,
...
}
On the side of the project that using it, you have to specify sideEffects in the package.json - read the doc to know how to config them. Beside that, the optimization in webpack has to has sideEffects: true, also read the doc here.
Hope this helps!

As you don't know which components of your Component Library (CL) will be needed by the adopters repositories you need to export everything but in a way
the adopters can execute a tree-shaking on your CL when they do their own build (and just include what they really need).
In a few words, you have to make your CL, tree-shakable. In order to achieve this, on your CL repo you have to:
Use bundlers that support tree-shaking (rollup, webpack, etc..)
Create the build for modules of type es/esm, NOT commonJS/cjs, etc..
Ensure no transpilers/compilers (babel,tsconfig, etc..) usually used as plugins, transform your ES module syntax to another module syntax.
By the default, the behavior of the popular Babel preset #babel/preset-env may break this rule, see the documentation for more details.
// babelrc.json example that worked for me
[
"#babel/preset-env",
{
"targets": ">0.2%, not dead, not op_mini all"
}
],
In the codebase, you always have to use import/export (no require) syntax, and import specifically the things you need only.
import arrayUtils from "array-utils"; //WRONG
import { unique, implode, explode } from "array-utils"; //OK
Configure your sideEffects on the package.json.
"sideEffects": ["**/*.css"], //example 1
"sideEffects": false, //example 2
DO NOT create a single-bundle file but keep the files separated after your build process (official docs don't say this but was the only solution that worked for me)
// rollup.config.js example
const config = [
{
input: 'src/index.ts',
output: [
{
format: 'esm', // set ES modules
dir: 'lib', // indicate not create a single-file
preserveModules: true, // indicate not create a single-file
preserveModulesRoot: 'src', // optional but useful to create a more plain folder structure
sourcemap: true, //optional
},
],
... }]
Additionally, you may need to change your module entry point in order the adopters can directly access to the proper index.js file where you are exporting everthing:
// package.json example
{
...
"module": "lib/index.js", //set the entrypoint file
}
Note: Remember that tree-shaking is executed by an adopter repository that has a build process that supports tree-shaking (eg: a CRA repo) and usually tree-shaking is just executed on prod mode (npm run build), no on dev mode. So be sure to properly test if this is working or not.

How do we include only required modules from lodash in a Nuxt?Vuejs Project?

We have built a Nuxt/VueJS project.
Nuxt has its own config file called nuxt.config.js within which we configure webpack and other build setup.
In our package.json, we have included the lodash package.
In our code, we have been careful to load only import what we require, for example:
import orderBy from 'lodash/orderBy'
In nuxt.config.js, lodash is add to the vendor list.
However when we create the build, webpack always includes the entire lodash library instead of including only what we have used in our code.
I have read numerous tutorials but haven't got the answer. Some of those answers will surely work if it was a webpack only project. But in our case, it is through nuxt config file.
Looking forward to some help.
Below is the partial nuxt.config.js file. Only relevant/important parts are included:
const resolve = require('resolve')
const webpack = require('webpack')
module.exports = {
/*
** Headers of the page
*/
head: {
},
modules: [
['#nuxtjs/component-cache', { maxAge: 1000 * 60 * 10 }]
],
plugins: [
{ src: '~/plugins/intersection', ssr: false },
],
build: {
vendor: ['moment', 'lodash'],
analyze: {
analyzerMode: 'static'
},
postcss: {
plugins: {
'postcss-custom-properties': false
}
},
plugins: [
new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/)
],
/*
** Run ESLINT on save
*/
extend (config, ctx) {
// config.resolve.alias['create-api'] = `./create-api-${ctx.isClient ? 'client' : 'server'}.js`
}
}
}

You can npm install only the required packages
Lodash can be split up per custom builds. You can find a list of already available ones here. You can use them like this: npm i -S lodash.orderby. I didn't check it but you would probably also need to change import orderBy from 'lodash/orderBy' to import orderBy from 'lodash.orderby'.

How do Webpack '!' imports work?

I have a React component that I'm installing and importing as a node_module in my project. This component requires an SVG and is transpiled to ES5 via babel before I require it.
The relevant transpiled code looks like this.
'use strict';
Object.defineProperty(exports, "__esModule", {
value: true
});
var _hudInline = require('./icons/hud.inline.svg');
var _hudInline2 = _interopRequireDefault(_hudInline);
_react2.default.createElement(_hudInline2.default, {
className: 'FooterNav-hud-icon',
style: {
width: '16px',
fill: 'white'
}
})
My main project that imports this code has a webpack config which has a babel loader rule for js files and a svg loader rule for svg files.
If I import the component using this syntax it works:
import MyComponent from '!babel-loader!my_node_modules_folder/MyComponent';
I do not understand why this works. I was under the assumption that using "!" bypasses the webpack config and uses the loaders I define. However, babel-loader shouldn't know how to process the SVG.
If I remove my svg loader in the webpack config, the above !babel-loader import does NOT work either. Does "!" only use the listed loader for the initial file, but as it traverses the dependency tree, subsequent requires use the webpack config?
If I don't use the above import style, but change my webpack config to NOT exclude /node_modules/ the code does not work. Webpack complains that it doesn't know how to deal with the SVG (unexpected character), making me believe that it is not hitting the correct svg-loader.
Am I misunderstanding how the webpack loaders / requires work? Could it be because the originally transpiled ES6 -> ES5 messes up the require?
Here are my js and svg loader options
const babelLoader = {
test: /\.jsx?$/,
use: [{
loader: require.resolve('babel-loader'),
query: {
babelrc: false,
presets: [['es2015', { modules: false }], 'react', 'stage-0'],
cacheDirectory: true
}
}]
};
const inlineSvgLoader = {
test: /\.inline.svg$/,
use: [{
loader: 'babel-loader',
query: {
presets: ['es2015']
}
}, {
loader: 'react-svg-loader',
query: {
jsx: true,
...
};
}]
EDIT:
My problem was that despite babel-loader including the .js files from my package in node_modules, the plugin webpack-node-externals needed to also whitelist the module on the server side webpack build so that it got included.

Does "!" only use the listed loader for the initial file, but as it traverses the dependency tree, subsequent requires use the webpack config?
Yes. Inline loaders only apply to the file that contains the module you are importing. Otherwise it wouldn't be possible to have that module require anything that needs a different loader.