Long story short, I cannot have certain characters like hyphens in our asset filenames. I'm not having the best of luck parsing through webpack documentation to figure out if it is possible to rename a file using a regex or something similar so I can strip out any hyphens from 3rd party packages where I do not control the source filename.
My super naive example would be something like this:
{
test: /\.(ttf|eot|woff|woff2)$/,
loader: `url-loader?limit=${ASSETS_LIMIT}&name=fonts/[name.replace(/-/)].[ext]`
}
Does anyone know if this is possible or how one would approach this requirement? Thanks!
The answer to this riddle appears to be found in the customInterpolateName loader option. With webpack#v3.4.1 the below was my end result for removing a hyphen.
This was the key tidbit:
plugins: [
// ... other plugins ...
new webpack.LoaderOptionsPlugin({
options: {
customInterpolateName: (loaderContext) => {
return loaderContext.replace(/-/g, '');
}
}
})
]
Here's a more complete example to give some context (note: the .css was appended to the font filenames intentionally as a workaround for yet another web resource name restriction in Dynamics CRM):
module.exports = {
// ... other config ...
module: {
loaders: [
// ... other loaders ...
{
test: /\.(ttf|eot|woff|woff2)$/,
loader: `url-loader?limit=${ASSETS_LIMIT}&name=fonts/[name].[ext].css`
}
]
},
plugins: [
// ... other plugins ...
new webpack.LoaderOptionsPlugin({
options: {
customInterpolateName: (loaderContext) => {
return loaderContext.replace(/-/g, '');
}
}
})
]
};
Related
After nest build or nest build --webpack dist folder does not contain all required modules and I got Error: Cannot find module '#nestjs/core' when trying to run node main.js.
I could not find any clear instructions on https://docs.nestjs.com/ on how to correctly build app for production, so maybe I missed something?
Out of the box, nest cli does not support including the node_modules dependencies into the dist bundle.
However, there are some community examples of custom webpack configs that include the dependencies in the bundle, e.g. bundled-nest. As described in this issue, it is necessary to include the webpack.IgnorePlugin to whitelist unused dynamic libraries.
bundle-nest has been archived/discontinued:
We've concluded that it is not recommended to bundle NestJS, or actually, NodeJS web servers in general. This is archived for historical reference during the period of time when the community was attempting to tree-shake, and bundle NestJS apps. Refer to #kamilmysliwiec comment for details:
In many real-world scenarios (depending on what libraries are being used), you should not bundle Node.js applications (not only NestJS applications) with all dependencies (external packages located in the node_modules folder). Although this may make your docker images smaller (due to tree-shaking), somewhat reduce the memory consumption, slightly increase the bootstrap time (which is particularly useful in the serverless environments), it won't work in combination with many popular libraries commonly used in the ecosystem. For instance, if you try to build NestJS (or just express) application with MongoDB, you will see the following error in your console:
Error: Cannot find module './drivers/node-mongodb-native/connection' at webpackEmptyContext
Why? Because mongoose depends on mongodb which depends on kerberos (C++) and node-gyp.
Well, about mongo, you can make some exceptions (leave some modules in node_modules), can you? It's not like it's all or nothing. But still, I'm not sure you want to follow this path. I've just succeeded with bundling a nestjs application. It was a proof of concept, I'm not sure if it'll go into production. And it was hard, I might have broken something in the process, but at first glance it works. The most complex part was adminjs. It has rollup and babel as dependencies. And in the app code they unconditionally call watch for some reason (UDP noop in production). Anyways, if you'd like to follow this path you should be ready to debug/inspect your packages' code. And you might need to add workarounds as new packages are added to the project. But it all depends on your dependencies, it may be easier than in my case. For a freshly created nestjs + mysql app it was relatively simple.
The config I ended up with (it overrides the nestjs defaults):
webpack.config.js (webpack-5.58.2, #nestjs/cli-8.1.4):
const path = require('path');
const MakeOptionalPlugin = require('./make-optional-plugin');
module.exports = (defaultOptions, webpack) => {
return {
externals: {}, // make it not exclude `node_modules`
// https://github.com/nestjs/nest-cli/blob/v7.0.1/lib/compiler/defaults/webpack-defaults.ts#L24
resolve: {
...defaultOptions.resolve,
extensions: [...defaultOptions.resolve.extensions, '.json'], // some packages require json files
// https://unpkg.com/browse/babel-plugin-polyfill-corejs3#0.4.0/core-js-compat/data.js
// https://unpkg.com/browse/core-js-compat#3.19.1/data.json
alias: {
// an issue with rollup plugins
// https://github.com/webpack/enhanced-resolve/issues/319
'#rollup/plugin-json': '/app/node_modules/#rollup/plugin-json/dist/index.js',
'#rollup/plugin-replace': '/app/node_modules/#rollup/plugin-replace/dist/rollup-plugin-replace.cjs.js',
'#rollup/plugin-commonjs': '/app/node_modules/#rollup/plugin-commonjs/dist/index.js',
},
},
module: {
...defaultOptions.module,
rules: [
...defaultOptions.module.rules,
// a context dependency
// https://github.com/RobinBuschmann/sequelize-typescript/blob/v2.1.1/src/sequelize/sequelize/sequelize-service.ts#L51
{test: path.resolve('node_modules/sequelize-typescript/dist/sequelize/sequelize/sequelize-service.js'),
use: [
{loader: path.resolve('rewrite-require-loader.js'),
options: {
search: 'fullPath',
context: {
directory: path.resolve('src'),
useSubdirectories: true,
regExp: '/\\.entity\\.ts$/',
transform: ".replace('/app/src', '.').replace(/$/, '.ts')",
},
}},
]},
// adminjs resolves some files using stack (relative to the requiring module)
// and actually it needs them in the filesystem at runtime
// so you need to leave node_modules/#adminjs/upload
// I failed to find a workaround
// it bundles them to `$prj_root/.adminjs` using `rollup`, probably on production too
// https://github.com/SoftwareBrothers/adminjs-upload/blob/v2.0.1/src/features/upload-file/upload-file.feature.ts#L92-L100
{test: path.resolve('node_modules/#adminjs/upload/build/features/upload-file/upload-file.feature.js'),
use: [
{loader: path.resolve('rewrite-code-loader.js'),
options: {
replacements: [
{search: /adminjs_1\.default\.bundle\('\.\.\/\.\.\/\.\.\/src\/features\/upload-file\/components\/edit'\)/,
replace: "adminjs_1.default.bundle('/app/node_modules/#adminjs/upload/src/features/upload-file/components/edit')"},
{search: /adminjs_1\.default\.bundle\('\.\.\/\.\.\/\.\.\/src\/features\/upload-file\/components\/list'\)/,
replace: "adminjs_1.default.bundle('/app/node_modules/#adminjs/upload/src/features/upload-file/components/list')"},
{search: /adminjs_1\.default\.bundle\('\.\.\/\.\.\/\.\.\/src\/features\/upload-file\/components\/show'\)/,
replace: "adminjs_1.default.bundle('/app/node_modules/#adminjs/upload/src/features/upload-file/components/show')"},
],
}},
]},
// not sure what babel does here
// I made it return standardizedName
// https://github.com/babel/babel/blob/v7.16.4/packages/babel-core/src/config/files/plugins.ts#L100
{test: path.resolve('node_modules/#babel/core/lib/config/files/plugins.js'),
use: [
{loader: path.resolve('rewrite-code-loader.js'),
options: {
replacements: [
{search: /const standardizedName = [^;]+;/,
replace: match => `${match} return standardizedName;`},
],
}},
]},
// a context dependency
// https://github.com/babel/babel/blob/v7.16.4/packages/babel-core/src/config/files/module-types.ts#L51
{test: path.resolve('node_modules/#babel/core/lib/config/files/module-types.js'),
use: [
{loader: path.resolve('rewrite-require-loader.js'),
options: {
search: 'filepath',
context: {
directory: path.resolve('node_modules/#babel'),
useSubdirectories: true,
regExp: '/(preset-env\\/lib\\/index\\.js|preset-react\\/lib\\/index\\.js|preset-typescript\\/lib\\/index\\.js)$/',
transform: ".replace('./node_modules/#babel', '.')",
},
}},
]},
],
},
plugins: [
...defaultOptions.plugins,
// some optional dependencies, like this:
// https://github.com/nestjs/nest/blob/master/packages/core/nest-application.ts#L45-L52
// `webpack` detects optional dependencies when they are in try/catch
// https://github.com/webpack/webpack/blob/main/lib/dependencies/CommonJsImportsParserPlugin.js#L152
new MakeOptionalPlugin([
'#nestjs/websockets/socket-module',
'#nestjs/microservices/microservices-module',
'class-transformer/storage',
'fastify-swagger',
'pg-native',
]),
],
// to have have module names in the bundle, not some numbers
// although numbers are sometimes useful
// not really needed
optimization: {
moduleIds: 'named',
}
};
};
make-optional-plugin.js:
class MakeOptionalPlugin {
constructor(deps) {
this.deps = deps;
}
apply(compiler) {
compiler.hooks.compilation.tap('HelloCompilationPlugin', compilation => {
compilation.hooks.succeedModule.tap(
'MakeOptionalPlugin', (module) => {
module.dependencies.forEach(d => {
this.deps.forEach(d2 => {
if (d.request == d2)
d.optional = true;
});
});
}
);
});
}
}
module.exports = MakeOptionalPlugin;
rewrite-require-loader.js:
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions#escaping
function escapeRegExp(string) {
return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
}
function processFile(source, search, replace) {
const re = `require\\(${escapeRegExp(search)}\\)`;
return source.replace(
new RegExp(re, 'g'),
`require(${replace})`);
}
function processFileContext(source, search, context) {
const re = `require\\(${escapeRegExp(search)}\\)`;
const _d = JSON.stringify(context.directory);
const _us = JSON.stringify(context.useSubdirectories);
const _re = context.regExp;
const _t = context.transform || '';
const r = source.replace(
new RegExp(re, 'g'),
match => `require.context(${_d}, ${_us}, ${_re})(${search}${_t})`);
return r;
}
module.exports = function(source) {
const options = this.getOptions();
return options.context
? processFileContext(source, options.search, options.context)
: processFile(source, options.search, options.replace);
};
rewrite-code-loader.js:
function processFile(source, search, replace) {
return source.replace(search, replace);
}
module.exports = function(source) {
const options = this.getOptions();
return options.replacements.reduce(
(prv, cur) => {
return prv.replace(cur.search, cur.replace);
},
source);
};
The supposed way to build the app is:
$ nest build --webpack
I didn't bother with source maps, since the target is nodejs.
It's not a config you can just copy-paste, you should figure out what's needed for your project yourself.
One more trick here, but well, you probably won't need it.
UPD adminjs seems to come with prebuilt bundles, so this config may be significantly simpler.
My babel+webpack config works fine, but the resulting bundle isn't runnable in IE11 as it contains const declarations. I thought having the es2015 preset was enough to fix this? Running $(npm bin)/babel test/some-es2015.js produces strict ES5.1 code, so Babel seems to work, but the actual code that borks in IE11 is in modules imported from node_modules.
When grepping for 'const ' in my resulting bundle I get certain lines like this (the eval is due to eval source mapping btw):
eval("\nObject.defineProperty(exports, \"__esModule\", { value: true });\nconst validator = __webpack_require__(/*! validator */ \"./node_modules/tcomb-additional-types/node_modules/validator/index.js\");\nconst t = __webpack_require__(/*! tcomb */ \"./node_modules/tcomb/index.js\");\nconst IP = t.refinement(t.String, validator.isIP);\nexports.IP = IP;\nexports.default = IP;\n//# sourceMappingURL=ip.js.map\n\n//# sourceURL=webpack:///./node_modules/tcomb-additional-types/lib/string/ip.js?");
The important part to note is the stuff such as const validator =. This isn't ES5.1 syntax. My own code seems to have been transpiled to ES5 just fine. I can see this file in /node_modules/tcomb-additional-types/lib/string/ip.js, where they use const, so this isn't Babel adding consts, but the source containing them. Most of the other packages are ES5.
So far, I have found that most consts are from material-ui and tcomb-additional-types.
Babel .babelrc:
{
"compact": false,
"presets": [
"es2015",
"es2017"
],
"plugins": [
["transform-runtime", {
"polyfill": false,
"regenerator": true
}],
"transform-class-properties",
"transform-react-jsx",
"transform-object-rest-spread"
]
}
Webpack config:
const path = require('path');
const CopyWebpackPlugin = require('copy-webpack-plugin');
/** #returns {String} an absolute path */
function toRoot(rootRelativeDir) {
return path.resolve(__dirname, '..', rootRelativeDir);
}
module.exports = {
entry: ['./src/app.js', './styles/flex.less'].map(toRoot),
output: {
filename: 'bundle.js',
path: toRoot('.webpack/dist')
},
resolve: {
extensions: ['.js', '.jsx'],
alias: {}
},
module: {
rules: [
{
test: /\.jsx?$/,
exclude: /node_modules/,
use: [
{
loader: 'babel-loader',
options: {
/* General options are read using .babelrc - only webpack loader specific here */
cacheDirectory: toRoot('.webpack/babel_cache')
}
}
]
}
]
},
plugins: [new CopyWebpackPlugin([toRoot('public')])]
};
My underlying problem was that some Node packages are not written using ES5 syntax, and the Babel transforms did not transform them for some reason. This is a normal issue
Finding why this happened was pretty easy (#Vincent's answer helped); I had exclude: /node_modules/ in the config. Of course, removing this would "fix" the issue, but it would introduce new issues, as the exclude is there for a reason, as you don't want Babel to process every file in there.
So what you want is this: selective filtering allowing some modules.
Trying to construct a regex that will allow a list of packages under node_modules, but restrict the rest is cumbersome and error prone. Thankfully the Webpack docs describe that the condition rules, of which exclude is one, can be
A string: To match the input must start with the provided string. I. e. an absolute directory path, or absolute path to the file.
A RegExp: It's tested with the input.
A function: It's called with the input and must return a truthy value to match.
An array of Conditions: At least one of the Conditions must match.
An object: All properties must match. Each property has a defined behavior.
Creating such a function is easy! So instead of having exclude: /node_modules, I changed it to be exclude: excludeCondition, where excludeCondition is the following function:
function excludeCondition(path){
const nonEs5SyntaxPackages = [
'material-ui',
'tcomb-additional-types'
]
// DO transpile these packages
if (nonEs5SyntaxPackages.some( pkg => path.match(pkg))) {
return false;
}
// Ignore all other modules that are in node_modules
if (path.match(toRoot("node_modules"))) { return true; }
else return false;
}
This fixed my issue, as there is just a tiny number of packages using ES2015 syntax, so adding them to the whitelist is manageable.
Addendum
Since people ask about the toRoot(), this is the verbatim code:
/** #returns {String} an absolute path */
function toRoot(rootRelativeDir) {
return path.resolve(__dirname, '..', rootRelativeDir);
}
Adapt to your own needs.
The fuller code:
const path = require('path');
const CopyWebpackPlugin = require('copy-webpack-plugin');
/** #returns {String} an absolute path */
function toRoot(rootRelativeDir) {
return path.resolve(__dirname, '..', rootRelativeDir);
}
function excludeCondition(path) {
const nonEs2015Packages = ['tcomb-additional-types', 'material-ui'];
// DO transpile these packages
if (nonEs2015Packages.some(pkg => path.match(pkg))) {
return false;
}
// Ignore all other modules that are in node_modules
return Boolean(path.match(toRoot('node_modules')));
}
module.exports = {
entry: ['./src/app.js', './styles/custom.less', './styles/flex.less', './styles/rc_slider.less'].map(toRoot),
output: {
filename: 'bundle.js',
path: toRoot('.webpack/dist')
},
resolve: {
extensions: ['.js', '.jsx'],
alias: {}
},
module: {
rules: [
{
test: /\.jsx?$/,
exclude: excludeCondition,
use: [
{
loader: 'babel-loader',
options: {
/* General options are read using .babelrc - only webpack loader specific here */
cacheDirectory: toRoot('.webpack/babel_cache')
}
}
]
}
]
},
plugins: [new CopyWebpackPlugin([toRoot('public')])]
};
I had a similar problem and I fixed it by renaming .babelrc.js to babel.config.js.
Apparently, .babelrc has smaller scope than babel.config.js, if you'd like to read more about that, check out this post:
When to use babel.config.js and .babelrc
The same problem happened to me as well. Some node modules don't provide browser support and target node versions that leverage newer ES syntax.
I came across that handy package that transpiles node modules code:
https://www.npmjs.com/package/babel-engine-plugin
It solved my problem regarding IE11 support, hope it helps
I'm new to webpack. I have been using gulp until now and trying to migrate to webpack.
In my .gulpfile I have:
var preProcess = require('gulp-preprocess');
gulp.src('app/config/app.constants.js')
.pipe(preProcess({
context: {
NODE_ENV: options.env
}
}))
I have the following lines in app/config/app.constants.js that need to be removed in production:
//#if NODE_ENV='development'
AppConstants.api = 'https://localhost:333/api';
AppConstants.webRoot = 'http://localhost:222';
//#endif
I am trying to accomplish this in wepack.config.js:
if (!isDev) {
config.module.rules.push([
{
test: ???, // Can't figure out what to put here
exclude: /(node_modules|bower_components|\.spec\.js)/,
use: [
{
loader: 'webpack-strip-block',
options: {
start: '#if NODE_ENV='development'',
end: '#endif'
}
}]
}
])
}
Two questions: How do I test for a single file? Is this the right way to replace gulp-preprocess?
To apply a loader to a single file, use include instead of exclude. This is whitelisting instead of blacklisting. You don't need to use test at all. For example...
{
include: 'path/to/your/file',
use: [{
loader: 'webpack-strip-block',
options: {
start: '#if NODE_ENV='development'',
end: '#endif'
}
}]
}
I'm writing a lib that has rxjs as a dependency. It only uses Subject - Is it possible for me to extract that one feature and include it within my lib, removing the need for rxjs as a dependency?
No. check inner dependencies Subject rely on (https://github.com/ReactiveX/rxjs/blob/master/src/internal/Subject.ts#L1-L8). It is pretty much requiring most of primitives in rx.
Put aside of availability, if you're depends on rxjs, what reason you'd like to not to specify it as dependency?
Looks like using Webpack's tree shaking feature did it for me.
https://webpack.js.org/guides/tree-shaking/
My Webpack configuration:
const UglifyJSPlugin = require('uglifyjs-webpack-plugin')
module.exports = {
entry: ['./src/index.js'],
output: {
filename: './dist/dist.bundle.js'
},
module: {
rules: [
{
test: /\.js$/,
exclude: /node_modules/,
loader: "babel-loader" ,
query: {
presets: ['env'],
plugins: ["transform-object-rest-spread"]
}
}
]
},
plugins: [
new UglifyJSPlugin()
]
}
And specified the location of the import like:
import { Subject } from "rxjs/subject"
Rather than
import { Subject } from "rxjs"
Bundle size went from 213kb to 14kb
In my SCSS file I need to use different base urls for different app environments which will be prepended to the image name.
Example:
For production environment
background: url(/prod/image.png);
For development environment
background: url(/dev/image.png);
The helper function which I'm using in the rest of my app returns the base path of the static assets and it looks like this:
static imagePath() {
let imagesPath;
if (this.isProduction()) {
basePath = '/prod';
} else {
basePath = '/dev';
}
return basePath
}
How to achieve this?
Edit:*
I'm using extract-text-webpack-plugin which won't let me output multiple css files.
For example you can have 2 main files (dev.scss and prod.scss) that will look like:
// prod.scss
$basePath: '/prod';
#import "style.scss";
and same for dev.scss.
Otherwise you can use some placeholder for path prefix and substitute it with actual prefix on post-processing step. For example you can use this plugin for PostCSS.
UPDATE:
Following discussion in comments here is (untested) example of how webpack configuration may look like:
module.exports = {
// ....
module: {
rules: [
// ....
{
test: /\.scss$/,
use: {
loader: StringReplacePlugin.replace({
replacements: [
{
pattern: /{urlPrefix}/ig,
replacement: () => process.env.NODE_ENV !== 'production' ? '/dev' : '/prod',
}
]
}, 'sass-loader'),
}
},
// ....
],
},
plugins: [
new StringReplacePlugin(),
// ....
],
// ....
};