I would like to use Webpack in a multi-page application in such a way that some pre-determined dependencies are bundled into a "vendor" chunk and the rest of the dependencies is bundled into a "commons" chunk.
For example, assuming two entry points (effectively representing a different page each), pageA.js and pageB.js contain both this code (in EC6, processed via Babel), followed by their own code:
import $ from 'jquery';
require('bootstrap/dist/css/bootstrap.css');
import angular from 'angular';
import uitree from 'angular-ui-tree';
I'd like jQuery and Bootstrap to be bundled into a "vendor" chunk, and the rest (whatever that is) to be bundled into a "commons" chunk.
The objectives are:
I would like to be able to have another separate build that would be able to rely on that same vendor chunk, without it needing to re-include the vendor libraries (I would explicitly declare that set of vendor libraries, to make it available to any sub-build that needs it).
I would also like not to have to re-process the vendor chunk every time I make a change to a page's script.
Here is the configuration I've tried:
module.exports = {
entry : {
"vendor" : [ "jquery", "bootstrap" ],
"pageA" : "./src/pageA.js",
"pageB" : "./src/pageB.js"
},
output : {
path : path.join(__dirname, "./dest"),
filename : "[name].chunk.js"
},
module : {
loaders : [ {
test : /bootstrap\/js\//,
loader : 'imports?jQuery=jquery'
},
/* ... Some modules for angular and CSS processing ... */
{
test : /\.js?$/,
include : [ path.resolve(__dirname, "./src") ],
loader : 'babel',
query : {
presets : [ 'es2015' ]
}
}
/* ... Some other settings for the fonts ... */ ]
},
plugins : [
new webpack.ProvidePlugin({
$ : "jquery",
jQuery : "jquery"
}),
new webpack.optimize.UglifyJsPlugin({
sourceMap : false,
mangle : false,
compress : false
}),
new CommonsChunkPlugin({
name : "vendor",
minChunks : Infinity
}),
new CommonsChunkPlugin({
name : "commons",
minChunks : 2
})
]
};
With the configuration above, I get jQuery and Bootstrap in vendor.chunk.js, as required, but the commons.chunk.js file is almost empty, all the rest of what's commonly used by pageA.js and pageB.js is then put into pageA.chunk.js and pageB.chunk.js (effectively duplicating that code).
If I swap the order of the last two plugins, commons.chunk.js now contains almost everything (unless what's actually specific to pageA.js and pageB.js), and vendor.chunk.js is almost empty:
plugins : [
// ...,
new CommonsChunkPlugin({
name : "commons",
minChunks : 2
}),
new CommonsChunkPlugin({
name : "vendor",
minChunks : Infinity
})
]
Is there a way to bundle a pre-defined list of libraries (e.g. [ "jquery", "jquery-ui", "bootstrap" ] into one particular chunk (in such a way that it can be used by completely independent scripts) and also have another common chunk for whatever else is in commonly used between the entry points?
The aim of all this would be to be able to build a completely separate piece of code for another page later, and tell it it doesn't need to re-bundle those pre-defined libraries.
Here is a diagram representing what I'm trying to achieve:
I would then use the generated scripts as follows on page A:
<script src="vendor.js"></script>
<script src="common.js"></script>
<script src="pageA.chunk.js"></script>
And on page C (built completely independently from pages A and B):
<script src="vendor.js"></script>
<script src="common2.js"></script>
<script src="pageC.chunk.js"></script>
(I am using Webpack 1.12.14.)
I have tried the solution suggested in the only answer so far. While this makes it indeed possible to separate the vendor chunk from the commons chunk, the vendor chunks (with the same definition) made from two separate builds generally cannot be swapped between each other. This does not make it possible to use only one of those vendor chunks across two builds (even though they are virtually the same).
I was working on something similar. I observed the behaviour you desire by having this configuration:
const entryPath = path.resolve(__dirname, "src", "modules"),
entryPoints = {
vendor: ["babel-polyfill", "react", "react-dom", "react-pure-render", "react-simpletabs", "react-redux", "redux", "redux-thunk"],
a: entryPath + "/a.js",
b: entryPath + "/b.js",
c: entryPath + "/c.js",
d: entryPath + "/d.js",
...
}, plugins = [
new CommonsChunkPlugin({
name: "commons",
filename: "commons.bundle.js",
minChunks: 2,
chunks: Object.keys(entryPoints).filter(key => key !== "vendor")
}),
new CommonsChunkPlugin("vendor", "vendor.bundle.js"),
new ExtractTextPlugin("[name].bundle.css", {
//allChunks: true
})
];
so this code above left in b.bundle.js still some import of React and other React libraries. after I added "fixed-data-table" to the vendor list, that disappeared and the only react source code there was was in vendor.bundle.js I assume that is what you were looking for? (in the end each vendor not listed in vendor list ended up in each own module.bundle.js or in commons.bundle.js if it was re-used in multiple bundles)
Regards
Jonas
As far as your request to be able to use the vendor chunk created in one webpack build with bundles from another build, the only way I have found to do this is through a combination of expose-loader and externals.
In my case, I successfully used this to bundle jQuery (and Bootstrap) into a "common.js" in one build, and then use that jQuery in modules belonging to another build, as follows:
1. Expose jQuery from build A chunk
module: {
loaders: [
{
// export jQuery globals for other modules to use as externals
test: require.resolve('jquery'),
loader: 'expose?$!expose?jQuery'
}
]
}
2. Consume jQuery in build B modules
externals: {
'jquery': 'jQuery'
}
Of course, the drawback to this is that you're bridging between the two sets of bundles by using manually-managed global variables, which you probably started using webpack to avoid :-)
However, I don't know of any other way at this point, given that every webpack build creates its own namespace of "private" IDs internally to reference modules, with no guarantee of stability or global unique-ness.
Related
We are using vue-loader (usually together with the html-webpack-plugin, but I omit this in the following because it is not the important part) to transpile multiple vue based applications within a single project. Our webpack configuration looks a little bit like the following:
const apps = [
{ html: 'app-1', js: 'app-1' },
{ html: 'app-2', js: 'app-2' },
...
]
module.exports = (_a, _b) =>({
entry: Object.fromEntries(apps.map(app => [app.js, './src/${app.js}.js'])),
...
plugins: [
new VueLoaderPlugin()
],
...
})
So for instance, app-1.html has a div with id app, which is referenced in app-1.js:
import Vue from 'vue'
import App1 from './app-1.vue'
...
new Vue({...}).$mount('#app')
This means our directory structure (very simplified) looks like the following:
src
app-1.html
app-1.js
app-1.vue
app-2.html
app-2.js
app-2.vue
...
package.json
webpack.config.js
Now here is our problem: We noticed a very high memory consumption, which increases drastically the more apps we have in our apps array. This is something like > 10 GB for a project containing 8 of these apps.
Could this be something like a memory leak in the vue-loader or are we somehow misusing the plugin? We are using version 15.10.0 of vue-loader.
It seems that vue-loader was not the culprit. I can reproduce the same high memory consumption when I remove the vue-loader completely and don't use any plugins at all. It seems that the terser plugin (which is automatically used by webpack) is the one using up that much memory (see also https://github.com/webpack/webpack/issues/13550). By default terser spawns a lot of threads. One can control this behaviour:
const TerserPlugin = require("terser-webpack-plugin")
...
module.exports = {
...
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
parallel: 2,
})
]
}
}
Alternative solution would be to use another minimizer, e.g. esbuild-loader.
The app is created with the default template for Kotlin React apps:
uses KTS-based Gradle build scripts;
Kotlin JS plugin 1.6.10;
Kotlin wrappers for React 17.0.2.
When using ./gradlew browserProductionWebpack without any additional tweaks, it generates a build/distributions directory with:
all resources (without any modifications);
index.html (without any modifications);
Kotlin sources compiled into one minified .js file.
What I want is to:
add some hash to the generated .js file;
minify the index.html file and refer the hashed .js file in it;
minify all resources (.json localization files).
Please prompt me some possible direction to do it. Looking to webpack configuration by adding corresponding scripts into webpack.config.d, but no luck yet: tried adding required dependencies into build.gradle.kts, i.e.:
implementation(devNpm("terser-webpack-plugin", "5.3.1"))
implementation(devNpm("html-webpack-plugin", "5.5.0"))
and describing webpack scripts:
const TerserPlugin = require("terser-webpack-plugin");
const HtmlWebpackPlugin = require('html-webpack-plugin');
module.exports = {
optimization: {
minimizer: [
new TerserPlugin(),
new HtmlWebpackPlugin({
minify: {
removeAttributeQuotes: true,
collapseWhitespace: true,
removeComments: true,
},
}),
],
}
}
Any hint will be appreciated.
A couple of things to put into consideration first.
If some flexible bundling configuration is needed, most likely it won't be possible to use Kotlin-wrapped (Gradle) solutions. After checking the org.jetbrains.kotlin.gradle.targets.js.webpack.KotlinWebpackConfig it turns out that there is only a limited set of things can be configured with it.
JS-based webpack configs require a little bit of reverse engineering to find out what is generated from the Kotlin/Gradle side and how to extend it.
For simple configurations (IMHO) there is almost no need in tweaking bundling options while using the Kotlin JS plugin.
Let's start from initial webpack configs. In my case (see the short environment description in the question above) they appear in ./build/js/packages/<project-name>/webpack.config.js. These original configs will also include all contents from JS files we create inside the ./webpack.config.d folder.
Some webpack configurations require external JS dependencies. We need to declare them in the dependencies block of build.gradle.kts. In my case they are represented with:
// Bundling.
implementation(devNpm("html-webpack-plugin", "5.5.0"))
implementation(devNpm("uglifyjs-webpack-plugin", "2.2.0"))
implementation(devNpm("terser-webpack-plugin", "5.3.1"))
implementation(devNpm("copy-webpack-plugin", "9.1.0" )) // newer versions don't work correctly with npm and Yarn
implementation(devNpm("node-json-minify", "3.0.0"))
I also dropped all commonWebpackConfigs from build.gradle.kts as they are going to be performed manually on the JS level.
All webpack JS configs (inside the ./webpack.config.d folder) are divided into 3 files:
common.js (dev server configuration for both dev and production builds):
// All route paths should fallback to the index page to make SPA's routes processed correctly.
const devServer = config.devServer = config.devServer || {};
devServer.historyApiFallback = true;
development.js:
// All configs inside of this file will be enabled only in the development mode.
// To check the outputs of this config, see ../build/processedResources/js/main
if (config.mode == "development") {
const HtmlWebpackPlugin = require("html-webpack-plugin");
// Pointing to the template to be used as a base and injecting the JS sources path into.
config.plugins.push(new HtmlWebpackPlugin({ template: "./kotlin/index.html" }));
}
and production.js:
// All configs inside of this file will be enabled only in the production mode.
// The result webpack configurations file will be generated inside ../build/js/packages/<project-name>
// To check the outputs of this config, see ../build/distributions
if (config.mode == "production") {
const HtmlWebpackPlugin = require("html-webpack-plugin"),
UglifyJsWebpackPlugin = require("uglifyjs-webpack-plugin"),
TerserWebpackPlugin = require("terser-webpack-plugin"),
CopyWebpackPlugin = require("copy-webpack-plugin"),
NodeJsonMinify = require("node-json-minify");
// Where to output and how to name JS sources.
// Using hashes for correct caching.
// The index.html will be updated correspondingly to refer the compiled JS sources.
config.output.filename = "js/[name].[contenthash].js";
// Making sure optimization and minimizer configs exist, or accessing its properties can crash otherwise.
config.optimization = config.optimization || {};
const minimizer = config.optimization.minimizer = config.optimization.minimizer || [];
// Minifying HTML.
minimizer.push(new HtmlWebpackPlugin({
template: "./kotlin/index.html",
minify: {
removeAttributeQuotes: true,
collapseWhitespace: true,
removeComments: true,
},
}));
// Minifying and obfuscating JS.
minimizer.push(new UglifyJsWebpackPlugin({
parallel: true, // speeds up the compilation
sourceMap: false, // help to match obfuscated functions with their origins, not needed for now
uglifyOptions: {
compress: {
drop_console: true, // removing console calls
}
}
}));
// Additional JS minification.
minimizer.push(new TerserWebpackPlugin({
extractComments: true // excluding all comments (mostly licence-related ones) into a separate file
}));
// Minifying JSON locales.
config.plugins.push(new CopyWebpackPlugin({
patterns: [
{
context: "./kotlin",
from: "./locales/**/*.json",
to: "[path][name][ext]",
transform: content => NodeJsonMinify(content.toString())
}
]
}));
}
I use styled components, so no CSS configs are provided. In other things these configs do almost the same minification as being done out-of-the-box without any additional configs. The differences are:
JS sources use a hash in their name: it is referenced correctly from the index page HTML template;
HTML template is minified;
locales (just simple JSON files) are minified.
It can look like an overhead slightly because as mentioned in the beginning, it does almost the same with minimal differences from the out-of-the-box configs. But as advantage we're getting more flexible configs which can be tweaked easier further.
I'm trying to package my Svelte app into a single Html file output.
I've managed to get the desired output with a configuration based on that answer :
Output Single HTML File from Svelte Project
With "npm run dev" everything is fine with the first build, but I'm having issues following (live-reload) builds: bundle['bundle.css'] is not filled in my inlineSvelte's generateBundle function.
I didn't manage to change the rollup-plugin-css-only for rollup-plugin-embed-css, which seemed to have an appropriate name for my needs.
Here's my rollup.config.js :
import svelte from 'rollup-plugin-svelte';
import livereload from 'rollup-plugin-livereload';
import css from 'rollup-plugin-css-only';
...
function inlineSvelte(templatePath, dest) {
return {
name: 'Svelte Inliner',
generateBundle(opts, bundle) {
const file = path.parse(opts.file).base;
const jsCode = bundle[file].code;
const cssCode = bundle['bundle.css'].source;
const template = fs.readFileSync(templatePath, 'utf-8');
bundle[file].code = template
.replace('%%script%%', jsCode)
.replace('%%style%%', cssCode);
}
}
}
export default {
input: 'src/main.js',
output: {
format: 'es',
file: outputDir + 'index.html',
name: 'app'
},
plugins: [
svelte({
compilerOptions: {
dev: !production
}
}),
css({ output: 'bundle.css' }),
resolve({
browser: true,
dedupe: ['svelte']
}),
commonjs(),
!production && livereload(outputDir),
inlineSvelte('./src/template.html')
],
watch: {
clearScreen: false
}
};
It is surely possible to embed the produced CSS file in your HTML, at least with some reasonably simple custom plugin.
However, if you only have CSS in your Svelte components, that is you don't have import 'whatever.css' anywhere in your code, you can just rely on Svelte injecting CSS from compiled JS code and be done with it.
This loses a little in terms of performance because such injected CSS will never be cached by the browser, but it avoids the added complexity / risk / coupling associated with a custom build step... And this kind of performance is often not so important in scenarios where you want all your app in a single HTML file.
To enable this, set emitCss: false on the Svelte plugin:
plugins: [
svelte({
emitCss: false,
...
}),
...
],
...
You won't need any Rollup plugin for CSS in this case.
I'm using Webpack with CommonChunksPlugin .
My problem is with lazy loaded modules.
When I load lazy modules ( 0.js and 1.js in image) , Where each uses the Http Module , it seems that Http is embedded/repeated in EACH module :
The entries in my file are :
entry: {
bundle: aot ? "./main.aot.ts" : "./main.ts",
vendor: "./vendor",
},
With CommonsChunkPlugin configured as :
new webpack.optimize.CommonsChunkPlugin({
name: ["vendor"],
}),
— I get this result where you can see multiple http.js both in bundle & 1.js& 0.js.
With CommonsChunkPlugin configured with async ( as written in here) :
new webpack.optimize.CommonsChunkPlugin({
async: true,
children: true,
}),
— I get this result where you can see tha 0.js and 1.js are OK but the bundle.js now contains dups as vendor(?!?!)
Question:
I've read the docs But as you can see , I'm trying to do something without success.
How can I fix the config so that common chunks will be extracted and not be repeated ?
webpack.config
Was having the same issue as you are/were facing.
In my case, I've removed the name parameter from the configuration and set async to true and children to true as well.
Also, make sure that the CommonChunksPlugin is the last plugin in the array of plugins.
I have one big bundled JS file which contains scripts that can run at the end of page loading.
In other words - I wanted to reduce the size of the first downloaded JS file by multiple entry points
So I've created two entry points :
module.exports = {
entry: {
a: "./scripts/entry.js",
b: "./scripts/windowEvents.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin("common.js")
],
output: {
path: path.join(__dirname, '/scripts/bundle'),
filename: "[name].entry.js"
},
module: {
loaders: [
]
}
};
So now I have :
a.entry.js
b.entry.js
common.js.entry.js
And my HTML file looks like :
<script src="/scripts/bundle/common.js.entry.js"></script>
<script src="/scripts/bundle/a.entry.js"></script>
//Html renders fast
//bottom page script
<script src="/scripts/bundle/b.entry.js"></script>
But here is the problem :
The first two scripts sections makes 2 requests to the server. I don't want that
Question:
How can I merge the first two bundles into one bundle there will be only one request ? In other words I want to merge a.entry.js & common.js.
Something like that :
<script src="/scripts/bundle/Common_AND_Entry_a.js"></script>
//Html renders fast
//bottom page script
<script src="/scripts/bundle/b.entry.js"></script>
You can give the CommonsChunkPlugin the name of an existing chunk and it will add it to that instead of creating a new file. So in your case it would get the name a (as shown in Explicit vendor chunk):
plugins: [
new webpack.optimize.CommonsChunkPlugin({
name: "a"
})
],
If you want to change the output filename as well, you can do that with the filename option, which accepts the same placeholders as output.filename. The following produces common_and_a.js and b.entry.js:
plugins: [
new webpack.optimize.CommonsChunkPlugin({
name: "a",
filename: "common_and_[name].js"
})
],
For all available options see CommonsChunkPlugin - Options
Recently, I ran into this same issue, but time has passed and CommonsChunkPlugin has been deprecated in favour of SplitChunksPlugin.
Currently there is no way to achieve the intended behaviour by using only Webpack's included functionalities, so I wrote a plugin that solves this problem and I'm glad to share it with who may need it!
You can find it HERE.