So i was moving my code to esbuild to fasten my builds.
When I do so, my website looks weird (when I run using esbuild).
I was comparing the diff between index.html of the webpack and esbuild and noticed that webpack one have this
<script src="/static/js/bundle.js"></script><script src="/static/js/vendors~main.chunk.js"></script><script src="/static/js/main.chunk.js"></script><script src="/static/js/bundle.js"></script><script src="/static/js/vendors~main.chunk.js"></script><script src="/static/js/main.chunk.js"></script></body>
Which is missing in esbuild.
This is my esbuild
// build.js
const esbuild = require('esbuild');
const inlineImage = require('esbuild-plugin-inline-image');
const { sassPlugin } = require('esbuild-sass-plugin');
const svgrPlugin = require('esbuild-plugin-svgr');
esbuild
.serve(
{
servedir: 'public',
port: 3000,
},
{
entryPoints: ['./src/index.tsx'],
outfile: './public/assets/app.js',
minify: true,
bundle: true,
define: {
'process.env.NODE_ENV': '"dev"',
'process.env.REACT_APP_BACKEND_URL': '"https://something.xyz.ai/"',
'process.env.REACT_APP_BACKEND_WSS': '"wss://something.xyz.ai/ws/"',
'process.env.REACT_APP_BACKEND_URL_STAGE': '"https://stage-new.something.xyz.ai/"',
'process.env.REACT_APP_HELP_AND_SUPPORT_URL': '"https://docs.something.xyz.ai/"',
},
loader: {
'.js': 'jsx',
'.ts': 'tsx',
},
plugins: [svgrPlugin({ exportType: 'named' }), inlineImage(), sassPlugin()],
},
)
.catch(() => process.exit(1));
I am curious, how does react put /static/js/bundle.js and other scripts and how can I make esbuild do the same?
The scripts present in webpack output are due to code splitting.
You can enable code splitting in esbuild by adding splitting: true to the config esbuild.serve(..., { splitting: true, ... }) or --splitting in case of command line.
One caveat though; currently esbuild does not support code splitting fully:
Code splitting is still a work in progress. It currently only works with the esm output format. There is also a known ordering issue with import statements across code splitting chunks. You can follow the tracking issue for updates about this feature.
When I do so, my website looks weird (when I run using esbuild).
Code splitting is a performance optimisation and as such rather unlikely cause of issues with altered functioning of the app. More likely cause is that some assets are not loaded properly. Depending on the complexity of previous webpack setup switching to esbuild may require performing additional steps to match webpack setup. The more non-standard setup the harder it gets.
Specifically so-called loaders (esbuild docs | webpack docs) may work differently for the same types of files.
Related
We are using vue-loader (usually together with the html-webpack-plugin, but I omit this in the following because it is not the important part) to transpile multiple vue based applications within a single project. Our webpack configuration looks a little bit like the following:
const apps = [
{ html: 'app-1', js: 'app-1' },
{ html: 'app-2', js: 'app-2' },
...
]
module.exports = (_a, _b) =>({
entry: Object.fromEntries(apps.map(app => [app.js, './src/${app.js}.js'])),
...
plugins: [
new VueLoaderPlugin()
],
...
})
So for instance, app-1.html has a div with id app, which is referenced in app-1.js:
import Vue from 'vue'
import App1 from './app-1.vue'
...
new Vue({...}).$mount('#app')
This means our directory structure (very simplified) looks like the following:
src
app-1.html
app-1.js
app-1.vue
app-2.html
app-2.js
app-2.vue
...
package.json
webpack.config.js
Now here is our problem: We noticed a very high memory consumption, which increases drastically the more apps we have in our apps array. This is something like > 10 GB for a project containing 8 of these apps.
Could this be something like a memory leak in the vue-loader or are we somehow misusing the plugin? We are using version 15.10.0 of vue-loader.
It seems that vue-loader was not the culprit. I can reproduce the same high memory consumption when I remove the vue-loader completely and don't use any plugins at all. It seems that the terser plugin (which is automatically used by webpack) is the one using up that much memory (see also https://github.com/webpack/webpack/issues/13550). By default terser spawns a lot of threads. One can control this behaviour:
const TerserPlugin = require("terser-webpack-plugin")
...
module.exports = {
...
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
parallel: 2,
})
]
}
}
Alternative solution would be to use another minimizer, e.g. esbuild-loader.
Situation
I would like to run some Database code (mongoDB(mongoose)) on server startup / during builds. Considering next js doesn't have any lifecycle hooks that you can hook into in an easy manner, I was trying to perform the database actions in my webpack (next.config.mjs) configuration. However I ran into some problems with importing local files.
Current setup
This is the code of my current next.config.mjs file. (PS. I have also tried the CommonJS way of requiring the needed files, but that also fails with error meessage "module not found".)
None of the lines that import a local typescript file appear to succeed and I have checked the paths multiple times. They always end up with the error message "ERR_MODULE_NOT_FOUND". Only if a node_module package is imported, it works as expected (the mongoose npm package).
Code
/** #type {import('next').NextConfig} */
const { EmployeesSchema } = await import("./mongodb_schemas/employee_schema");
import { EmployeesSchema } from "./mongodb_schemas/employee_schema";
import "./util/test"
import mongoose from "mongoose";
const nextConfig = {
experimental: {
externalDir: true,
},
reactStrictMode: true,
swcMinify: true,
images: {
domains: ["*", "**", "www.google.com"],
},
webpack: (
config,
{ buildId, dev, isServer, defaultLoaders, nextRuntime, webpack }
) => {
if (isServer) {
console.log(process.cwd());
}
return config;
},
};
export default nextConfig;
Anyone got a clue to why this might end up happening / have any possible solutions to the problem? I have also tried with a normal JavaScript file instead of a Typescript file, which also didn't work. I have found some similar asked questions on Stack Overflow but which were all left unanswered.
My guess for the reason why this occurs: during the build of the project, so when "npm run dev" is ran, the next.config.mjs is copied to a different location into the file structure, which means that the relative paths aren't correct anymore and thus the files can't be found.
PS. My apologize if the question is unclear / in an unusual format, it is my first post so not used to it.
The app is created with the default template for Kotlin React apps:
uses KTS-based Gradle build scripts;
Kotlin JS plugin 1.6.10;
Kotlin wrappers for React 17.0.2.
When using ./gradlew browserProductionWebpack without any additional tweaks, it generates a build/distributions directory with:
all resources (without any modifications);
index.html (without any modifications);
Kotlin sources compiled into one minified .js file.
What I want is to:
add some hash to the generated .js file;
minify the index.html file and refer the hashed .js file in it;
minify all resources (.json localization files).
Please prompt me some possible direction to do it. Looking to webpack configuration by adding corresponding scripts into webpack.config.d, but no luck yet: tried adding required dependencies into build.gradle.kts, i.e.:
implementation(devNpm("terser-webpack-plugin", "5.3.1"))
implementation(devNpm("html-webpack-plugin", "5.5.0"))
and describing webpack scripts:
const TerserPlugin = require("terser-webpack-plugin");
const HtmlWebpackPlugin = require('html-webpack-plugin');
module.exports = {
optimization: {
minimizer: [
new TerserPlugin(),
new HtmlWebpackPlugin({
minify: {
removeAttributeQuotes: true,
collapseWhitespace: true,
removeComments: true,
},
}),
],
}
}
Any hint will be appreciated.
A couple of things to put into consideration first.
If some flexible bundling configuration is needed, most likely it won't be possible to use Kotlin-wrapped (Gradle) solutions. After checking the org.jetbrains.kotlin.gradle.targets.js.webpack.KotlinWebpackConfig it turns out that there is only a limited set of things can be configured with it.
JS-based webpack configs require a little bit of reverse engineering to find out what is generated from the Kotlin/Gradle side and how to extend it.
For simple configurations (IMHO) there is almost no need in tweaking bundling options while using the Kotlin JS plugin.
Let's start from initial webpack configs. In my case (see the short environment description in the question above) they appear in ./build/js/packages/<project-name>/webpack.config.js. These original configs will also include all contents from JS files we create inside the ./webpack.config.d folder.
Some webpack configurations require external JS dependencies. We need to declare them in the dependencies block of build.gradle.kts. In my case they are represented with:
// Bundling.
implementation(devNpm("html-webpack-plugin", "5.5.0"))
implementation(devNpm("uglifyjs-webpack-plugin", "2.2.0"))
implementation(devNpm("terser-webpack-plugin", "5.3.1"))
implementation(devNpm("copy-webpack-plugin", "9.1.0" )) // newer versions don't work correctly with npm and Yarn
implementation(devNpm("node-json-minify", "3.0.0"))
I also dropped all commonWebpackConfigs from build.gradle.kts as they are going to be performed manually on the JS level.
All webpack JS configs (inside the ./webpack.config.d folder) are divided into 3 files:
common.js (dev server configuration for both dev and production builds):
// All route paths should fallback to the index page to make SPA's routes processed correctly.
const devServer = config.devServer = config.devServer || {};
devServer.historyApiFallback = true;
development.js:
// All configs inside of this file will be enabled only in the development mode.
// To check the outputs of this config, see ../build/processedResources/js/main
if (config.mode == "development") {
const HtmlWebpackPlugin = require("html-webpack-plugin");
// Pointing to the template to be used as a base and injecting the JS sources path into.
config.plugins.push(new HtmlWebpackPlugin({ template: "./kotlin/index.html" }));
}
and production.js:
// All configs inside of this file will be enabled only in the production mode.
// The result webpack configurations file will be generated inside ../build/js/packages/<project-name>
// To check the outputs of this config, see ../build/distributions
if (config.mode == "production") {
const HtmlWebpackPlugin = require("html-webpack-plugin"),
UglifyJsWebpackPlugin = require("uglifyjs-webpack-plugin"),
TerserWebpackPlugin = require("terser-webpack-plugin"),
CopyWebpackPlugin = require("copy-webpack-plugin"),
NodeJsonMinify = require("node-json-minify");
// Where to output and how to name JS sources.
// Using hashes for correct caching.
// The index.html will be updated correspondingly to refer the compiled JS sources.
config.output.filename = "js/[name].[contenthash].js";
// Making sure optimization and minimizer configs exist, or accessing its properties can crash otherwise.
config.optimization = config.optimization || {};
const minimizer = config.optimization.minimizer = config.optimization.minimizer || [];
// Minifying HTML.
minimizer.push(new HtmlWebpackPlugin({
template: "./kotlin/index.html",
minify: {
removeAttributeQuotes: true,
collapseWhitespace: true,
removeComments: true,
},
}));
// Minifying and obfuscating JS.
minimizer.push(new UglifyJsWebpackPlugin({
parallel: true, // speeds up the compilation
sourceMap: false, // help to match obfuscated functions with their origins, not needed for now
uglifyOptions: {
compress: {
drop_console: true, // removing console calls
}
}
}));
// Additional JS minification.
minimizer.push(new TerserWebpackPlugin({
extractComments: true // excluding all comments (mostly licence-related ones) into a separate file
}));
// Minifying JSON locales.
config.plugins.push(new CopyWebpackPlugin({
patterns: [
{
context: "./kotlin",
from: "./locales/**/*.json",
to: "[path][name][ext]",
transform: content => NodeJsonMinify(content.toString())
}
]
}));
}
I use styled components, so no CSS configs are provided. In other things these configs do almost the same minification as being done out-of-the-box without any additional configs. The differences are:
JS sources use a hash in their name: it is referenced correctly from the index page HTML template;
HTML template is minified;
locales (just simple JSON files) are minified.
It can look like an overhead slightly because as mentioned in the beginning, it does almost the same with minimal differences from the out-of-the-box configs. But as advantage we're getting more flexible configs which can be tweaked easier further.
We're using the style-loader in webpack, which, when compiled seems to place information about the current directory in the source code that injects style tags when modules are loaded/unloaded. It looks roughly like this:
if(false) {
// When the styles change, update the <style> tags
if(!content.locals) {
module.hot.accept("!!./../../../../../node_modules/css-loader/index.js!./../../../../../node_modules/sass-loader/index.js?includePaths[]=/var/deploy/referrals/web_head/releases/20151118202441/node_modules/patternity/node_modules/node-neat/node_modules/node-bourbon/node_modules/bourbon/app/assets/stylesheets&includePaths[]=/var/deploy/referrals/web_head/releases/20151118202441/node_modules/patternity/node_modules/node-neat/node_modules/bourbon-neat/app/assets/stylesheets&includePaths[]=/var/deploy/referrals/web_head/releases/20151118202441/node_modules/patternity&includePaths[]=/var/deploy/referrals/web_head/releases/20151118202441/node_modules/infl-fonts!./campaigns.scss", function() {
var newContent = require("!!./../../../../../node_modules/css-loader/index.js!./../../../../../node_modules/sass-loader/index.js?includePaths[]=/var/deploy/referrals/web_head/releases/20151118202441/node_modules/patternity/node_modules/node-neat/node_modules/node-bourbon/node_modules/bourbon/app/assets/stylesheets&includePaths[]=/var/deploy/referrals/web_head/releases/20151118202441/node_modules/patternity/node_modules/node-neat/node_modules/bourbon-neat/app/assets/stylesheets&includePaths[]=/var/deploy/referrals/web_head/releases/20151118202441/node_modules/patternity&includePaths[]=/var/deploy/referrals/web_head/releases/20151118202441/node_modules/infl-fonts!./campaigns.scss");
if(typeof newContent === 'string') newContent = [[module.id, newContent, '']];
update(newContent);
});
}
// When the module is disposed, remove the <style> tags
module.hot.dispose(function() { update(); });
}
The thing to note is the directory listed in the accept string.
Now, this code ultimately gets removed once uglify is run (because of the if (false)) which is great. Where my problem lies however is that this compilation happens on 2 machines and the chunk hashing appears to happen before uglification, because the hash generated on 2 different machines (or even on the same machine but when in a different folder) is different. This obviously won't work if I'm deploying to, say, a production machine and need both of these machines to serve up an asset with the same digest.
Just to clarify, when not minified, the code is different, thus generates a different hash, but minification does in fact make the files identical, however the chunk hashing appears to have happened before minification.
Does anyone know how I can get the chunkhash to be generated after the uglify plugin is run. I'm using a config like so:
...
output: {
filename: '[name]-[chunkhash].js'
...
with the command:
webpack -p
edit
So after looking over this. I'm seeing now that this has to do with us adding includePaths to our style loader, it looks like this:
var includePaths = require('node-neat').includePaths;
module: {
loaders: [
{ test: /\.scss$/, loader: "style!css!sass?includePaths[]=" + includePaths },
]
}
So I think we know why we're getting these absolute URLs, but I think the original question still stands, IMO webpack should be hashing chunks AFTER minification, not before.
I am having trouble with webpacks code splitting functionality. I am trying to have 2 named chunks for two routes in my application which are not often visited. mysite.com/settings and mysite.com/access.
here is my webpack.config.coffee
module.exports =
contentBase: "#{__dirname}/src/"
cache: true
entry:
app: './src/coffee/app'
head: './src/coffee/head'
output:
path: path.join(__dirname, 'build')
publicPath: '/'
filename: '[name].js'
chunkFilename: '[name]-[chunkhash].js'
plugins: []
And here is my router.coffee
access: (slug) ->
_this = #
require.ensure ['../view/page/access-page.coffee'], (require) ->
AccessPage = require '../view/page/access-page.coffee'
accessPage = AccessPage.getInstance()
accessPage.render() unless accessPage.isRendered
_this.showPage accessPage
, 'access'
settings: (slug) ->
_this = #
require.ensure ['../view/page/settings-page.coffee'], (require) ->
SettingsPage = require '../view/page/settings-page.coffee'
settingsPage = SettingsPage.getInstance()
settingsPage.render() unless settingsPage.isRendered
_this.showPage settingsPage
, 'settings'
I am not using the webpack dev-server, instead I am watching simply by using the following cmd-line tool
webpack -d --progress --colors --watch
The problem is that it ignores the names when requiring the files, as you can see the format is '[name]-[hash].js' it generates files with the correct format e.g. settings-2j3nekj2n3ejkn2.js but during development, when I attempt to load the page, the browser complains that '-2j3nekj2n3ejkn2.js' cannot be found, somehow the mapping of the files, ignores the names. If I leave out the names, then it works.
So the question is how can I setup mulitple named chunks correctly. Thanks in advance.
Note I have checked out their examples in the docs at https://github.com/webpack/docs/wiki/code-splitting
and I have followed their optimization docs aswell at
https://github.com/webpack/docs/wiki/optimization
But I am stuck
Well the simple answer is - [name= is not supported in chunkName.
The awesome guys at Webpack have actually heard my cries and implemented it
Here is the commit
https://github.com/webpack/webpack/commit/03c87c11a4219ae6ec6bfe87e570a0dacceac859
As a result of the following issue I made
https://github.com/webpack/webpack/issues/358
It is already available as of Beta ^1.3.2