We Keep Coding

Next.js (React) - Can't import local typescript file into config file

Situation
I would like to run some Database code (mongoDB(mongoose)) on server startup / during builds. Considering next js doesn't have any lifecycle hooks that you can hook into in an easy manner, I was trying to perform the database actions in my webpack (next.config.mjs) configuration. However I ran into some problems with importing local files.
Current setup
This is the code of my current next.config.mjs file. (PS. I have also tried the CommonJS way of requiring the needed files, but that also fails with error meessage "module not found".)
None of the lines that import a local typescript file appear to succeed and I have checked the paths multiple times. They always end up with the error message "ERR_MODULE_NOT_FOUND". Only if a node_module package is imported, it works as expected (the mongoose npm package).
Code
/** #type {import('next').NextConfig} */
const { EmployeesSchema } = await import("./mongodb_schemas/employee_schema");
import { EmployeesSchema } from "./mongodb_schemas/employee_schema";
import "./util/test"
import mongoose from "mongoose";
const nextConfig = {
experimental: {
externalDir: true,
},
reactStrictMode: true,
swcMinify: true,
images: {
domains: ["*", "**", "www.google.com"],
},
webpack: (
config,
{ buildId, dev, isServer, defaultLoaders, nextRuntime, webpack }
) => {
if (isServer) {
console.log(process.cwd());
}
return config;
},
};
export default nextConfig;
Anyone got a clue to why this might end up happening / have any possible solutions to the problem? I have also tried with a normal JavaScript file instead of a Typescript file, which also didn't work. I have found some similar asked questions on Stack Overflow but which were all left unanswered.
My guess for the reason why this occurs: during the build of the project, so when "npm run dev" is ran, the next.config.mjs is copied to a different location into the file structure, which means that the relative paths aren't correct anymore and thus the files can't be found.
PS. My apologize if the question is unclear / in an unusual format, it is my first post so not used to it.

Use different output.libraryTarget for a Web Worker

I have this Webpack configuration:
{
output: {
libraryTarget: "system",
...
},
...
}
I'm trying to use a Web Worker. I'm using the standard Webpack 5 syntax:
new Worker(new URL('./MyWorker', import.meta.url));
Now Webpack outputs the Web Worker as a System.js module. How can I change it to something different, like ES module, without affecting the main bundle?

You can use chunkFormat to specify what the chunk formats are, workers are by default array-push, https://webpack.js.org/configuration/output/#outputchunkformat.
You can also create multiple configs with different targets for different entries.
const config = {
//
}
module.exports = (env) => {
if (env.module) {
config.entry = //path/to/index
config.output.libraryTarget = 'module'
} else {
config.entry = //path/to/worker
config.output.libraryTarget = 'umd'
}
return config
}
Then you can separately compile your web workers or chunks different from others. You can also use chunkFileName: () => along with that.
If you want to compile it in a single run, without having to run it twice using different configs, you can also manually invoke the webpack compiler with both configs.
import {Compiler} from 'webpack'
// or import webpack from 'webpack'
compiler.run(config)
compiler.run(config2)
Then you can run both of them at the same time and compile everything.
Another possible option is enabledChunkLoadingTypes, https://webpack.js.org/configuration/output/#outputenabledchunkloadingtypes, which will allow you to specify the available loading types for chunks which webpack will automatically use based on the entry function. I've never used that myself so I don't know if that'll work but it's something you can try.

How to include manual import() in Webpack Bundle

I am quite new to Webpack, so bear with me if thats a stupid question.
My goal is to transform my old, AMD based codebase to a ES6 Module based solution. What I am struggling with is handling dynamic import()s. So my app router works on a module basis, i.e. each route is mapped to a module path and then required. Since I know what modules will be included, I just add those dynamically imported modules to my r.js configuration and am able to build everything in a single file, with all require calls still working.
Now, I am trying to do the same with ES6 modules and Webpack. With my devmode this is no problem as I can just replace require() with import(). However I cannot get this to work with bundling. Either Webpack splits my code (and still fails to load the dynamic module anyways), or - if I use the Array format for the entry config, the dynamic module is included in the bundle but loading still fails: Error: Cannot find module '/src/app/DynClass.js'
This is how my Webpack config looks like:
const webpack = require('webpack');
const path = require('path');
module.exports = {
mode: "development",
entry: ['./main.js', './app/DynClass.js'],
output: {
filename: 'main.js',
path: path.resolve(__dirname, "../client/")
},
resolve: {
alias: {
"/src": path.resolve(__dirname, '')
}
},
module: {
rules: [
{
test: /\.tpl$/i,
use: 'raw-loader',
},
]
}
};
So basically I want to tell Webpack: "hey, there is another module (or more) that is to be loaded dynamically and I want it to be included in the bundle"
How can I do this?

So yeah, after much fiddling there seems to be a light at the end of the tunnel. Still, this is not a 100% solution and it is surely not for the faint of heart, as it is quite ugly and fragile. But still I want to share my approach with you:
1) manual parsing of my routes config
My router uses a config file looking like this:
import StaticClass from "/src/app/StaticClass.js";
export default {
StaticClass: {
match: /^\//,
module: StaticClass
},
DynClass: {
match: /^\//,
module: "/src/app/DynClass.js"
}
};
So as you can see the export is an object, with keys acting as the route id, and an object that contains the matches (regex based) and the module which should be executed by the router if the route matches. I can feed my router with both a Constructor function (or an object) for modules which are available immediatly (i.e. contained in the main chunk) or if the module value is a string, this means that the router has to load this module dynamically by using the path specified in the string.
So as I know what modules could be potentially loaded (but not if and when) I can now parse this file within my build process and transform the route config to something webpack can understand:
const path = require("path");
const fs = require("fs");
let routesSource = fs.readFileSync(path.resolve(__dirname, "app/routes.js"), "utf8");
routesSource = routesSource.substr(routesSource.indexOf("export default"));
routesSource = routesSource.replace(/module:\s*((?!".*").)*$/gm, "module: undefined,");
routesSource = routesSource.replace(/\r?\n|\r/g, "").replace("export default", "var routes = ");
eval(routesSource);
let dummySource = Object.entries(routes).reduce((acc, [routeName, routeConfig]) => {
if (typeof routeConfig.module === "string") {
return acc + `import(/* webpackChunkName: "${routeName}" */"${routeConfig.module}");`;
}
return acc;
}, "") + "export default ''";
(Yeah I know this is quite ugly and also a bit brittle so this surely could be done better)
Essentially I create a new, virtual module where every route entry which demands a dynamic import is translated, so:
DynClass: {
match: /^\//,
module: "/src/app/DynClass.js"
}
becomes:
import(/* webpackChunkName: "DynClass" */"/src/app/DynClass.js");
So the route id simply becomes the name of the chunk!
2) including the virtual module in the build
For this I use the virtual-module-webpack-plugin:
plugins: [
new VirtualModulePlugin({
moduleName: "./app/dummy.js",
contents: dummySource
})
],
Where dummySource is just a string containing the sourcecode of my virtual module I just have generated. Now, this module is pulled in and the "virtual imports" can be processed by webpack. But wait, I still need to import the dummy module, but I do not have any in my development mode (where I use everything natively, so no loaders).
So in my main code I do the following:
let isDev = false;
/** #remove */
isDev = true;
/** #endremove */
if (isDev) { import('./app/dummy.js'); }
Where "dummy.js" is just an empty stub module while I am in development mode. The parts between that special comments are removed while building (using the webpack-loader-clean-pragma loader), so while webpack "sees" the import for dummy.js, this code will not be executed in the build itself since then isDev evaluates to false. And since we already defined a virtual module with the same path, the virtual module is included while building just like I want, and of course all dependencies are resolved as well.
3) Handling the actual loading
For development, this is quite easy:
import routes from './app/routes.js';
Object.entries(routes).forEach(async ([routeId, route]) => {
if (typeof route.module === "function") {
new route.module;
} else {
const result = await import(route.module);
new result.default;
}
});
(Note that this is not the actual router code, just enough to help me with my PoC)
Well, but for the build I need something else, so I added some code specific to the build environment:
/** #remove */
const result = await import(route.module);
new result.default;
/** #endremove */
if (!isDev) {
if (typeof route.module === "string") { await __webpack_require__.e(routeId); }
const result = __webpack_require__(route.module.replace("/src", "."));
new result.default;
}
Now, the loading code for the dev environment is just stripped out, and there is another loading code that uses webpack internally. I also check if the module value is a function or string, and if it is the latter I invoke the internal require.ensure function to load the correct chunk: await __webpack_require__.e(routeId);. Remember that I named my chunks when generating the virtual module? Now thats why I still can find them now!
4) more needs to be done
Another thing I encountered is when several dynamically loaded modules have the same dependencies, webpack tries to generate more chunks with names like module1~module2.bundle.js, breaking my build. To counter this, I needed to make sure that all those shared modules go into a specific named bundle I called "shared":
optimization: {
splitChunks: {
chunks: "all",
name: "shared"
}
}
And when in production mode, I simply load this chunk manually before any dynamic modules depending on it are requested:
if (!isDev) {
await __webpack_require__.e("shared");
}
Again, this code only runs in production mode!
Finally, I have to prevent webpack renaming my modules (and chunks) to something like "1", "2" etc, but rather keep the names I just have defined:
optimization: {
namedChunks: true,
namedModules: true
}
Se yeah, there you have it! As I said this wasn't pretty but seems to work, at least with my simplified test setup. I really hope there aren't any blockers ahead of me when I do all the rest (like ESLint, SCSS etc)!

Skipping entry with js files in WebPack

I have to use Webpack for one of my projects to build front-end bundles for js, css and other static assets. It does the job well, but in my early stages of the project I've got only some css and static images and no js files yet. Here is my full webpack.config.js
const Webpack = require("webpack");
const Glob = require("glob");
const path = require("path");
const CopyWebpackPlugin = require("copy-webpack-plugin");
const configurator = {
entries: function(){
var entries = {
application: [
'./assets/dummy.js',
],
}
return entries
},
plugins() {
var plugins = [
new CopyWebpackPlugin([{from: "./assets",to: ""}], {copyUnmodified: true,ignore: ["css/**", "js/**", "**.js"] }),
];
return plugins
},
moduleOptions: function() {
return {
rules: [
]
}
},
buildConfig: function(){
const env = process.env.NODE_ENV || "development";
var config = {
mode: env,
entry: configurator.entries(),
output: {filename: "[name].[hash].js", path: `${__dirname}/public/assets`},
plugins: configurator.plugins(),
module: configurator.moduleOptions()
}
return config
}
}
module.exports = configurator.buildConfig()
Practically what it does for me is copying assets to public dir. I don't have any javascripts yet, but they will be in the future. So, I tried commenting entry, setting it to null or empty string with no luck. It seems Webpack needs to process js files so badly. My current solution is creating an empty dummy.js file and feeding it to Webpack. Annoyingly it generates some 3.3kb application.afff4a3748b8d5d33a3a.js file with some boilerplate js code, despite that my source js file is totally empty.
I understand that this is an edge use case for Webpack and Webpack was primarily created for processing javascript, but I bet many people still use it not just for bundling javascripts. So, my question, is there a better, more elegant way to skip bundling js files in Webpack?
p.s.
I think, I've found a related question without an answer here How to make WebPack copy a library instead of bundling?
p.s.#2
The suggested duplicate question has an accepted answer with an invalid Webpack config Invalid configuration object., so, I can't use it to solve my issue.
Moreover, the answer reads
webpack will create a dummy javascript file
and I'm specifically asking how to avoid creating unnecessary files.

How can I get PurifyCSSPlugin to remove my unused css in Angular6?

I'm trying to remove a lot of unused css within my sass files in my Angular 6 project.
I'm learned that there is a webpack plugin called PurifyCss.
Currently now, I'm unable to eject the webpack config in my angular project so I'm using ngw to help add the necessary plugins (to angular's webpack config) needed to extract the unused css in my sass files.
ngw.config.ts
import * as webpack from 'webpack';
import { Path } from '#angular-devkit/core';
import { NormalizedBrowserBuilderSchema } from '#angular-devkit/build-angular';
import * as PurifyCSSPlugin from 'purifycss-webpack';
import * as path from 'path';
import * as glob from 'glob';
export type WebpackOptions<T = NormalizedBrowserBuilderSchema> = {
root: Path,
projectRoot: Path,
options: T;
};
const command = process.argv[2].toLowerCase();
export default function (config: webpack.Configuration, options: WebpackOptions) {
if (command === 'test') {
console.log('Test configuration is running');
}
console.log('To modify webpack build, you can use ngw.config.ts');
console.log('check path:', glob.sync(path.join(__dirname, 'src/**/*.html')));
config.plugins.push(
new PurifyCSSPlugin({
// This was suggested to help it actually remove the css from: https://github.com/webpack-contrib/purifycss-webpack/issues/54
// Although there is an error of: Error: data['resolveExtensions'] should NOT have additional properties
resolveExtensions: ['.html', '.js'],
// This causes the build to run but does not remove the unused css
paths: glob.sync(path.join(__dirname, 'src/**/*.html'))
}),
);
return config;
}
Using the paths property alone doesn't work and it was suggested to add resolveExtensions from here.
Although this leads to the error below when doing a ngw prod build:
Error: data['resolveExtensions'] should NOT have additional properties
How can I configure the PurifyCSSPlugin to remove unused css within sass files in an Angular-6-cli environement?
Note: I don't have much experience with webpack, so I'm not sure if this config only works with css files instead of scss files. (If so please correct me).
Thanks