Does webpack expose the dependency tree before the build is sealed? I have searched all through the compiler instance, but have found nothing about a dependency tree. It seems that there should be one hidden in that object somewhere, because webpack has to know what this tree is in order to later output the stats.json.
I've tried using the dependency-tree npm package, but it doesn't support some things I have in my webpack config, so the tree is incomplete.
TL;DR: Yes, you can access the dependency tree just before it's being sealed.
To do this, add the following code to your webpack.config.js:
class AccessDependenciesPlugin {
apply (compiler) {
compiler.hooks.compilation.tap('AccessDependenciesPlugin', compilation => {
compilation.hooks.finishModules.tap('AccessDependenciesPlugin', modules => {
/*
|---------------------------------------------------
| Here we go, `modules` is what we're looking for!
|---------------------------------------------------
*/
})
})
}
}
module.exports = {
// ...
plugins: [
new AccessDependenciesPlugin()
]
}
For more details, see the explanation below.
The hook we're looking for
We can access the pre-sealed dependency tree with the finishModules compilation hook.
How do we know?
Since the webpack hook docs are very minimal (to say the least), we had to read webpack source code to be sure it's what we're looking for:
The last thing the compiler does before sealing the dependency tree is "finishing" it.
This happens in Compiler.js, line 492
Finishing the dependency tree offers a hook on the compilation.
finishModules hook in the webpack docs
The hook is being called in Compilation.js, line 828
Code example
We create a plugin called AccessDependenciesPlugin:
// Basic webpack plugin structure
class AccessDependenciesPlugin {
apply (compiler) {
}
}
To use a compilation hook, we need to get access to the compilation object first. We do that with the compilation hook:
class AccessDependenciesPlugin {
apply (compiler) {
compiler.hooks.compilation.tap('AccessDependenciesPlugin', compilation => {
// We have access to the compilation now!
})
}
}
Now we tap the finishModules hook of the compilation:
class AccessDependenciesPlugin {
apply (compiler) {
compiler.hooks.compilation.tap('AccessDependenciesPlugin', compilation => {
compilation.hooks.finishModules.tap('AccessDependenciesPlugin', modules => {
// Here we go, `modules` is what we're looking for!
})
})
}
}
The modules parameter of that hook is an array of webpack modules with their dependencies and basically all other data available about them.
Last but not least, we need to add the plugin to our webpack configuration:
module.exports = {
plugins: [
new AccessDependenciesPlugin()
]
}
And we're done. 🎉
Hope this helps.
Bonus Content: webpack 3
Per request in the comments: Here's the version of the AccessDependenciesPlugin for the legacy plugin system of webpack 3.
class AccessDependenciesPlugin {
apply (compiler) {
compiler.plugin('compilation', compilation => {
compilation.plugin('finish-modules', modules => {
/*
|---------------------------------------------------
| Here we go, `modules` is what we're looking for!
|---------------------------------------------------
*/
})
})
}
}
Perhaps the line of code from this GitHub thread will help you out:
"Compile with webpack --profile --json > stats.json
(node.js API: { profile: true } and stats.toJson())
Go to http://webpack.github.io/analyse/#modules
Load your stats file (It's not uploaded, the analyse tools is a client only tool).
Wait a bit until the graph has been stabilized."
If that's not what you need, then I'd look into #Loilo 's answer - more complicated, but probably more what you need too.
Related
Situation
I would like to run some Database code (mongoDB(mongoose)) on server startup / during builds. Considering next js doesn't have any lifecycle hooks that you can hook into in an easy manner, I was trying to perform the database actions in my webpack (next.config.mjs) configuration. However I ran into some problems with importing local files.
Current setup
This is the code of my current next.config.mjs file. (PS. I have also tried the CommonJS way of requiring the needed files, but that also fails with error meessage "module not found".)
None of the lines that import a local typescript file appear to succeed and I have checked the paths multiple times. They always end up with the error message "ERR_MODULE_NOT_FOUND". Only if a node_module package is imported, it works as expected (the mongoose npm package).
Code
/** #type {import('next').NextConfig} */
const { EmployeesSchema } = await import("./mongodb_schemas/employee_schema");
import { EmployeesSchema } from "./mongodb_schemas/employee_schema";
import "./util/test"
import mongoose from "mongoose";
const nextConfig = {
experimental: {
externalDir: true,
},
reactStrictMode: true,
swcMinify: true,
images: {
domains: ["*", "**", "www.google.com"],
},
webpack: (
config,
{ buildId, dev, isServer, defaultLoaders, nextRuntime, webpack }
) => {
if (isServer) {
console.log(process.cwd());
}
return config;
},
};
export default nextConfig;
Anyone got a clue to why this might end up happening / have any possible solutions to the problem? I have also tried with a normal JavaScript file instead of a Typescript file, which also didn't work. I have found some similar asked questions on Stack Overflow but which were all left unanswered.
My guess for the reason why this occurs: during the build of the project, so when "npm run dev" is ran, the next.config.mjs is copied to a different location into the file structure, which means that the relative paths aren't correct anymore and thus the files can't be found.
PS. My apologize if the question is unclear / in an unusual format, it is my first post so not used to it.
I'm on webpack#4.43.0 and am trying to use this web assembly library https://github.com/vislyhq/stretch As per docs I am importing some classes from it i.e.
import { Allocator, Node } from 'stretch-layout';
class Base {
public layoutNode;
public constructor() {
this.layoutNode = new Node(allocator, {});
}
}
However when I am trying to build it with webpack (not using any loaders and I have .wasm in my resolve.extensions webpack config) I am getting following error
WebAssembly module is included in initial chunk. This is not allowed,
because WebAssembly download and compilation must happen asynchronous.
Add an async splitpoint (i. e. import()) somewhere between your
entrypoint and the WebAssembly module:
To my understanding I need to use import() for this module, but how do I make these imports available to my class? Application will fail since we need to await that import? If I do something like this I get an error saying that Allocator and Node are not constructors.
let Allocator: any = null;
let Node: any = null;
import('stretch-layout').then(module => {
Allocator = module.Allocator;
Node = module.Node;
});
I was able to solve this using latest beta of webpack 5, by adding following experimental flags to config
experiments: {
asyncWebAssembly: true,
importAsync: true
}
this way you don't have to worry about any async import() statements at all
A user tries to use my package for nuxt.js, but gets the error: document is not defined.
I found the first issue. When I build the bundle with "build-bundle": "vue-cli-service build --target lib --name index ./src/index.js",
vue-style-loader is being used. This, however, results in the error for using nuxt projects. This part is failing:
function addStyle (obj /* StyleObjectPart */) {
var update, remove
var styleElement = document.querySelector('style[' + ssrIdKey + '~="' + obj.id + '"]')
Document is not defined since we are using server rendering. But the question is how can I build up my package so that I can use it with nuxt?
I need:
index.common.js
index.umd.js
index.umd.min.js
This is due to the server-side rendering. If you need to specify that you want to import a resource only on the client-side, you need to use the process.client variable.
For example, in your .vue file:
if (process.client) {
require('external_library')
// do something
}
The above is the fundamental solution to document is not defined.
I checked some information and found that, this problem is not caused by your package. In fact, the problem lies on the cache-loader package in the user’s nuxt project.
For some reason cache-loader incorrectly determined the current environment as browser and not node so that vue-style-loader is confused and used client implementation instead.
So try to let users add the following configuration to the nuxt.config.js file to disable stylesheet caches on server-side:
build: {
...
cache: true,
extend(config, { isServer, isDev, isClient }) {
...
if (isServer) {
for (const rules of config.module.rules.filter(({ test }) =>
/\.((c|le|sa|sc)ss|styl.*)/.test(test.toString())
)) {
for (const rule of rules.oneOf || []) {
rule.use = rule.use.filter(
({ loader }) => loader !== 'cache-loader'
)
}
}
}
...
}
...
}
I found a solution but it is not using the vue-cli service. Instead, the files are compiled by rollup. I found using the cli service much easier. The only problem with the cli service is it will adjust the "flow" of your repo. However, you can modify the rollup.config.js to amend the folder structure.
The problem with rollup is that it isn't webpack. Therefore, all components using a webpack configuration need to be adjusted or rollup.config.js needs to be amended to include the additional functionality
I'm trying to create a custom transform for Jest, but running into a documentation roadblock which has me asking myself if I'm even on the right track.
Problem
I have a Rails project which is serving a Vue JS app. I want to write Jest tests for the JS app. In order to pass config variables from Rails to the app, I'm using ERB to template a small number of .js files. For example:
// in server-routes.js.erb
export default {
reports: '<%= Rails.application.config.relative_url_root %><%= Rails.application.routes.url_helpers.reports_path %>',
...
In my Webpack build for the Vue app, I use rails-erb-loader to preprocess the *.erb files before they get passed to the rest of the build process.
However, when I run my JS tests, Jest doesn't know anything about ERB loaders (reasonably enough). So my goal is to add a custom transform for Jest to convert the ERB files when running npm test.
Approach
I thought I might be able to use rails-erb-loader as a Jest transform:
// package.json
"jest": {
"moduleFileExtensions": [
"js",
"json",
"vue"
],
"moduleDirectories": [
"<rootDir>/node_modules"
],
"transform": {
".*\\.(vue)$": "vue-jest",
"^.+\\.js$": "babel-jest",
"^.+\\.js\\.erb$": "rails-erb-loader"
},
This doesn't work, however, because Jest transforms and Webpack loaders seemingly have different signatures. In particular, Jest expects a process function:
$ npm test
FAIL app/javascript/components/__tests__/dummy.test.js
● Test suite failed to run
TypeError: Jest: a transform must export a `process` function.
> 101 | import ServerRoutes from '../server-routes.js.erb';
| ^
at ScriptTransformer._getTransformer (node_modules/#jest/transform/build/ScriptTransformer.js:291:15)
at ScriptTransformer.transformSource (node_modules/#jest/transform/build/ScriptTransformer.js:353:28)
at ScriptTransformer._transformAndBuildScript (node_modules/#jest/transform/build/ScriptTransformer.js:457:40)
at ScriptTransformer.transform (node_modules/#jest/transform/build/ScriptTransformer.js:513:25)
at app/javascript/components/related-media.vue:101:1
at Object.<anonymous> (app/javascript/components/related-media.vue:232:3)
And this is where I get stuck, because I can't see where it's documented what the API and behaviour of a process function should be. In the documentation for the transform config option there's a single not very helpful example, and that's it as far as docs go, unless I've missed something.
I also note that babel-jest has a createTransformer function which sounds like it might be helpful, or at least illuminating, but again I can't find any docs on what it does.
If anyone has pointers on the details of creating custom Jest transforms, or at least some better docs, that would be great! Or, if I'm going about this the wrong way, what should I be doing?
You could look at ts-jest. https://github.com/kulshekhar/ts-jest/blob/master/src/ts-jest-transformer.ts. It is in typescript so is typed
Alternatively find the jest code that initiates the transform process. I don't think it is that difficult to find.
I think the transformer is either created with class constructor or via the factory function createTransformer.
From my understanding for the ts-jest-transformer and jest-erb-transformer it seems you need to export an object with public process or to export createTransformer methods which create a transformer object that have a process method.
a simple code example that runs before ts-jest
transform-example.js
const tsJest = require('ts-jest');
const t = tsJest.createTransformer();
module.exports = {
process(fileContent, filePath, jestConfig) {
const res = t.process(fileContent, filePath, jestConfig)
console.log(filePath);
return res;
}
}
jest.config.js
module.exports = {
transform: {
'^.+\\.tsx?$': ['<rootDir>/transform-example']
}
}
running this would run typescript tests (just like ts-jest) and log all the file paths transformed in the test.
Suppose I have the following module:
var modulesReq = require.context('.', false, /\.js$/);
modulesReq.keys().forEach(function(module) {
modulesReq(module);
});
Jest complains because it doesn't know about require.context:
FAIL /foo/bar.spec.js (0s)
● Runtime Error
- TypeError: require.context is not a function
How can I mock it? I tried using setupTestFrameworkScriptFile Jest configuration but the tests can't see any changes that I've made in require.
I had the same problem, then I've made a 'solution'.
I'm pretty sure that this is not the best choice. I ended up stopping using it, by the points answered here:
https://github.com/facebookincubator/create-react-app/issues/517
https://github.com/facebook/jest/issues/2298
But if you really need it, you should include the polyfill below in every file that you call it (not on the tests file itself, because the require will be no global overridden in a Node environment).
// This condition actually should detect if it's an Node environment
if (typeof require.context === 'undefined') {
const fs = require('fs');
const path = require('path');
require.context = (base = '.', scanSubDirectories = false, regularExpression = /\.js$/) => {
const files = {};
function readDirectory(directory) {
fs.readdirSync(directory).forEach((file) => {
const fullPath = path.resolve(directory, file);
if (fs.statSync(fullPath).isDirectory()) {
if (scanSubDirectories) readDirectory(fullPath);
return;
}
if (!regularExpression.test(fullPath)) return;
files[fullPath] = true;
});
}
readDirectory(path.resolve(__dirname, base));
function Module(file) {
return require(file);
}
Module.keys = () => Object.keys(files);
return Module;
};
}
With this function, you don't need to change any require.context call, it will execute with the same behavior as it would (if it's on webpack it will just use the original implementation, and if it's inside Jest execution, with the polyfill function).
After spending some hours trying each of the answers above. I would like to contribute.
Adding babel-plugin-transform-require-context plugin to .babelrc for test env fixed all the issues.
Install - babel-plugin-transform-require-context here https://www.npmjs.com/package/babel-plugin-transform-require-context (available with yarn too)
Now add plugin to .babelrc
{
"env": {
"test": {
"plugins": ["transform-require-context"]
}
}
}
It will simply transform require-context for test env into dummy fn calls so that code can run safely.
If you are using Babel, look at babel-plugin-require-context-hook. Configuration instructions for Storybook are available at Storyshots | Configure Jest to work with Webpack's require.context(), but they are not Storyshots/Storybook specific.
To summarise:
Install the plugin.
yarn add babel-plugin-require-context-hook --dev
Create a file .jest/register-context.js with the following contents:
import registerRequireContextHook from 'babel-plugin-require-context-hook/register';
registerRequireContextHook();
Configure Jest (the file depends on where you are storing your Jest configuration, e.g. package.json):
setupFiles: ['<rootDir>/.jest/register-context.js']
Add the plugin to .babelrc
{
"presets": ["..."],
"plugins": ["..."],
"env": {
"test": {
"plugins": ["require-context-hook"]
}
}
}
Alternatively, add it to babel.config.js:
module.exports = function(api) {
api.cache(true)
const presets = [...]
const plugins = [...]
if (process.env.NODE_ENV === "test") {
plugins.push("require-context-hook")
}
return {
presets,
plugins
}
}
It may be worth noting that using babel.config.js rather than .babelrc may cause issues. For example, I found that when I defined the require-context-hook plugin in babel.config.js:
Jest 22 didn't pick it up;
Jest 23 picked it up; but
jest --coverage didn't pick it up (perhaps Istanbul isn't up to speed with Babel 7?).
In all cases, a .babelrc configuration was fine.
Remarks on Edmundo Rodrigues's answer
This babel-plugin-require-context-hook plugin uses code that is similar to Edmundo Rodrigues's answer here. Props to Edmundo! Because the plugin is implemented as a Babel plugin, it avoids static analysis issues. e.g. With Edmundo's solution, Webpack warns:
Critical dependency: require function is used in a way in which dependencies cannot be statically extracted
Despite the warnings, Edmundo's solution is the most robust because it doesn't depend on Babel.
Extract the call to a separate module:
// src/js/lib/bundle-loader.js
/* istanbul ignore next */
module.exports = require.context('bundle-loader?lazy!../components/', false, /.*\.vue$/)
Use the new module in the module where you extracted it from:
// src/js/lib/loader.js
const loadModule = require('lib/bundle-loader')
Create a mock for the newly created bundle-loader module:
// test/unit/specs/__mocks__/lib/bundle-loader.js
export default () => () => 'foobar'
Use the mock in your test:
// test/unit/specs/lib/loader.spec.js
jest.mock('lib/bundle-loader')
import Loader from 'lib/loader'
describe('lib/loader', () => {
describe('Loader', () => {
it('should load', () => {
const loader = new Loader('[data-module]')
expect(loader).toBeInstanceOf(Loader)
})
})
})
Alrighty! I had major issues with this and managed to come to a solution that worked for me by using a combination of other answers and the Docs. (Took me a good day though)
For anyone else who is struggling:
Create a file called bundle-loader.js and add something like:
module.exports = {
importFiles: () => {
const r = require.context(<your_path_to_your_files>)
<your_processing>
return <your_processed_files>
}
}
In your code import like:
import bundleLoader from '<your_relative_Path>/bundle-loader'
Use like
let <your_var_name> = bundleLoader.importFiles()
In your test file right underneath other imports:
jest.mock('../../utils/bundle-loader', () => ({
importFiles: () => {
return <this_will_be_what_you_recieve_in_the_test_from_import_files>
}
}))
Installing
babel-plugin-transform-require-context
package and adding the plugin in the .babelrc resolved the issue for me.
Refer to the documentation here:
https://www.npmjs.com/package/babel-plugin-transform-require-context
The easiest and fastest way to solve this problem will be to install require-context.macro
npm install --save-dev require-context.macro
then just replace:
var modulesReq = require.context('.', false, /\.js$/);
with:
var modulesReq = requireContext('.', false, /\.js$/);
Thats it, you should be good to go!
Cheers and good luck!
Implementation problems not mentioned:
Jest prevents out-of-scope variables in mock, like __dirname.
Create React App limits Babel and Jest customization. You need to use src/setupTests.js which is run before every test.
fs is not supported in the browser. You will need something like browserFS. Now your app has file system support, just for dev.
Potential race condition. Export after this import. One of your require.context imports includes that export. I'm sure require takes care of this, but now we are adding a lot of fs work on top of it.
Type checking.
Either #4 or #5 created undefined errors. Type out the imports, no more errors. No more concerns about what can or can't be imported and where.
Motivation for all this? Extensibility. Keeping future modifications limited to one new file. Publishing separate modules is a better approach.
If there's an easier way to import, node would do it. Also this smacks of premature optimization. You end up scrapping everything anyways because you're now using an industry leading platform or utility.
If you're using Jest with test-utils in Vue.
Install these packages:
#vue/cli-plugin-babel
and
babel-plugin-transform-require-context
Then define babel.config.js at the root of the project with this configuration:
module.exports = function(api) {
api.cache(true);
const presets = [
'#vue/cli-plugin-babel/preset'
];
const plugins = [];
if (process.env.NODE_ENV === 'test') {
plugins.push('transform-require-context');
}
return {
presets,
plugins
};
};
This will check if the current process is initiated by Jest and if so, it mocks all the require.context calls.
I faced the same issue with an ejected create-react-app project
and no one from the answers above helped me...
My solution were to copy to config/babelTransform.js the follwoing:
module.exports = babelJest.createTransformer({
presets: [
[
require.resolve('babel-preset-react-app'),
{
runtime: hasJsxRuntime ? 'automatic' : 'classic',
},
],
],
plugins:["transform-require-context"],
babelrc: false,
configFile: false,
});
Simpleset Solution for this
Just Do
var modulesReq = require.context && require.context('.', false, /\.js$/);
if(modulesReq) {
modulesReq.keys().forEach(function(module) {
modulesReq(module);
});
}
So Here I have added extra check if require.context is defined then only execute By Doing this jest will no longer complain