we are in the process of uplifting parts of a huge code base. We are bringing in a module which has been built with webpack. In order to avoid code duplication we have used webpacks externals option.
When we start to integrate our module into the main codebase which is currently using browserify, we have an issue where a shared dependency is included twice and causes issues.
Is there are a way to have webpack use the packaged version of the dependency? So in the final browserified bundle we just have the dependency included once?
It seems to me like this may not be possible, if so I will push for moving the rest of our codebase onto webpack (it is underway already).
The only solution I have come up with so far is to have the webpack module also export the shared dependency, and then have the main app use that export, but this is not ideal.
I have ensured that the dependency in both node_modules folders are at the same version and I still get 2 instances.
I need to be able to tell Browserify to only resolve my apps node_modules, or tell it to resolve from top to bottom, i.e. look in the top level node_modules first, is this possible?
I have tried setting the NODE_PATH option when using the cli to no affect.
** update **
So the issue is that when Browserify hits the require() statement in the webpack bundle it resolves from the local node_modules folder, so we end up with 2 instances of the dependency. I can fix this by using a relative path in either the apps require() or webpacks external option and ensuring they use the same file.
So it seems the issue was caused by the fact that I had symlinked (with npm link), the module as I was working on this also. It seems that when Browserify resolves the require in the symlinked module it resolves to its own node_modules, and we end up with 2 copies.
When I install the module normally it all works fine, so this is OK as other consumers of the module should have no issues. It is annoying that it behaves this way, but I just need to ensure that I point at the same dependency in my main app when developing alongside the module.
So my require statement in the main app (while symlinking) looks something like this:
require('./node_modules/my-module/node_modules/shared-dependency/index.js');
I can require as normal when not symlinking.
Related
I work in a large monorepo that has very many files that are processed by Webpack. One of our main patterns is that we use fully-qualified filenames for every module name and import. For example, if I have a file with path "/path/to/module/MyCoolModule.js", the import would simply look like import MyCoolModule from 'MyCoolModule' without needing any explicit path to the module. This is enabled by recursively scanning the project's directories for all relevant .js files/folders at Webpack init and then passing in the scanned files into resolve.modules and resolve.alias.
We like this pattern, and it works well for general usage when webpack is run in watch mode; however, it breaks down when new folders and files are added to the project while Webpack is already running. Because the files and folders are only scanned at Webpack init, Webpack is unable to resolve imports for new files/folders added since it started, so compilation will fail. This requires us to restart Webpack, which takes 1-2 minutes on our project. Across many engineers when switching between git branches or updating changes from master, this can add up to a significant slowdown in raw time and in breaking flow.
I'd like to write a plugin that will dynamically update the resolve.modules and resolve.alias such that we do not have to restart Webpack when new folders and files are added to the project. However, I am at a loss for how to do this. It seems like I have three main questions:
What hook(s) can I tap into to be notified when Webpack detects new files or folders?
Where would I put this plugin to access the above hooks? Would it exist in the top-level plugins config? resolve.plugins? etc.. As I'm understanding it, where the plugin is placed/configured will affect what hooks are available to it
How would I make Webpack aware of these dynamic updates? I am guessing there's a reference to the resolve obj somewhere that holds resolve.modules and resolve.aliases that Webpack actively references, but I'm not sure where that would live or where to access it (this could be completely wrong too)
(note: we are technically using Webpack 4, but answers for Webpack 5 are welcomed because I know we're behind there, and I'd guess a Webpack 5 solution would still point me in the right direction)
Thanks!
We have several websites, each in its own project, and we are looking to migrate them all to using Vue.js. Each website has its own directory with .vue files that are then bundled using Webpack. We have a Webpack config in place that converts the .vue files, bundles, lints and pipes it all through babel and it works fine.
However, now that we have been moving things over for several weeks we have noticed that there are several components and core javascript files that are very similar and ideally we want to pull these out into a shared library of vue components and functions.
We have extracted several .vue into a folder alongside the websites, and put them together as a basic npm module with its own package.json, and include them using an npm file include, so in the package.json it just looks like: "vue-shared": "file:../CommonJavascript/Vue". Now when we try to use Webpack to build the bundle, we get the error:
ERROR in ../CommonJavascript/Vue/index.js
Module build failed (from ./node_modules/eslint-loader/index.js):
Error: Failed to load plugin react: Cannot find module 'eslint-plugin-react'
I'm not sure where this error is coming from, because we aren't using react anywhere, and it seemed happy enough to build fine before we moved the files out. At the moment the only dependency in the shared module is moment, and it only contains 4 .vue, and a basic wrapper to bundle them up:
import button from 'Button.vue'
import loading from 'Loading.vue'
import modal from 'Modal.vue'
import progressBar from 'ProgressBar.vue'
export default {
button,
loading,
modal,
progressBar,
}
But, I was curious so I decided to add the package (even though we don't need it) to see if it would fix the issue, but I then get a new error:
ERROR in ../CommonJavascript/Vue/index.js
Module build failed (from ./node_modules/babel-loader/lib/index.js):
ReferenceError: Unknown plugin "transform-runtime" specified in "base" at 0, attempted to resolve relative to "C:\Projects\Tmo\Code\CommonJavascript\Vue"
Now, that one makes a little more sense, we do use the babel runtime transform on the main project, but it isn't required by anything in the shared project and even if it was, surely the fact it is included in the main project means it should still build.
Partly, it seems perhaps I'm just not understanding the way npm resolves dependencies. It seems to be trying to now resolve some dependencies by looking in the shared files project and I dont know why. Also I have no idea where this strange dependency on eslint-plugin-react has come from.
So I guess this is a multi-part question. What is up with the way npm is trying to resolve the dependencies? Am I doing things right by moving the .vue files into a separate project, wrapping it up as a module and requiring it in the main project? and if not, what is the best way to have shared dependencies like this?
This was caused by a mixture of two separate issues:
The import statements didn't reference the file properly, the correct syntax is: import button from './Button.vue' (note the change to file path)
When you add a local package to npm via a path, it creates a symlink to the folder rather than copying the files over (this has been the behaviour since npm v5+). This then changes the way webpack tries to resolve dependencies since it then looks up from the location of the shared files to try and resolve dependencies including thing like eslint and babel.
The eslint-plugin-react dependency was because in visual studio code I had installed the eslint plugin, which it seems had created a .eslintrc file which reference the react plugin in my user folder (c:\users\<username>). Eslint will then use this as the default if it can't find a config file (which it couldn't because it was looking above the shared files because of the pathing issues described above)
We have decided we will be using a git submodule for these files going forward
I have a common module in an angular 2 application. This is how the project is structured:
main
--app
/common
--config
//.ts configs for modules
--services
//angular services
--models
--store
//ngrx store
/component1
/component2
app.module.ts
main.ts
--js
//systemjs dependencies
Gruntfile.js
package.json
tconfig.json
The project is configured to transpile to bundle.js and works as expected. However, now I want to bundle out the contents of common folder and package it in isolation. Such that, I can just load its dependency using system js, and then include it is app.module.ts to use in components.
I need help in getting started, is it some gruntfile/tsconfig changes that I need to make? Or would I have to create a separate package.json/index.ts to transpile .ts dependencies?
The feature I believe you are looking for is the Bundle Arithmetic feature of the systemjs-builder. It is built into SystemJS and works very well.
However, it will mean using SystemJS to bundle your app instead of a grunt.
This has many advantages and actually simplifies many scenarios, like your specific use case, but it may take you a little while to get everything set up.
TypeScript does not support splitting bundles at this time.
Splitting your app into multiple npm packages is not necessary and will introduce significant complexity into development process if your application is small.
I'm struggling to get my head around how npm manages dependencies - in terms of how they are actually referenced in HTML.
Say I have a specific version of a plugin installed, which includes a version number in its path or file name - if npm is configured to update to a new minor release - the files referenced via script tags will no longer be present.
I've also read that exposing node_modules path is incorrect and should be avoided.
How then should these files be referenced so that they are loaded and so version updates do not break a site?
The idea is that you use these modules in your code. Let's say you have a main.js file which has your application, then you import modules using import $ from 'jquery'; (this could depend on your configuration, you could also use 'require'). Then use a tool like browserify which is going resolve all your dependencies for you and package it into a nice file which can then be loaded into your browser.
This is only one setup out of many so this could vary, for example if you use webpack this will be different but the idea is the same, you import what you need into your main.js.
npm uses package.json file as reference to build dependency map. And installs all dependencies in node_modules folder. When you publish an update to your module, you also publish a new version of package.json file which will include modifications to dependencies.
So short answer is - package.json file... I hope you can figure things out from this.
I recently published a private module to NPM which has some common code we use across a few of our services.
The code is written in ES6 and so we need to transpile it using babel before publishing to NPM. I've got a prepublish script which transpiles src in to lib.
There's no index.js file in this module since it's just some common code.
The problem I'm having is that when I install the module from NPM, using require('#ourorg/ourmodule/somecode') doesn't work (module can't be found). I instead have to use require('#ourorg/ourmodule/lib/somecode').
I've tried changing the main field in package.json to many variations of lib, but it doesn't seem to work unless I include an index.js file, in which case require('#ourorg/ourmodule') returns whatever is exported there. One workaround I can see would be to export all the common code in an index.js file, but this isn't maintainable at all.
The main field in package.json follows the same rules as normal Node imports - either it should point at a single file in particular, or it can point at a directory that has an index.js in it.
As far as I know, there is no way to make importing your package simply be an alias for a directory. If there was, what would require("#ourorg/ourmodule") return?
If it's absolutely driving you crazy having to type lib every time you import something, maybe you could add a step to your build process that autogenerates an index.js that re-exports everything at the root?