I'm struggling to get my head around how npm manages dependencies - in terms of how they are actually referenced in HTML.
Say I have a specific version of a plugin installed, which includes a version number in its path or file name - if npm is configured to update to a new minor release - the files referenced via script tags will no longer be present.
I've also read that exposing node_modules path is incorrect and should be avoided.
How then should these files be referenced so that they are loaded and so version updates do not break a site?
The idea is that you use these modules in your code. Let's say you have a main.js file which has your application, then you import modules using import $ from 'jquery'; (this could depend on your configuration, you could also use 'require'). Then use a tool like browserify which is going resolve all your dependencies for you and package it into a nice file which can then be loaded into your browser.
This is only one setup out of many so this could vary, for example if you use webpack this will be different but the idea is the same, you import what you need into your main.js.
npm uses package.json file as reference to build dependency map. And installs all dependencies in node_modules folder. When you publish an update to your module, you also publish a new version of package.json file which will include modifications to dependencies.
So short answer is - package.json file... I hope you can figure things out from this.
Related
I don't get all the connections. Where do they resides and what is the impact of their usage. Are any of following statments correct?
the loader will reside in browser(after loaded)? So when the main app javascript needs some module it will load it behind the scenes
if I use bundler(command line tool) it will create one file from all of my modules(and all common ones used), so no async loading, as there is only the big bundled.js?
why do I need for example bower.json and package.json in the same project(is npm used just to download build utilities)
from where do they take the modules (npm repository or all have its own?)
package managers are just to download the modules to your project for further processing
npm modules holds only .js files
you always needs way to orchastrate the build(gulp, grunt, npm scripts)
So far I have not found resource with information like : packagers do that, loaders do that.
the loader will reside in browser(after loaded)? So when the main app javascript needs some module it will load it behind the scenes
Loaders is a webpack feature, these loaders are used to process code that webpack does not understand by default (css, sass, images, etc.). They are just used when your code is compiled, when you publish your code it does not contains loaders with it. Loaders just run during build time.
--> Edit: this does not apply to the question since the question refeers to require.js for example, but still relevant overall.
if I use bundler(command line tool) it will create one file from all of my modules(and all common ones used), so no async loading, as there is only the big bundled.js?
You can have prefetch, prelight, you can load chunks (pieces of your bundle) only when required (look for code splitting using dynamic import import(...). This makes the code be loadede/requested only when it is going to be loaded.
why do I need for example bower.json and package.json in the same project(is npm used just to download build utilities)
Nowadays you don't need bower, most (if not all) the things that bower has you can find on the npm repositories.
from where do they take the modules (npm repository or all have its own?)
Every package you download was uploaded to npm, you can look at npm as a "github" but for libraries.
packagers are just to download the modules to your project for further processing
Yes
npm modules holds only .js files
False, look at bootstrap.
you always needs way to orchastrate the build(gulp, grunt, npm scripts)
Yes, but if you want to transpile content, you can use only babel-cli.
We have several websites, each in its own project, and we are looking to migrate them all to using Vue.js. Each website has its own directory with .vue files that are then bundled using Webpack. We have a Webpack config in place that converts the .vue files, bundles, lints and pipes it all through babel and it works fine.
However, now that we have been moving things over for several weeks we have noticed that there are several components and core javascript files that are very similar and ideally we want to pull these out into a shared library of vue components and functions.
We have extracted several .vue into a folder alongside the websites, and put them together as a basic npm module with its own package.json, and include them using an npm file include, so in the package.json it just looks like: "vue-shared": "file:../CommonJavascript/Vue". Now when we try to use Webpack to build the bundle, we get the error:
ERROR in ../CommonJavascript/Vue/index.js
Module build failed (from ./node_modules/eslint-loader/index.js):
Error: Failed to load plugin react: Cannot find module 'eslint-plugin-react'
I'm not sure where this error is coming from, because we aren't using react anywhere, and it seemed happy enough to build fine before we moved the files out. At the moment the only dependency in the shared module is moment, and it only contains 4 .vue, and a basic wrapper to bundle them up:
import button from 'Button.vue'
import loading from 'Loading.vue'
import modal from 'Modal.vue'
import progressBar from 'ProgressBar.vue'
export default {
button,
loading,
modal,
progressBar,
}
But, I was curious so I decided to add the package (even though we don't need it) to see if it would fix the issue, but I then get a new error:
ERROR in ../CommonJavascript/Vue/index.js
Module build failed (from ./node_modules/babel-loader/lib/index.js):
ReferenceError: Unknown plugin "transform-runtime" specified in "base" at 0, attempted to resolve relative to "C:\Projects\Tmo\Code\CommonJavascript\Vue"
Now, that one makes a little more sense, we do use the babel runtime transform on the main project, but it isn't required by anything in the shared project and even if it was, surely the fact it is included in the main project means it should still build.
Partly, it seems perhaps I'm just not understanding the way npm resolves dependencies. It seems to be trying to now resolve some dependencies by looking in the shared files project and I dont know why. Also I have no idea where this strange dependency on eslint-plugin-react has come from.
So I guess this is a multi-part question. What is up with the way npm is trying to resolve the dependencies? Am I doing things right by moving the .vue files into a separate project, wrapping it up as a module and requiring it in the main project? and if not, what is the best way to have shared dependencies like this?
This was caused by a mixture of two separate issues:
The import statements didn't reference the file properly, the correct syntax is: import button from './Button.vue' (note the change to file path)
When you add a local package to npm via a path, it creates a symlink to the folder rather than copying the files over (this has been the behaviour since npm v5+). This then changes the way webpack tries to resolve dependencies since it then looks up from the location of the shared files to try and resolve dependencies including thing like eslint and babel.
The eslint-plugin-react dependency was because in visual studio code I had installed the eslint plugin, which it seems had created a .eslintrc file which reference the react plugin in my user folder (c:\users\<username>). Eslint will then use this as the default if it can't find a config file (which it couldn't because it was looking above the shared files because of the pathing issues described above)
We have decided we will be using a git submodule for these files going forward
we are in the process of uplifting parts of a huge code base. We are bringing in a module which has been built with webpack. In order to avoid code duplication we have used webpacks externals option.
When we start to integrate our module into the main codebase which is currently using browserify, we have an issue where a shared dependency is included twice and causes issues.
Is there are a way to have webpack use the packaged version of the dependency? So in the final browserified bundle we just have the dependency included once?
It seems to me like this may not be possible, if so I will push for moving the rest of our codebase onto webpack (it is underway already).
The only solution I have come up with so far is to have the webpack module also export the shared dependency, and then have the main app use that export, but this is not ideal.
I have ensured that the dependency in both node_modules folders are at the same version and I still get 2 instances.
I need to be able to tell Browserify to only resolve my apps node_modules, or tell it to resolve from top to bottom, i.e. look in the top level node_modules first, is this possible?
I have tried setting the NODE_PATH option when using the cli to no affect.
** update **
So the issue is that when Browserify hits the require() statement in the webpack bundle it resolves from the local node_modules folder, so we end up with 2 instances of the dependency. I can fix this by using a relative path in either the apps require() or webpacks external option and ensuring they use the same file.
So it seems the issue was caused by the fact that I had symlinked (with npm link), the module as I was working on this also. It seems that when Browserify resolves the require in the symlinked module it resolves to its own node_modules, and we end up with 2 copies.
When I install the module normally it all works fine, so this is OK as other consumers of the module should have no issues. It is annoying that it behaves this way, but I just need to ensure that I point at the same dependency in my main app when developing alongside the module.
So my require statement in the main app (while symlinking) looks something like this:
require('./node_modules/my-module/node_modules/shared-dependency/index.js');
I can require as normal when not symlinking.
I recently published a private module to NPM which has some common code we use across a few of our services.
The code is written in ES6 and so we need to transpile it using babel before publishing to NPM. I've got a prepublish script which transpiles src in to lib.
There's no index.js file in this module since it's just some common code.
The problem I'm having is that when I install the module from NPM, using require('#ourorg/ourmodule/somecode') doesn't work (module can't be found). I instead have to use require('#ourorg/ourmodule/lib/somecode').
I've tried changing the main field in package.json to many variations of lib, but it doesn't seem to work unless I include an index.js file, in which case require('#ourorg/ourmodule') returns whatever is exported there. One workaround I can see would be to export all the common code in an index.js file, but this isn't maintainable at all.
The main field in package.json follows the same rules as normal Node imports - either it should point at a single file in particular, or it can point at a directory that has an index.js in it.
As far as I know, there is no way to make importing your package simply be an alias for a directory. If there was, what would require("#ourorg/ourmodule") return?
If it's absolutely driving you crazy having to type lib every time you import something, maybe you could add a step to your build process that autogenerates an index.js that re-exports everything at the root?
I am new to JavaScript development and could do with some advice regarding how best to work with multiple modules of my own creation.
Module1
--src
----a.js
----b.js
Module2
--src
----c.js
----d.js
Module3
--src
----e.js
----f.js
Module4
--src
----g.js
----h.js
Now each module is a collection of js files that use module.exports to export functions that they need to.
In this example Module1 and Module2 are library module. Module2 depends on Module1. Both of these modules are platform agnostic and can be ran inside a browser or nodejs.
Module3 is a project module which depends on both Module2 and Module1. It is designed for a browser and uses browserify to combine them all into a bundle.
Module4 is another project module which also depends on Module2 and Module1. This is designed to run under nodejs.
Each module has its own git repo.
Now the problem I am having is to do with relative paths inside the require. For example c.js currently does require("../../Module1/src/a.js"); Similarly h.js does require("../../Module2/src/c.js");
This causes foldering structure to be an absolute pain and each project has to be cloned from git in the correct setup.
The following is what I am trying to achieve.
To do away with relative paths so I simply do require ("ModuleX/src/foo.js"); This needs to work with both browserify and nodejs.
To clone a project module from git and have it get all of its dependent modules (perhaps git submodules?). Not caring about folder structure as this should be solved using the point mentioned above.
Once a project and it's dependent modules have been cloned to be able to edit each of these modules, make changes and push them to their individual git repo.
I imagine what I am trying to do is pretty standard. Creating my own modules and reusing them in different projects. Am I trying to go about solving it in the standard way? I've read that there are different ways to achieve this using NODE_PATH which is supported by both node and browserify but both discourage it. Browserify supports a paths option but doesn't work for node. Also read about putting the modules inside node_modules but not sure how that would help with relative paths.
Any advice would be greatly appreciated
Thanks
What you probably want to do is commit your reusable code to git (here GitHub) and then npm install <git remote url>:. An example would be npm install git+https://isaacs#github.com/npm/npm.git. On the other hand your repo needs a package.json and a index.js holding your code or pointing to it. Alternatively you could define the location of your main file in the package.json via main:"" key.
The dependencies between those projects would be defined in the respective package.jsons. Upon install npm does it's magic to avoid circular dependencies, caching etc.
Once you've done all that you would be able to just var x = require("ModuleX") to get ModuleX/src/foo.js if you so wish to.
Alternatively use npms private modules.