I have a package (package-a) that depends on another package (package-b) which is not published to npm but is on my file system. When I run npm install from package-a, package-b's dependencies are not installed. I have to navigate to package-b's directory and run npm install manually. Is there a way to install both packages' dependencies with a single npm command?
Here's my directory structure:
/
...
shared/
...
javascript/
...
package-b/
package.json
package-a/
package.json
Per the docs, I placed the following in package-a/package.json. (I'm using npm 5+)
dependencies: {
package-b: "file:../shared/javascript/package-b",
}
When I navigate to /package-a and run npm install, it installs all of package-a's dependencies like normal and also copies the package-b directory to package-a/node_modules. This is presumably what lets me type require('package-b') instead of require('../shared/javascript/package-b') .
However, as I stated before, package-bs dependencies are not installed, so if I try to use package-a, I get an error when package-b is required because it is trying to use dependencies that do not exist locally.
Once again, to solve this, I can navigate to package-b and run npm-install, but I'm hoping there's a better solution as I may have many such sub packages and I'd like to avoid having to write a shell script to go install all my dependencies if I can do this with an npm command. (perhaps I just did something wrong and npm install should be working?)
Follow up question: when I run npm install from package-b's directory, the packages are installed there, but not in the version of package-b that got copied to /package-a/node_modules during the first npm install, yet everything still works. So now it seems like when I require('package-b') it is actually resolving to /shared/javascript/package-b and NOT /package-a/node_modules/package-b. So what's the point of copying the file in the first place?
Update
It turns out this is a bug in npm 5. It only occurrs when installing from a package-lock.json file. (Github Issue)
The files are (probably) not being copied, they're being symbolically linked (symlink). This essentially creates an alias/shortcut that looks like a real directory, but points to another path.
This is how the older npm link feature worked. The reason is the code stays "live"; Changes in the linked module are reflected whenever you run the module that's referencing them, meaning you don't have to npm update all the time.
Related
My problem is that when I try to install a package of mine with the version set to a certain git branch my build is breaking because the repository doesn't house the dist folder which my npm package does.
Here is how I want to install the package.
#me/myPkg": "github:me/myPkg#myBranch
So should my repository include the dist folder or what's the best practice in this case? I don't want to publish my dist package onto github because then I would have weird merge conflicts because the dist folder is containg all of my type declaration and minified javascript files, but maybe this the preferred way?
Another thought I had was to have a postinstall script in my package.json which built the source files but then I would have to have the devDependencies installed as well and npm install doesn't install them for dependencies.
How are you publishing your package? For instance, I have a GitHub Actions setup on my repositories that, when commits are made to master, will automatically perform several steps
Run an npm install Setup Git Credential variables (for later
process)
Run npm test (and fail whole process if tests fail)
Run
npm run release (I use standard-version for auto version
incrementing and Changelog creation, and do this prior to build so
the version can be reflected in the built code)
Run npm run build
Run npm run publish to push to npm
Commit my version update as a
tag to Git
My GitHub Action does all of this in a temp container it spins up for the build process. Once it is done, the whole container is tossed. I use an .npmignore file to tell publish what I don't want in the final package (note files, build tools, etc).
Some background:
I started my project with a clone of this repository. Here's an example of the code. After some time, I realized that the vue-cli-service it used appeared to be old. (They even renamed it to '#vue/cli-service'). So, I ran npm update. The code has me use npm run serve to start the service. It's apparently equivalent to npx vue-cli-service serve. When I did that, it couldn't load some bootstrap stuff. I reinstalled bootstrap (npm install bootstrap#version (don't have access to the exact line I used). It then said it couldn't find '#vue/cli-plugin-babel'. I saw it right in the node_modules folder.
I fixed it using this process:
I ran npm update.
It first said that it could not find a font (no longer have the specific details). The error directed to the bootstrap.css file.
I reinstalled bootstrap.
I run npm run serve, and it says "Cannot find module '#vue/cli-plugin-babel'"
I see '#vue/cli-plugin-babel' showing in the node_modules folder
I run npm list --depth=0, and '#vue/cli-plugin-babel' does not show up.
I*’m seeing this
npm cache clean --force alone did not cause the site to resume. It still said “ Cannot find module '#vue/cli-plugin-babel'”.
Following the top SO answer here got my site to work again.
rm -rf node_modules/
npm cache clean (then replaced with npm cache verify on npm's prompt)
npm install (I wonder if it was possible to ignore ‘rm -rf node_modules/’ and just do the last two steps).
I run npm run serve, and the site works again.
After running npm list --depth=0 again, there are no more UNMET DEPENDENCY issues, like with #6.
Also on that SO Q&A, people talked about restructuring (reordering) the package.json file. In the top answer he says that a possible solution is to “Re-structure your package.json. Place all the high-level modules (serves as a dependency for others modules) at the bottom.”. I wonder if that relates to my issue.
My expected vs. actual results:
When I updated it, I actually sort of expected it to break. I got that.
But what I also expected was:
If the node module is there, it will not give me an error.
If I run npm install, all dependencies will be handled automatically.
If npm install doesn't do the trick, then npm cache clean --force should do the trick. (Note: I later learned about npm cache verify).
I got:
The '#vue/cli-plugin-babel' module was there and it gave an error saying it couldn't find it.
(And also 3.) Running npm install and npm install [unmet-dependency] did not work until after 'rm -rf node_modules/' -> `npm verify cache' and THEN -> 'npm install'.
Here are my guesses:
#vue/cli service doesn’t check the folders directly to see if a module exists (75% chance it’s true)
#vue/cli service uses a proxy, something like the packages.json file (but different) to see if a node exists (40%).
#vue/cli service caches commonly used stuff. So, when I ran it again after the update, it didn’t check the new stuff in my folder, it checked its cache. It didn’t line up. (10%).
For everything above, replace #vue/cli service with npm (10%).
Again, my biggest question is "How could the vue-cli-service crash because of the '#vue/cli-plugin-babel' module, if the node module is there?"
And "How did removing the node_modules folder, verifying the cache, and reinstalling with npm cause npx vue-cli-service serve to successfully run my site again (and find #vue/vue-plugin-babel)?".
I am doing an internship in a company.
I need to create a node server.
I installed node on the computer (Windows) and I should install some plugins like:
- nodejs-webpack
- colors
- uglify
Normally I need to enter a command like : npm install "theModule"
But the software can not access the internet (due to company restrictions) and support service can not authorize the software (or do not want).
Can I install modules in any other way ? (download from Google and slide archives in the correct folder for example).
If the answer is no, do you know how can i get around this security?
You need a private npm repository.
Check out this answer:
can you host a private repository for your organization to use with npm?
I found it !
Just for exemple, we will install 'nodejs-websocket' :
1) You just have to download it here.
2) Put files into your Node's directory (for me it's "C:\Program Files\nodejs\node_modules\npm\node_modules")
3) in your .js file just add this line : var ws = require("C:/Program Files/nodejs/node_modules/npm/node_modules/nodejs-websocket/")
Done ! Thanks for all :D
I added this as a comment on your own answer, but I figured I should add a real answer with a better explanation.
Normally when you run npm install package-name npm installs the package to a node_modules directory in the directory you are in at that moment. So if your app was located at C:\code\my-app then you would cd into that directory and run npm install package-name. This would create a node_modules directory at C:\code\my-app\node_modules if it didn't already exist. Then it would install package-name into that directory at C:\code\my-app\node_modules\package-name.
As long as the module is in the node_modules directory for your app, you can require the module in your code without entering a big long file path.
var ws = require('nodejs-websocket');
The place you manually installed your module is the global node_modules directory. It's where npm would install a module if you did npm install -g package-name. The global node_modules directory is added to your system path when you install npm. In other words, any modules you install in there would be accessible from the command line like any other command. For example:
npm install -g bower
That would install the "bower" package to the global npm module directory. Bower would then be accessible as a command line tool for you to use. For example:
bower install angularjs
The global directory is more for tools like that and not really for modules that you intend to use in your code. Technically you can require a module from anywhere by including the full path in the require call like you did, but the standard practice would be to place it in the node_modules directory in the root of your application, and then require it with just its name and not a full path.
Edit: Here's another tip that you might like to take advantage of as well.
When you normally install a module with npm install package-name, your application usually has a package.json file at the root of it. If it does, you can do npm install package-name --save and npm will add the package-name module to a list in your app's package.json file. Once a module is listed in your app's package.json file it's called a "dependency" of your app because it basically says your app depends on package-name.
Normally, when you have dependencies listed in package.json, you can completely delete your app's node_modules directory and then simply run npm install from within your app's root directory and npm will automatically install all dependencies it finds listed in your app's package.json file. Since your corporate firewall won't allow this automatic downloading of modules, you won't get that benefit. However it is still good practice to follow the same conventions.
A trick you can do to create your package.json file is to manually install your dependencies into your app's node_modules directory. Once you have the modules your app needs, you can instruct npm to create a package.json file for your app by simply running npm init. It will walk you through a few little prompts to fill out your package.json file with details about your app. It will then peek inside your node_modules directory to see if you've already installed any modules before having a package.json file. If it finds any, it will automatically add them to the dependencies field in the package.json file it creates :D
With a proper package.json in place you'll always have a nice tidy list of what dependencies your application needs, even if your node_modules directory gets deleted. Normally people add their app's node_modules directory to their .gitignore file, only checking in the package.json file. This way they don't store their dependencies in source control but they can still be easily installed on new machines that clone it by simply running npm install from inside the app's directory. In your case though you may want to just add node_modules to your source control since you can't let npm install dependencies automatically.
When I want to add a package (and check in the dependency into git), where does it belong - into package.json or into bower.json?
From what I gather,
running bower install will fetch the package and put it in /vendor directory,
running npm install it will fetch it and put it into /node_modules directory.
This SO answer says bower is for front-end and npm is for backend stuff.
Ember-app-kit seems to adhere to this distinction from the first glance... But instructions in gruntfile for enabling some functionality give two explicit commands, so I'm totally confused here.
Intuitively I would guess that
npm install --save-dev package-name would be equivalent to adding the package-name to my package.json
bower install --save package-name might be the same as adding the package to my bower.json and running bower install?
If that is the case, when should I ever install packages explicitly like that without adding them to the file that manages dependencies (apart from installing command line tools globally)?
Npm and Bower are both dependency management tools. But the main difference between both is npm is used for installing Node js modules but bower js is used for managing front end components like html, css, js etc.
A fact that makes this more confusing is that npm provides some packages which can be used in front-end development as well, like grunt and jshint.
These lines add more meaning
Bower, unlike npm, can have multiple files (e.g. .js, .css, .html, .png, .ttf) which are considered the main file(s). Bower semantically considers these main files, when packaged together, a component.
Edit: Grunt is quite different from Npm and Bower. Grunt is a javascript task runner tool. You can do a lot of things using grunt which you had to do manually otherwise. Highlighting some of the uses of Grunt:
Zipping some files (e.g. zipup plugin)
Linting on js files (jshint)
Compiling less files (grunt-contrib-less)
There are grunt plugins for sass compilation, uglifying your javascript, copy files/folders, minifying javascript etc.
Please Note that grunt plugin is also an npm package.
Question-1
When I want to add a package (and check in the dependency into git), where does it belong - into package.json or into bower.json
It really depends where does this package belong to. If it is a node module(like grunt,request) then it will go in package.json otherwise into bower json.
Question-2
When should I ever install packages explicitly like that without adding them to the file that manages dependencies
It does not matter whether you are installing packages explicitly or mentioning the dependency in .json file. Suppose you are in the middle of working on a node project and you need another project, say request, then you have two options:
Edit the package.json file and add a dependency on 'request'
npm install
OR
Use commandline: npm install --save request
--save options adds the dependency to package.json file as well. If you don't specify --save option, it will only download the package but the json file will be unaffected.
You can do this either way, there will not be a substantial difference.
Update for mid 2016:
The things are changing so fast that if it's late 2017 this answer might not be up to date anymore!
Beginners can quickly get lost in choice of build tools and workflows, but what's most up to date in 2016 is not using Bower, Grunt or Gulp at all! With help of Webpack you can do everything directly in NPM!
Google "npm as build tool" result:
https://medium.com/#dabit3/introduction-to-using-npm-as-a-build-tool-b41076f488b0#.c33e74tsa
Webpack: https://webpack.github.io/docs/installation.html
Don't get me wrong people use other workflows and I still use GULP in my legacy project(but slowly moving out of it), but this is how it's done in the best companies and developers working in this workflow make a LOT of money!
Look at this template it's a very up-to-date setup consisting of a mixture of the best and the latest technologies:
https://github.com/coryhouse/react-slingshot
Webpack
NPM as a build tool (no Gulp, Grunt or Bower)
React with Redux
ESLint
the list is long. Go and explore!
Your questions:
When I want to add a package (and check in the dependency into git),
where does it belong - into package.json or into bower.json
Everything belongs in package.json now
Dependencies required for build are in "devDependencies" i.e. npm install require-dir --save-dev (--save-dev updates your package.json by adding an entry to devDependencies)
Dependencies required for your application during runtime are in "dependencies" i.e. npm install lodash --save (--save updates your package.json by adding an entry to dependencies)
If that is the case, when should I ever install packages explicitly like that without adding them to the file that manages dependencies (apart from installing command line tools globally)?
Always. Just because of comfort. When you add a flag (--save-dev or --save) the file that manages deps (package.json) gets updated automatically. Don't waste time by editing dependencies in it manually. Shortcut for npm install --save-dev package-name is npm i -D package-name and shortcut for npm install --save package-name is npm i -S package-name
I love Bundler, it's great at dependency management. I love npm, installing node packages is easy! I have a nodejs app and would love to be able to specify my apps dependencies and easily install / update them wherever I deploy my app. This isn't a library I'm releasing, it's a full fledged web-app.
I'm aware of the npm bundle command, but that just seems to simply override the directory where packages are installed.
I'm used to using bundler in this fashion:
# Gemfile
gem "rails", "3.0.3"
Installs rails v3.0.3 and any other required gems on the host machine only if it doesn't already exist
> bundle install
How can I achieve something similar with npm?
As of npm 1.0 (which is now what you get by default if you follow the steps in the README file), "bundle" is no longer a segregated thing -- it's just "how it works".
So:
Put a package.json file in the root of your project
List your deps in that file
{ "name" : "my-project"
, "version" : "1.0.0"
, "dependencies" : { "express" : "1.0.0" } }
npm install Since you're calling this with no args, and not in global mode, it'll just install all your deps locally.
require("express") and be happy.
Edit: This only applies to npm versions < 1.0
It was quite difficult to figure this out, but NPM makes this possible.
You need three components
A subdirectory in your repository (i.e. deps/)
A package.json file in the above directory that lists dependencies
An index.js file in the above directory that requires your dependencies
Example
Imagine that express is your only dependency
deps/package.json
note: Increment the version # each time you modify the dependencies
{
"name": "myapp_dependencies",
"version": "0.0.1",
"engines": {
"node": "0.4.1"
},
"dependencies":{
"express": "2.0.0beta2"
}
}
deps/index.js
export.modules = {
express: require('express')
//add more
}
Now you should be able to install your dependencies using npm. You could even make this part of your deployment process
cd deps
npm install
Then within your app code you can get access to your specific version of express like this:
var express = require('myapp_dependencies').express;
You should read these two articles from Isaacs(author npm) blog. I think they are really good, and I believe tell you how to achieve your goal:
http://blog.izs.me/post/1675072029/10-cool-things-you-probably-didnt-realize-npm-could-do
http://foohack.com/2010/08/intro-to-npm/
I believe link #1(point #11) explains this:
11: Bundle all your dependencies into the package itself
When you use the
npm bundle command, npm will put all
your dependencies into the
node_modules folder in your package.
But it doesn’t stop there.
If you want to depend on something
that’s not on the registry, you can do
that. Just do this:
npm bundle install
http://github.com/whoever/whatever/tarball/master
This will install the contents of that
tarball into the bundle, and then you
can list it as a dependency, and it
won’t try to install it when your
package gets installed.
This also is handy if you have your
own fork of something, and would
prefer not to change the name.
In fact, you can run almost any npm
command at the bundle. To see what’s
inside, you can do npm bundle ls. To
remove something, do npm bundle rm
thing. And, of course, you can install
multiple versions and activate the one
you want.
As of Npm version 1.1.2 , there's a new command npm shrinkwrap which creates an npm-shrinkwrapped.json file, analogous to Gemfile.lock. It's important to make one, to prevent software rot (see Bundler's rationale). Particularly as Nodejs has such a fast moving community.
While bundle install creates a Gemfile.lock automatically, npm install won't create npm-shrinkwrapped.json (but will use it when it exists). Hence you need to remember to use npm shrinkwrap.
Read a full guide at http://blog.nodejs.org/2012/02/27/managing-node-js-dependencies-with-shrinkwrap/
It seems to me that the simplest solution is to use a package.json file with the private flag (added to npm just last month) set to true. That way, you can run npm install or npm bundle to grab your project's dependencies, but you prevent anyone from accidentally publishing your non-public project.
Here's an example package.json:
{
"name": "yourProject"
,"version": "1.0.0"
,"dependencies": { "express" : ">=2.1.0" }
,"private": true
}
Running npm install will install express on the local system if it doesn't already exist; running npm publish gives an error because of the "private": true.
You and your team can use the version tag internally to track dependency changes over time—each time you change a dependency, bump the version. To see which version you've installed, use npm ls installed.
Publish your app with npm as well, and list its dependencies in your package.json file.
When someone uses npm to install your package, npm will take care of resolving its dependencies.
Packages spec: http://wiki.commonjs.org/wiki/Packages/1.0