npm install on modules not in node_modules/ - javascript

I'm building a project and I want to split it up into different modules. I know I should put modules into node_modules/ but I want to commit the modules to the main version control for the time being (I'm using .gitignore to ignore the node_modules/ directory at the moment).
My current project setup:
services/
services/service1/package.json
services/service1/index.js
services/service2/package.json
services/service2/index.js
node_modules/*
app.js
package.json
My problem is it works with relative requires but the dependences in the services/*/package.json aren't loaded. Only the main package.json are loaded.
Is this a good approach? Is there a better one? What command can I run to install all the dependences? This is more of a problem for my CI/CD setup.
Thanks.

i found similar feature request here https://github.com/npm/npm/issues/4017 and seems you can't do it
one approach i see is to create soft links from node_modules to services you want and maintain them somehow
maybe you can use https://www.npmjs.org/doc/misc/npm-scripts.html postinstall script for this?

Related

Running node app without node_modules folder

My professor wants us to submit our project for our node.js app but we are not allowed to include node_modules in the submission.
When I try to run the app without the folder in there, it crashes due to missing dependencies.
Am i missing a step?
Presumably either
you're not supposed to use third-party dependencies in your solution (just the core Node.js modules), or
you're not supposed to include node_modules in your submission (which is good practice anyway), but instead just have a package.json (and possibly a lockfile) so your instructor can npm i / yarn to install the packages.
When building an app that has node_modules you aren't supposed to include this anyway.
Install any node_modules that are required for your application to run and when sibmitting just leave out the node_modules folder.
Your package file will include everything that your instructor can check over or install on their end if necessary.
I found this random example on github: https://github.com/hayanisaid/nextjs-intro-example
As you can see by looking they have there package files but no node_modules, when you clone this repo you would run npm install which would create and install the packages
package.json contains information about node modeuless.
If u deleted node modules, you just have to run npm i --save.
All the dependency will be downloaded through this.
Just use npm/yarn install if you remove the folder.

Can i move my node_modules folder to another project?

I had some issues with a current iOS React Native project that i spent a long time working on. I've decided to start afresh. However i did spend a lot of time ensuring i got the correct versions of various react native modules and configuring these to work. These sit in my old 'node_modules' folder.
Can i copy these modules/folders (from the old 'node_modules') over to my new project? Do i need to update 'package.json' or link using 'react-native link xxxxx'? It it as simple as that in theory?
You don't need to copy over your node_modules directory. You can if you'd like, but it is not considered best practice. In any case, you shouldn't make any modifications to the files inside node_modules.
Preferably you need to only copy over your package.json file and optionally, your package-lock.json (or yarn.lock if you're using Yarn) file so that your project will be easily installable and upgradeable on other computers.
When you have a package.json or package-lock.json file, you can run npm install (or yarn install) to install the packages to your node_modules directory.
Copying your package-lock.json file over as well would ensure that the exact same versions of all of the packages that you have installed in your previous project will be installed in the new project as well. See this for more information on the package-lock.json file.
Unfortunately, I don't know much about react-native and linking react-native dependencies, but from this answer it seems that you would have to link any dependencies that contain native code again after you've run npm install.
No, I wouldn't recommend you to copy specific module from node_modules folder as once installed it has entries in .bin folder and files which you will miss while copying and it will be of no help in new project as they will be downloaded and installed again due to missing indexes.
Solutions :
You can use same package.json and package-lock in another project if you are sure that the dependencies version in it are exactly compatible or one you want, and install those dependencies in new project. The package-lock.json will ensure the version you choose.
Yes you can copy whole node_modules (have done it multiple times) from one project to another and use same package.json and package-lock (Will only save time in dependencies installation/download)
As you are starting fresh what i will recommend is create a new package.json and copy only those dependencies which are important for the project initialization, and as you progress add specific dependencies you need while developing. This will save you from huge garbage of unused dependencies which are hard to keep track of once the project inflates.
Maybe you can move it but you can't copy it, at least not with a naive cp -r! The node_modules/.bin directory ought to be full of symlinks to scripts in their respective directories in node_moudles, but after a cp -r these will be copies, which will now resolve relative module paths incorrectly.
For example, npm install typescript installs a executable script with the contents
#!/usr/bin/env node
require('../lib/tsc.js')
in two locations. Before a copy, that looks like this:
$ ls -l node_modules/.bin/tsc node_modules/typescript/bin/tsc
-rwxr-xr-x 45 node_modules/.bin/tsc
-rwxr-xr-x 45 node_modules/typescript/bin/tsc
and after a cp -r copy, like this:
$ ls -l node_modules/.bin/tsc node_modules/typescript/bin/tsc
lrwxr-xr-x 21 node_modules/.bin/tsc -> ../typescript/bin/tsc
-rwxr-xr-x 45 node_modules/typescript/bin/tsc
which makes the ../typescript/bin/tsc no longer exist, resulting in
$ tsc -b
...
Error: Cannot find module '../lib/tsc.js'
...

npm publish ignores files inside node_modules

I've created npm package. In this package I'm using some modules which I put to node_modules to be able to require them as "modules", for example I have modules node_modules/my-module.js which I require in my code as require('my-module'). Now I do "npm publish" and then in another project I do "npm i" to install my module. It is installed but there are no my modules which I put to node_modules. I tried to add the next lines to .gitignore and to .npmignore, but it did not help:
node_modules/*
!node_modules/my-module.js
what do I do wrong?
I believe that all files in node_modules/ are ignored by NPM, so setting rules for node_modules in .gitignore and .npmignore will have no effect.
Additionally, everything in node_modules is ignored, except for bundled dependencies. npm automatically handles this for you, so don't bother adding node_modules to .npmignore.
Source
It's not clear to me why you want to have a package in the node_modules directory and not in the package.json file, but the short answer is that it's not possible, because that directory will be always ignored, as gauge said.
So there's two ways to go, you can put those modules in another directory and require it from his relative path or you publish those in NPM and then you require them from the package.json file.
Have you tried using something like:
!node_modules/
node_modules/*
!node_modules/my-module.js
I have that in a gitignore of mine and it has allowed publishing.

Why is my es6 code not compiling after npm install?

I have created a npm module with es6. You can see it on github here.
There are some scripts in the package.json file which should ensure that the es6 modules get built. It works when running npm publish and npm install when in the context (directory) of the npm module. Running npm install in another project (ie including the module as a dependency) does not however build the es6 modules. There don't seem to be any errors in the npm-debug.log file either.
I'd really like to understand why.
It was caused by the lack of a .npmignore file. As there wasn't one, npm was using the .gitignore file instead. This filters out the dist/ folder which I don't want in source control. This commit fixed the issue
Your script is missing an install (or postinstall) script.
If you are building the code with a script on the prepublish hook, then you don't need it to be built again when others install it. It should ship with the built ES6 code.
In fact, you probably want to add your src directory to your .npmignore file, so that it doesn't ship with any JS that can't be used directly.

NPM Pack / Nodejitsu ignores buried Node_modules folders

It seems that NPM pack, and by extension, nodejitsu, do not like my node_modules folders. :*(
I currently am building a web app.
My web app's project folder structure is as follows:
Root
Engine(folder) Server(folder) readme.md package.json
(Multiple Folders) (folder)(folder) node_modules(folder)
easyimage,mongodb,mysql(folders) socket.io (folder)
node_modules(folder, NPM Pack ignores this) node_modules(folder, NPM Pack ignores this)
Socket.io-client (folder, NPM Pack ignores this)
I hope everyone can see this structure alright!
The problem I am having is that when I run NPM Pack at the root directory, the entire directory structure is packed correctly, except for all of the node_modules folders below the first node_modules folder.
It is as if NPM pack completely ignores those node_modules folders. (The one below socket.io for instance).
Due to the fact NPM pack ignores these npm folders, jitsu is also ignoring them and I can't get my web app started.
How can I get NPM pack/nodejitsu to correctly package all of the node_modules folders correctly?
My current package.json file, at the root directory, looks like follows:
http://pastebin.com/SAU6rwb5
As you can see, I have attempted to use bundleDependencies to tell NPM Pack that I am trying to include some node_modules folders (modules?), but pack still ignores all of them... Also, if i include anything under "dependencies", NPM start creates a new (??) node_modules folder at the root directory... but at the root directory nothing needs node_modules... as you can see node_modules are used inside of the server folder.
How can I get NPM Pack to recognize the files and folders inside of all of the node_modules folders and pack them correctly?
(jump to last paragraph if you want a really easy solution)
I'm having a hard time understanding your app structure. I think I get what your trying to do though.
From https://npmjs.org/doc/folders.html , it actually goes into detail about when and why sub modules will or wont show up.
When installing locally, npm first tries to find an appropriate prefix folder. This is so that npm install foo#1.2.3 will install to the sensible root of your package, even if you happen to have cded into some other folder.
Starting at the $PWD, npm will walk up the folder tree checking for a folder that contains either a package.json file, or a node_modules folder. If such a thing is found, then that is treated as the effective "current directory" for the purpose of running npm commands. (This behavior is inspired by and similar to git's .git-folder seeking logic when running git commands in a working dir.)
If no package root is found, then the current folder is used.
When you run npm install foo#1.2.3, then the package is loaded into the cache, and then unpacked into ./node_modules/foo. Then, any of foo's dependencies are similarly unpacked into ./node_modules/foo/node_modules/....
Bear With me for another quote...
Cycles are handled using the property of node's module system that it walks up the directories looking for node_modules folders. So, at every stage, if a package is already installed in an ancestor node_modules folder, then it is not installed at the current location.
Consider the case above, where foo -> bar -> baz. Imagine if, in addition to that, baz depended on bar, so you'd have: foo -> bar -> baz -> bar -> baz .... However, since the folder structure is: foo/node_modules/bar/node_modules/baz, there's no need to put another copy of bar into .../baz/node_modules, since when it calls require("bar"), it will get the copy that is installed in foo/node_modules/bar.
This shortcut is only used if the exact same version would be installed in multiple nested node_modules folders. It is still possible to have a/node_modules/b/node_modules/a if the two "a" packages are different versions. However, without repeating the exact same package multiple times, an infinite regress will always be prevented.
As far as the bundledDependencies:
Upon publishing [this also applies to packing], npm will look in the node_modules folder. If any of the items there are not in the bundledDependencies array, then they will not be included in the package tarball.
I take this to mean that it will only bundle the listed modules in ./node_modules/ and specific sub-modules for a ./package.json . Then of course as written above, it recursively walks down the directory tree again... so if it sees another package.json file in this directory npm will look and see if that has any bundled deps to include in the pack.
So as I understand it right now, since you have no packages in your base directory, your bundled dependencies in package.json doesn't do anything, and actually having items in your bundledDependencies field does more harm than good.
To fix you need to edit the package.json files to include these bundles at each level.
I have had this issue before when trying to get a packed then unpacked meteor app working on nodejitsu. I solved it in a different way. In the root folder of my app I included all of the top level node modules and explicitly set their version in my package.json file.
From what I understand your file structure is such:
app
+-- Engine
+-- Server
| +-- socket.io
| | `-- package.json
| | +-- node_modules
`-- readme.md
`-- package.json
If this is so then you need to be editing the package.json under socket.io to include the bundled deps you want. Generally though you can trust the package maintainers to keep valid versions.(but in this case you can't?)
As for Socket.io-client not being packed that's a result of it being a dependency of socket.io.
If I was to suggest a way for you to make this easier for yourself, i'd suggest you to, in your MAIN top level package.json file, to include the dependencies you need for your app # the specific version you need. If you need them bundled for some reason, add them into the bundled section, if you need sub modules at a different version than what the author intended. Consider making a folder called package or vendor and then placing the modules in there, in those you can edit the package.json and bundles their dependencies to your hearts content. Be sure though that you do not ignore any files or folders under you vendor or packages directory with .npmignore or .gitignore files.
Alternatively, if this is too difficult (editing all theses files and specifying certain versions can be a pain) I'd suggest hosting your vendor packages some where where you could download them with a script, and then execute this script in the postinstall part of your package.json (take a read at https://npmjs.org/doc/scripts.html ... you would add this in the same section you have your "start" script.
I hope that clarifies things.

Categories