Right now I got a NodeJS backend for a system I built. The problem is that I need to also maintain another instance of the backend for some client specific requirements, both of the instances share like 70-80% of the code. At the moment I'm using git branches to keep them apart, I know git is not meant to do this, so I would like to know if there is something that allows me to have two separate projects sharing some codebase, similar to flavors in Android.
There are few options to do this:
1. Install your own module as separate dependency with npm via package.json dependency.
create own reusable code as separate project on your git space
specify it as dependency in package.json
use it by installing it with npm install, in same fashion you do with regular npm modules
2. Use docker.
Docker is container virtualisation engine, that allows you to create images of vritual environments with pre-installed infrastructure/file system
You just crete image with some linux os inside, node and your module preinstalled, and all you need is to mount your unique code as "volume" to the container and thats it.
use nodejs offcial image - it have everything basic node.js env would need - to create own image. In the folder where you have /reusable_code folder and package.json create Dockerile file
Dockerfile:
FROM node:6.9.2
RUN mkdir app
COPY ./reusable_code /app/reusable_code
COPY ./package.json /app/package.json
WORKDIR /app
RUN npm install -g <your global modules> && npm install
now run docker build -t base-image-with-reusable-code .
It will create and tag the image as base-image-with-reusable-code
Now once you want to use it with any unique code you should do from the folder where the code is (this assuming all unique code use same package.json dependencies used in previous step - if not this will need extra step)
docker run -ti -v ./app.js:/app/app.js -v ./unique_code:/app/unique_code base-image-with-reusable-code node app.js
Of course names should be corrected, and if you have different project structure then the changes should reflect that.
3. Link the reusable code module folder via OS
Simply put, just ln -s /path/to/reusable/code ./resuable_code from your unique code project root folder, and then use it assuming it residing at root of every unique project you have linked it to.
4. Link the reusable code module folder via npm link
as suggested by #Paul :
node native way to do #3 is via npm link, which both sets up the symlink and makes reference to it in your package.json so that your code can treat it as an external module
Assuming you have reusable code folder in same folder where unique code folders are located:
Modified example of the npm link docs:
cd ~/projects/unique_project1 # go into the dir of your main project
npm link ../reusable_code # link the dir of your dependency
Note : all solutions assume you have separate git project for your reusable code. Generally speaking , it's a good practice.
Related
I have a mono repo and want to link a component library inside a package which contains this library as a dependency. But as we control this library and make changes, I want to be able to test it locally.
cd <library_path>
npm link
cd <monorepo_path>
npm link --workspace <target_package> <library>
When I execute find: find -name '<library>' I can find the library in the <target_package> node_modules directory as well as in the roots node_modules directory, which means the one in the <target_package> will take precedence, but this is the published one, not the linked one. The linked one lives in the root.
When I install my node_modules without linking the package, the library lives only inside the root node_modules.
I tried several things without success.
How can I override the workspace dependency temporarily to use the local library from npm link?
EDIT:
I found a temporary workaround in our webpack configuration with hardcoding which path to use for this dependency.
However, this is not a satisfying solution in my opinion.
I want to do something like this, where, I want to keep all my packages globally just like node package itself. So for example in my package.json I have a package name called "Highcharts" I want to install it globally I don't want to create a local node_modules folder and use it but I want to access it from outside so next time whenever I want to create a copy of my project folder I should be able to use highcharts directly without using npm install. Is it possible?
globally installed node_modules - > Users/user/AppData/Roaming/node_modules/highcharts
app
src
node_modules (I don't want to keep it)
package.json
tsconfig.json
angular.json
How to link these globally installed node_modules with the current app or any app which we want to create?
Any help will be appreciated. Thank you so much :)
local packages are installed in the project directory
global packages are installed in a single place in your system
Usually it is a good idea to have all npm packages required for your project installed locally (project folder). This makes sure, that you can have dozens of applications which are running a different versions of each package if needed.
export NODE_PATH='yourdir'/node_modules
Hello, if am getting right, you want to keep all dependencies global.
You can just run install with -g command. Those libraries will be available in node installation folder.
From the Node docs
If the NODE_PATH environment variable is set to a colon-delimited list of absolute paths, then node will search those paths for modules if they are not found elsewhere. (Note: On Windows, NODE_PATH is delimited by semicolons instead of colons.)
Additionally, node will search in the following locations:
1: $HOME/.node_modules
2: $HOME/.node_libraries
3: $PREFIX/lib/node
Where $HOME is the user's home directory, and $PREFIX is node's configured node_prefix.
These are mostly for historic reasons. You are highly encouraged to place your dependencies locally in node_modules folders. They will be loaded faster, and more reliably.
I hope I answered, you just need to manage the paths to node_modules wherever you have kept it.
What i want is to have a library locally that when i change it those changes are reflected in the project that is using the library.
i have check out this library here in my local machine: https://github.com/manfredsteyer/angular-oauth2-oidc
So what i'm doing right now, is that i go to the library directory and then
npm link
And then get in my project directory and do
npm link angular-oauth2-oidc
The library folder appears inside my node_modules folder but i can't manage to use it, since when i start the app ng serve it says:
Cannot find module 'angular-oauth2-oidc'
I'm importing like this:
import { OAuthModule } from 'angular-oauth2-oidc';
I've tried to add the the path under the compilerOptions of the tsconfig.json file but haven't been sucessful.
Any ideas on what i'm missing here? I've tried several suggestions i've found on angular github issues but none solved my problem.
Thanks in advance
npm link in a package folder will create a symlink in the global folder {prefix}/lib/node_modules/ that links to the package where the npm link command was executed
Dont use npm link to add a library to your project, use npm install :
npm install angular-oauth2-oidc --save
You have to install it not just link it, so use this line to with flag --save to ensure that it will be saved in your package.json
npm install [package_name] --save
You can get the package name from the source website or from
https://www.npmjs.com/package/angular2
When you say:
So what i'm doing right now, is that i go to the library directory and
then npm link
Do you mean you are executing npm link in the folder you cloned the repository in? Because if so, that's likely your issue as that's the source directory and not what's actually published as a package. You must build the library, change directory into the distribution folder for the package, and then run npm link. Then when you run builds of that library, any Angular applications with that linked will automatically have the last version of your build in their node_modules.
Also, in your Angular applications where you are using the linked library you'll want to make sure you are setting preserveSymlinks to true in your angular.json.
While you can create multiple projects (e.g. an Angular app and an Angular library) under one Angular project to make this process a bit easier, I prefer to separating these two since I like one git repository to present one module.
First, you need to link your modules to your project's package.json file. Here's how to link files locally in general:
Local dependency in package.json
Linking a plain Typescript library is pretty straight forward as you just create an entry point (usually index.ts) file and export everything you want from there. This file needs to be in the same folder as the package.json file in your project.
An Angular library is a bit different as angular modules needs to be compiled before it can be properly exported. If you just import the module to your project without compiling you will get an error stating this: cannot read property 'ɵmod'. This happens at least at the time of writing this.
So we need to compile the library and then link it:
open two terminal windows
in the first terminal, go to your Angular library's root folder and run ng build --watch
check the output folder of the compiled module, usually something like dist/[library name]
change your Angular project's package.json to point to the output folder e.g. "my-angular-library": "file:../my-angular-library/dist/my-angular-library"
run npm install in the same folder
Add path to your Angular project's tsconfig.json e.g:
compilerOptions: {
"paths": {
"my-angular-library": ["./node_modules/my-angular-library"]
}
}
Otherwise you'll get errors like Error: Symbol MyComponent declared in /path/to/library/my.component.d.ts is not exported from my-angular-library
in the second terminal, go to your Angular project's root folder and run ng serve. Make sure you serve the project only after you have installed the local dependency.
You should now be able to use components, services etc. exported via your library module.
TL;DR
for the library ng build --watch
make the library dependency to point to the output folder e.g. "my-angular-library": "file:../my-angular-library/dist/my-angular-library"
npm i
Add path to your Angular project's tsconfig.json e.g:
compilerOptions: {
"paths": {
"my-angular-library": ["./node_modules/my-angular-library"]
}
}
ng serve
I am doing an internship in a company.
I need to create a node server.
I installed node on the computer (Windows) and I should install some plugins like:
- nodejs-webpack
- colors
- uglify
Normally I need to enter a command like : npm install "theModule"
But the software can not access the internet (due to company restrictions) and support service can not authorize the software (or do not want).
Can I install modules in any other way ? (download from Google and slide archives in the correct folder for example).
If the answer is no, do you know how can i get around this security?
You need a private npm repository.
Check out this answer:
can you host a private repository for your organization to use with npm?
I found it !
Just for exemple, we will install 'nodejs-websocket' :
1) You just have to download it here.
2) Put files into your Node's directory (for me it's "C:\Program Files\nodejs\node_modules\npm\node_modules")
3) in your .js file just add this line : var ws = require("C:/Program Files/nodejs/node_modules/npm/node_modules/nodejs-websocket/")
Done ! Thanks for all :D
I added this as a comment on your own answer, but I figured I should add a real answer with a better explanation.
Normally when you run npm install package-name npm installs the package to a node_modules directory in the directory you are in at that moment. So if your app was located at C:\code\my-app then you would cd into that directory and run npm install package-name. This would create a node_modules directory at C:\code\my-app\node_modules if it didn't already exist. Then it would install package-name into that directory at C:\code\my-app\node_modules\package-name.
As long as the module is in the node_modules directory for your app, you can require the module in your code without entering a big long file path.
var ws = require('nodejs-websocket');
The place you manually installed your module is the global node_modules directory. It's where npm would install a module if you did npm install -g package-name. The global node_modules directory is added to your system path when you install npm. In other words, any modules you install in there would be accessible from the command line like any other command. For example:
npm install -g bower
That would install the "bower" package to the global npm module directory. Bower would then be accessible as a command line tool for you to use. For example:
bower install angularjs
The global directory is more for tools like that and not really for modules that you intend to use in your code. Technically you can require a module from anywhere by including the full path in the require call like you did, but the standard practice would be to place it in the node_modules directory in the root of your application, and then require it with just its name and not a full path.
Edit: Here's another tip that you might like to take advantage of as well.
When you normally install a module with npm install package-name, your application usually has a package.json file at the root of it. If it does, you can do npm install package-name --save and npm will add the package-name module to a list in your app's package.json file. Once a module is listed in your app's package.json file it's called a "dependency" of your app because it basically says your app depends on package-name.
Normally, when you have dependencies listed in package.json, you can completely delete your app's node_modules directory and then simply run npm install from within your app's root directory and npm will automatically install all dependencies it finds listed in your app's package.json file. Since your corporate firewall won't allow this automatic downloading of modules, you won't get that benefit. However it is still good practice to follow the same conventions.
A trick you can do to create your package.json file is to manually install your dependencies into your app's node_modules directory. Once you have the modules your app needs, you can instruct npm to create a package.json file for your app by simply running npm init. It will walk you through a few little prompts to fill out your package.json file with details about your app. It will then peek inside your node_modules directory to see if you've already installed any modules before having a package.json file. If it finds any, it will automatically add them to the dependencies field in the package.json file it creates :D
With a proper package.json in place you'll always have a nice tidy list of what dependencies your application needs, even if your node_modules directory gets deleted. Normally people add their app's node_modules directory to their .gitignore file, only checking in the package.json file. This way they don't store their dependencies in source control but they can still be easily installed on new machines that clone it by simply running npm install from inside the app's directory. In your case though you may want to just add node_modules to your source control since you can't let npm install dependencies automatically.
It seems that NPM pack, and by extension, nodejitsu, do not like my node_modules folders. :*(
I currently am building a web app.
My web app's project folder structure is as follows:
Root
Engine(folder) Server(folder) readme.md package.json
(Multiple Folders) (folder)(folder) node_modules(folder)
easyimage,mongodb,mysql(folders) socket.io (folder)
node_modules(folder, NPM Pack ignores this) node_modules(folder, NPM Pack ignores this)
Socket.io-client (folder, NPM Pack ignores this)
I hope everyone can see this structure alright!
The problem I am having is that when I run NPM Pack at the root directory, the entire directory structure is packed correctly, except for all of the node_modules folders below the first node_modules folder.
It is as if NPM pack completely ignores those node_modules folders. (The one below socket.io for instance).
Due to the fact NPM pack ignores these npm folders, jitsu is also ignoring them and I can't get my web app started.
How can I get NPM pack/nodejitsu to correctly package all of the node_modules folders correctly?
My current package.json file, at the root directory, looks like follows:
http://pastebin.com/SAU6rwb5
As you can see, I have attempted to use bundleDependencies to tell NPM Pack that I am trying to include some node_modules folders (modules?), but pack still ignores all of them... Also, if i include anything under "dependencies", NPM start creates a new (??) node_modules folder at the root directory... but at the root directory nothing needs node_modules... as you can see node_modules are used inside of the server folder.
How can I get NPM Pack to recognize the files and folders inside of all of the node_modules folders and pack them correctly?
(jump to last paragraph if you want a really easy solution)
I'm having a hard time understanding your app structure. I think I get what your trying to do though.
From https://npmjs.org/doc/folders.html , it actually goes into detail about when and why sub modules will or wont show up.
When installing locally, npm first tries to find an appropriate prefix folder. This is so that npm install foo#1.2.3 will install to the sensible root of your package, even if you happen to have cded into some other folder.
Starting at the $PWD, npm will walk up the folder tree checking for a folder that contains either a package.json file, or a node_modules folder. If such a thing is found, then that is treated as the effective "current directory" for the purpose of running npm commands. (This behavior is inspired by and similar to git's .git-folder seeking logic when running git commands in a working dir.)
If no package root is found, then the current folder is used.
When you run npm install foo#1.2.3, then the package is loaded into the cache, and then unpacked into ./node_modules/foo. Then, any of foo's dependencies are similarly unpacked into ./node_modules/foo/node_modules/....
Bear With me for another quote...
Cycles are handled using the property of node's module system that it walks up the directories looking for node_modules folders. So, at every stage, if a package is already installed in an ancestor node_modules folder, then it is not installed at the current location.
Consider the case above, where foo -> bar -> baz. Imagine if, in addition to that, baz depended on bar, so you'd have: foo -> bar -> baz -> bar -> baz .... However, since the folder structure is: foo/node_modules/bar/node_modules/baz, there's no need to put another copy of bar into .../baz/node_modules, since when it calls require("bar"), it will get the copy that is installed in foo/node_modules/bar.
This shortcut is only used if the exact same version would be installed in multiple nested node_modules folders. It is still possible to have a/node_modules/b/node_modules/a if the two "a" packages are different versions. However, without repeating the exact same package multiple times, an infinite regress will always be prevented.
As far as the bundledDependencies:
Upon publishing [this also applies to packing], npm will look in the node_modules folder. If any of the items there are not in the bundledDependencies array, then they will not be included in the package tarball.
I take this to mean that it will only bundle the listed modules in ./node_modules/ and specific sub-modules for a ./package.json . Then of course as written above, it recursively walks down the directory tree again... so if it sees another package.json file in this directory npm will look and see if that has any bundled deps to include in the pack.
So as I understand it right now, since you have no packages in your base directory, your bundled dependencies in package.json doesn't do anything, and actually having items in your bundledDependencies field does more harm than good.
To fix you need to edit the package.json files to include these bundles at each level.
I have had this issue before when trying to get a packed then unpacked meteor app working on nodejitsu. I solved it in a different way. In the root folder of my app I included all of the top level node modules and explicitly set their version in my package.json file.
From what I understand your file structure is such:
app
+-- Engine
+-- Server
| +-- socket.io
| | `-- package.json
| | +-- node_modules
`-- readme.md
`-- package.json
If this is so then you need to be editing the package.json under socket.io to include the bundled deps you want. Generally though you can trust the package maintainers to keep valid versions.(but in this case you can't?)
As for Socket.io-client not being packed that's a result of it being a dependency of socket.io.
If I was to suggest a way for you to make this easier for yourself, i'd suggest you to, in your MAIN top level package.json file, to include the dependencies you need for your app # the specific version you need. If you need them bundled for some reason, add them into the bundled section, if you need sub modules at a different version than what the author intended. Consider making a folder called package or vendor and then placing the modules in there, in those you can edit the package.json and bundles their dependencies to your hearts content. Be sure though that you do not ignore any files or folders under you vendor or packages directory with .npmignore or .gitignore files.
Alternatively, if this is too difficult (editing all theses files and specifying certain versions can be a pain) I'd suggest hosting your vendor packages some where where you could download them with a script, and then execute this script in the postinstall part of your package.json (take a read at https://npmjs.org/doc/scripts.html ... you would add this in the same section you have your "start" script.
I hope that clarifies things.