In the npm ecosystem it is becoming a common practice to configure various libraries using either dotfiles, package.json entries, or *.config.js files.
In order to establish common practices across different projects in our organization, I would like to be able to provide defaults for these configs (eslint, babel, vue, npmrc, editorconfig, jest.config, postcss.config, etc?) as a package that can be installed using npm and affect projects using it.
I can't just add those files in a dependency, but I could write a command to copy a group of files to the root of a repository. I'm wondering if there are standard/accepted ways of approaching this problem.
Related
Let's say we have a project with lots of sub-packages:
👩🍳 project root
🍔sup package a
package.json
tsconfig.json
webpack.config.js
🍟 sub package b
package.json
tsconfig.json
webpack.config.js
This kind of structure most likely contains lots of redundancies, in both, settings and dependencies. I would like to have a way to manage these common settings and dependencies in a centralized way.
For example:
If a new sup package 🌮 is added, it would be nice, if common settings could be inherited from some kind of a centralized setting-thingy
If the version of a dependency, that is used in several sub packages, is changed, I would like to have a way to change the version at some central point, so that all sub packages "see" the new version.
I would like to change a property in all tsconfig.json's in one step, instead of editing all tsconfig.json's manually.
🤩In essence, I want to apply the DRY-principle to a project configuration.
I looked into npm workspaces, lerna, yarn, webpack, but didn't find anything that would do all of that. How could this be achieved?
🙄In Java, I would come close to this using Maven. There I could at least manage the versions of my dependencies in a central config. Isn't there something like that in the JavaScript universe?
We have been working on our projects all tied to a single lerna repository, as:
lerna
|----- app1 - application 1
|----- app2 - application 2
|----- appN - application N
|----- commondb (common database libraries for app1, app2 to appN)
|----- commonux (common ux libraries for app1, app2 to appN)
|----- commonauth (common authentication libraries for app1, app2 to appN)
As the code grew a lot, lerna is really full of packages (40+ packages) and too much code.
We're now trying to split lerna into smaller pieces and we're looking for alternatives. Doing that, applications would need a way to import common libraries as we do today.
Certainly NPM seens to be a solution (making each common package independent and publishing it on NPM), but we want to keep our code in our environment without third party services or clouds (we have our own git server instance).
What are the current options to manage javascript libraries that we can make use of? What would be the recommended one in such a scenario?
Your decision can be greatly affected by answering the following - do you want your apps to be running the same version of the shared libraries? Or do want autonomy within the libraries, and to be able to publish and manage different versions of the libraries, where it is the responsibility on the consumer app to manage which version of the library it is using?
If it is the former, my suggestion would be to stick with a Mono-repo approach, maybe consider something like NX, where it has some nice tooling for only linting, testing, building and deploying only the affected modules, whilst sharing a common single package.json and therefore common libraries shared across multiple apps and libs
Otherwise you are looking at potentially managing multiple repos, multiple versions of each library, multiple pipelines, multiple workspace configs.
You could simply use pnpm workspaces: protocol which allows you to create a monorepo structure but without even requiring you to publish when used this way: "foo": "workspace:*". So if you wish to use this approach, then you could have a monorepo structure and just keep local packages, it will use the local code without downloading anything. I used this concept in a Vue pnpm Workspace demo. With this approach you could stay with local packages and/or go with Lerna to publish if you also want to (I prefer Lerna-Lite which is what I use) since they now both support pnpm/yarn workspace: protocol. This approach is flexible and allows you to switch to publishing (to npm or a private registry) at any point in the time in the future if you wish to, in other words there's no major code refactoring to add Lerna after the fact when using a pnpm workspace structure.
I'm new to JS development and was wondering given that the size of node modules is an issue of concern where one has to even go on deleting the folder when many projects pile up and is not used, why can't we have a global node modules folder, like m2 for maven and then reference it in our projects?
Why do we need multiple node modules folders inside each project?
You can install packages globally and use them with the -g flag when using npm install.
But this isn't done often, and often isn't recommended, because the same package can have many, many different versions. SomeLibrary version 5.6.1 can be significantly different from SomeLibrary version 5.7.1. If you have multiple projects, they'll very often have different dependencies, and different dependency versions. If you try to source them all from the same global install, you'll usually run into problems very quickly. Having a separate node_modules folder for each project solves this problem for you; often, package versioning for each project you have will "just work" without any extra configuration.
There are also managers like Yarn which can cache package downloads, so you don't re-download huge numbers of duplicate megabytes over multiple projects
Variety of reasons. Just FYI, you can install npm packages globally using the -g flag when installing.
Here's some reasons:
Different project may require different versions of a package.
A single machine can consist of multiple different environment, i.e
containers or virtual machines.
Sharing code between projects/environments introduces potential security risks.
For the above reasons, and others, you might not actually want a projects
packages to leave the scope of that project.
since java script is not compiled, we will need to have all the plugins in each deployment verses maven is compiled so we can compile it to machine code with the plugin globally installed
I have a Lerna powered monorepo which contains various packages that are published to NPM. Within this have various helper functions that live on the root of the repo and are shared by a few packages.
The issue I have is that when each package is built with Babel, the transpiled code uses require to get those helpers. Obviously this doesn't work as when a package is published it needs to be completely self contained - those helpers are not there.
Is there a way to solve this? I don't want to have to duplicate helpers between packages.
When using lerna bootstrap --hoist I still have the same issue.
Well the easiest solution i see is to create an helper workspace that you do not publish when using lerna publish.
You will be able to set this "helper" workspace as a dependency of each of your workspace, therefore avoiding code duplication.
I am exploring AngularJS tutorial project and found it has node_modules directory inside, which size if 60 megabytes.
Does simple clientside javascript project really need so huge corpus of unknown data?
I tried to delete this directory and project still works. I suspect it somehow relates with node.js and it's npm but how? Suppose I need to run my project on some conventional web server (not node.js), then how to know, which files/directories are unneeded?
Many javascript libraries require to use bower to install them. If I use bower, does this mean I need to keep node_modules?
The node_modules directory is only for build tools.
The package.json file in the app root defines what libraries will be installed into node_modules when you run npm install.
Very often with an angular app, on your dev machine or on a build server, you use other Javascript libraries from npm (a node.js package manager) to build your angular app. Tasks can be concatenating resources, using CSS preprocessors like LESS or SASS, minification, replacing of values, etc. etc. The most common tools for managing and running these tasks are called grunt and gulp, which are installed through npm as well.
When you deploy your app, you only distribute the resulting build, not any of the source files or build tools.
It is of course possible to write an AngularJS app without building anything.
edit from comments: When you dive into Angular more, there are more advanced techniques of using libraries installed by npm even in the client app, you then selectively choose the ones you need, not the whole 50MB+ thing. I'd recommend staying with the basic approaches until you get a good grasp on them though.
NPM is the node package manager, which installs packages locally into a project, specifically, into the node_modules folder. From there the package code can be included into a project, yes, can is the important word there.
The browser has no way to include modules in code (yet), so you need to use a library that can expose node's commonJS style modules. Browserify and Webpack are two popular methods of doing so.
Angular complicates this by introducing its own module system, which more closely resembles AMD-style modules. There are ways around this, so that you can use node-style modules, maybe your project uses those.
Using npm for managing dependencies is a great idea, its a fantastic package manager. It is likely in your case though that the project is only built using node and that the node_modules folder contains dependencies only related to the build.