In order to refactor a client-side project, i'm looking for a safe way to find (and delete) unused code.
What tools do you use to find unused/dead code in large react projects? Our product has been in development for some years, and it is getting very hard to manually detect code that is no longer in use. We do however try to delete as much unused code as possible.
Suggestions for general strategies/techniques (other than specific tools) are also appreciated.
Thank you
Solution:
For node projects, run the following command in your project root:
npx unimported
If you're using flow type annotations, you need to add the --flow flag:
npx unimported --flow
Source & docs: https://github.com/smeijer/unimported
Outcome:
Background
Just like the other answers, I've tried a lot of different libraries but never had real success.
I needed to find entire files that aren't being used. Not just functions or variables. For that, I already have my linter.
I've tried deadfile, unrequired, trucker, but all without success.
After searching for over a year, there was one thing left to do. Write something myself.
unimported starts at your entry point, and follows all your import/require statements. All code files that exist in your source folder, that aren't imported, are being reported.
Note, at this moment it only scans for source files. Not for images or other assets. As those are often "imported" in other ways (through tags or via css).
Also, it will have false positives. For example; sometimes we write scripts that are meant to simplify our development process, such as build steps. Those aren't directly imported.
Also, sometimes we install peer dependencies and our code doesn't directly import those. Those will be reported.
But for me, unimported is already very useful. I've removed a dozen of files from my projects. So it's definitely worth a shot.
If you have any troubles with it, please let me know. Trough github issues, or contact me on twitter: https://twitter.com/meijer_s
Solution for Webpack: UnusedWebpackPlugin
I work on a big front-end React project (1100+ js files) and stumbled upon the same problem: how to find out which files are unused anymore?
I've tested the next tools so far:
findead
deadfile
unrequired
None of them really worked. One of the reason is that we use "not standard" imports. In additional to the regular relative paths in our imports we also use paths resolved by the webpack resolve feature which basically allows us to use neat import 'pages/something' rather than cumbersome import '../../../pages/something'.
UnusedWebpackPlugin
So here is the solution I've finally come across thanks to Liam O'Boyle (elyobo) #GitHub:
https://github.com/MatthieuLemoine/unused-webpack-plugin
It's a webpack plugin so it's gonna work only if your bundler is webpack.
I personaly find it good that you don't need to run it separately but instead it's built into your building process throwing warnings when something is not ok.
Our research topic: https://github.com/spencermountain/unrequired/issues/6
Libraries such as unrequired and deadcode only support legacy code.
In order to find the unused assets, to remove manually, you can use deadfile
library:https://m-izadmehr.github.io/deadfile/
Out of box support for ES5, ES6, React, Vue, ESM, CommonJs.
It supports import/require and even dynamic import.
It can simply find unused files, in any JS project.
Without any config, it supports ES6, React, JSX, and Vue files:
First of all, very good question, in large project coders usually try many lines of code test and at the end of result, hard to find the unused code.
There is two possible that must be work for you - i usually do whenever i need to remove and reduce the unused code into my project.
1st way WebStorm IDE:
If you're using the web-storm IDE for JS development or React JS / React Native or Vue js etc it's tell us and indicate us alote of mention with different color or red warning as unused code inside the editor
but it's not works in your particular scenario there is another way to remove the unused code .
2nd Way unrequired Library: (JSX is not supported)
The second way to remove the unused code inside the project is unrequired library you can visit here : unrequired github
another library called depcheck under NPM & github here
Just follow their appropriate method - how to use them you will fix this unused issue easily
Hopefully that helps you
I think the easiest solution for a create-react-app bootstrapped application is to use ESLint. Tried using various webpack plugins, but ran into out of memory issues with each plugin.
Use the no-unused-modules which is now a part of eslint-plugin-import.
After setting up eslint, installing eslint-plugin-import, add the following to the rules:
"rules: {
...otherRules,
"import/no-unused-modules": [1, {"unusedExports": true}]
}
My approach is an intensive use of ESlint and make it run both on IDE ad before every push.
It points out unused variables, methods, imports and so on.
Webpack (which has too nice plugins for dead code detection) take care about avoiding to bundle unimported code.
findead
With findead you can find all unused components in your project. Just install and run:
Install
npm i -g findead
Usage
findead /path/to/search
This question recalls me that react by default removes the deadcode from the src when you run the build command.
Notes:
you need to run build command only when you want to ship your app to production.
Related
I am pretty new to js. And I am looking to learn js. Recently I came across one library called bounce.js which is animation library. Which require NPM to install but why? I am dont want to use NPM (or any packet Manager) they havent provided min.js file to direct import in scrpit tag?? Why??. Similarly for Tailwind it require NPM. And as NPM require it means I need vercel to deploy and all stuff.
2) As I use django I dont know how to install NPM modules in my templates.
Please help me clear out from this mess.
When all I can find is NPM based installation guides I like to search the library name followed by CDN which typically brings up some minified results. In your case I tried searching for bounce.js CDN which brought up lots of results including this one:
https://cdnjs.com/libraries/bounce.js/0.8.2
which points to
https://cdnjs.cloudflare.com/ajax/libs/bounce.js/0.8.2/bounce.min.js
You should be able to do some searching and find the CDN you wish to use. If you want the source JS to serve from your own server you can visit the .js link and right click and download (or copy and paste into your own file).
One of the advantages of using npm over a direct download is facilitating the integration into your workflow.
For instance, if you use a module bundler, the bundler will generate an "optimised", minified version for you: Getting rid of unused code for you (=> Tree shaking) reducing the size of your resulting code, solving some import issues, and more
NPM will also help you keep track of your imported library. You know if you use an up-to-date or outdated version. It will also inform you about
Eventual securities issues. And much more.
There are many, many advantages of using npm over direct download.
I have started to learn react and I am very confused with the concept of packages.
and why we can't just use a simple link as cdn and there is a module which i don't understand it and what's npm and why i have to use it with react
Not trying to give the definite answer here, but trying to explain the 3 terms as simple as I can:
A module is just a file containing lines of JavaScript code.
A library uses one or many modules to provide a set of features.
A package is a downloadable, versioned library. Think of someone putting it in a box and shipping it to you, so you can import it and use it in combination with your own code.
so I came with conclusion and hope you tell me if I get it right or not .
-Module : it is justba javascript file but it's different from normal script that it has its own scope so you have to use import or export to exchange information between modules.
-Library : it is a group of modules or scripts that it is responsible for the function you want .
-package : can be one or more libraries but it is also contain files that don't deal with the functionality but it's only role to make sure the libraries and functional file work properly .
like react package it is come with react library deals with the functionality and also has babel compiler to make browser read and understand react library.
It is very much possible to use a simple link such as a CDN. Many packages also have links available, such as material UI. However, it quickly becomes unmanageable to use CDN links when your project grows, and it can affect performance and load times of your site.
Npm stands for Node package manager. It handles packaging for Node, where it would not be suitable to use a simple link.
It turns out that it is possible to also use npm for web applications, by combining it with a bundler. The bundler (such as webpack) takes all of your modules (JavaScript files and npm packages) and bundles it together so that you get a single script which you can run in the browser.
I want to write a node cli application and im wondering how i should structure the application. Im fairly new to node and im a confused with all the design patterns used when building such a application.
I want to be able to call the application from the command line, but also use it as a node module for better testing.
Currently i have one file with lots of functions that get called directly from the cli, but i feel this is rather difficult to maintain.
Is there any good writing on how to do such things? i looked at rimraf but it confused me even more. Thanks for your time
I don't know if there is a "right" way to do it but I can tell you how I have dealt with a problem similar to yours. I wanted to create a CLI and a visual studio code plugin so people would be able to use the functionality both from VSC and from the CLI (for those that don't use VSC), so the approach I took was to put all the logic in its own package and then create two other packages that included the first one, one for CLI and one VSC plugin that required the "logic" package.
In the CLI package you would only have code strictly related to command handling and then the real meat happens in the logic package. In my case the VSC plugin package had very few lines of code, just configuration and the calls to the needed functions.
Then regarding the structure of the code some recommendations:
expose only what is strictly necessary
isolate your code in different files/classes based on common functionality (and go to point 1)
test your code
lint your code
But those are common sense and language independent recommendations.
There's no one "standard" way to structure Node.js apps, however you will notice that many authors follow similar patterns. Instead of having one file containing all code, it should be split out into modules, grouped by function. Have a look at this repo on Github, it has some very good suggestions about Node.js best practice https://github.com/i0natan/nodebestpractices#1-project-structure-practices.
A couple more pointers I would add: Ensure you're logging any errors, consider using something like Winston.js for this purpose. Also have some mechanism in place to restart the service if a critical error occurs, e.g. Forever.js.
Ensure likewise you're unit testing, there are some good test frameworks, Jasmine, Mocha, Cucumber.js.
I bought an HTML template recently, which contains many plugins placed inside a bower_components directory and a package.js file inside. I wanted to install another package I liked, but decided to use npm for this purpose.
When I typed:
npc install pnotify
the node_modules directory was created and contained about 900 directories with other packages.
What are those? Why did they get installed along with my package? I did some research and it turned out that those were needed, but do I really need to deliver my template in production with hundreds of unnecessary packages?
This is a very good question. There are a few things I want to point out.
The V8 engine, Node Modules (dependencies) and "requiring" them:
Node.js is built on V8 engine, which is written in C++. This means that Node.js' dependencies are fundamentally written in C++.
Now when you require a dependency, you really require code/functions from a C++ program or js library, because that's how new libraries/dependencies are made.
Libraries have so many functions that you will not use
For example, take a look at the express-validator module, which contains so many functions. When you require the module, do you use all the functions it provides? The answer is no. People most often require packages like this just to use one single benefit of it, although all of the functions end up getting downloaded, which takes up unnecessary space.
Think of the node dependencies that are made from other node dependencies as Interpreted Languages
For example, JavaScript is written in C/C++, whose functions and compilers are in turn originally written in assembly. Think of it like a tree. You create new branches each time for more convenient usage and, most importantly, to save time . It makes things faster. Similarly, when people create new dependencies, they use/require ones that already exist, instead of rewriting a whole C++ program or js script, because that makes everything easier.
Problem arises when requiring other NPMs for creating a new one
When the authors of the dependencies require other dependencies from here and there just to use a few (small amount) benefits from them, they end up downloading them all, (which they don't really care about because they mostly do not worry about the size or they'd rather do this than explicitly writing a new dependency or a C++ addon) and this takes extra space. For example you can see the dependencies that the express-validator module uses by accessing this link.
So, when you have big projects that use lots of dependencies you end up taking so much space for them.
Ways to solve this
Number 1
This requires some expert people on Node.js. To reduce the amount of the downloaded packages, a professional Node.js developer could go to the directories that modules are saved in, open the javascript files, take a look at their source code, and delete the functions that they will not use without changing the structure of the package.
Number 2 (Most likely not worth your time)
You could also create your own personal dependencies that are written in C++, or more preferably js, which would literally take up the least space possible, depending on the programmer, but would take/waste the most time, in order to reduce size instead of doing work. (Note: Most dependencies are written in js.)
Number 3 (Common)
Instead of Using option number 2, you could implement WebPack.
Conclusion & Note
So, basically, there is no running away from downloading all the node packages, but you could use solution number 1 if you believe you can do it, which also has the possibility of screwing up the whole intention of a dependency. (So make it personal and use it for specific purposes.) Or just make use of a module like WebPack.
Also, ask this question to yourself: Do those packages really cause you a problem?
No, there is no point to add about 900 packages dependencies in your project just because you want to add some template. But it is up to you!
The heavyness of a template is not challenging the node.js ecosystem nor his main package system npm.
It is a fact that javascript community tend to make smallest possible module to be responsible for one task, and just one.
It is not a bad thing I guess. But it could result of a situation where you have a lot of dependencies in your project.
Nowadays hard drive memory is cheap and nobody cares any more about making efficient/small apps.
As always, it's only a matter of choice.
What is the point of delivering hundreds of packages weighing hundreds of MB for a few kB project.
There isn't..
If you intend to provide it to other developers, just gitignore (or remove from shared package) node_modules or bower_components directories. Developers simply install dependencies again as required ;)
If it is something as simple as an HTML templates or similar stuff, node would most likely be there just for making your life as a developer easier providing live reload, compiling/transpiling typescript/babel/SCSS/SASS/LESS/Coffee... ( list goes on ;P ) etc.
And in that case dependencies would most likely only be dev_dependencies and won't be required at all in production environment ;)
Also many packages come with separate production and dev dependencies, So you just need to install production dependencies...
npm install --only=prod
If your project does need many projects in production, and you really really wanna avoid that stuff, just spend some time and include css/js files your your project needs(this can be a laborious task).
Update
Production vs default install
Most projects have different dev and production dependencies,
Dev dependencies may include stuff like SASS, typescript etc. compilers, uglifiers (minification), maybe stuff like live reload etc.
Where as production version will not have those things reducing the size node_modules directory.
** No node_modules**
In some html template kind of projects, you may not need any node_modules in production, so you skip doing an npm install.
No access to node_modules
Or in some cases, when server that serves exists in node_modules itself, access to it may be blocked (coz there is no need to access these from frontend).
What are those? Why did they get installed along with my package?
Dependencies exists to facilitate code reuse through modularity.
... do I need to deliver my template in production with hundreds of unnecessary packages?
One shouldn't be so quick to dismiss this modularity. If you inline your requires and eliminate dead code, you'll lose the benefit of maintenance patches for the dependencies automatically being applied to your code. You should see this as a form of compilation, because... well... it is compilation.
Nonetheless, if you're licensed to redistribute all of your dependencies in this compiled form, you'll be happy to learn those optimisations are performed by a compiler which compile Javascript to Javascript. The Closure Compiler, as the first example I stumbled across, appears to perform advanced compilation, which means you get dead code removal and function inlining... That seems promising!
This does however have another side effect when you are required to justify the licensing of all npm modules..so when you have hundreds of npm modules due to dependencies this effort also becomes a more cumbersome task
Very old question but I happened to come across very similar situation just as RA pointed out.
I tried to work with node.js framework using vscode and the moment when I tried to install start npm using npm init -y, it generated so many different dependencies. In my case, it was vscode extension ESlint that I added to prior to running npm init -y
Uninstalling ESlint
Restarted vscode to apply that uninstallation
removed previously generated package.json and node-modules folder
do npm init -y again
This solved my problem of starting out with so many dependencies.
I am thinking of extending the format of package.json to include dynamic package (plugin) loading on client side and I would like to understand whether this idea contradicts with npm vision or not. In other words I want to load a bunch of modules, that share common metadata, in browser runtime. Solutions like system.js and jspm are good for modules management, but what I seek is dynamic packages management on client side.
Speaking in details I would like to add a property like "myapp-clientRuntimeDependencies" that would allow to specify dependencies that would be loaded by browser instead of standard prepackaging (npm install->browserify-like solution).
package.json example:
{
name: "myapp-package",
version: "",
myapp-clientRuntimeDependencies: {
"myapp-plugin": "file:myapp-plugin",
"myapp-anotherplugin": "file:myapp-anotherplugin"
},
peerDependencies: {
"myapp-core": "1.0.0"
}
}
The question:
Does this idea contradict with "npm" and "package.json" vision? If yes then why?
Any feedback from npm community is very much appreciated.
References:
Extending package.json: http://blog.npmjs.org/post/101775448305/npm-and-front-end-packaging
EDIT:
The question was not formulated too well, the better way to ask this is:
What is the most standard way (e.g. handled by some existing tools, likely to be supported by npm) to specify run-time dependencies between 2 dynamically loaded front-end packages in package.json?
What is the most standard way to attach metadata in JSON format to front-end packages, that are loaded dynamically?
I wouldn't say that it conflicts with the vision of package.json, however it does seem to conflict a bit with how it's typically used. As you seem to be aware, package.json is normally used pre-runtime. In order to load something from package.json into your runtime, you'd have to load the package.json into your frontend code. If you're storing configurations that you don't want visible to frontend via a simple view source, this could definitely present a problem.
One thing that didn't quite click with me on this: you said that system.js and jspm are good for module management but that you were looking for package management. In the end, packages and modules tend to be synonymous, as a package becomes a module.
I may be misunderstanding what it is that you're looking for, but from what I can gather, I recommend you take a look at code-splitting...which is essentially creating separate js files that will be loaded dynamically based on what is needed instead of bundling all your javascript into a single file. Here's some docs on how to do this with webpack (I'm sure browserify does it as well).
If I understand correctly, your question is about using the package.json file to include your own app configuration. In the example you describe, you would use such configuration to let your app know which dependencies can be loaded at runtime.
There is really nothing preventing you from inserting your own fields in the package.json file, except for the risk of conflict with names that are used by npm for other meanings. But if you use a very specific name (like in your example), you should be safe enough. Actually, many lint and build tools have done so already. It is even explicitly written in the post you refer to:
If your tool needs metadata to make it work, put it in package.json. It would be rude to do this without asking, but we’re inviting you to do it, so go ahead. The registry is a schemaless store, so every field you add is on an equal footing to all the others, and we won’t strip out or complain about new fields (as long as they don’t conflict with existing ones).
But if you want to be even safer, you could resort to use a different file (like Bower did for example).