I've got a collection of javascript files from a 3rd party, and I'd like to remove all the unused methods to get size down to a more reasonable level.
Does anyone know of a tool that does this for Javascript? At the very least give a list of unused/used methods, so I could do the manually trimming? This would be in addition to running something like the YUI Javascript compressor tool...
Otherwise my thought is to write a perl script to attempt to help me do this.
No. Because you can "use" methods in insanely dynamic ways like this.
obj[prompt("Gimme a method name.")]();
Check out JSCoverage . Generates code coverage statistics that show which lines of a program have been executed (and which have been missed).
I'd like to remove all the unused methods to get size down to a more reasonable level.
There are a couple of tools available:
npm install -g fixmyjs
fixmyjs <filename or folder>
A configurable module that uses JSHint (Github, docs) to flag functions that are unused and perform clean up as well.
I'm not sure that it removes undefined functions as opposed to flagging them. though it is a great tool for cleanup, it appears to lack compatibility with later versions of ECMAScript (more info below).
There is also the Google Closure Compiler which claims to remove dead JS but this is more of a build tool.
Updated
If you are using something like Babel, consider adding ESLint to your text editor, which can trigger a warning on unused methods and even variables and has a --fix CLI option for autofixing some errors and style issues.
I like ESLint because it contains multiple plugins for alternate libs (like React warnings if you're missing a prop), allowing you to catch bugs in advance. They have a solid ecosystem.
As an example: on my NodeJS projects, the config I use is based off of the Airbnb Style Guide.
You'll have to write a perl script. Take no notice of the nay-sayers above.
Such a tool could work with libraries that are designed to only make function calls explicitly. That means no delegates or pointers to functions would be allowed, the use of which in any case only results in unreadable "spaghetti code" and is not best practice. Even if it removes some of these hidden functions you'll discover most if not all of them in testing. The ones you dont discover will be so infrequently used that they will not be worth your time fixing them. Dont obsess with perfection. People go mad doing that.
So applying this one restriction to JavaScript (and libraries) will result in incredible reductions in page size and therefore load times, not to mention readability and maintainability. This is already the case for tools that remove unused CSS such as grunt_CSS and unCSS (see http://addyosmani.com/blog/removing-unused-css/) and which report typical reductions down to one tenth the original size.
Its a win/win situation.
Its noteworthy that all interpreters must address this issue of how to manage self modifying code. For the life of me I dont understand why people want to persist with unrestrained freedom. As noted by Triptych above Javascript functions can be called in ways that are literally "insane". This insane fexibility corrupts the fundamental doctrine of separation of code and data, enables real-time code injection, and invalidates any attempt to maintain code integrity. The result is always unreadable code that is impossible to debug and the side effect to JavaScript - removing the ability to run automatic code pre-optimisation and validation - is much much worse than any possible benefit.
AND - you'd have to feel pretty insecure about your work to want to deliberately obsficate it from both your collegues and yourself. Browser clients that do work extremely well take the "less is more" approach and the best example I've seeen to date is Microsoft Office combination of Access Web Forms paired with SharePoint Access Servcies. The productivity of having a ubiquitous heavy tightly managed runtime interpreter client and its server side clone is absolutely phenomenal.
The future of JavaScript self modifying code technologies therfore is bringing them back into line to respect the...
KISS principle of code and data: Keep It Seperate, Stupid.
Unless the library author kept track of dependencies and provided a way to download the minimal code [e.g. MooTools Core download], it will be hard to to identify 'unused' functions.
The problem is that JS is a dynamic language and there are several ways to call a function.
E.g. you may have a method like
function test()
{
//
}
You can call it like
test();
var i = 10;
var hello = i > 1 ? 'test' : 'xyz';
window[hello]();
I know this is an old question by UglifyJS2 supports removing unused code which may be what you are looking for.
Also worth noting that eslint supports an option called no-unused-vars which actually does some basic handling of detecting if functions are being used or not. It definitely detects it if you make the function anonymous and store it as a variable (but just be aware that as a variable the function declaration doesn't get hoisted immediately)
In the context of detecting unused functions, while extreme, you can consider breaking up a majority of your functions into separate modules because there are packages and tools to help detect unused modules. There is a little segment of sindreshorus's thoughts on tiny modules which might be relevant to that philosophy but that may be extreme for your use case.
Following would help:
If you have fully covered test cases, running Code Coverage tool like istanbul (https://github.com/gotwarlost/istanbul) or nyc (https://github.com/istanbuljs/nyc), would give a hint of untouched functions.
At least the above will help find the covered functions, that you may thought unused.
Related
I would like to validate some JavaScript code for syntactic correctness, but without enforcing a specific coding-style to the user.
My first approach was to use esvalidate that comes included with esprima. This does the job partially, as it detects unexpected tokens, such as:
constx foo = {};
What it does not detect is among other things usage of variables that have never been declared, such as:
const foox = {
bar() {}
};
foo.bar();
A tool such as eslint would detect this, but it's very difficult to configure ESLint in a way that no specific style is enforced to the user (I don't say it's not possible, I just say it's a huge amount of work since you need to check every single rule and decide whether to enable or disable it, and with hundreds of rules this is… yes, well, a huge amount of work).
What other options do I have? How could I validate the code without this effort?
By the way: What is it that I want to validate here? It's not the syntax (I mean, syntactically, everything is fine, it just does not make sense), but it's also not the semantics. What would be the correct term for this type of checks?
Years ago Linters were only there to check the style of your code. Nowadays they do more, a lot more. Even static analysis. ESLint is such a powerful tool and IMHO it is exactly what you're looking for.
You may think the initial configuration may be costly, but as the ESLint page says:
No rules are enabled by default.
Besides, having a consistent coding style across your time is very beneficial and if you and your team found a common basline you can share the .eslintrc between projects.
Enabling the no-undef rule and using a plugin for your IDE, like one of those should solve your issue -> static code analysis at development time :)
To answer your last question, I think that what you want is static analysis. This will be harder with JavaScript in general because of its dynamic nature and lack of types, and relative lack of maturity in tooling. Decades of work have gone into writing static analyzers for C, for example, and that work won't immediately carry over into others languages.
Something like jshint may be what you want because its goal is to help developers "write complex programs without worrying about typos and language gotchas".
Hope this helps!
ESLint Recommended set: Recommended rules have checkmark next to them do not include any stylistic rules. You can use that to verify validity of the script without enforcing any specific style. At the very least it's a good starting point, and you can always turn off the rules you don't like.
I don't understand WHY and in what scenario this would be used..
My current web setup consists of lots of components, which are just functions or factory functions, each in their own file, and each function "rides" the app namespace, like : app.component.breadcrumbs = function(){... and so on.
Then GULP just combines all the files, and I end up with a single file, so a page controller (each "page" has a controller which loads the components the page needs) can just load it's components, like: app.component.breadcrumbs(data).
All the components can be easily accessed on demand, and the single javascript file is well cached and everything. This way of work seems extremely good, never saw any problem with this way of work. of course, this can (and is) be scaled nicely.
So how are ES6 imports for functions any better than what I described?
what's the deal with importing functions instead of just attaching them to the App's namespace? it makes much more sense for them to be "attached".
Files structure
/dist/app.js // web app namespace and so on
/dist/components/breadcrumbs.js // some component
/dist/components/header.js // some component
/dist/components/sidemenu.js // some component
/dist/pages/homepage.js // home page controller
// GULP concat all above to
/js/app.js // this file is what is downloaded
Then inside homepage.js it can look like this:
app.routes.homepage = function(){
"use strict";
var DOM = { page : $('#page') };
// append whatever components I want to this page
DOM.page.append(
app.component.header(),
app.component.sidemenu(),
app.component.breadcrumbs({a:1, b:2, c:3})
)
};
This is an extremely simplified code example but you get the point
Answers to this are probably a little subjective, but I'm going to do my best.
At the end of the day, both methods allow support creating a namespace for a piece of functionality so that it does not conflict with other things. Both work, but in my view, modules, ES6 or any other, provide a few extra benefits.
Explicit dependencies
Your example seems very bias toward a "load everything" approach, but you'll generally find that to be uncommon. If your components/header.js needs to use components/breadcrumbs.js, assumptions must be made. Has that file been bundled into the overall JS file? You have no way of knowing. You're two options are
Load everything
Maintain a file somewhere that explicitly lists what needs to be loaded.
The first option is easy and in the short term is probably fine. The second is complicated for maintainability because it would be maintained as an external list, it would be very easy to stop needing one of your component file but forget to remove it.
It also means that you are essentially defining your own syntax for dependencies when again, one has now been defined in the language/community.
What happens when you want to start splitting your application into pieces? Say you have an application that is a single large file that drives 5 pages on your site, because they started out simple and it wasn't big enough to matter. Now the application has grown and should be served with a separate JS file per-page. You have now lost the ability to use option #1, and some poor soul would need to build this new list of dependencies for each end file.
What if you start using a file in a new places? How do you know which JS target files actually need it? What if you have twenty target files?
What if you have a library of components that are used across your whole company, and one of they starts relying on something new? How would that information be propagated to any number of the developers using these?
Modules allow you to know with 100% certainty what is used where, with automated tooling. You only need to package the files you actually use.
Ordering
Related to dependency listing is dependency ordering. If your library needs to create a special subclass of your header.js component, you are no longer only accessing app.component.header() from app.routes.homepage(), which would presumable be running at DOMContentLoaded. Instead you need to access it during the initial application execution. Simple concatenation offers no guarantees that it will have run yet. If you are concatenating alphabetically and your new things is app.component.blueHeader() then it would fail.
This applies to anything that you might want to do immediately at execution time. If you have a module that immediately looks at the page when it runs, or sends an AJAX request or anything, what if it depends on some library to do that?
This is another argument agains #1 (Load everything) so you start having to maintain a list again. That list is again going to be a custom things you'll have come up with instead of a standardized system.
How do you train new employees to use all of this custom stuff you've built?
Modules execute files in order based on their dependencies, so you know for sure that the stuff you depend on will have executed and will be available.
Scoping
Your solution treats everything as a standard script file. That's fine, but it means that you need to be extremely careful to not accidentally create global variables by placing them in the top-level scope of a file. This can be solved by manually adding (function(){ ... })(); around file content, but again, it's one more things you need to know to do instead of having it provided for you by the language.
Conflicts
app.component.* is something you've chosen, but there is nothing special about it, and it is global. What if you wanted to pull in a new library from Github for instance, and it also used that same name? Do you refactor your whole application to avoid conflicts?
What if you need to load two versions of a library? That has obvious downsides if it's big, but there are plenty of cases where you'll still want to trade big for non-functional. If you rely on a global object, it is now up to that library to make sure it also exposes an API like jQuery's noConflict. What if it doesn't? Do you have to add it yourself?
Encouraging smaller modules
This one may be more debatable, but I've certainly observed it within my own codebase. With modules, and the lack of boilerplate necessary to write modular code with them, developers are encouraged to look closely on how things get grouped. It is very easy to end up making "utils" files that are giant bags of functions thousands of lines long because it is easier to add to an existing file that it is to make a new one.
Dependency webs
Having explicit imports and exports makes it very clear what depends on what, which is great, but the side-effect of that is that it is much easier to think critically about dependencies. If you have a giant file with 100 helper functions, that means that if any one of those helpers needs to depend on something from another file, it needs to be loaded, even if nothing is ever using that helper function at the moment. This can easily lead to a large web of unclear dependencies, and being aware of dependencies is a huge step toward thwarting that.
Standardization
There is a lot to be said for standardization. The JavaScript community has moved heavily in the direction of reusable modules. This means that if you hope into a new codebase, you don't need to start off by figuring out how things relate to eachother. Your first step, at least in the long run, won't be to wonder whether something is AMD, CommonJS, System.register or what. By having a syntax in the language, it's one less decision to have to make.
The long and short of it is, modules offer a standard way for code to interoperate, whether that be your own code, or third-party code.
Your current process is to concatenate everything always into a single large file, only ever execute things after the whole file has loaded and you have 100% control over all code that you are executing, then you've essentially defined your own module specification based on your own assumptions about your specific codebase. That is totally fine, and no-one is forcing you to change that.
No such assumptions can be made for the general case of JavaScript code however. It is precisely the objective of modules to provide a standard in such a way as to not break existing code, but to also provide the community with a way forward. What modules offer is another approach to that, which is one that is standardized, and one that offers clearer paths for interoperability between your own code and third-party code.
One of the developers on our team (me) keeps accidentally checking in fdescribe and fit in Jasmine tests. Which sometimes results in masking broken tests.
Is there an easy way to either fail a build (TFS) if fdescribe is used, or (better?) configure jasmine server side to treat fdescribe (and fit) as regular describe (and it)?
I would rather use those then fall back to ?spec approach.
I also would like to take this opportunity to apologize for doing that.
We've approached it with static code analysis and eslint.
There is a specific plugin: eslint-plugin-jasmine, see:
Disallow use of focused tests (no-focused-tests)
Sample output:
test/e2e/specs/test.spec.js
5:0 error Unexpected fdescribe jasmine/no-focused-tests
Currently, the plugin also checks for disabled tests and duplicate suite names.
If you actually want to make fit and fdescribe behave like it and describe respectively, you can override one with the other in a separate file and insert it before your actual specs (how to insert depends on your test runner):
// assuming Jasmine is already loaded and has published it's public API
fit = it;
fdescribe = describe;
But I would not suggest to use this approach, because if you really want to focus some spec, you'd have to comment these things or exclude the file from a runner. I would rather add this as another step for the build process, if you have one, or pre-commit hook, which would run some tool, like #alecxe has suggested, analyzing the code for existing fit and fdescribe and fail a build / reject a commit.
Let's say you would get a bunch of .js files and now it is your job to sort them into groups like:
requires at least JavaScript 1.85
requires at least E4X (ECMAScript 4 EX)
requires at least ECMAScript 5
or something like this.
I am interested in any solution, but especially in those which work using JavaScript or PHP. This is used for creation of automated specifications, but it shouldn't matter - this is a nice task which should be easy to solve - however, I have no idea how and it is not easy for me. So, if this is easy to you, please share any hints.
I would expect something like this - http://kangax.github.com/es5-compat-table/# - just not for browsers, rather for a given file to be checked against different implementations of JavaScript.
My guess is, that each version must have some specifics, which can be tested for. However, all I can find is stuff about "what version does this browser support".
PS: Don't take "now it is your job" literally, I used it to demonstrate the task, not to imply that I expect work done for me; while in the progress of solving this, it would be just nice to have some help or direction.
EDIT: I took the easy way out, by recquiring ECMAScript 5 to be supported at least as good as by the current FireFox for my projekt to work as intendet and expected.
However, I am still intereseted in any solution-attemps or at least an definite answer of "is possible(, with XY)" or "is not possible, because ..."; XY can be just some Keyword, like FrameworkXY or DesignPatternXY or whatever or a more detailed solution of course.
Essentially you are looking to find the minimum requirements for some javascript file. I'd say that isn't possible until run time. JavaScript is a dynamic language. As such you don't have compile time errors. As a result, you can't tell until you are within some closure that something doesn't work, and even then it would be misleading. Your dependencies could in fact fix many compatibility issues.
Example:
JS File A uses some ES5 feature
JS File B provides a shim for ES5 deficient browsers or at least mimics it in some way.
JS File A and B are always loaded together, but independently A looks like it won't work.
Example2:
Object.create is what you want to test
Some guy named Crockford adds create to Object.prototype
Object.create now works in less compatible browsers, and nothing is broken.
Solution 1:
Build or find a dependency map. You definitely already have a dependency map, either explicitly or you could generate it by iterating over you HTML files.
Run all relevant code paths in environments with decreasing functionality (eg: ES5, then E4X, then JS 1.x, and so forth).
Once a bundle of JS files fail for some code path you know their minimum requirement.
Perhaps you could iterate over the public functions in your objects and use dependency injection to fill in constructors and methods. This sounds really hard though.
Solution 2:
Use webdriver to visit your pages in various environments.
Map window.onerror to a function that tells you if your current page broke while performing some actions.
On error you will know that there is a problem with the bundle on the current page so save that data.
Both these solutions assume that you always write perfect JS that never has errors, which is something you should strive for but isn't realistic. This might; however, provide you with some basic "smoke testing" though.
This is not possible in an exact way, and it also is not a great way of looking at things for this type of issue.
Why its not possible
Javascript doesn't have static typing. But properties are determined by the prototype chain. This means that for any piece of code you would have to infer the type of an object and check along the prototype chain before determining what function would be called for a function call.
You would for instance, have to be able to tell that $(x).bind() o $(x).map are not making calls to the ecmascript5 map or bind functions, but the jQuery ones. This means that you would really have to parse out the whole code and make inferences on type. If you didn't have the whole code base this would be impossible. If you had a function that took an object and you called bind, you would have no idea if that was supposed to be Function.prototype.bind or jQuery.bind because thats not decided till runtime. In fact its possible (though not good coding practice) that it could be both, and that what is run depends on the input to a function, or even depends on user input. So you might be able to make a guess about this, but you couldn't do it exactly.
Making all of this even more impossible, the eval function combined with the ability to get user input or ajax data means that you don't even know what types some objects are or could be, even leaving aside the issue that eval could attempt to run code that meets any specification.
Here's an example of a piece of code that you couldn't parse
var userInput = $("#input").val();
var objectThatCouldBeAnything = eval(userInput);
object.map(function(x){
return !!x;
});
There's no way to tell if this code is parsing a jQuery object in the eval and running jQuery.map or producing an array and running Array.prototype.map. And thats the strength and weakness of a dynamically typed language like javascript. It provides tremendous flexibility, but limits what you can tell about the code before run time.
Why its not a good strategy
ECMAScript specifications are a standard, but in practice they are never implemented perfectly or consistently. Different environments implement different parts of the standard. Having a "ECMAScript5" piece of code does not guarantee that any particular browser will implement all of its properties perfectly. You really have to determine that on a property by property basis.
What you're much better off doing is finding a list of functions or properties that are used by the code. You can then compare that against the supported properties for a particular environment.
This is still a difficult to impossible problem for the reasons mentioned above, but its at least a useful one. And you could gain value doing this even using a loose approximation (assuming that bind actually is ecmascript5 unless its on a $() wrap. Thats not going to be perfect, but still might be useful).
Trying to figure out a standard thats implemented just isn't practical in terms of helping you decide whether to use it in a particular environment. Its much better to know what functions or properties its using so that you can compare that to the environment and add polyfills if necessary.
I have a complex dojo app that works correctly uncompiled, but after compiling with Google's Closure Compiler, I get subtle differences in some behaviours.
As it is, it's exceedingly difficult to debug, and I've been unable to find any information about possible functional differences between compiled and uncompiled Javascript with Google Closure.
Can anyone point me in the direction of known differences, or share any similar experiences and some ideas of where to start looking?
General Closure Compiler Debugging Tips
Use the VERBOSE warning level. This turns on all of the checks.
Use the debug flag. It makes renamed symbols ridiculously long, but they are named in such a way you can find the original. If code works with the debug flag but not without it, it is almost certainly a renaming issue.
Definitely use formatting=PRETTY_PRINT. Debugging compacted code is painful without it.
Use source maps
Disable the type based optimizations with --use_types_for_optimization false. Incorrect type annotations can cause the compiler to make invalid assumptions.
UPDATE: As of the 20150315 compiler release, the type based optimizations are enabled by default.
With the help of Chad's answer, I found a bug where my working code like so:
a = [b, b = a][0]; // swap variable values
Was compiled to:
a = b;
It might be fixed in later versions, because tests with the online Closure compiler app don't demonstrate the same bug. Fixed it by not trying to be clever, and using a 3rd variable to hold the old value while swapping values.
A couple problems that I've seen with dojo 1.3 (pre-closure):
If you have a class property named class it needs to be quoted. So {class: "css"} needs to be written as {"class": "css"} this includes any widget fields.
Make sure you remove any debugger statements.