What I've got
A large (proprietary unfortunately) JS library, the many small modules that get rolled up into it during the build process, the accompanying source map, and over 300 examples that use the built version of the library.
The goal
A form of dependency tracking, I guess? I need to be able to modify one of the small modules, rebuild the large file, and then only re-verify the examples that were affected by this change. Note: I don't care whether this requires static analysis or if I have to run all examples thru a headless browser to extract something or so - I'm fine as long as it can be automated.
What I've tried so far
I've read answers to questions like this and tried pre-existing tools like
Madge, but none of them seem to work for my case. Madge in particular is great for telling me which of the modules depend on which modules, but that's not what I'm looking for. Most solutions online are based on the assumption that you're already using something like require.js or similar on which they can piggy-back, but in my case the library is simply just a giant blob.
My current approach is instrumenting the built version of the library by simply appending every line with something like neededModules["the_file_this_line_comes_from.module.js"] = true similar to how code coverage tools do it. However, that fails because of several parts like this:
Points.prototype = Object.assign( Object.create( Info.prototype ), {
plot: ( function () {
var static = new Background();
return function plot( line, physics ) {
<code>
};
}() ),
copy: function () {
return new this.constructor( this.info, this.history ).copy( this );
}
} );
The copy function is tracked/skipped just fine, but because the plot function is an IIFE(right?), the line var static = new Background(); always gets executed, even if there is absolutely no connection to the Points module.
What I'm looking for
Either some help with my current approach and its problems with IIFE or a different solution altogether. I've seen Facebook's Jest does offer dependency tracking, maybe someone has experience with that one, or there's some way to incorporate the source map?
Again, as long as it's automatable and finishes in let's say < 5 min, I'm totally fine with it no matter if it's static analysis or just some hacky script or whatever :)
Thanks!
Related
In my add-on, I'm using XUL to display dialog windows because I can customize their appearance to suit the add-on's general style (like a custom titlebar).
Using the migration guide, I'm able to do this easily. The thing is, I would like to call certain functions in the add-on's main module from the XUL dialog.
After a bit of searching I found the loader module, which seems to be able to do exactly what I want. But, I'm experiencing trouble in using it to access the main module.
First, I tried the obvious approach as mentioned in the documentation;
xul_dialog.js:
let {Loader} = Components.utils.import('resource://gre/modules/commonjs/toolkit/loader.js');
let loader = Loader.Loader({
paths: {
'toolkit/': 'resource://gre/modules/commonjs/toolkit/',
'': 'resource:///modules/',
'./': 'resource://<my-addon-name>/root/'
}
});
let main = Loader.main(loader, './main');
I got an error that './main' was not found at resource://<my-addon-name>/root/. Figuring that I was using the incorrect paths, I experimented a bit until I could remove all path associated errors.
xul_dialog.js:
let {Loader} = Components.utils.import('resource://gre/modules/commonjs/toolkit/loader.js');
let loader = Loader.Loader({
paths: {
'toolkit/': 'resource://gre/modules/commonjs/toolkit/',
'': 'resource://gre/modules/commonjs/',
'./': 'resource://<my-addon-id>-at-jetpack/<my-addon-name>/lib/'
}
});
let main = Loader.main(loader, './main');
This time I got a rather confusing error at loader.js, line 279.
Components is not available in this context.
Functionality provided by Components may be available in an SDK
module: https://jetpack.mozillalabs.com/sdk/latest/docs/
However, if you still need to import Components, you may use the
`chrome` module's properties for shortcuts to Component properties:
Shortcuts:
Cc = Components.classes
Ci = Components.interfaces
Cu = Components.utils
CC = Components.Constructor
Example:
let { Cc, Ci } = require('chrome');
I get the same error when I use Loader.Require(loader, {id: './main'}) instead of Loader.main. I even tried passing Components as globals when instantiating the loader, but without much luck.
I'm fairly certain that I'm doing a lot of things wrong. I don't understand why I'm getting the error, even after spending quite a bit of time in loader.js. Plus, I also think that there would be a better alternative than having to use the add-on id for the path to main.js; hard-coding it like that doesn't seem right.
Any help would be really appreciated.
What you have to do is to find a specific instance of Loader, not create a new one.
At main.js
const { id, name, prefixURI } = require("#loader/options");
//pass these to the XUL dialog
At xul.js (or whatever is the name of the xul dialog script)
Components.utils.import("resource://gre/modules/addons/XPIProvider.jsm");
var extensionscope = XPIProvider.bootstrapScopes[id];
var mainjssandbox = extensionscope.loader.sandboxes[prefixURI + name + "/lib/main.js"];
Assuming there is a foo function at main.js, you can call it like
mainjssandbox.foo();
Of course don't expect things to work as if XUL and Add-on SDK actually blended into one thing.
If it is your XUL dialog that should interact with your add-on, then please don't use the Loader stuff and in particular do not go the XPIProvider.bootstrapScopes #paa suggested. While this might work (for now), it should be noted that it relies on tons of implementation details that are subject to change at any point making this solution extremely fragile.
Instead there are a couple of other options (not an exhaustive list):
If the SDK part opens the windows, you may use .openDialog which supports passing arguments to the created window, and these arguments can even be objects and functions. Also, you can have the window dispatch (custom) events, and which your SDK part can listen to by calling addEventListener on the window the .openDialog call returns.
If the window is created from somewhere else (e.g. from the AddonManager because of em:optionsURL) then the nsIObserverService is another way to communicate. The window could e.g. .notifyObservers on DOMContentLoaded containing a reference to itself. The SDK parts would just have to observe such notifications by addObserver.
Another way, a bit hacky but working, is the SDK part listening to new windows via nsIWindowWatcher.registerNotification and then injecting some API into e.g. browser.xul windows by XPCNativeWrapper.unwrap(subject.QueryInterface(Ci.nsIDOMWindow)).myAddonAPI = something.
Be sure to handle unloading or your add-on well - it is still restartless -, i.e. reverse any changes you've done and remove any observers and so on again.
If you want to interact with SDK addons not created by you, then the XPIProvider route might be the only feasible. But still, it might be worth contacting the add-on author first asking for the addition of some public API instead of hacking deep into the AddonManager and SDK loader internals.
PS:
Considering passing require or the global scope to your window via openDialog if you want to. Get the global scope by placing this into your main.js:
const globalScope = this;
I use TypeScript to code my javascript file with Object Oriented Programing.
I want to use the node module https://npmjs.org/package/typescript-require to require my .ts files from other files.
I want to share my files in both server and client side. (Browser) And that's very important. Note that the folder /shared/ doesn't mean shared between client and server but between Game server and Web server. I use pomelo.js as framework, that's why.
For the moment I'm not using (successfully) the typescript-require library.
I do like that:
shared/lib/message.js
var Message = require('./../classes/Message');
module.exports = {
getNewInstance: function(message, data, status){
console.log(requireTs);// Global typescript-require instance
console.log(Message);
return new Message(message, data, status);
}
};
This file need the Message.js to create new instances.
shared/classes/Message.ts
class Message{
// Big stuff
}
try{
module.exports = Message;
}catch(e){}
At the end of the fil I add this try/catch to add the class to the module.exports if it exists. (It works, but it's not really a good way to do it, I would like to do better)
If I load the file from the browser, the module.export won't exists.
So, what I did above is working. Now if I try to use the typescript-require module, I'll change some things:
shared/lib/message.js
var Message = requireTs('./../classes/Message.ts');
I use requireTs instead of require, it's a global var. I precise I'm using .ts file.
shared/classes/Message.ts
export class Message{
// Big stuff
}
// remove the compatibility script at the end
Now, if I try like this and if I take a look to the console server, I get requireTs is object and Message is undefined in shared/lib/message.js.
I get the same if I don't use the export keyword in Message.ts. Even if I use my little script at the end I get always an error.
But there is more, I have another class name ValidatorMessage.ts which extends Message.ts, it's not working if I use the export keyword...
Did I did something wrong? I tried several other things but nothing is working, looks like the typescript-require is not able to require .ts files.
Thank you for your help.
Looking at the typescript-require library, I see it hasn't been updated for 9 months. As it includes the lib.d.ts typing central to TypeScript (and the node.d.ts typing), and as these have progressed greatly in the past 9 months (along with needed changes due to language updates), it's probably not compatible with the latest TypeScript releases (just my assumption, I may be wrong).
Sharing modules between Node and the browser is not easy with TypeScript, as they both use very different module systems (CommonJS in Node, and typically something like RequireJS in the browser). TypeScript emits code for one or the other, depending on the --module switch given. (Note: There is a Universal Module Definition (UMD) pattern some folks use, but TypeScript doesn't support this directly).
What goals exactly are you trying to achieve, and I may be able to offer some guidance.
I am doing the same and keep having issues whichever way I try to do things... The main problems for me are:
I write my typescript as namespaces and components, so there is no export module with multiple file compilation you have to do a hack to add some _exporter.ts at the end to add the export for your library-output.js to be importable as a module, this would require something like:
module.exports.MyRootNamespace = MyRootNamespace
If you do the above it works, however then you get the issue of when you need to reference classes from other modules (such as MyRootNamespace1.SomeClass being referenced by MyRootNamespace2.SomeOtherClass) you can reference it but then it will compile it into your library-output2.js file so you end up having duplicates of classes if you are trying to re-use typescript across multiple compiled targets (like how you would have 1 solution in VS and multiple projects which have their own dll outputs)
Assuming you are not happy with hacking the exports and/or duplicating your references then you can just import them into the global scope, which is a hack but works... however then when you decide you want to test your code (using whatever nodejs testing framework) you will need to mock out certain things, and as the dependencies for your components may not be included via a require() call (and your module may depend upon node_modules which are not really usable with global scope hacking) and this then makes it difficult to satisfy dependencies and mock certain ones, its like an all or nothing sort of approach.
Finally you can try to mitigate all these problems by using a typescript framework such as appex which allows you to run your typescript directly rather than the compile into js first, and while it seems very good up front it is VERY hard to debug compilation errors, this is currently my preferred way but I have an issue where my typescript compiles fine via tsc, but just blows up with a max stack size exception on appex, and I am at the mercy of the project maintainer to fix this (I was not able to find the underlying issue). There are also not many of these sort of projects out there however they make the issue of compiling at module level/file level etc a moot point.
Ultimately I have had nothing but problems trying to wrestle with Typescript to get it to work in a way which is maintainable and testable. I also am trying to re-use some of the typescript components on the clientside however if you go down the npm hack route to get your modules included you then have to make sure your client side uses a require compatible resource/package loader. As much as I would love to just use typescript on my client and my server projects, it just does not seem to want to work in a nice way.
Solution here:
Inheritance TypeScript with exported class and modules
Finally I don't use require-typescript but typescript.api instead, it works well. (You have to load lib.d.ts if you use it, else you'll get some errors on the console.
I don't have a solution to have the script on the browser yet. (Because of export keyword I have some errors client side) I think add a exports global var to avoid errors like this.
Thank you for your help Bill.
Original Question
This maybe a stupid question but is there a way in VS 2013 to unminify JavaScript?
Just making sure we are all on the same page here.
Minify:
var flashVer=-1;if(navigator.plugins!=null&&navigator.plugins.length>0){if(navigator.plugins["Shockwave Flash 2.0"]||navigator.plugins["Shockwave Flash"]){var swVer2=navigator.plugins["Shockwave Flash 2.0"]?"
That's just an example to make sure we all know what I'm on about. As far as I can tell there is no way to be able to do this. I have only been using VS 2013 for around 3 weeks so there is probably still stuff that is hidden to me.
If there is no way to do this within the program what is the next best thing for this?
I did see on another similar post that recommends the site http://jsbeautifier.org/ , so may have to give that ago but would make life easier if it was built into VS 2013
Thanks in advance as I know someone will be able to help me out here.
Update:
I have looked around VS 2013 and found nothing that can help me with this problem, like I said before they maybe some things I have missed (certain settings) so I guess if it cannot be done in VS what's the next best thing for the job? I seem to run into a fair amount of JS that is minifed and would like the quickest and best way to get the job done. I couple sites I have tried seem to have problems with it, is there a program I could install that would just allow me to short cut it with a hot-key or something. That would be pretty handy.
Update 2:
So I think its safe to say this cannot be done within VS2013, or for that matter at all due to missing var names and so on. So I have seen a few links and programs that allow you to format the code. Is there a way to do with within VS2013? And again if not what is the most reliable website/program that I can use to do this. Like I said I can see there have been answers and I appreciate all of them. I will be leaving this question open for a while to get more people to look at it and possibly give a better answer. Keep it up guys!
Update 3:
If anyone has any more information on this please do share. I am still looking around now and then waiting for someone to come up with something amazing for this. One day people.... One day!
The thing is that you cannot really "unminify" your code since some data was already lost - e.g. variable names. You can reformat it to more readable form though.
According to this question, since VisualStudio 2012 you can just use Ctrl+E, D keyboard shortcut
If the above is not right, there is this extension for VS 2010: http://visualstudiogallery.msdn.microsoft.com/41a0cc2f-eefd-4342-9fa9-3626855ca22a but I am not sure if it works with VS 2013
There is an extension to VisualStudio called ReSharper which can reformat javascript in a few different manners.
Also there are online formatters already mentioned in other answers (if your code is confidential, I would advise some paranoia manifested by downloading sources and using them locally).
Also you may always try to find unminified version of desired library on the interwebs
Also, there is the WebStorm IDE from JetBrains that is able to reformat JS - you may download a trial for the sole purpose of reformatting your minified scripts :)
If that's just to make debugging easier, you may want to use source maps
Also, here is a bunch of related questions:
How to automatically indent source code? <-- this is for VS2010, but it looks promising, maybe it will help you if it supports JavaScript (and it does since VS2012 according to MS support):
Ctrl+E, D - Format whole doc
Ctrl+K, Ctrl+F - Format selection
reindent(reformat) minimized jquery/javascript file in visual studio
Visual Studio 2010 can't format complex JavaScript documents
Visual Studio code formatter
how to make visual studio javascript formatting work?
I am not sure if they figured out a working way to reformat JS, but I've seen a few answers which might be helpful - I am just pasting this in here just FYI.
Added 03.06.2014:
http://www.jsnice.org/
This tool could be useful too, it even tries to infer minified names. As stated on their website:
We will rename variables and parameters to names that we learn from thousands of open source projects.
Personally I can't think of a reason to ever unminify code^:
If you're using a compiled js file (a-la google closure) and want more readable code to debug, use source maps available for well-supported libraries (speaking of jQuery, if it is served from a google CDN it already maps to the correct source)
If you're using a whitespace-only minified js file and want more readable code to debug, you could just toggle pretty print in-browser. This seems to best fit your question.
If you're using either of the above and want to modify the source code for a third-party js file, don't. Any future release will cancel out your change - instead consider one of the many patterns to extend a framework (or, perhaps, do some duck punching depending on the exact scenario.)
The other answers seem to cover the "unminification" process (maxification?) well, but it's worth making sure it's a necessary step first.
^ - Except when version control falls over, there are no backups and the only version of the file left is a minified copy in browser cache. Don't ask.
Its just a one way transformation .... sorry in normal cases you will not get something understandable back from minified JavaScript !
Make just a quick look at JQuery source for a second:
(function( window, undefined ) {
// Can't do this because several apps including ASP.NET trace
// the stack via arguments.caller.callee and Firefox dies if
// you try to trace through "use strict" call chains. (#13335)
// Support: Firefox 18+
//"use strict";
var
// The deferred used on DOM ready
readyList,
// A central reference to the root jQuery(document)
rootjQuery,
// Support: IE<10
// For `typeof xmlNode.method` instead of `xmlNode.method !== undefined`
core_strundefined = typeof undefined,
// Use the correct document accordingly with window argument (sandbox)
location = window.location,
document = window.document,
docElem = document.documentElement,
// Map over jQuery in case of overwrite
_jQuery = window.jQuery,
// Map over the $ in case of overwrite
_$ = window.$,
// [[Class]] -> type pairs
class2type = {},
// List of deleted data cache ids, so we can reuse them
core_deletedIds = [],
core_version = "1.10.2",
------
And now at the minify source:
(function(e,t){var n,r,i=typeof t,o=e.location,a=e.document,s=a.documentElement,
l=e.jQuery,u=e.$,c={},p=[],f="1.10.2", ....
I think now you see it =>
window => e
undefined => t
readyList => n
rootjQuery => r
core_strundefined => i
location => o
document => a
So its mapped somehow to make it more shorter look here to minify something
People normally use this so there is no way back
you can just format it look here
If the code has only been minified then the best you can do automatically is reformat to make it more readable. One way of doing this is using an online formatter/beautifier. E.g. Copy and paste the line of code you posted into http://jsbeautifier.org/ or http://www.jspretty.com/ and it'll produce something like this:
var flashVer = -1;
if (navigator.plugins != null && navigator.plugins.length > 0) {
if (navigator.plugins["Shockwave Flash 2.0"]
|| navigator.plugins["Shockwave Flash"]) {
var swVer2 = navigator.plugins["Shockwave Flash 2.0"] ? ""
But of course what these don't do is put any comments that have been removed by the minifier back in. And if the code has also been obfuscated then it will be a lot less readable since the variable names will have changed (e.g. var a instead of var flashVer). See here for further details.
As you can see from the other answers, there is no way to reconstitute minified Javascript back into its original form, it is a lossy compression. The best you can do is make it readable by reformatting it.
If the code is open source, then it is likely that the code will exists in a raw state on some form of version control site or as a zip. Why not just download the raw version if available?
There is an online tool to unminify Javascripts
http://jsbeautifier.org/
And also for CSS
http://mrcoles.com/blog/css-unminify/
I am trying to unit test a JavaScript web tracking function because we intend to expose it as a library and want to be sure its interface remains stable. JavaScript web tracking works similarly to image preloading like this:
if (document.images) {
var image = new Image();
image.onload = function () {
...
};
image.src = pixelUrl;
}
When in a unit test (with js-testdriver) I am now trying to validate that an image has been created with the right pixelUrl I thought I had to do something like this:
call_the_reporting_function();
assertNotUndefined(document.images[0]);
assertNotUndefined(document.images[0].src);
My problem is now that the first assertion already fails. It seems that the new image is not added to document.images immediately.
Does anybody has an idea how to do proper unit testing on such things in JavaScript?
Thanks
Stefan
One possible solution, which is the way we just took, is to circumvent the problem by extracting the part above doing the html manipulation and mocking it.
While this solution is fine, it certainly has at least two downsides:
- there is no point using a browser-based unit test framework (like jsTestDriver, which is, what we are using) if you do not use the browser-specific parts
- the code is untested and you have to ensure that it works either manually or using runtime-expensive Selenium tests (where you can configure a proxy before the HttpClient and simply catch all web tracking requests issued by the browser. Assertions are done on the caught requests)
We develop an application in an embedded environment. It is a high level computing environment with a complete webbrowser on top of a busybox Linux system. The only exception is that the system has a limited amount of system memory.
Our application is built in JavaScript and runs inside a Webkit based webbrowser and consists of a lot of javascript modules that are loaded in sequence (Which is not very efficient).
Some modules provide common functionality that is used by several modules. We are in the process of converting our current javascript loader with requirejs, but there is one specific need we have to address first.
Is it possible to unload a module when it has been loaded using requirejs? Assume that we dynamically loads a module using :
require(["somemodule.js"], function(m) { m.run(); } );
That works well for loading and running 'somemodule' and also resolving all dependencies for 'somemodule' and the requirejs framework will store a reference to 'somemodule' for future requests.
If we at some point need to reclaim memory, e.g to be able to load and run an infinite number of modules, we have to start removing some of them after some time. Is that possible with requirejs without altering the internal implementation?
Has anyone dealt with this kind of problem before? Most single page JS apps runs in a webbrowser on a desktop PC where memory usage usually is not a major concern.
RequireJS does not have a built-in unload feature, but it could be added perhaps as an additional part you could build into it. If you would like to have that feature, feel free to propose it in the mailing list or as a GitHub issue.
If you wanted to experiment to see if it helps your situation, what you need to do is the following:
1) Remove the defined module from the RequireJS module cache. If you are not using the multiversion support, you can do something like:
var context = require.s.contexts['_'];
delete context.defined[moduleName];
delete context.specified[moduleName];
delete context.loaded[moduleName];
2) Then you can try removing the script tag to see if that helps:
var scripts = document.getElementsByTagName('script');
for (var i = scripts.length - 1; i >= 0; i--) {
var script = scripts[i];
if (script.getAttribute('data-requiremodule') === moduleName) {
script.parentNode.removeChild(script);
break;
}
}
Note that the module may not be garbage collected if another module holds on to it via the closure function(){} that defines that other module. That other module would need to be removed too.
You can try to limit that impact by not passing in the module as a function argument, but just use require("somemodule") inside the function definition whenever you want to get a hold of dependent modules, and not holding on to that require return value for too long.
Also, in your example above, for modules that use require.def to define themselves, it should look like this (without the .js suffix):
require(["somemodule"], function(m) { m.run(); } );
Try this: require.undef(moduleName)