What would I use to find which resources are required by a NodeJS file?
For example, if I had a file called "file.js" containing this:
import x from './x';
const y = require('./y');
// Some more code
How do I parse that file and extract './x' and './y'?
Why would you do this?
I'm playing with the idea of an architectural tool. To do this, I want to know which files are being required by the targeted source code.
I know that Webpack follows this information when it creates bundles, so that it can stack the required files in an appropriate order in a single concatenated (well, minified) file.
I don't need to do the concatenation, but I want to find which files would be used.
When I find out which files are being used by which files, I plan to assist a user in organising them in an orderly manner (e.g. by pointing out circular dependencies).
For trivial cases, you could try feeding the source to some JS parser and search the AST for calls to require(); as long as require() is called with a string constant as a parameter, it shouldn't be hard to determine the dependencies. More complex situations could cause problems, though.
I have a javascript build file, that is unminified; it contains prototype class definitions. Is there any tool i can use to break the build file into separate files, assuming one file per prototype/class definition?
This build file looks like the following, with many objects defined in the following way, all concatenated into the single file. I would like to break this file apart. I don't have access to the original source code, I was basically just given a dump in the form of this build file but its unmanageable in this form as its 10k+ lines of code.
MyClass = function(){
}
MyClass.prototype.foo = function(){
}
I would go with the suggested (in comments) do it manually manner, but if you insist to automate it some way, you can always use javaScript for the job.
One way would be to look up for constructors throw a regex like ([^\.=\s]+[^=]*=[\s]*function[\s]*[(]+[^)]*[)][\s]*[{]*) and extract all 'class'.prototype.'something' from the file, parse the entire file then write each group in separate files after doing any kind of ordering you would prefer.
Another manner would be to use a javaScript parser and group relevant function definitions throw token examination (this one is overkill, but might be interesting for learning purposes).
I have a directory where some other programs write XML files that I have to process when they're complete.
Until now I avoided the handling of incompletely written files by asking the writing programs to first use a temporary name and only at end rename the files in ".xml". My code looks like this :
var fs = require("fs");
var handleFiles = function(){
fs.readdirSync(args.in).forEach(function(filename) {
if (filename.slice(-4)!=='.xml') return;
// handle XML file here
});
}
fs.watch(args.in, handleFiles);
But some new programs I have to support are unable to write with a temporary name.
How can I ensure I handle the files when they're completely written in an efficient, reliable, cross-browser (windows & linux) and not timeout-based (i.e. not testing a rename every 10 ms until it works) way ?
Writing operations are one-shot, so I guess what I want (for linux and more importantly for windows) is to be notified when there are new files not being write-locked.
On linux you could use the inotify facilities. See inotify(7).
Maybe using incrontab could be worthwhile.
One technique for dealing with this is to use a zero-byte "signal file" which is written after the main file is done. For example
mybigfile.txt
mybigfile.txt.done
The ".done" file is 0 bytes long and is created by a "touch" or similar process when it has finished writing the data file. You scan for the .done file and don't do anything if it is missing. Unfortunately you imply that you can't control the writing processes, so this might not be a solution you can use.
I've been reading about the module pattern, but everything I read assumes that the entire contents of the module will be in a single file. I want to have one file per class.
I've resorted to doing this at the top of every file:
if(window.ModuleName === undefined) { window.ModuleName = {}; }
ModuleName.ClassName = function () { ... }
But this allows files to be included without their dependencies, which is also annoying. For example, lets say there is ClassA which uses ClassB, and "ClassB.js" is left out of the HTML, then ClassA will throw up errors. As far as I'm aware Javascript lacks an import statement, so in this case I actually want everything to be in a single file.
I assume that large javascript projects are broken up into multiple files, so there has to be a way around this. How is it generally done? Is there some tool that will combine multiple class files into a single module file?
This is a big topic but let me explain as much as I can. Javascript requires that you have preloaded anything you intended to use, which is why your module pattern has all the "things" in the same file. But if you plan to separate them in different files then you have to manage it before using. I suggest the following approaches
Concatenate them before serving them in the server. For example in jsp, you can create a servlet that returns contenttype = "text/javascript", inside that servlet you can append all the scripts you need in one dynamically generated script then return it to the client.
In your ant or maven builds etc, there are configurations where in you can concatenate them the files you want together. This is a common practice therefore you should find many reference in the internet.
Lazy-load javascripts. This is my preferred way. I use Lazyload javascript library. Basically I declare the dependencies of certain codes much like "import" in Java, then before i call any of them i load their dependencies. This allows for optimized dependency loading without scripts redundancies. The problem is you need to write some fairly complicated scripts to do this.
Hope to help.
In complex client side projects, the number of Javascript files can get very large. However, for performance reasons it's good to concatenate these files, and compress the resulting file for sending over the wire. I am having problems in concatenating these as the dependencies are included after they are needed in some cases.
For instance, there are 2 files:
/modules/Module.js <requires Core.js>
/modules/core/Core.js
The directories are recursively traversed, and Module.js gets included before Core.js, which causes errors. This is just a simple example where dependencies could span across directories, and there could be other complex cases. There are no circular dependencies though.
The Javascript structure I follow is similar to Java packages, where each file defines a single Object (I'm using MooTools, but that's irrelevant). The structure of each javascript file and the dependencies is always consistent:
Module.js
var Module = new Class({
Implements: Core,
...
});
Core.js
var Core = new Class({
...
});
What practices do you usually follow to handle dependencies in projects where the number of Javascript files is huge, and there are inter-file dependencies?
Using directories is clever, however, I think you might run into problems when you have multiple dependencies. I found that I had to create my own solution to handle this. So, I created a dependency management tool that is worth checking out. (Pyramid Dependency Manager documentation)
It does some important things other javascript dependency managers don't do, mainly
Handles other files (including inserting html for views...yes, you can separate your views during development)
Combines the files for you in javascript when you are ready for release (no need to install external tools)
Has a generic include for all html pages. You only have to update one file when a dependency gets added, removed, renamed, etc
Some sample code to show how it works during development.
File: dependencyLoader.js
//Set up file dependencies
Pyramid.newDependency({
name: 'standard',
files: [
'standardResources/jquery.1.6.1.min.js'
]
});
Pyramid.newDependency({
name:'lookAndFeel',
files: [
'styles.css',
'customStyles.css'
]
});
Pyramid.newDependency({
name:'main',
files: [
'createNamespace.js',
'views/buttonView.view', //contains just html code for a jquery.tmpl template
'models/person.js',
'init.js'
],
dependencies: ['standard','lookAndFeel']
});
Html Files
<head>
<script src="standardResources/pyramid-1.0.1.js"></script>
<script src="dependencyLoader.js"></script>
<script type="text/javascript">
Pyramid.load('main');
</script>
</head>
This may be crude, but what I do is keep my separate script fragments in separate files. My project is such that I'm willing to have all my Javascript available for every page (because, after all, it'll be cached, and I'm not noticing performance problems from the parse step). Therefore, at build time, my Ant script runs Freemarker via a little custom Ant task. That tasks roots around the source tree and gathers up all the separate Javascript source files into a group of Maps. There are a few different kinds of sources (jQuery extensions, some page-load operations, so general utilities, and so on), so the task groups those different kinds together (getting its hints as to what's what from the script source directory structure.
Once it's built the Maps, it feeds those into Freemarker. There's a single global template, and via Freemarker all the script fragments are packed into that one file. Then that goes through YUI compressor, and bingo! each page just grabs that one script, and once it's cached there's no more script fetchery over my entire site.
Dependencies, you ask? Well, that Ant task orders my source files by name as it builds those maps, so where I need to ensure definition-use ordering I just prefix the files with numeric codes. (At some point I'm going to spiff it up so that the source files can keep their ordering info, or maybe even explicitly declared dependencies, inside the source in comment blocks or something. I'm not too motivated because though it's a little ugly it really doesn't bother anybody that much.)
There is a very crude dependency finder that I've written based on which I am doing the concatenation. Turns out the fact that its using MooTools is not so irrelevant after all. The solution works great because it does not require maintaining dependency information separately, since it's available within the javascript files itself meaning I can be super lazy.
Since the class and file naming was consistent, class Something will always have the filename Something.js. To find the external dependencies, I'm looking for three things:
does it Implement any other
classes
does it Extend any other
classes
does it instantiate other classes
using the new keyword
A search for the above three patterns in each javascript file gives its dependent classes. After finding the dependent classes, all Javascript files residing in any folder are searched and matched with this class name to figure out where that class is defined. Once the dependencies are found, I build a dependency graph and use the topological sort algorithm to generate the order in which files should be included.
I say just copy and paste this files to a one file in an ordered way. Each file will have a starting and ending comment to distinguish each particular code.
Each time you updated one of the files, you'll need to updated this file. So, this file need to contain only finish libraries, that not going to changes in the near time.
Your directory structure is inverted...
Core dependencies should be in the root and modules are in subdirs.
scripts/core.js
scripts/modules/module1.js
and your problem is solved.
Any further dependency issues will be indicative of defective 'class'/dependency design.
Similar to Mendy, but I create combined files on server-side. The created files will also be minified, and will have a unique name to omit cache issues after an update.
Of course, this practice only makes sense in a whole application or in a framework.
I think your best bet if at all possible, would be to redesign to not have a huge number of javascript files with interfile dependencies. Javascript just wasn't intended to go there.
This is probably too obvious but have you looked at the mootools Core Depender: http://mootools.net/docs/more/Core/Depender
One way to break the parse-time or load-time dependencies is with Self-Defining Objects (a variation on Self-Defining Functions).
Let's say you have something like this:
var obj = new Obj();
Where this line is in someFile.js and Obj is defined in Obj.js. In order for this to parse successfully you must load or concatenate Obj.js before someFile.js.
But if you define obj like this:
var obj = {
init: function() {
obj = new Obj();
}
};
Then at parse or load time it doesn't matter what order you load the two files in as long as Obj is visible at run-time. You will have to call obj.init() in order to get your object into the state you want it, but that's a small price to pay for breaking the dependency.
Just to make it clearer how this works here is some code you can cut and paste into a browser console:
var Obj = function() {
this.func1 = function ( ) {
console.log("func1 in constructor function");
};
this.init = function () {
console.log("init in constructor function");
}
};
var obj = {
init: function() {
console.log("init in original object");
obj = new Obj();
obj.init();
}
};
obj.init();
obj.func1();
And you could also try a module loader like RequireJS.