Goal
I am the author of a JavaScript library which can be consumed through AMD or ESM within various runtime environments (Browser, Node.js, Dev Servers). My library needs to spawn WebWorkers and AudioWorklets using the file it is contained. The library detects in which context it is running and sets up the required stuff for the execution context.
This works fine as long users (user=integrator of my library) do not bring bundlers like WebPack into the game. To spawn the WebWorker and AudioWorklet I need the URL to file in which my library is contained in and I need to ensure the global initialization routines of my library are called.
I would prefer to do as much as possible of the heavy lifting within my library, and not require users to do a very specialized custom setup only for using my library. Offloading this work to them typically backfires instantly and people open issues asking for help on integrating my library into their project.
Problem 1: I am advising my users to ensure my library is put into an own chunk. Users might setup the chunks based on their own setup as long the other libs don't cause any troubles or side effects in the workers. Especially modern web frameworks like React, Angular and Vue.js are typical problem children here, but also people tried to bundle my library with jQuery and Bootstrap. All these libraries cause runtime errors when included in Workers/Worklets.
The chunking is usually done with some WebPack config like:
config.optimization.splitChunks.cacheGroups.alphatab = {
chunks: 'all',
name: 'chunk-mylib',
priority: config.optimization.splitChunks.cacheGroups.defaultVendors.priority + 10,
test: /.*node_modules.*mylib.*/
};
The big question mylib now has: What is the absolute URL of the generated chunk-mylib.js as this is now the quasi-entrypoint to my library now with bundling and code splitting in place:
document.currentScript points usually to some entry point like an app.js and not the chunks.
__webpack_public_path__ is pointing to whatever the user sets it to in the webpack config.
__webpack_get_script_filename__ could be used if the chunk name would be known but I haven't found a way to get the name of the chunk my library is contained in.
import.meta.url is pointing to some absolute file:// url of the original .mjs of my library.
new URL(import.meta.url, import.meta.url) causes WebPack to generate an additional .mjs file with some hash. This additional file is not desired and also the generated .mjs contains some additional code breaking its usage in browsers.
I was already thinking to maybe create a custom WebPack plugin which can resolve the chunk my library is contained in so I can use it during runtime. I would prefer to use as much built-in features as possible.
Problem 2: Assuming that problem 1 is solved I could now spawn a new WebWorker and AudioWorklet with the right file. But as my library is wrapped into a WebPack module my initialization code will not be executed. My library only lives in a "chunk" and is not an entry and I wouldn't know that this splitting would allow mylib to run some code after the chunk was loaded by the browser.
Here I am rather clueless. Maybe chunks are not the right way of splitting for this purpose. Maybe some other setup is needed I am not yet aware of that its possible?
Maybe also this could be done best with a custom WebPack plugin.
Visual Representation of the problem: With the proposed chunking rule we get an output as shown in the blocks. Problem 1 is the red part (how to get this URL) and Problem 2 is the orange part (how to ensure my startup logic is called when the background worker/worklet starts)
Actual Project I want to share my actual project for better understanding of my use case. I am talking about my project alphaTab, a music notation rendering and playback library. On the Browser UI thread (app.js) people integrate the component into the UI and they get an API object to interact with the component. One WebWorker does the layouting and rendering of the music sheet, a second one synthesizes the audio samples for playback and the AudioWorklet sends the buffered samples to the audio context for playback.
I think the worker code should be handled as an assets instead of a source code. Maybe you could add a simple CLI to generate a ".alphaTab" folder on the root of the project and add instructions for your user to copy that to the "dist"or "public folder".
Even if come up with a Webpack specific solution, you would have to work your way around other bundlers/setups (Vite, rollup, CRA, etc).
EDIT: You would also need to add an optional parameter to the initialization for passing the script path. Not fully automated, but simpler that having to setup complex bundler configs
Disabling import.meta
Regarding import.meta.url, this link might help. It looks like you'd disable it in your webpack config by setting module.parser.javascript.importMeta to false.
Reworking Overall Architecture
For the rest, it sounds like a bit of a mess. You probably shouldn't be trying to import the same exact chunk code into your workers/worklets, since this is highly dependent on how webpack generates and consumes chunks. Even if you manage to get it to work today, it might break in the future if the webpack team changes how they internally represent chunks.
Also from a user's perspective, they just want to import the library and have it just work without fiddling with all of the different build steps.
Instead, a cleaner way would to be to generate separate files for the main library, the AudioWorklet, and the Web Worker. And since you already designed the worklet and web worker to use your library, you can just use the prebuilt, non-module library for them, and have a separate file for the entry point for webpack/other bundlers.
The most straightforward way would be to have users add your original non-module js library in with the bundle that they build, and have the es module load Web Workers and Audio Worklets using that non-module library's url.
Of course, from a user's perspective, it'd be easier if they didn't have to copy over additional files and put them in the right directory (or configure a scripts directory). The straightforward way would be to load the web worker or worklet from a CDN (like https://unpkg.com/#coderline/alphatab#1.2.2/dist/alphaTab.js), but there are restrictions from loading web workers cross origin, so you'd have to use a work around like fetching it and then loading it from a blob url (like that found here). This unfortunately makes initializing the Worker/Worklet asynchronous.
Bundling Worker code
If this isn't an option, you can bundle a library, Web Worker/Worklet code into one file by stringifying the Worker/Worklet code and loading it via a blob or data url. In your particular use case, it's a little painful from an efficiency standpoint considering how much code will be duplicated in the bundled output.
For this approach, you'd have multiple build steps:
Build the library that's used by your Web Worker and/or Audio Worklet.
Build the single library by stringifying the previous libraries/library.
This is all complicated by there being only one entry file for the library, web worker, and audio worklet. In the long term, you'd probably benefit by rewriting entry points for these different targets, but for now, we could reuse the current workflow and change the build steps by using different plugins. For the first build, we'll make a plugin that returns a dummy string when it tries to import the worker library, for the second, we'll have it return the stringified contents of that library. I'll use rollup, since that's what your project uses. The code below is mostly for illustrative purposes (which saves the worker library as dist/worker-library.js); I haven't actually tested it.
First plugin:
var firstBuildPlugin = {
load(id) {
if (id.includes('worker-library.js')) {
return 'export default "";';
}
return null;
}
}
Second plugin:
var secondBuildPlugin = {
transform(code, id) {
if (id.includes('worker-library.js')) {
return {
code: 'export default ' + JSON.stringify(code) + ';',
map: { mappings: '' }
};
}
return null;
}
}
Using these plugins, we can import the web worker/audio worklet library via import rawCode from './path/to/worker-library.js';. For your case, since you'd be reusing the same library, you may want to create a new file with an export, so the to prevent multiple bundling of the same code:
libraryObjectURL.js:
import rawCode from '../dist/worker-library.js'; // may need to tweak the path here
export default URL.createObjectURL(
new Blob([rawCode], { type: 'application/javascript' })
);
And to actually use it:
import libraryObjectURL from './libraryObjectURL.js'; // may need to tweak the path here
//...
var worker = new Worker(libraryObjectURL);
To then actually build it, your rollup.config.js would look something like:
module.exports = [
{
input: `dist/lib/alphatab.js`,
output: {
file: `dist/worker-library.js`,
format: 'iife', // or maybe umd
//...
plugins: [
firstBuildPlugin,
//...
]
}
},
{
input: `dist/lib/alphatab.js`,
output: {
file: `dist/complete-library.mjs`,
format: 'es',
//...
plugins: [
secondBuildPlugin,
//...
]
}
},
// ...
Preserving old code
Finally, for your other builds, you may still want to preserve the old paths. You can use #rollup/plugin-replace for this, by using a placeholder that will be replaced in the build process.
In your files, you could replace:
var worker = new Worker(libraryObjectURL);
with:
var worker = new Worker(__workerLibraryURL__);
and in the build process use:
// ...
// for the first build:
plugins: [
firstBuildPlugin,
replace({ __workerLibraryURL__: 'libraryObjectURL')
// ...
],
// ...
// for the second build:
plugins: [
secondBuildPlugin,
replace({ __workerLibraryURL__: 'libraryObjectURL')
// ...
],
// ...
// for all other builds:
plugins: [
firstBuildPlugin,
replace({ __workerLibraryURL__: 'new URL(import.meta.url)') // or whatever the old code was
// ...
],
You may need to use another replacement for your AudioWorklet url if it's different. In cases where the worker-library file isn't used, the imported libraryObjectURL will be tree shook out.
Future work:
You may want to look into having multiple outputs for your different targets: web worker, audio worklet, and library code. They really aren't supposed load the same exact file. This would negate the need for the first plugin (that ignores certain files), and it might make things more manageable and efficient.
More Reading:
Loading files as raw strings (you can see/use TrySound's plugin here; it's a simple plugin)
Loading strings as blob or data URLs (see https://stackoverflow.com/a/10372280/)
I found a way to solve the described problem but there are still some open pain points because the WebPack devs are rather trying to avoid vendor specific expressions and prefer to rely on "recognizable syntax constructs" to rewrite the code as they see fit.
The solution does not work in a fully local environment, but it works together with NPM:
I am launching my worker now with /* webpackChunkName: "alphatab.worker" */ new Worker(new URL('#coderline/alphatab', import.meta.url))) where #coderline/alphatab is the name of the library installed through NPM. This syntax construct is detected by WebPack and will trigger generation of a new special JS file containing some WebPack bootstrapper/entry-point which loads the library for startup. So effectively it looks after compilation like this:
For this to work, users should configure the WebPack to place the library in an own chunk. Otherwise it can happen that the library is maybe inlined into the webpack generated worker file instead of also loaded from a common chunk. It would work also without a common chunk, but it would defy the benefits of even using webpack because it duplicates the code of the library to spawn it as worker (double loading time and double disk usage).
Unfortunately this currently only works for Web Workers for now because WebPack has no support for Audio Worklet at this point.
Also there are some warnings due to cyclic dependencies produced by WebPack because there seem to be a a cycle between chunk-alphatab.js and alphatab.worker.js. In this setup it should not be a problem.
In my case there is no difference between the UI thread code and the one running in the worker. If users decide to render to an HTML5 canvas through a setting, rendering happens in the UI thread, and if SVG rendering is used it is off-loaded to a worker. The whole layout and rendering pipeline is the same on both sides.
In complex client side projects, the number of Javascript files can get very large. However, for performance reasons it's good to concatenate these files, and compress the resulting file for sending over the wire. I am having problems in concatenating these as the dependencies are included after they are needed in some cases.
For instance, there are 2 files:
/modules/Module.js <requires Core.js>
/modules/core/Core.js
The directories are recursively traversed, and Module.js gets included before Core.js, which causes errors. This is just a simple example where dependencies could span across directories, and there could be other complex cases. There are no circular dependencies though.
The Javascript structure I follow is similar to Java packages, where each file defines a single Object (I'm using MooTools, but that's irrelevant). The structure of each javascript file and the dependencies is always consistent:
Module.js
var Module = new Class({
Implements: Core,
...
});
Core.js
var Core = new Class({
...
});
What practices do you usually follow to handle dependencies in projects where the number of Javascript files is huge, and there are inter-file dependencies?
Using directories is clever, however, I think you might run into problems when you have multiple dependencies. I found that I had to create my own solution to handle this. So, I created a dependency management tool that is worth checking out. (Pyramid Dependency Manager documentation)
It does some important things other javascript dependency managers don't do, mainly
Handles other files (including inserting html for views...yes, you can separate your views during development)
Combines the files for you in javascript when you are ready for release (no need to install external tools)
Has a generic include for all html pages. You only have to update one file when a dependency gets added, removed, renamed, etc
Some sample code to show how it works during development.
File: dependencyLoader.js
//Set up file dependencies
Pyramid.newDependency({
name: 'standard',
files: [
'standardResources/jquery.1.6.1.min.js'
]
});
Pyramid.newDependency({
name:'lookAndFeel',
files: [
'styles.css',
'customStyles.css'
]
});
Pyramid.newDependency({
name:'main',
files: [
'createNamespace.js',
'views/buttonView.view', //contains just html code for a jquery.tmpl template
'models/person.js',
'init.js'
],
dependencies: ['standard','lookAndFeel']
});
Html Files
<head>
<script src="standardResources/pyramid-1.0.1.js"></script>
<script src="dependencyLoader.js"></script>
<script type="text/javascript">
Pyramid.load('main');
</script>
</head>
This may be crude, but what I do is keep my separate script fragments in separate files. My project is such that I'm willing to have all my Javascript available for every page (because, after all, it'll be cached, and I'm not noticing performance problems from the parse step). Therefore, at build time, my Ant script runs Freemarker via a little custom Ant task. That tasks roots around the source tree and gathers up all the separate Javascript source files into a group of Maps. There are a few different kinds of sources (jQuery extensions, some page-load operations, so general utilities, and so on), so the task groups those different kinds together (getting its hints as to what's what from the script source directory structure.
Once it's built the Maps, it feeds those into Freemarker. There's a single global template, and via Freemarker all the script fragments are packed into that one file. Then that goes through YUI compressor, and bingo! each page just grabs that one script, and once it's cached there's no more script fetchery over my entire site.
Dependencies, you ask? Well, that Ant task orders my source files by name as it builds those maps, so where I need to ensure definition-use ordering I just prefix the files with numeric codes. (At some point I'm going to spiff it up so that the source files can keep their ordering info, or maybe even explicitly declared dependencies, inside the source in comment blocks or something. I'm not too motivated because though it's a little ugly it really doesn't bother anybody that much.)
There is a very crude dependency finder that I've written based on which I am doing the concatenation. Turns out the fact that its using MooTools is not so irrelevant after all. The solution works great because it does not require maintaining dependency information separately, since it's available within the javascript files itself meaning I can be super lazy.
Since the class and file naming was consistent, class Something will always have the filename Something.js. To find the external dependencies, I'm looking for three things:
does it Implement any other
classes
does it Extend any other
classes
does it instantiate other classes
using the new keyword
A search for the above three patterns in each javascript file gives its dependent classes. After finding the dependent classes, all Javascript files residing in any folder are searched and matched with this class name to figure out where that class is defined. Once the dependencies are found, I build a dependency graph and use the topological sort algorithm to generate the order in which files should be included.
I say just copy and paste this files to a one file in an ordered way. Each file will have a starting and ending comment to distinguish each particular code.
Each time you updated one of the files, you'll need to updated this file. So, this file need to contain only finish libraries, that not going to changes in the near time.
Your directory structure is inverted...
Core dependencies should be in the root and modules are in subdirs.
scripts/core.js
scripts/modules/module1.js
and your problem is solved.
Any further dependency issues will be indicative of defective 'class'/dependency design.
Similar to Mendy, but I create combined files on server-side. The created files will also be minified, and will have a unique name to omit cache issues after an update.
Of course, this practice only makes sense in a whole application or in a framework.
I think your best bet if at all possible, would be to redesign to not have a huge number of javascript files with interfile dependencies. Javascript just wasn't intended to go there.
This is probably too obvious but have you looked at the mootools Core Depender: http://mootools.net/docs/more/Core/Depender
One way to break the parse-time or load-time dependencies is with Self-Defining Objects (a variation on Self-Defining Functions).
Let's say you have something like this:
var obj = new Obj();
Where this line is in someFile.js and Obj is defined in Obj.js. In order for this to parse successfully you must load or concatenate Obj.js before someFile.js.
But if you define obj like this:
var obj = {
init: function() {
obj = new Obj();
}
};
Then at parse or load time it doesn't matter what order you load the two files in as long as Obj is visible at run-time. You will have to call obj.init() in order to get your object into the state you want it, but that's a small price to pay for breaking the dependency.
Just to make it clearer how this works here is some code you can cut and paste into a browser console:
var Obj = function() {
this.func1 = function ( ) {
console.log("func1 in constructor function");
};
this.init = function () {
console.log("init in constructor function");
}
};
var obj = {
init: function() {
console.log("init in original object");
obj = new Obj();
obj.init();
}
};
obj.init();
obj.func1();
And you could also try a module loader like RequireJS.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
As JavaScript frameworks like jQuery make client side web applications richer and more functional, I've started to notice one problem...
How in the world do you keep this organized?
Put all your handlers in one spot and write functions for all the events?
Create function/classes to wrap all your functionality?
Write like crazy and just hope it works out for the best?
Give up and get a new career?
I mention jQuery, but it's really any JavaScript code in general. I'm finding that as lines upon lines begin to pile up, it gets harder to manage the script files or find what you are looking for. Quite possibly the biggest propblems I've found is there are so many ways to do the same thing, it's hard to know which one is the current commonly accepted best practice.
Are there any general recommendations on the best way to keep your .js files as nice and neat as the rest of your application? Or is this just a matter of IDE? Is there a better option out there?
EDIT
This question was intended to be more about code organization and not file organization. There has been some really good examples of merging files or splitting content around.
My question is: what is the current commonly accepted best practice way to organize your actual code? What is your way, or even a recommended way to interact with page elements and create reuseable code that doesn't conflict with each other?
Some people have listed namespaces which is a good idea. What are some other ways, more specifically dealing with elements on the page and keeping the code organized and neat?
It would be a lot nicer if javascript had namespaces built in, but I find that organizing things like Dustin Diaz describes here helps me a lot.
var DED = (function() {
var private_var;
function private_method()
{
// do stuff here
}
return {
method_1 : function()
{
// do stuff here
},
method_2 : function()
{
// do stuff here
}
};
})();
I put different "namespaces" and sometimes individual classes in separate files. Usually I start with one file and as a class or namespace gets big enough to warrant it, I separate it out into its own file. Using a tool to combine all you files for production is an excellent idea as well.
I try to avoid including any javascript with the HTML. All the code is encapsulated into classes and each class is in its own file. For development, I have separate <script> tags to include each js file, but they get merged into a single larger package for production to reduce the overhead of the HTTP requests.
Typically, I'll have a single 'main' js file for each application. So, if I was writing a "survey" application, i would have a js file called "survey.js". This would contain the entry point into the jQuery code. I create jQuery references during instantiation and then pass them into my objects as parameters. This means that the javascript classes are 'pure' and don't contain any references to CSS ids or classnames.
// file: survey.js
$(document).ready(function() {
var jS = $('#surveycontainer');
var jB = $('#dimscreencontainer');
var d = new DimScreen({container: jB});
var s = new Survey({container: jS, DimScreen: d});
s.show();
});
I also find naming convention to be important for readability. For example: I prepend 'j' to all jQuery instances.
In the above example, there is a class called DimScreen. (Assume this dims the screen and pops up an alert box.) It needs a div element that it can enlarge to cover the screen, and then add an alert box, so I pass in a jQuery object. jQuery has a plug-in concept, but it seemed limiting (e.g. instances are not persistent and cannot be accessed) with no real upside. So the DimScreen class would be a standard javascript class that just happens to use jQuery.
// file: dimscreen.js
function DimScreen(opts) {
this.jB = opts.container;
// ...
}; // need the semi-colon for minimizing!
DimScreen.prototype.draw = function(msg) {
var me = this;
me.jB.addClass('fullscreen').append('<div>'+msg+'</div>');
//...
};
I've built some fairly complex appliations using this approach.
You can break up your scripts into separate files for development, then create a "release" version where you cram them all together and run YUI Compressor or something similar on it.
Inspired by earlier posts I made a copy of Rakefile and vendor directories distributed with WysiHat (a RTE mentioned by changelog) and made a few modifications to include code-checking with JSLint and minification with YUI Compressor.
The idea is to use Sprockets (from WysiHat) to merge multiple JavaScripts into one file, check syntax of the merged file with JSLint and minify it with YUI Compressor before distribution.
Prerequisites
Java Runtime
ruby and rake gem
You should know how to put a JAR into Classpath
Now do
Download Rhino and put the JAR ("js.jar") to your classpath
Download YUI Compressor and put the JAR (build/yuicompressor-xyz.jar) to your classpath
Download WysiHat and copy "vendor" directory to the root of your JavaScript project
Download JSLint for Rhino and put it inside the "vendor" directory
Now create a file named "Rakefile" in the root directory of the JavaScript project and add the following content to it:
require 'rake'
ROOT = File.expand_path(File.dirname(__FILE__))
OUTPUT_MERGED = "final.js"
OUTPUT_MINIFIED = "final.min.js"
task :default => :check
desc "Merges the JavaScript sources."
task :merge do
require File.join(ROOT, "vendor", "sprockets")
environment = Sprockets::Environment.new(".")
preprocessor = Sprockets::Preprocessor.new(environment)
%w(main.js).each do |filename|
pathname = environment.find(filename)
preprocessor.require(pathname.source_file)
end
output = preprocessor.output_file
File.open(File.join(ROOT, OUTPUT_MERGED), 'w') { |f| f.write(output) }
end
desc "Check the JavaScript source with JSLint."
task :check => [:merge] do
jslint_path = File.join(ROOT, "vendor", "jslint.js")
sh 'java', 'org.mozilla.javascript.tools.shell.Main',
jslint_path, OUTPUT_MERGED
end
desc "Minifies the JavaScript source."
task :minify => [:merge] do
sh 'java', 'com.yahoo.platform.yui.compressor.Bootstrap', '-v',
OUTPUT_MERGED, '-o', OUTPUT_MINIFIED
end
If you done everything correctly, you should be able to use the following commands in your console:
rake merge -- to merge different JavaScript files into one
rake check -- to check the syntax of your code (this is the default task, so you can simply type rake)
rake minify -- to prepare minified version of your JS code
On source merging
Using Sprockets, the JavaScript pre-processor you can include (or require) other JavaScript files. Use the following syntax to include other scripts from the initial file (named "main.js", but you can change that in the Rakefile):
(function() {
//= require "subdir/jsfile.js"
//= require "anotherfile.js"
// some code that depends on included files
// note that all included files can be in the same private scope
})();
And then...
Take a look at Rakefile provided with WysiHat to set the automated unit testing up. Nice stuff :)
And now for the answer
This does not answer the original question very well. I know and I'm sorry about that, but I've posted it here because I hope it may be useful to someone else to organize their mess.
My approach to the problem is to do as much object-oriented modelling I can and separate implementations into different files. Then the handlers should be as short as possible. The example with List singleton is also nice one.
And namespaces... well they can be imitated by deeper object structure.
if (typeof org === 'undefined') {
var org = {};
}
if (!org.hasOwnProperty('example')) {
org.example = {};
}
org.example.AnotherObject = function () {
// constructor body
};
I'm not big fan of imitations, but this can be helpful if you have many objects that you would like to move out of the global scope.
The code organization requires adoption of conventions and documentation standards:
1. Namespace code for a physical file;
Exc = {};
2. Group classes in these namespaces javascript;
3. Set Prototypes or related functions or classes for representing real-world objects;
Exc = {};
Exc.ui = {};
Exc.ui.maskedInput = function (mask) {
this.mask = mask;
...
};
Exc.ui.domTips = function (dom, tips) {
this.dom = gift;
this.tips = tips;
...
};
4. Set conventions to improve the code. For example, group all of its internal functions or methods in its class attribute of an object type.
Exc.ui.domTips = function (dom, tips) {
this.dom = gift;
this.tips = tips;
this.internal = {
widthEstimates: function (tips) {
...
}
formatTips: function () {
...
}
};
...
};
5. Make documentation of namespaces, classes, methods and variables. Where necessary also discuss some of the code (some FIs and Fors, they usually implement important logic of the code).
/**
* Namespace <i> Example </i> created to group other namespaces of the "Example".
*/
Exc = {};
/**
* Namespace <i> ui </i> created with the aim of grouping namespaces user interface.
*/
Exc.ui = {};
/**
* Class <i> maskdInput </i> used to add an input HTML formatting capabilities and validation of data and information.
* # Param {String} mask - mask validation of input data.
*/
Exc.ui.maskedInput = function (mask) {
this.mask = mask;
...
};
/**
* Class <i> domTips </i> used to add an HTML element the ability to present tips and information about its function or rule input etc..
* # Param {String} id - id of the HTML element.
* # Param {String} tips - tips on the element that will appear when the mouse is over the element whose identifier is id <i> </i>.
*/
Exc.ui.domTips = function (id, tips) {
this.domID = id;
this.tips = tips;
...
};
These are just some tips, but that has greatly helped in organizing the code. Remember you must have discipline to succeed!
Following good OO design principals and design patterns goes a long way to making your code easy to maintain and understand.
But one of the best things I've discovered recently are signals and slots aka publish/subscribe.
Have a look at http://markdotmeyer.blogspot.com/2008/09/jquery-publish-subscribe.html
for a simple jQuery implementation.
The idea is well used in other languages for GUI development. When something significant happens somewhere in your code you publish a global synthetic event which other methods in other objects may subscribe to.
This gives excellent separation of objects.
I think Dojo (and Prototype?) have a built in version of this technique.
see also What are signals and slots?
I was able to successfully apply the Javascript Module Pattern to an Ext JS application at my previous job. It provided a simple way to create nicely encapsulated code.
Dojo had the module system from the day one. In fact it is considered to be a cornerstone of Dojo, the glue that holds it all together:
dojo.require — the official doc.
Understanding dojo.declare, dojo.require, and dojo.provide.
Introducing Dojo.
Using modules Dojo achieves following objectives:
Namespaces for Dojo code and custom code (dojo.declare()) — do not pollute the global space, coexist with other libraries, and user's non-Dojo-aware code.
Loading modules synchronously or asynchronously by name (dojo.require()).
Custom builds by analyzing module dependencies to create a single file or a group of interdependent files (so-called layers) to include only what your web application needs. Custom builds can include Dojo modules and customer-supplied modules as well.
Transparent CDN-based access to Dojo and user's code. Both AOL and Google carry Dojo in this fashion, but some customers do that for their custom web applications as well.
Check out JavasciptMVC.
You can :
split up your code into model, view and controller layers.
compress all code into a single production file
auto-generate code
create and run unit tests
and lots more...
Best of all, it uses jQuery, so you can take advantage of other jQuery plugins too.
My boss still speaks of the times when they wrote modular code (C language), and complains about how crappy the code is nowadays! It is said that programmers can write assembly in any framework. There is always a strategy to overcome code organisation. The basic problem is with guys who treat java script as a toy and never try to learn it.
In my case, I write js files on a UI theme or application screen basis, with a proper init_screen(). Using proper id naming convention, I make sure that there are no name space conflicts at the root element level. In the unobstrusive window.load(), I tie the things up based on the top level id.
I strictly use java script closures and patterns to hide all private methods. After doing this, never faced a problem of conflicting properties/function definitions/variable definitions. However, when working with a team it is often difficult to enforce the same rigour.
I'm surprised no one mentioned MVC frameworks. I've been using Backbone.js to modularize and decouple my code, and it's been invaluable.
There are quite a few of these kinds of frameworks out there, and most of them are pretty tiny too. My personal opinion is that if you're going to be writing more than just a couple lines of jQuery for flashy UI stuff, or want a rich Ajax application, an MVC framework will make your life much easier.
"Write like crazy and just hope it works out for the best?", I've seen a project like this which was developed and maintained by just 2 developers, a huge application with lots of javascript code. On top of that there were different shortcuts for every possible jquery function you can think of. I suggested they organize the code as plugins, as that is the jquery equivalent of class, module, namespace... and the whole universe. But things got much worse, now they started writing plugins replacing every combination of 3 lines of code used in the project.
Personaly I think jQuery is the devil and it shouldn't be used on projects with lots of javascript because it encourages you to be lazy and not think of organizing code in any way. I'd rather read 100 lines of javascript than one line with 40 chained jQuery functions (I'm not kidding).
Contrary to popular belief it's very easy to organize javascript code in equivalents to namespaces and classes. That's what YUI and Dojo do. You can easily roll your own if you like. I find YUI's approach much better and efficient. But you usualy need a nice editor with support for snippets to compensate for YUI naming conventions if you want to write anything useful.
I create singletons for every thing I really do not need to instantiate several times on screen, a classes for everything else. And all of them are put in the same namespace in the same file. Everything is commented, and designed with UML , state diagrams. The javascript code is clear of html so no inline javascript and I tend to use jquery to minimize cross browser issues.
In my last project -Viajeros.com- I've used a combination of several techniques. I wouldn't know how to organize a web app -- Viajeros is a social networking site for travellers with well-defined sections, so it's kind of easy to separate the code for each area.
I use namespace simulation and lazy loading of modules according to the site section. On each page load I declare a "vjr" object, and always load a set of common functions to it (vjr.base.js). Then each HTML page decides which modules need with a simple:
vjr.Required = ["vjr.gallery", "vjr.comments", "vjr.favorites"];
Vjr.base.js gets each one gzipped from the server and executes them.
vjr.include(vjr.Required);
vjr.include = function(moduleList) {
if (!moduleList) return false;
for (var i = 0; i < moduleList.length; i++) {
if (moduleList[i]) {
$.ajax({
type: "GET", url: vjr.module2fileName(moduleList[i]), dataType: "script"
});
}
}
};
Every "module" has this structure:
vjr.comments = {}
vjr.comments.submitComment = function() { // do stuff }
vjr.comments.validateComment = function() { // do stuff }
// Handlers
vjr.comments.setUpUI = function() {
// Assign handlers to screen elements
}
vjr.comments.init = function () {
// initialize stuff
vjr.comments.setUpUI();
}
$(document).ready(vjr.comments.init);
Given my limited Javascript knowledge, I know there must be better ways to manage this, but until now it's working great for us.
Organising your code in a Jquery centric NameSpace way may look as follows... and will not clash with other Javascript API's like Prototype, Ext either.
<script src="jquery/1.3.2/jquery.js" type="text/javascript"></script>
<script type="text/javascript">
var AcmeJQ = jQuery.noConflict(true);
var Acme = {fn: function(){}};
(function($){
Acme.sayHi = function()
{
console.log('Hello');
};
Acme.sayBye = function()
{
console.log('Good Bye');
};
})(AcmeJQ);
// Usage
// Acme.sayHi();
// or
// Say Hello
</script>
Hope this helps.
Good principal of OO + MVC would definitely go a long way for managing a complex javascript app.
Basically I am organizing my app and javascript to the following familiar design (which exists all the way back from my desktop programming days to Web 2.0)
Description for the numeric values on the image:
Widgets representing the views of my application. This should be extensible and separated out neatly resulting good separation that MVC tries to achieve rather than turning my widget into a spaghetti code (equivalent in web app of putting a large block of Javascript directly in HTML). Each widget communicate via others by listening to the event generated by other widgets thus reducing the strong coupling between widgets that could lead to unmanageable code (remember the day of adding onclick everywhere pointing to a global functions in the script tag? Urgh...)
Object models representing the data that I want to populate in the widgets and passing back and forth to the server. By encapsulating the data to its model, the application becomes data format agnostics. For example: while Naturally in Javascript these object models are mostly serialized and deserialized into JSON, if somehow the server is using XML for communication, all I need to change is changing the serialization/deserialization layer and not necessarily needs to change all the widget classes.
Controller classes that manage the business logic and communication to the server + occasionally caching layer. This layer control the communication protocol to the server and put the necessary data into the object models
Classes are wrapped neatly in their corresponding namespaces. I am sure we all know how nasty global namespace could be in Javascript.
In the past, I would separate the files into its own js and use common practice to create OO principles in Javascript. The problem that I soon found that there are multiple ways to write JS OO and it's not necessarily that all team members have the same approach. As the team got larger (in my case more than 15 people), this gets complicated as there is no standard approach for Object Oriented Javascript. At the same time I don't want to write my own framework and repeat some of the work that I am sure smarter people than I have solved.
jQuery is incredibly nice as Javascript Framework and I love it, however as project gets bigger, I clearly need additional structure for my web app especially to facilitate standardize OO practice. For myself, after several experiments, I find that YUI3 Base and Widget (http://yuilibrary.com/yui/docs/widget/ and http://yuilibrary.com/yui/docs/base/index.html) infrastructure provides exactly what I need. Few reasons why I use them.
It provides Namespace support. A real need for OO and neat organization of your code
It support notion of classes and objects
It gives a standardize means to add instance variables to your class
It supports class extension neatly
It provides constructor and destructor
It provides render and event binding
It has base widget framework
Each widget now able to communicate to each other using standard event based model
Most importantly, it gives all the engineers an OO Standard for Javascript development
Contrary to many views, I don't necessarily have to choose between jQuery and YUI3. These two can peacefully co-exist. While YUI3 provides the necessary OO template for my complex web app, jQuery still provides my team with easy to use JS Abstraction that we all come to love and familiar with.
Using YUI3, I have managed to create MVC pattern by separating classes that extend the Base as the Model, classes that extends Widget as a View and off course you have Controller classes that are making necessary logic and server side calls.
Widget can communicate with each other using event based model and listening to the event and doing the necessary task based on predefined interface. Simply put, putting OO + MVC structure to JS is a joy for me.
Just a disclaimer, I don't work for Yahoo! and simply an architect that is trying to cope with the same issue that is posed by the original question. I think if anyone finds equivalent OO framework, this would work as well. Principally, this question applies to other technologies as well. Thank God for all the people who came up with OO Principles + MVC to make our programming days more manageable.
I use Dojo's package management (dojo.require and dojo.provide) and class system (dojo.declare which also allows for simple multiple inheritance) to modularize all of my classes/widgets into separate files. Not only dose this keep your code organized, but it also lets you do lazy/just in time loading of classes/widgets.
A few days ago, the guys at 37Signals released a RTE control, with a twist. They made a library that bundles javascript files using a sort of pre-processor commands.
I've been using it since to separate my JS files and then in the end merge them as one. That way I can separate concerns and, in the end, have only one file that goes through the pipe (gzipped, no less).
In your templates, check if you're in development mode, and include the separate files, and if in production, include the final one (which you'll have to "build" yourself).
Create fake classes, and make sure that anything that can be thrown into a separate function that makes sense is done so. Also make sure to comment a lot, and not to write spagghetti code, rather keeping it all in sections. For example, some nonsense code depicting my ideals. Obviously in real life I also write many libraries that basically encompass their functionality.
$(function(){
//Preload header images
$('a.rollover').preload();
//Create new datagrid
var dGrid = datagrid.init({width: 5, url: 'datalist.txt', style: 'aero'});
});
var datagrid = {
init: function(w, url, style){
//Rendering code goes here for style / width
//code etc
//Fetch data in
$.get(url, {}, function(data){
data = data.split('\n');
for(var i=0; i < data.length; i++){
//fetching data
}
})
},
refresh: function(deep){
//more functions etc.
}
};
Use inheritance patterns to organize large jQuery applications.
I think this ties into, perhaps, DDD (Domain-Driven Design). The application I'm working on, although lacking a formal API, does give hints of such by way of the server-side code (class/file names, etc). Armed with that, I created a top-level object as a container for the entire problem domain; then, I added namespaces in where needed:
var App;
(function()
{
App = new Domain( 'test' );
function Domain( id )
{
this.id = id;
this.echo = function echo( s )
{
alert( s );
}
return this;
}
})();
// separate file
(function(Domain)
{
Domain.Console = new Console();
function Console()
{
this.Log = function Log( s )
{
console.log( s );
}
return this;
}
})(App);
// implementation
App.Console.Log('foo');
For JavaScript organization been using the following
Folder for all your javascript
Page level javascript gets its' own file with the same name of the page. ProductDetail.aspx would be ProductDetail.js
Inside the javascript folder for library files I have a lib folder
Put related library functions in a lib folder that you want to use throughout your application.
Ajax is the only javascript that I move outside of the javascript folder and gets it's own folder. Then I add two sub folders client and server
Client folder gets all the .js files while server folder gets all the server side files.
I'm using this little thing. It gives you 'include' directive for both JS and HTML templates. It eleminates the mess completely.
https://github.com/gaperton/include.js/
$.include({
html: "my_template.html" // include template from file...
})
.define( function( _ ){ // define module...
_.exports = function widget( $this, a_data, a_events ){ // exporting function...
_.html.renderTo( $this, a_data ); // which expands template inside of $this.
$this.find( "#ok").click( a_events.on_click ); // throw event up to the caller...
$this.find( "#refresh").click( function(){
widget( $this, a_data, a_events ); // ...and update ourself. Yep, in that easy way.
});
}
});
You can use jquery mx (used in javascriptMVC) which is a set of scripts that allows you to use models, views, and controllers. I've used it in a project and helped me create structured javascript, with minimal script sizes because of compression. This is a controller example:
$.Controller.extend('Todos',{
".todo mouseover" : function( el, ev ) {
el.css("backgroundColor","red")
},
".todo mouseout" : function( el, ev ) {
el.css("backgroundColor","")
},
".create click" : function() {
this.find("ol").append("<li class='todo'>New Todo</li>");
}
})
new Todos($('#todos'));
You can also use only the controller side of jquerymx if you aren't interested in the view and model parts.
Your question is one that plagued me late last year. The difference - handing the code off to new developers who had never heard of private and public methods. I had to build something simple.
The end result was a small (around 1KB) framework that translates object literals into jQuery. The syntax is visually easier to scan, and if your js grows really large you can write reusable queries to find things like selectors used, loaded files, dependent functions, etc.
Posting a small framework here is impractical, so I wrote a blog post with examples (My first. That was an adventure!). You're welcome to take a look.
For any others here with a few minutes to check it out, I'd greatly appreciate feedback!
FireFox recommended since it supports toSource() for the object query example.
Cheers!
Adam
I use a custom script inspired by Ben Nolan's behaviour (I can't find a current link to this anymore, sadly) to store most of my event handlers. These event handlers are triggered by the elements className or Id, for example.
Example:
Behaviour.register({
'a.delete-post': function(element) {
element.observe('click', function(event) { ... });
},
'a.anotherlink': function(element) {
element.observe('click', function(event) { ... });
}
});
I like to include most of my Javascript libraries on the fly, except the ones that contain global behaviour. I use Zend Framework's headScript() placeholder helper for this, but you can also use javascript to load other scripts on the fly with Ajile for example.
You don't mention what your server-side language is. Or, more pertinently, what framework you are using -- if any -- on the server-side.
IME, I organise things on the server-side and let it all shake out onto the web page. The framework is given the task of organising not only JS that every page has to load, but also JS fragments that work with generated markup. Such fragments you don't usually want emitted more than once - which is why they are abstracted into the framework for that code to look after that problem. :-)
For end-pages that have to emit their own JS, I usually find that there is a logical structure in the generated markup. Such localised JS can often be assembled at the start and/or end of such a structure.
Note that none of this absolves you from writing efficient JavaScript! :-)
Lazy Load the code you need on demand. Google does something like this with their google.loader