This is a complete noob question, but I gotta ask it anyway
I started playing with backbone.js a few days ago and I was really fascinated. As I got over the "ToDo", I started working on a project of my own. Coming from the world of Java, I prefer keeping everything in as many separate files as possible. Therefore, I split my models views, and routers into separate files, into separate folders.
The problem came when I tried to combine those fiels into one single applciation.js file. Again, coming from the Java world, I love when I can automate stuff, and even more, when I can use familiar tools like ant, to setup build processes for my javascript projects.
I got a sample ant build template which concatenates and minifies all the files in an arbitrary order. When it finished, I tried to run my JS app, and not surprisingly, it failed with a bunch of errors. Many of my models and views try to extend each other, others depende on them as components. If they are not defined in a proper order, the app just reaches a point where it is trying to execute extend of an undefined
I know from before that for JavaScript the order is very important, but somehow I was left with the impression that if all the scripts are in one single file, the JS parser will load all the stuff first and then will try to execute whatever is to be executed. Well, my assumption was wrong.
It is possible to list all the files in the specific order I want them, but do I really need to go for such a primitive step? Unfortunately after spending a few hours researching, I couldn't find anything better.
Is it really possible to concatenate JS files, which depend on each other, in an arbitrary order, without them clashing? I guess, the biggest problem is the fact that the extend function is actually being called, rather than each script simply defining and object literal
So, what's the solution?
UPDATE: I just saw that Sproutcore has its own builder. If SC is roughly similar to BB, in the way one creates and extends entities, how does the SC builder work without clashing?
There are many ways to do this, but here's my recipe. I prefix my development files with a number, starting from the one with no dependencies (base "classes", models that will be depended upon from other models, then views using these models, then routers calling those views, etc.).
Then I use uglify-js (available as a node.js library, that you install using npm install uglify-js) to minify all my js in one file (don't know from your question if you use node.js server-side, though). Then I cat *.js | uglifyjs -o min/myfile.min.js. This will send the content of all my .js files (respecting the order of dependencies because of my prefix) to uglify, which will minify it and save it to a single file.
Since I, too, like automation, I have this set up in a Makefile, though I guess it could be done using Ant (not too familiar with it). The relevant part of the Makefile look like this:
TARGET_MIN_FILE = public/js/min/myfile.min.js
JS = $(shell echo public/js/*.js)
public/js/min/myfile.min.js: $(JS)
cat $(JS) | uglifyjs -o $(TARGET_MIN_FILE)
clean:
rm -f $(TARGET_MIN_FILE)
.PHONY: clean
On the other hand, if you go for the asynchronous module definition (AMD) format, you can require() your modules and it will manage for you the dependency loading in the correct order (see Require.js for more info), as mentioned by TheShelfishMeme.
Your "assumption" is only true for var statements and functions of the form function name(a,b) {}. Those two get hoisted to the top of the script (or function block they are in) and are evaluated first.
If your files depend on other files being loaded first, it stands to reason that when you concatenate them they must be in that order in the final file.
Have a look at requirejs. It takes some time to set up but it should help you with your problem.
This article should help with the implementation.
Related
While trying to get started with Reason, in one JavaScript project, I've got an extremely light file that tries to be a Reason-typed interface to the existing, heavy, library:
/* TheLibrary.re */
type engine
external addEngine : string -> engine -> unit = "" [##bs.val] [##bs.module "../"]
However, when I try to use that library in a ReasonReact project (having added #org/the-library to the bsconfig.json bs-dependencies),
/* AComponent.re */
[#bs.val] [#bs.module "#org/game-engine/dist/game-engine.js"]
external gameEngine : TheLibrary.engine = "default";
/* Further down, a React lifecycle method, */
TheLibrary.addEngine("Game", gameEngine);
I get errors about ../ being not found, relative to that React component:
./src/components/main-menu/AComponent.re
Module not found: Can't resolve '../' in '/Users/ec/Work/reason-reacty/src/components/main-menu'
I've also tried, instead of ../ in TheLibrary.re's external declaration:
#bs.module "./index.js" (the direct, ES6 entry-point for the untyped-JavaScript side of the package in question,)
#bs.module "#org/the-library", the entire name of said library (even though I'm typing inside that library???)
Please help! I'd love to be able to further adopt ML, but I'm having the hardest time wrapping my head around ReasonReact's dependency-resolution!
Additional context:
So, we're trying to build our first ReasonReact project, and we've successfully added baby's-first-opaque-types to one of our internal libraries and include that in the ReasonReact page with something like the following — which works, by the way:
/* Imports.re */
type engine;
[#bs.val] [#bs.module "#org/game-engine/dist/game-engine.js"]
external gameEngine : engine = "default";
[#bs.val] [#bs.module "#org/the-library"] [#bs.scope "default"]
external addEngine : (string, engine) => unit = "";
This yields, when we Imports.(addEngine("Game", gameEngine)), the global setup line we need: TheLibrary.addEngine("Game", GameEngine). I'm in the very first stages of trying to upstream that typing-information into the parent project, and publish that code to npm, so that all consuming projects can start to use Reason.
t sounds like you might be a bit confused about the different tools that make up your toolchain, so let's first go through them one by one to put them in their place:
ReasonReact is a library of opinionated, "thick" bindings to react.js, which despite the name isn't actually all that Reason-specific, except for its integration with Reason's JSX syntax. It would be more accurate to call it a BuckleScript library.
Reason is mostly just the syntax you use, but is often also used more broadly to refer to the ecosystem around it, and usually also imply that BuckleScript is being used.
OCaml is the underlying language. The "semantics" of Reason, if you will.
BuckleScript is the OCaml-to-JavaScript compiler. It compiles ONE source file, which is considered a module, into ONE JavaScript module, but also requires the type information of other OCaml modules as input.
Now, I suspect you already know most of that, but what you do not seem to know is that NONE of these actually do ANY dependency resolution. These next parts of your toolchain are what does that:
The BuckleScript Build System, or bsb, is what finds all the modules in your local project according to what you've specified in src and any BuckleScript libraries you've listed in bs-dependecies in bsconfig.json. It will figure out the dependency order of all these and feed them to the compiler in the correct order to produce one JavaScript module for each OCaml module (along with some other artefacts containing type information and such). But it will not resolve any JavaScript dependencies.
Lastly, webpack, or some other JavaScript bundler, is what you likely use to combine all the JavaScript modules into a single file, and which therefore needs to resolve any JavaScript dependencies. And this is likely where the error message comes from.
Using [#bs.module "some-module"] will make the BuckleScript compiler emit var ... = require('some-module') (or import ... from 'some-module' if es6 is used), but BuckleScript itself will not do anything more with it. The string you pass to #bs.module is the same string you would pass to require if it had been an ordinary CommonJS module (or whatever other module format you have configured).
Also note that the import is not emitted where the external is defined, but where it's used. You can work around, or "ground" it in a module by re-exporting it as an ordinary definition, ie. let addEngine = addEngine.
In order to precisely answer your question I would need to know which bundler you use, where you've configured BuckleScript to output its JavaScript artefacts, where the externals are used, not just defined, and where the external JavaScript module is located. But I hope all this underlying knowledge will make it easy for you and future readers to identify and resolve the problem yourself. If you're still a bit unsure, look at the compiled JavaScript artefacts and just treat them as ordinary JavaScript modules. At this point that's really all they are.
I followed the documentation to put the constants in the lib/constants.js file.
Question:
How to access these constants in my client side html and js files?
Variables in Meteor are file-scoped.
Normally a var myVar would go in the global Node context, however in Meteor it stays enclosed in the file (which makes it really useful to write more transparent code). What happens is that Meteor will wrap all files in an IIFE, scoping the variables in that function and thus effectively in the file.
To define a global variable, simply remove the var/let/const keyword and Meteor will take care to export it. You have to create functions through the same mechanism (myFunc = function myFunc() {} or myFunc = () => {}). This export will either be client-side if the code is in the client directory, or server-side if it is in the server directory, or both if it is in some other not-so-special directories.
Don't forget to follow these rules:
HTML template files are always loaded before everything else
Files beginning with main. are loaded last
Files inside any lib/ directory are loaded next
Files with deeper paths are loaded next
Files are then loaded in alphabetical order of the entire path
Now you may run into an issue server-side if you try to access this global variable immediately, but Meteor hasn't yet instantiated it because it hasn't run over the file defining the variable. So you have to fight with files and folder names, or maybe try to trick Meteor.startup() (good luck with that). This means less readable, fragile location-dependant code. One of your colleague moves a file and your application breaks.
Or maybe you just don't want to have to go back to the documentation each time you add a file to run a five-step process to know where to place this file and how to name it.
There is two solutions to this problem as of Meteor 1.3:
1. ES6 modules
Meteor 1.3 (currently in beta) allows you to use modules in your application by using the modules package (meteor add modules or api.use('modules')).
Modules go a long way, here is a simple example taken directly from the link above:
File: a.js (loaded first with traditional load order rules):
import {bThing} from './b.js';
// bThing is now usable here
File: b.js (loaded second with traditional load order rules):
export const bThing = 'my constant';
Meteor 1.3 will take care of loading the b.js file before a.js since it's been explicitly told so.
2. Packages
The last option to declare global variables is to create a package.
meteor create --package global_constants
Each variable declared without the var keyword is exported to the whole package. It means that you can create your variables in their own files, finely grain the load order with api.addFiles, control if they should go to the client, the server, or both. It also allows you to api.use these variables in other packages.
This means clear, reusable code. Do you want to add a constant? Either do it in one of the already created file or create one and api.addFiles it.
You can read more about package management in the doc.
Here's a quote from "Structuring your application":
This [using packages] is the ultimate in code separation, modularity, and reusability. If you put the code for each feature in a separate package, the code for one feature won't be able to access the code for the other feature except through exports, making every dependency explicit. This also allows for the easiest independent testing of features. You can also publish the packages and use them in multiple apps with meteor add.
It's amazing to combine the two approaches with Meteor 1.3. Modules are way easier and lighter to write than packages since using them is one export line and as many imports as needed rather than the whole package creation procedure, but not as dumb-error-proof (forgot to write the import line at top of file) as packages.
A good bet would be to use modules first, then switch to a package as soon as they're tiring to write or if an error happened because of it (miswritten the import, ...).
Just make sure to avoid relying on traditional load order if you're doing anything bigger than a POC.
You will need to make them global variables in order for other files to see them.
JavaScript
/lib/constants.js
THE_ANSWER = 42; // note the lack of var
/client/some-other-file.js
console.log(THE_ANSWER);
CoffeeScript
/lib/constants.coffee
#THE_ANSWER = 42
/client/some-other-file.coffee
console.log THE_ANSWER
For my team at work, I'm trying to set up a semi-automated JavaScript script and dependency management system with the help of Gulp and Browserify.
I'm not even sure if what I'm trying to achieve is possible with the currently available set of tools (and my limited JavaScript knowledge). I believe what I'm trying to achieve is a pretty common scenario, but I haven't been able to find the information I've been looking for.
Consider the following diagram:
The lines indicate depedencies. For shared modules, such as Module-v and Module-y, I don't want the scripts to be duplicated by being included in each of their respective bundles.
I know that using Browserify I can manually ignore or exclude modules, which is fine when the project is young - but as the project grows managing which dependencies need to be included where is going to become very cumbersome.
A similar Q&A here I think has the same essence of what I'm trying to ask, but to me, it isn't quite clear. It also references gulp-browserify which has since been blacklisted.
In my diagram, I can see that I have three Browserify entry points, but my lack of Gulp/Node/Browserify experience means I'm struggling to wrap my head around how I can try to achieve what I want to.
I'm happy to do the work to try and piece it together, as I already have been trying - however project managers are breathing down my neck so I'm having to hack together a temporary "solution" until I can implement something a little more automated and robust.
Thanks in advance.
Edit
It seems from Browserify's plugin documentation that this might be able to be achieved by using factor-bundle which substack pointed out to me; however again due to my lack of Node/Browserify/Gulp experience I am struggling to pull all the pieces together.
Related Questions
How can I use factor-bundle with browserify programmatically?
Figured it out, sharing the learns:
Code example:
var gulp = require('gulp'),
source = require('vinyl-source-stream'),
browserify = require('browserify'),
factor = require('factor-bundle');
gulp.task('browserify', function(){
return browserify({
entries: ['blog.js', 'page.js']
})
.plugin(factor, {
// File output order must match entry order
o: ['bundle/blog.js', 'bundle/page.js']
})
.bundle({
debug: true
})
.pipe(source('common.js'))
.pipe(gulp.dest('bundle/'));
});
The key difference between this output and the diagram, is that the common.js file is automatically generated based on common depenedencies between blog.js and page.js. This is described in the factor-bundle documentation.
Notes:
I found that Node would throw an error if the output files didn't already exist. I'm unsure why as I would have assumed that factor-bundle would simply write a stream to the outputting file regardless of whether it was there or not. As a workaround, after 'cleaning' the output directory, I simply created 'placeholder' files to keep it happy.
I haven't figured out how to access the factor-bundle stream event when using it as a browserify plugin (it may not even be possible), so any further work on the output files (such as uglifying etc) would likely need to be done somewhere else in the pipeline, possibly with another browserify plugin, or even after the fact.
I've been reading about the module pattern, but everything I read assumes that the entire contents of the module will be in a single file. I want to have one file per class.
I've resorted to doing this at the top of every file:
if(window.ModuleName === undefined) { window.ModuleName = {}; }
ModuleName.ClassName = function () { ... }
But this allows files to be included without their dependencies, which is also annoying. For example, lets say there is ClassA which uses ClassB, and "ClassB.js" is left out of the HTML, then ClassA will throw up errors. As far as I'm aware Javascript lacks an import statement, so in this case I actually want everything to be in a single file.
I assume that large javascript projects are broken up into multiple files, so there has to be a way around this. How is it generally done? Is there some tool that will combine multiple class files into a single module file?
This is a big topic but let me explain as much as I can. Javascript requires that you have preloaded anything you intended to use, which is why your module pattern has all the "things" in the same file. But if you plan to separate them in different files then you have to manage it before using. I suggest the following approaches
Concatenate them before serving them in the server. For example in jsp, you can create a servlet that returns contenttype = "text/javascript", inside that servlet you can append all the scripts you need in one dynamically generated script then return it to the client.
In your ant or maven builds etc, there are configurations where in you can concatenate them the files you want together. This is a common practice therefore you should find many reference in the internet.
Lazy-load javascripts. This is my preferred way. I use Lazyload javascript library. Basically I declare the dependencies of certain codes much like "import" in Java, then before i call any of them i load their dependencies. This allows for optimized dependency loading without scripts redundancies. The problem is you need to write some fairly complicated scripts to do this.
Hope to help.
In complex client side projects, the number of Javascript files can get very large. However, for performance reasons it's good to concatenate these files, and compress the resulting file for sending over the wire. I am having problems in concatenating these as the dependencies are included after they are needed in some cases.
For instance, there are 2 files:
/modules/Module.js <requires Core.js>
/modules/core/Core.js
The directories are recursively traversed, and Module.js gets included before Core.js, which causes errors. This is just a simple example where dependencies could span across directories, and there could be other complex cases. There are no circular dependencies though.
The Javascript structure I follow is similar to Java packages, where each file defines a single Object (I'm using MooTools, but that's irrelevant). The structure of each javascript file and the dependencies is always consistent:
Module.js
var Module = new Class({
Implements: Core,
...
});
Core.js
var Core = new Class({
...
});
What practices do you usually follow to handle dependencies in projects where the number of Javascript files is huge, and there are inter-file dependencies?
Using directories is clever, however, I think you might run into problems when you have multiple dependencies. I found that I had to create my own solution to handle this. So, I created a dependency management tool that is worth checking out. (Pyramid Dependency Manager documentation)
It does some important things other javascript dependency managers don't do, mainly
Handles other files (including inserting html for views...yes, you can separate your views during development)
Combines the files for you in javascript when you are ready for release (no need to install external tools)
Has a generic include for all html pages. You only have to update one file when a dependency gets added, removed, renamed, etc
Some sample code to show how it works during development.
File: dependencyLoader.js
//Set up file dependencies
Pyramid.newDependency({
name: 'standard',
files: [
'standardResources/jquery.1.6.1.min.js'
]
});
Pyramid.newDependency({
name:'lookAndFeel',
files: [
'styles.css',
'customStyles.css'
]
});
Pyramid.newDependency({
name:'main',
files: [
'createNamespace.js',
'views/buttonView.view', //contains just html code for a jquery.tmpl template
'models/person.js',
'init.js'
],
dependencies: ['standard','lookAndFeel']
});
Html Files
<head>
<script src="standardResources/pyramid-1.0.1.js"></script>
<script src="dependencyLoader.js"></script>
<script type="text/javascript">
Pyramid.load('main');
</script>
</head>
This may be crude, but what I do is keep my separate script fragments in separate files. My project is such that I'm willing to have all my Javascript available for every page (because, after all, it'll be cached, and I'm not noticing performance problems from the parse step). Therefore, at build time, my Ant script runs Freemarker via a little custom Ant task. That tasks roots around the source tree and gathers up all the separate Javascript source files into a group of Maps. There are a few different kinds of sources (jQuery extensions, some page-load operations, so general utilities, and so on), so the task groups those different kinds together (getting its hints as to what's what from the script source directory structure.
Once it's built the Maps, it feeds those into Freemarker. There's a single global template, and via Freemarker all the script fragments are packed into that one file. Then that goes through YUI compressor, and bingo! each page just grabs that one script, and once it's cached there's no more script fetchery over my entire site.
Dependencies, you ask? Well, that Ant task orders my source files by name as it builds those maps, so where I need to ensure definition-use ordering I just prefix the files with numeric codes. (At some point I'm going to spiff it up so that the source files can keep their ordering info, or maybe even explicitly declared dependencies, inside the source in comment blocks or something. I'm not too motivated because though it's a little ugly it really doesn't bother anybody that much.)
There is a very crude dependency finder that I've written based on which I am doing the concatenation. Turns out the fact that its using MooTools is not so irrelevant after all. The solution works great because it does not require maintaining dependency information separately, since it's available within the javascript files itself meaning I can be super lazy.
Since the class and file naming was consistent, class Something will always have the filename Something.js. To find the external dependencies, I'm looking for three things:
does it Implement any other
classes
does it Extend any other
classes
does it instantiate other classes
using the new keyword
A search for the above three patterns in each javascript file gives its dependent classes. After finding the dependent classes, all Javascript files residing in any folder are searched and matched with this class name to figure out where that class is defined. Once the dependencies are found, I build a dependency graph and use the topological sort algorithm to generate the order in which files should be included.
I say just copy and paste this files to a one file in an ordered way. Each file will have a starting and ending comment to distinguish each particular code.
Each time you updated one of the files, you'll need to updated this file. So, this file need to contain only finish libraries, that not going to changes in the near time.
Your directory structure is inverted...
Core dependencies should be in the root and modules are in subdirs.
scripts/core.js
scripts/modules/module1.js
and your problem is solved.
Any further dependency issues will be indicative of defective 'class'/dependency design.
Similar to Mendy, but I create combined files on server-side. The created files will also be minified, and will have a unique name to omit cache issues after an update.
Of course, this practice only makes sense in a whole application or in a framework.
I think your best bet if at all possible, would be to redesign to not have a huge number of javascript files with interfile dependencies. Javascript just wasn't intended to go there.
This is probably too obvious but have you looked at the mootools Core Depender: http://mootools.net/docs/more/Core/Depender
One way to break the parse-time or load-time dependencies is with Self-Defining Objects (a variation on Self-Defining Functions).
Let's say you have something like this:
var obj = new Obj();
Where this line is in someFile.js and Obj is defined in Obj.js. In order for this to parse successfully you must load or concatenate Obj.js before someFile.js.
But if you define obj like this:
var obj = {
init: function() {
obj = new Obj();
}
};
Then at parse or load time it doesn't matter what order you load the two files in as long as Obj is visible at run-time. You will have to call obj.init() in order to get your object into the state you want it, but that's a small price to pay for breaking the dependency.
Just to make it clearer how this works here is some code you can cut and paste into a browser console:
var Obj = function() {
this.func1 = function ( ) {
console.log("func1 in constructor function");
};
this.init = function () {
console.log("init in constructor function");
}
};
var obj = {
init: function() {
console.log("init in original object");
obj = new Obj();
obj.init();
}
};
obj.init();
obj.func1();
And you could also try a module loader like RequireJS.