We Keep Coding

Assemble every module into a single .js file

I want to minimize the number of HTTP requests from the client to load scripts in the browser. This is going to be a pretty general question but I still hope I can get some answers because module management in javascript has been a pain so far.
Current situation
Right now, in development, each module is requested individually from the main html template, like this:
<script src="/libraries/jquery.js"></script>
<script src="/controllers/controllername.js"></script>
...
The server runs on Node.js and sends the scripts as they are requested.
Obviously this is the least optimal way of doing so, since all the models, collections, etc. are also separated into their own files which translates into numerous different requests.
As far as research goes
The libraries I have come across (RequireJS using AMD and CommonJS) can request modules from within the main .js file sent to the client, but require a lot of additional work to make each module compliant with each library:
;(function(factory){
if (typeof define === 'function' && define.amd) define([], factory);
else factory();
}(function(){
// Module code
exports = moduleName;
}));
My goal
I'd like to create a single file on the server that 'concatenates' all the modules together. If I can do so without having to add more code to the already existing modules that would be perfect. Then I can simply serve that single file to the client when it is requested.
Is this possible?
Additionally, if I do manage to build a single file, should I include the open source libraries in it (jQuery, Angular.js, etc.) or request them from an external cdn on the client side?

What you are asking to do, from what I can tell, is concat your js files into one file and then in your main.html you would have this
<script src="/pathLocation/allMyJSFiles.js"></script>
If my assumption is correct, then the answer would be to use one of the two following items
GULP link or GRUNT link
I use GULP.
You can either use gulp on a case by case basis, which means calling gulp from the command line to execute gulp code, or use a watch to do it automatically on save.
Besides getting gulp to work and including the gulp files you need to do what you need, I will only provide a little of what I use to get your answer.
In my gulp file I would have something like this
var gulp = require('gulp');
var concat = require('gulp-concat');
...maybe more.
Then I have the file paths I need to be reduced into one file.
var onlyProductionJS = [
'public/application.js',
'public/directives/**/*.js',
'public/controllers/**/*.js',
'public/factories/**/*.js',
'public/filters/**/*.js',
'public/services/**/*.js',
'public/routes.js'
];
and I use this info in a gulp task like the one below
gulp.task('makeOneFileToRuleThemAll', function(){
return gulp.src(onlyProductionJS)
.pipe(concat('weHaveTheRing.js'))
.pipe(gulp.dest('public/'));
});
I then run the task in my command line by calling
gulp makeOneFileToRuleThemAll
This call runs the associated gulp task which uses 'gulp-concat' to get all the files together into one new file called 'weHaveTheRing.js' and creates that file in the destination 'public/'
Then just include that new file into your main.html
<script src="/pathLocation/weHaveTheRing.js"></script>
As for including all your files into one file, including your vendor files, just make sure that your vendor code runs first. It's probably best to keep those separate unless you have a sure fire way of getting your vendor code to load first without any issues.
UPDATE
Here is my gulp watch task.
gulp.task('startTheWatchingEye', function () {
gulp.watch(productionScripts, ['makeOneFileToRuleThemAll']);
});
Then I start up my server like this (yours may differ)
npm start
// in a different terminal window I then type
gulp startTheWatchfuleye
NOTE: you can use ANY movie or show reference you wish! :)
Now just code it up, every time you make a change in the specified files GULP will run your task(s).
If you want to say run Karma for your test runner...
add the following to your gulp file
var karma = require('karma').server;
gulp.task('karma', function(done){
karma.start({
configFile: __dirname + '/karma.conf.js'
}, done);
});
Then add this task karma to your watch I stated above like this...
gulp.task('startTheWatchingEye', function(){
gulp.watch(productionScripts, ['makeOneFileToRuleThemAll', 'karma']);
});
ALSO
Your specific settings may require a few more gulp modules. Usually, you install Gulp globally, as well as each module. Then use them in your various projects. Just make sure that your project's package.json has the gulp modules you need in dev or whatever.
There are different articles on whether to use Gulp or Grunt. Gulp was made after Grunt with a few additions that Grunt was lacking. I don't know if Grunt lacks them anymore. I like Gulp a lot though and find it very useful with a lot of documentation.
Good luck!

I'd like to create a single file on the server that 'concatenates' all the modules together. If I can do so without having to add more code to the already existing modules that would be perfect.
Sure you can. You can use Grunt or Gulp to do that, more specifically grunt-contrib-concat or gulp-concat
Here's an example of a Gruntfile.js configuration to concat every file under a js directory:
grunt.initConfig({
concat: {
dist: {
files: {
'dist/built.js': ['js/**/**.js'],
},
},
},
});
Also, you can minify everything after concatenating, using grunt-contrib-minify.
Both libraries support source maps so, in the case a bug gets to production, you can easily debug.
You can also minify your HTML files using grunt-contrib-htmlmin.
There's also an extremely useful library called grunt-usemin. Usemin let's you use HTML comments to "control" which files get minified (so you don't have to manually add them).
The drawback is that you have to explicitely include them in your HTML via script tags, so no async loading via javascript (with RequireJS for instance).
Additionally, if I do manage to build a single file, should I include the open source libraries in it (jQuery, Angular.js, etc.) or request them from an external cdn on the client side?
That's debatable. Both have pros and cons. Concatenating vendors assures that, if for some reason, the CDN isn't available, your page works as intended. However the file served is bigger so you consume more bandwidth.
In my personal experience, I tend to include vendor libraries that are absolutely essential for the page to run such as AngularJS for instance.

If I understand you correctly, you could use a task runner such as Grunt to concatenate the files for you.
Have a look at the Grunt Concat plugin.
Example configuration from the docs:
// Project configuration.
grunt.initConfig({
concat: {
dist: {
src: ['src/intro.js', 'src/project.js', 'src/outro.js'],
dest: 'dist/built.js',
}
}
});
Otherwise, as you have stated, a 'module loader' system such as Require JS or Browserify may be the way to go.

Bundling and Minifying Durandal Applications

I'm using MVC5/Durandal and wondering what the recommended approach to bundling/minifying a durandal application would be. Ive seen docs on using Weyland but will be deploying to an Azure Website and don't see how to leverage this in my .net-based deployment process. How can I go about configuring automated bundling/minification of my durandal application when deploying to Azure?

I've spent a bit of time trying to optimize an AngularJS application for one of the biggest banks in Holland. Although it's no Durandal, this might still give you some ideas.
So what did we use for bundling and minification? Out-of-the-box bundling and minifcation from ASP.NET MVC ( which is from the system.web.optimization namespace )
You need to get a couple of things in order to leverage this:
Organize your files
Organize your code files in a way that they can easily be bundled. We had a large tree structure under the /app folder in the web project. Something like:
- App
|- Modules
| |-Common
| | |- Directives
| | |- Templates
| | |- Filters
| --User
| ...
| app.js
So the application skeleton was inside the app.js and all the other JS files were required by the application. Point being: all SPA code is separated from vendor javscript files and the rest of course
Set up the budling inside the bundle configuration
That's a breeze now, just do regular-old-bundling from your Global.asax.cs:
Make sure there's a line in the Application_Start() with:
BundleConfig.RegisterBundles(BundleTable.Bundles);
That calls into your BundleConfig class which only needs 1 bundle to pack up the whole /app folder:
bundles.Add(new ScriptBundle("~/bundles/app")
.Include("~/app/*.js")
.IncludeDirectory("~/app", "*.js", true));
We needed the app.js to load first - therefore we put it explicitly at the top. Don't worry, it will not be requested twice.
For bundling - only the sequence of files can be important. However, through including that file explicitly, we could control that and it worked like a charm.
Minification
Now for minification we had to do some code changes. AngularJS can be used with different types of syntax - some of which can be minified, others give problems.
Example:
angular.module('myapp').controller(function($http,$scope) { ... });
can not be minified. The minifyer will change the name of $http so something shorter, after which the injector cannot do dependency injection anymore, since it only knows stuff called $http and $scope and not the minified variable name.
So for Angular you need to use a different syntax:
angular.module('myapp').controller(['$http', '$scope', function($http,$scope) { ... }]);
With this, the injector will know that the first argument of the function is '$http' because that's the first string variable in the array. OK, but that's Angular and you're looking for Durandal.
I've heard that Durandal uses AMD right? So within a module, minification shouldn't be a problem, because it should be smart enough. However, if you're using external things, you want to make sure everything still works. I've read here that you'll want to use te following syntax for your AMDs:
define("someModule", ["jquery", "ko"], function($,ko) { ... });
And that gave us a reduction of 80% of the requests and around the same number for the Javascript payload.
Added AngularJS bonus
This might not be of interest to you, but maybe for other readers. The reason we didn't get a 99% reduction of requests is because AngularJS uses something called 'directives'. These are like HTML templates. Those HTML templates still needed to be downloaded every time they were used.
They were also included in our /app folder - hence we had to add an IgnoreRoute in the routeconfig:
routes.IgnoreRoute("app/");
I Googled, but couldn't find anything similair for Durandal. So Angular will go and get all of the small HTML files, but will first check its $templatecache. In case the HTML content is not in the cache, it goes out and downloads it and places it in the cache, so it needs to be downloaded only once.
We, well I, wrote a T4 generator that outputs a JS file in which all the HTML files in the /app folder are added to the $templatecache. So the output would look like:
angular.module('myapp').run(function($templateCache) {
/// For all *.html files in the /app folder and its children
$templateCache.put('...filename...', '...content of html file ...');
});
Because this .JS file was inside the /app folder, it would immediately get bundled with the application, no more configuration required. This got our requests down for the whole application to just 1. Since the amount of HTML was quite small, it seemed to be faster to do 1 larger request, then multiple smaller ones.
Point is: if Durandal has something similair and it will look for some templates, find the caching mechanism ( because it will have it ) and try to tap into that.
Controlling bundling and minification
I'll quote this site: http://www.asp.net/mvc/overview/performance/bundling-and-minification
Bundling and minification is enabled or disabled by setting the value
of the debug attribute in the compilation Element in the Web.config
file. In the following XML, debug is set to true so bundling and
minification is disabled.
<system.web>
<compilation debug="true" />
</system.web>
So for your release build - this flag shouldn't be set and thus bundling + minification should happen.
But of course, you will want to test locally - you can either remove this from your web.config or override it with BundleTable.EnableOptimizations = true;
Deployment to Azure
Then you mention deployment to Azure. I don't know how this would be any different from deploying to a local server. We used web-deploy. Bundling and minification doesn't happen build-time, so there are no changes in the build process. Also, the optimization framework is being deployed with the site - so no difficult things for deployment either.
Maybe one thing though: you could consider adding the CDN for the libraries you are using:
bundles.Add(new ScriptBundle("~/bundles/jquery", "http://ajax.aspnetcdn.com/ajax/jQuery/jquery-1.7.1.min.js")
In case the CDN location of jQuery was already cached by the client's browser, you'll save another request.
Measuring the performance was easy: just open up the network tab on Chrome and reload the page ( make sure it's not caching ). Check the total number of requests and the total amount of data downloaded. Like I said: we saw a huge improvement.
Well, hope it helps or points you in a right direction.

The below answers are pretty complicated. I've just gone through this with a simple(r) approach here:
https://lifelibertycode.wordpress.com/2015/04/14/how-to-bundle-up-your-mvc-durandal-app/
The steps below:
Step 1: Install Node
Step 2: Install Gulp
$ npm install --global gulp
$ npm install --save-dev gulp
Step 3: Create your gulpfile.js
This should be at the root of your project, and should initially contain this:
var gulp = require('gulp');
gulp.task('default', function() {
// place code for your default task here
});
Step 4: Install gulp-durandal
npm install gulp-durandal --save-dev
Step 5: Update your gulpfile.js
var durandal = require('gulp-durandal');
gulp.task('durandal', function(){
durandal({
baseDir: 'app', //same as default, so not really required.
main: 'main.js', //same as default, so not really required.
output: 'main.js', //same as default, so not really required.
almond: true,
minify: true
})
.pipe(gulp.dest('dir/to/save/the/output'));
});
Step 6: Add a post-build event to your project
if '$(Configuration)'=='Release' (
cd $(ProjectDir)
gulp durandal
)
Step 7: Add a pre-build event to your project
I needed this because occasionally gulp would hang when generating the new main-built.js on top of an existing version. So I just delete the old version before the build begins:
if '$(Configuration)'=='Release' (
cd $(ProjectDir)/app
del main-built.js
del main-built.js.map
)
Now, when you build your project, you’ll generate a new main-built.js file each time that can be served down to your clients. Sweet.
At this point, you probably have some concerns.
How do I keep my files un-bundled when I’m debugging?
#if (HttpContext.Current.IsDebuggingEnabled) {
<script type="text/javascript" src="~/Scripts/require.js" data-main="/App/main"></script>
} else {
#Scripts.Render("~/Scripts/main-built")
}
Where ‘main-built’ is defined in your BundleConfig:
bundles.Add(new ScriptBundle("~/Scripts/main-built").Include(
"~/app/main-built.js"));
How do I bust cache when I have new stuff to ship?
If you’re using the above approach, bundling will take care of this for you. ASP.NET will detect a change to your main-built.js file and append a unique identifier to your bundles to bust the cache.
What if my client has downloaded my SPA, and then I ship an update. Won’t the (outdated) client-side code stick around until they refresh?
Yup. Unless you leverage build versioning to tell the client when it’s out of date, and then tell the user.

I happen have written a blog post about this:
https://javascriptkicks.com/articles/4230
Hopefully that helps you out

Yii2: Registering Asset Bundle vs registering external Js file

Hi I wanted to know the advantage of registering Asset Bundle following the process described in the docs like
Process one
in AppAsset.php
public $js = [
'js/myjsfile.js'
];
then in the view file
adding Namespace like
namespace app\assets;
and then adding the use statement like
use app\assets\AppAsset;
AppAsset::register($this);
Instead of doing all this if I use
Process Two
$this->registerJs('js/myjsfile.js', $this::POS_READY);
it works fine.
So why should I use Process One.
Any advantage and reason for this will be greatly appreciated.
If I follow the process one Do I need to add all the js files in
AppAsset.php individually.
Thanks.

Asset Bundles have some advantages over normal registering. Apart from what #deacs said in his/her answer here are others:
Assets Bundles can publish the file to assets if its not in web accessible directory
Assets Bundle can deal with less files (in case of CSS) as well as compressing the assets.
Makes Code Elegant especially in solving dependencies and hence reusability
All the features that makes bundles shine are found in docs

One of the main reasons for using an Asset Bundle is that your assets' paths will always be correct. Consider:
$this->registerJsFile('js/myjsfile.js', ['position'=>$this::POS_READY]);
will generate something like:
<script src="js/myjsfile.js"></script>
Which works great for non urlManager enabled urls, e.g. http://localhost/yiiproject/index.php?r=user/update&id=8 because your browser looks for the js file at: /yiiproject/js/myjsfile.js
But if you enable urlManager, your url will look like http://localhost/yiiproject/user/update/8, which means your browser will look for your js file at: /yiiproject/user/update/8/js/myjsfile.js.
You could overcome this problem by using:
$this->registerJsFile(Yii::$app->request->baseUrl.'/js/myjsfile.js', ['position'=>$this::POS_READY]);
But the Asset Bundle basicly does that for you.

Using Asset Bundles, you can also get the latest version from 'vendor' folder, so if you need to update some lib you don't need to manually do this since composer already do this.

Sharing common code across pages with Browserify

I have a fairly large multi-page javascript applications that uses requirejs to organize code. I am researching moving to browserify because I like the simplicity that it offers and I am already used to the node.js module system.
Currently on each page I have javascript that goes like this
<script data-main="/scripts/config/common" src="/scripts/lib/require.js">
<script data-main="/scripts/config/page-specific-js" src="/scripts/lib/require.js">
and I have a common build step and a build for each page. This way the majority of the JS is cached for every page.
Is it possible to do something similar with browserify? Is caching like this even worth it, or is it better to bundle everything into one file across all pages (considering that maybe only one page can depend on a large external library)?

You can use factor-bundle to do exactly this. You will just need to split your code up into different entry points for each file.
Suppose you have 3 pages, /a, /b, and /c. Each page corresponds to an entry point file of /browser/a.js, /browser.b.js, and /browser/c.js. With factor-bundle, you can do:
browserify -p [ factor-bundle -o bundle/a.js -o bundle/b.js -o bundle/c.js ] \
browser/a.js browser/b.js browser/c.js > bundle/common.js
any files used by more than 1 of the entry points will be factored out into bundle/common.js, while all the page-specific code will be located in the page-specific bundle file. Now on each page you can put a script tag for the common bundle and a script tag for the page-specific bundle. For example, for /a you can put:
<script src="bundle/common.js"></script>
<script src="bundle/a.js"></script>
If you don't want to use the command-line version, you can also use factor-bundle from the API:
var browserify = require('browserify');
var fs = require('fs');
var files = [ './files/a.js', './files/b.js', './files/c.js' ];
var b = browserify(files);
b.plugin('factor-bundle', {
outputs: [ 'bundle/a.js', 'bundle/b.js', 'bundle/c.js' ]
});
b.bundle().pipe(fs.createWriteStream('bundle/common.js'));

CakePHP AssetCompress: Not generating compressed files in cache folders

Has anyone here used Mark Story's Asset Compress (https://github.com/markstory/asset_compress/) plugin ?
I've followed the installation instructions to the last bit and have the plugin up and running - but it simply won't generate the combined JS files to the specified cache (cache_js) folder.
I'm using Cake 1.3 and v0.2 of AssetCompress (the latest available download from github).
The plugin has been placed in the app/plugins/asset_compress folder
Cache folders - cache_js and cache_css created in WEBROOT
JsMin and CssMin filters downloaded and added to app/vendors/JsMin and app/vendors/CssMin respectively
Config file setup to point to the cache folders and filters
Routes configured as per requirements
Debug mode set to 1
My config.ini:
[Javascript]
searchPaths[] = WEBROOT/js/
searchPaths[] = WEBROOT/js/jquery/
searchPaths[] = WEBROOT/js/jquery/plugins/
stripComments = true
cacheFilePath = WEBROOT/cache_js/
cacheFiles = false
filters[] = JsMin
[Css]
searchPaths[] = WEBROOT/css/
stripComments = true
cacheFilePath = WEBROOT/cache_css/
cacheFiles = false
filters[] = CssMin
Still no output in the cache folders.
Any ideas why ?
Thanks,
m^e

I did not use this plugin yet
but just few comments.
-sometimes minification leads to the hells (javascript errors)especially if you mimnify an already minified version
-minification makes comments and license agreements disappear which makes things illegal.
I personally, do not prefer to compress assets using plugins.
plugin in cakePHP by definition is a semi application not just a utility class (helper, component, behavior, or any vendor utility class)
currently I am compressing the concatenated javascript files (resp css files) in the AppController by using just one function

Finally got it to work.
Turns out I was messing around with v0.2 which is what you get by default when you hit the DOWNLOAD button at the GitHub repository of Asset Compress.
You've to check out the latest version from GitHub using a git client like msysGit (if you are on Windows) and then be extra careful about where you are placing the asset inclusion commands.
Here are the steps you need to take:
Place the contents of the download in a folder named asset_compress under your app's plugins folder.
Include the plugin as a helper (preferably in your app_controller.php)
public $helpers = array(
'AssetCompress.AssetCompress',
);
In your layout file, place the asset inclusion commands, e.g.
$this->AssetCompress->script( filename );
Just before the point where you place echo $scripts_for_layout in your layout, place the statement,
echo $this->AssetCompress->includeJs();
...and you're good to go.
Cheers,
m^e