On a recent project I have been working on in C#/ASP.NET I have some fairly complicated JavaScript files and some nifty Style Sheets. As these script resources grow in size it is advisable to minify the resources and keep your web pages as light as possible, of course. I know many developers who hand-feed their JavaScript resources into compressors after debugging and then deploy their applications.
When it comes to source control and automated builds in the satisfying world of continuous integration (thank you CruiseControl.NET); hand compression will simply not do. The only way to maintain source control and offer compressed resources is to keep JS/CSS source & their minified brethren in a separate directory structure. Then register only one set of resources or the other in code-behind. However, if a developer makes a change to JS/CSS source and then fails to re-compact it and check in both versions, then you’re code-line is now out of sync. Not to mention inelegant.
I am thinking that it would be nice to write a custom executable (if one does not exist yet) for the CC.NET task block which would find and compress all JavaScript and CSS resources in the target directory after the build action but before the asp.net publish to target. This way, developers would only work on JS and CSS source and users would only get the minified resources.
Is there an application that already performs this task and if not, what kind of resource(s) should I look to install on the build server to have CC.NET execute?
(The closest question I could find here to this one required NAnt, which is not an option in my case.)
EDIT:
Dave Ward now has a great article on how to automatically minify in Visual Studio at his site.
The MSBuildCommunityTasks Project has a few MSBuild tasks that may do what you are looking for including Merge and JSCompress.
You could add these into your MSBuild project in the AfterBuild target to allow the project to perform this action every time the project is built and nothing would ever be out of sync. Your web application could then reference the compacted version for run but the developers would edit the full versions.
Nothing else would be needed on the server except the MSBuild community tasks assembly. You can put this assembly in your own source tree and reference from there and your CI build should get that assembly and everything it needs when it builds.
Another JS (and CSS!) compression library for MSBuild:
http://www.codeplex.com/YUICompressor
This is a .NET port of the java-based Yahoo! compressor.
Not a perfect answer, but if you're using MVC4 they've built this in as a new feature. When running a Debug configuration, it outputs individual files with comments and such but when you switch to Release, it will automatically bundle, minify, and change in page references to the minified files. You can setup separate bundles for, say, jquery and your own js. This works with CSS and JS files.
http://www.asp.net/mvc/tutorials/mvc-4/bundling-and-minification
If MVC4 doesn't work for you, you can also find packages on Nuget that can help such as this:
https://www.nuget.org/packages?q=minify
Related
I came across a strange issue in my recent project.
I was using Trent Richardson's Timepicker control to avail time picker functionality in my MVC 4 application. I had relevant JQuery file bundled using MVC bundling feature. I found this working quite well in development environment (Visual Studio 2012).
But when I deployed the website on IIS, I started facing a strange issue, and there was a javascript error "function expected" in that particular bundle. I could see the bundle got loaded because developer tool was showing javascript code when that bundle was selected in "scripts" tab.
Finally, when I referenced the JQuery file directly instead of bundle, it started working fine on IIS. Though the problem got solved, I am now curious to know what was wrong with that particular file if bundled, and if MVC bundling was actually an issue, then why it was working well in development environment, but not in IIS?
Any lights on this much appreciated.
Minification is a complex process by making scripts/styles smaller using techniques such variable name shortening, white space elimination, comments removal, etc... It uses ASP.NET Web Optimization that depends on WebGrease for minification. Of course, there can have issues but I personnaly never noticed that.
Here are some situations, where you should not use bundling
There is only one file in your bundle. Why bundling ?
You are using only famous frameworks such as JQuery or jQuery UI. Do not redistribute scripts that are already served by someone else. Google/Microsoft/Amazon/... already provide CDN for the most popular, open-source JavaScript libraries.
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.1/jquery.min.js"></script>
Your bundle takes only a few Bytes. Web performance Optimization suggests to limit the number of web requests. Everything has a cost. Not very optimal, but sometimes it's better to put inline scripts in your page.
In some architectures. Bundles requests contains a unique identifier used for caching. If any file in the bundle changes, the ASP.NET optimization framework will generate a new token, guaranteeing that browser requests for the bundle will get the latest bundle. When working with some architectures, JS updates can be frequent and will invalidate all your bundles.
On Dev Environment. It's is really really painful to debug a bundle.
Along with Cybermaxs' reply, I have also received following response while posted on ASP.NET Forum In case, if that helps the visitors.
What bundling suppose to do is to put together the script/stylesheet files in a single bundle into a single request and send it to the client so that the browser has to make less calls to get those required script files.
In a development environment, when you do debugging in visual studio. It doesn't do the above process unless you specify it to do so. But in a production environment, when the debug is set to false in the web.config file. it will start to do the above process.
There can be some other reasons as well. such as the script might have two versions. one for debugging and one for production. I came across such a situation with knockout. in my development enviornment I had referenced the debug version of the script. But when I put it into the production enviornment, everything came to a hault. There was a release version for the knockout script file and I had to reference that to make everything work again.
I'm very keen to make use of some build techniques in my Javascript/Web App development such as
Concatentation
Minification
Image replacement with data:uri's
Build vs Source *
App Cache Manifest generation *
It's those last two that I haven't found an answer for yet.
Build vs Source
By this I mean having a "source" version of my HTML and Javascript that is untouched so that I do not have to build each time to preview a change. All of my JS files are separate <script> tags as usual with the build vs updating these script sections with the final concatenated versions. To be honest I feel like I'm missing something here with all of these new Javascript build systems as this seems like an obvious need but I can't find anyone else talking about it. How is everyone else dealing with this?.. Build on each change during development?? surely not.
App Cache Manifest generation
This explains itself - walk through my source tree and build up a manifest and insert it into my <html> tag.
I've searched for these two with no luck - any pointers?
I'd be on the road with a killer build system if it wasn't for those two.
Thanks!!!
Re: Build vs Source
It sounds like you're already familiar with grunt. You may want to consider looking into the grunt node-build-script plugin.
It adds a number of new tasks, notably grunt mkdirs and grunt copy which duplicates your project directory into a separate staging folder and then copies your optimised project into a publish folder. If I'm not mistaken, this is what you mean by keeping an 'untouched' version of your source files?
Running grunt server will then serve up the contents of your publish files on localhost. You could always point your web server to your initial project directory if you want to examine your application in its unoptimised state.
node-build-script adds a bunch of other super convenient tasks, such as image optimisation, automatic file revving and substitution. It's incredibly easy to use and super customisable.
I have a basic single page template which uses node-build-script which also may be of interest.
Re: App Cache Manifest generation
I believe this used to be part of node-build-script but was since removed, see 1, 2
There would be nothing stopping you from creating a custom grunt task that utilised something like confess.js however.
Finally, it looks like Google's upcoming Yeoman might be worth keeping an eye on if you're not already!
Let's say I write a jQuery plugin and add it to my repository (Mercurial in my case). It's a single file, say jquery.plugin.js. I'm using BitBucket to manage this repository, and one of its features is a Downloads page. So, I add jquery.plugin.js as one of the downloads.
Now I want to make available a minified version of my plugin, but I'm not sure what the best practice is. I know that it should be available on the Downloads page as jquery.plugin.min.js, but should I also version control it each time I update it to reflect the unminified version?
The most obvious problem I see with version controlling the minified version is that I might forget to update it each time I make a change to the unminified version.
So, should I version control the minified file?
No, you should not need to keep generated minimized versions under source control.
We have had problems when adding generated files into source control (TFS), because of the way TFS sets local files to be read-only. Tools that generate files as part of the build process then have write access problems (this is probably not a problem with other version control systems).
But importantly, all the:
tools
scripts
source code
resources
third party libraries
and anything else you need to build, test and deploy your product should be under version control.
You should be able to check out a specific version from source control (by tag or revision number or the equivalent) and recreate the software exactly as it was at that point in time. Even on a 'fresh' machine.
The build should not be dependent on anything which is not under source control.
Scripts: build-scripts whether ant, make, MSBuild command files or whatever you are using, and any deployment scripts you may have need to be under version control - not just on the build machine.
Tools: this means the compilers, minimizers, test frameworks - everything you need for your build, test and deployment scripts to work - should be under source control. You need the exact version of those tools to be available to recreate to a point in time.
The book 'Continuous Delivery' taught me this lesson - I highly recommend it.
Although I believe this is a great idea - and stick to it as best as possible - there are some areas where I am not 100% sure. For example the operating system, the Java JDK, and the Continuous Integration tool (we are using Jenkins).
Do you practice Continuous Integration? It's a good way to test that you have all the above under control. If you have to do any manual installation on the Continuous Integration machine before it can build the software, something is probably wrong.
My simple rule of thumb:
Can this be automatically generated during a build process?
If yes, then it is a resource, not a source file. Do not check it in.
If no, then it is a source file. Check it in.
Here are the Sensible Rules for Repositories™ that I use for myself:
If a blob needs to be distributed as part of the source package in order to build it, use it, or test it from within the source tree, it should be under version control.
If an asset can be regenerated on demand from versioned sources, do that instead. If you can (GNU) make it, (Ruby) rake it, or just plain fake it, don't commit it to your repository.
You can split the difference with versioned symlinks, maintenance scripts, submodules, externals definitions, and so forth, but the results are generally unsatisfactory and error prone. Use them when you have to, and avoid them when you can.
This is definitely a situation where your mileage may vary, but the three Sensible Rules work well for me.
I am not even sure if something like I want is possible, so I am asking you guys to just let me know if anyone did that before. So, my goal is to when I click on "Publish" website in VS2010, to have all javascript files compressed into one, same with css and then in my layout file change the references from all different js and css files to only those two merged ones. Is that doable? Or maybe it's doable but in more manual way?
Of course the goal here is to have only two calls to external files on the website, but when I develop I need to see all files so that I can actually work with it. I guess I could do it manually before each push, but I'd rather have it done automatically using some script or something. I didn't try anything yet, and I am not looking for ready solution, I am just looking to get to know the problem better and maybe some tips.
Thanks a lot!
This is built into ASP.net 4.5. But in the mean time, you should look at the following projects
YUI Compressor
The objective of this project is to compress any Javascript and Cascading Style Sheets to an efficient level that works exactly as the original source, before it was minified.
Cassette
Cassette automatically sorts, concatenates, minifies, caches and versions all your JavaScript, CoffeeScript, CSS, LESS and HTML templates.
RequestReduce
Super Simple Auto Spriting, Minification and Bundling solution
No need to tell RequestReduce where your resources are
Your CSS and Javascript can be anywhere - even on an external host
RequestReduce finds them at runtime automatically
SquishIt
SquishIt lets you squish some JavaScript and CSS. And also some LESS and CoffeeScript.
Combres
.NET library which enables minification, compression, combination, and caching of JavaScript and CSS resources for ASP.NET and ASP.NET MVC web applications. Simply put, it helps your applications rank better with YSlow and PageSpeed.
Chirpy
Mashes, minifies, and validates your javascript, stylesheet, and dotless files. Chirpy can also auto-update T4MVC and other T4 templates.
Scott Hanselman wrote a good overview blog post about this topic a while back.
I voted up the answer that mentioned Cassette but I'll detail that particular choice a little more. Cassette is pretty configurable, but under the most common option, it allows you to reference CSS and Javascript resources through syntax like this:
Bundles.Reference("Scripts/aFolderOfScriptsThatNeedsToLoadFirst", "first");
Bundles.Reference("Scripts/aFolderOfScripts");
Bundles.Reference("Styles/aFolderOfStyles");
You would then render these in your master or layout pages like this:
#Bundles.RenderStylesheets()
#Bundles.RenderScripts("first")
#Bundles.RenderScripts()
During development, your scripts and styles will be included as individual files, and Cassette will try to help you out by detecting changes and trying to make the browser reload those files. This approach is great for debugging into libraries like knockout when they're doing something you don't expect. And, the best part, when you launch the site, you just change the web.config and Cassette will minify and bundle all your files into as few bundles as possible.
You can find more detail in their documentation (which is pretty good though sometimes lags behind development): http://getcassette.net/documentation/getting-started
Have a look at YUI compressor # codeplex.com this could be really helpful.
What I have done before is setup a post-build event, have it run a simple batch file which minimizes your source files. Then if you're in release mode (not in debug mode), you would reference the minimized source files. http://www.west-wind.com/weblog/posts/2007/Jan/19/Detecting-ASPNET-Debug-mode
I haven't heard about publish minification. I think use should choose between dynamical minification like SquishIt or compile time like YuiCompressor or AjaxMinifier.
I prefer compile time. I don't think it's very critical to have to compile time changing files. If you have huge css/js code lines you can choose this action only for release compilation and if it helps publish this files only in needed build cinfigurations.
I don't know if there is any possible way to somehow hook into the functionality from that 'Publish' button/whatever it is, but it's surely possible to have that kind of 'static build process'.
Personally I'm using Apache ANT to script exactly what you've described there. So you're developing on your uncompressed js/html/css files and when you're done, you call like ant build which then minifies, compresses, stripes and publishes your whole web application.
Example script: https://github.com/jAndreas/typeof-NaN-2.0/blob/master/build/build.xml
Is there any software package/library that will produce a consolidated, minified JavaScript file for a production environment, while leaving the original files/references as-is in a development environment (so developers can work independently)?
JAWR does this (and more) for a Java/Groovy environment, but I haven't seen anything like it for the Microsoft .NET/IIS7 stack. Any pointers would be helpful. Thanks!
If you're looking for a good way to automatically compress and combine css & js files here are some options:
Xpedite Not bad: has one big disadantage: you can't combine files (js/css) that are included in usercontrols with the files in your page.
Shinkansen, I don't have a lot of experience with it, but I know it has a lot of configuration options.
The ClientDependency Framework was originally written for Umbraco. Now there is a package available via NuGet for both WebForms and MVC. It works really well and this is my favorite.
We use YUICompressor to minify our Javascript (and CSS) and it works well.
However, we've had to write our own HttpHandler to decide whether to minify or not on the fly, depending on a config setting (but it could equally be on whether it was a DEBUG or RELEASE build).
In fact, we cache the file once minified (or not), so we don't have to do the same process on every request.