We Keep Coding

Dyncamically loading HTML, JS etc. files

I am trying to implement a SPA (single page application) without using any framework. I figured that I need to first download all my application resources (HTML, JS, CSS etc.) on my first page load and then use them later.
Now, since I have to pre-load resource and use it later, so I think I have 2 options:
Option 1: Download them using "script" or "link" tag etc. and then refer the downloaded resource later.
Option 2: Download then using xhr or jquery.get(), put them in a global variable and then use those global variables later.
Problems with options 1:
First and biggest challenge is how do I refer the downloaded resource later. Lets say I have somehow downloaded all my HTML, JS etc. but later, dynamically, how will I refer or read or load them later? I will read it from cache? But what if user has disabled caching of resources?
I know I can download JS files using <script> tag but how do I download HTML resource dynamically? I know some templating engine which can download but I do not want to use any external library.
Problems with options 2:
I could pretty much achieve this except below issues:
I downloaded my resources using jquery.get but since I wanted to refer them before DOM rendering so I couldn't use asynchronous mode. I had to download them synchronously. But then there is warning from XHR that synchronous downloading is deprecated. So, then how I can download a application resource synchronously?
I have to keep the content of downloaded resource in a global variable. So, I am worried that will it be a bad idea because it will consume my browser memory? How does the frameworks like Backbone.js or AngularJS does it?

Problems with options 1:
First and biggest challenge is how do I refer the downloaded resource
later. Lets say I have somehow downloaded all my HTML, JS etc. but
later, dynamically, how will I refer or read or load them later? I
will read it from cache? But what if user has disabled caching of
resources?
First of all, you can't directly download HTML using <script> or <link> tags as far as I know.
You can download scripts and css, the scripts will be compiled and executed once downloaded, and CSS will be applied to the web page. There is no need to refer to them later.
I know I can download JS files using <script> tag but how do I
download HTML resource dynamically? I know some templating engine
which can download but I do not want to use any external library.
In the above bullet you stated you can download HTML using <script> tags and now you're saying you can't in the very next bullet of same option. This is already answered, You can't.
Problems with options 2:
I could pretty much achieve this except below issues:
I downloaded my resources using jquery.get but since I wanted to
refer them before DOM rendering so I couldn't use asynchronous mode.
I had to download them synchronously. But then there is warning from
XHR that synchronous downloading is deprecated. So, then how I can
download a application resource synchronously?
"since I wanted to
refer them before DOM rendering" - consider DOM rendering is the process of downloading and processing the mandatory resources - resources that is needed to present the initial state, and load the resources that is going to be needed later. These will be downloaded by browser (think of the index.html and the <script> and <link> tag resources in it)
I have to keep the content of downloaded resource in a global
variable. So, I am worried that will it be a bad idea because it will
consume my browser memory? How does the frameworks like Backbone.js
or AngularJS does it?
There are techniques like name spacing, IIFE etc used to avoid global variables. And regarding memory, download the extra resources after the document is ready, when required. You wouldn't need any extra resources before document is ready.
Angular has some sort of optional cache, I haven't came across anything like that in backbone.
To conclude, simply load JS and CSS resources that are mandatory using <script> and <link> tags in index.html. And load any other resources required afterwards using AJAX, you wouldn't need to refer to JS or CSS since that is processed by the browser when injected via <script> and <link> tags. You can keep a reference to HTML strings simply using variables like var myHTML= ajaxResponse.
With all that said, look into libraries like requireJS, lazyload etc that already handles stuff like these.

html5: a good loading approach?

I'm writing my first HTML5 + jquery.mobile web app.
The app is basically a menu which redirects to internal pages (data-role="page") defined in the same index.html. I do not write pages as external files to avoid reloading and rewriting the - substantially - same <head>: I suppose it's faster to jump to an internal tag than loading a new page...
Now, I have a page which needs some specific jquery plugins and some specific css. No other page needs these plugins or css.
Of course I could load these js/css in the main <head> section, but this approach would slow the first page loading, uselessly.
I could solve the problem with CSS, with:
$('head:first').append('<link rel="stylesheet" type="text/css" href="' + file + '" />');
I could even solve the problem with JS, but only for 'standard' JavaScript, with something like:
<script>
$(document).ready(function() {
$('#page-availability').live('pageinit', function () {
$.getScript("js/jqm-datebox.core.js");
$.getScript("js/jqm-datebox.mode.calbox.js");
$.getScript("js/jquery.mobile.datebox.i18n.en.utf8.js");
$('#datepicker').data({
"mode": "calbox",
...
});
...
});
...
});
Unfortunately this approach seems not to work (firebug croaks: "TypeError: a.mobile.datebox is undefined"...) with jquery plugins: it looks like they are not evaluated... (even if they are there, before the end of the <head> section, viewing at the "Generated Source"...).
I'm using Firefox (15) to debug, but I suppose this isn't the point...
Any hint?

The one page approach can be good for mobile if:
You don't have to load too much extra content in order to support all the content the user might show from that one page.
You don't have to load too much code to support all the behaviors.
The typical user actually does go to several different virtual pages so the scheme saves them load time and makes things quicker on subsequent virtual page loads.
Done well, the user gets OK performance on loading the first page and very quick performance when going to the other "embedded" pages that don't have to load new content over the network.
The one page approach is not so good if:
The initial load time is just more than it's worth because of the volume of stuff that must be loaded.
You have to dynamically load content for the sub-pages anyway.
You have SEO issues because the search engine can't really find/properly index all your virtual pages.
So, in the end, it's a real tradeoff and depends very much on how big things are, how many things you're loading and what the actual performance comes out to be. A compact mobile site can serve server-loaded page views from one page to the next pretty quickly if the pages are kept very lightweight and there are very few requests that must be satisfied for each page.
In general, you want to pursue these types of optimizations:
Compress/minify all javascript.
Reduce the number of separate items that must be loaded (stylesheets, javascript files, images).
Reduce the number of sequential things that must be loaded (load one, wait for it to load, load another). Mobile is bad at round-trips and loading lots of resources. It's OK at loading a few things.
Make it easy for the browser to cache javascript files. Use a few common javascript files that each serve the needs of many pages. Loading a little more at the start and then allowing the javascript file to be loaded from cache on all future pages loads is way, way better if the user will be visiting many successive pages on your site. The same is true for external CSS files.
Very very careful of lots of images, even small images. Lots of http requests in order to load a page is bad for load time on mobile and every image you request is an http request (unless it comes from the browser cache).
Make sure your server is configured to maximize browser caching for things that can be effectively cached.
Other things to be aware of:
By default dynamic loading of script files is asynchronous and unordered. If your script files must execute in a specific order, then you will have to either not load them dynamically or you will have to write code (or use a library) that serializes their execution in the desired order.

$.getscript is a shorthand AJAX function, it takes a callback as the second parameter.
Check out the docs:
http://dochub.io/#jquery/jquery.getscript
You could concatenate those scripts and then do your stuff in the callback.

This is not so dissimilar to old Flash asset loading issues.
My strategy for that? load only whats necessary for the initial page view. When its loaded and the page / app is viewable by the user, progressively load all other assets.
If the assets were particularly heavy, then I would disable the link to that specific page until its required assets were loaded.
In this case, you might disable the link to the particular page at the outset, initiate the load of its assets, and when they are ready, enable the link.
Not sure if you're having any syntax issues, but you can certainly just inject a new script element into the head with the correct source, and it will instigate a download (like you are doing with css. But you probably know that ;D )
Cheers

I would just combine/minify and compress all the JS in one file and always load that. This is something (with correct caching) which is only downloaded once so you don't have to worry about performance much.
Of course I could load these js/css in the main section
I often just add it just before the </body> and tag. Also note that besides the fact that .live() is deprecated it is also slow as hell. So don't use it, but use .on().

What are the benefits of putting jQuery references in Masterpage?

I'm currently developing a website with ASP.NET and I always check its performance through Firebug...
Now, my question is,
is it better to put all referencing jquery references in Masterpage?
(Reference all first)
or is it better to put specific jquery reference to a specific Content Page?
(Reference specific only)
Thank you!

I would think it's best to put your stylesheets in the master page, as long as they are site-wide styles. You should also think about compressing them into one download, so you reduce HTTP requests.

Maintainance-wise, I would put each reference where it belongs, depending on the scope of your jquery-referencing object.
For instance, if it's something to do with, say, the main navigation, that's present on all rendered pages, and therefore certainly in your master page, then you have better have your $("#navBar") close to your <div id="navBar">...</div>, i.e. on your MasterPage.
If on the other hand it's something related to a specific content page, let's say that shiny carousel (and its specific jQuery plugin) you need on your homepage, you have better have $('myCarousel').carousel(2); close to you <div id="carousel">, i.e. on your HomePage.aspx content page.
And while you're at dispatching stuff to where they belong, if you can tell for sure that carousel plugin is only required on your homepage, you have better include the plugin on your HomePage.aspx content page only.
Not only will you ease your maintainance, but you will also get benefits performance-wise, as you will be more likely to be initializing variables only when they are used, therefore puting a little bit less memory overhead on the browser. Same stuff about loading plugin-related resources (you may not want each and every page load bloated because your master links to every stuff required somewhere on your site).

Generally speaking, I would encourage you to identify any libraries or plugins which you would be using on multiple pages and include them in a place which will automatically put them within those pages.
For maintainability, I usually mash/minify all of my third-party libraries (such as jQuery, jQuery-UI, Backbone, etc) into a single JS file, along with any plugins for them which I know I will be using throughout the site. The downside to doing this is that you may have one very large JS file which loads the first time the user loads the page - the upside: client-side cache that file, and the user doesn't have to load it again.
The general rule of thumb is: minify the number of bytes that the user has to download, and minify the number of HTTP requests which the user has to make throughout your site. So - by compressing these kinds of files into a single download, and letting it exist on every page with the same URL - you can have a single request which generally gets a 302 response and no download. This is far better than having 5 different plugins which are loaded on different pages, each of which makes a separate HTTP call - even if those calls all receive 302 responses.

It is not a proper Approach to referencing jquery references on Content Page.
Once give jquery references on Master Page and use them on all content Pages. It is, Proper and Optimized approach.

Where to place my JS code and where/how to load multiple jQuery plugins?

I have a couple of questions that are somewhat related so I'm posting them all on a single question on SO...
Question 1:
I'm currently doing this Facebook application where I'm using jQuery UI Tabs, there's only 4 where 2 of them are loaded through Ajax. The main page is index.html, this is where the tabs code is placed and for the 2 tabs loaded through Ajax, I have two different files, tab1.html and tab2.html.
Currently, the jQuery tabs initialization and Facebook JavaScript initialization is done on index.html. Both tab1.html and tab2.html have JavaScript code that belongs to those pages. For instance, tab2.html has a form and there's some JS (with jQuery) code to validate the form, this code is irrelevant to tab1.html as the JS code on tab1.html is irrelevant to tab2.html.
My question is, should I keep doing this or maybe aggregate all the JS/jQuery code in index.html, tab1.html and tab2.html in a single global.js file and then include it in index.html?
I though of doing this but there will be irrelevant code loaded if the user never opens tab1 or tab2. The benefit of using a single global.js file is that I could pack/minify the file, which I couldn't do if I included each code block in each respective tabX.html file.
Question 2:
As I'm using jQuery, I'm also using lots of plugins (actually only 3 for now, but that number can grow). Some of them provide a minified JS and I use those when available, when they are not, I use the normal versions of course.
There's also the requests problem. If I have lots of plugins, say 10, it will be 10 requests for those plugins. And there is also the fact that some plugins are used in tab1.html but not on tab2.html and vice-verse.
How should I load all the plugins in a minified/packed version on a single web request? Should I do that manually before publishing my app (packing and merging them into a single file) or could I use the PHP version of Dean Edwards's Packer and pack/merge all plugins on the fly? Would this be a good approach?
Question 3:
If the answer on Q1 was something like "merge all code in a single global.js file", should I include the global.js file in the packing/merging script I described above on Q2?
Doing this would simplify everything. I could have my development environment properly organized with all .js files, for the plugins and the global.js in the appropriate folders without bothering with anything else. The packing/merging should take care of the rest (pull the files from the respective folders, send the respective JS headers and output one single packed .js file).
The one thing that's confusing me the most is that not all plugins are used for every tab, not all code is for every tab too. Still, a chunk of the code is global to every tab and the index. This also simplifies everything as: a) I don't have to worry to add the needed code to each tabX.html file and can I simply look at them as HTML templates and nothing else; b) I don't have to be bothered in including the necessary plugins where I need them as I'm currently using $.getScript() from jQuery to load the plugins I need when and only when I need them, but I'm not sure this is a good approach and the code feels dirty and ugly like this.

Question 1:
Pack them all into a single .js file. This will make maintenance easier, and the tiny bit of overhead for the user loading a little js they they potentially may not use does not matter. I would also let Google load the jQuery library for you and then have all of your js code in a single separate file.
Question 2:
As these plugins don't really change I would manually combine them. Closure Compiler is good at this. When minifying use the highest setting that does not give any warnings.
Question 3:
Yes you will want to minify the global.js
When the browser downloads the global.js it's cached for an amount of time. Thus when you call the entire global.js again on a different page, its not re-downloaded it looks at your local copy first. So you do a little bit more work at first on the initial download, but from then on, it should be quicker.

Generally best practices related to javascript for speeding up website loads are:
Minify all javascript and put all of it into a single file (make as much of your javascript external as possible).
Put javascript at the bottom of the document.
Force web server to assign expiration date in the future and use a timestamped query string to invalidate old versions of javascript files, this will prevent unnecessary requests for your javascript if it has not changed. (ie: in httpd.conf ExpiresByType application/x-javascript "access plus 1 year", in your document: <script type="text/javascript" src="/allmy.js?v=1285877202"></script>)
Configure your web server to gzip all text files.

The main reason why you should keep too much javascript away from tab pages is because it will kill user experience. When a user clicks on a tab for the first time it will grab all the components needed on the fly which makes it kinda sluggish.
You're question is only semi-specific as we don't know a lot of things about your site like exact file sizes, how the modules are really used.
The general idea would be to find balance between modularity and speed.
When you're combining modules together these are the general ideas you should consider:
how often does this module change?
how often is this module used?
how big is this module (filesize)?
Then put the most used, stable codebase and merge it into one. Then you should include the rest site specific functionality on the tab pages.
Also, make sure to load javascript asynchronously as it won't block rendering of the page (and tabs).

Another combined answer:
if adding all the JS together in packed/minifed version generates no more than 30k of file size you're better off combining it. A single extra connection for a file (assuming it's not cached) is worth 10-20k of extra JS download. This has to do with browsers opening and closing connections vs streaming extra 20k on an established connection. The threshold also depends on your user distribution. If you have a lot of dial-up or low bandwidth users your threshold will be smaller.
I typically recommend combining and loading as 1 file unless the library is very obscure and requires a very edge case for it to be triggered on a page. Ex: Hover triggers functionality Y but it's on a feedback widget that gets less than 1% of traffic- don't bother combining.
Minifying and Packing is a little overrated these days. With the vast majority of browsers supporting gZip the amount of data consolidation gZip provides of the file over the wire during browser transmission has virtually the same effect as min/pack. However, there is a small cost on the browser to unpack it. Having said that, it's still good practice to min/pack the code since not all browsers support it, you may not want the file to be gZip enabled, etc.
I've used online packers against 3rd party module and it works fairly well. However, there are times when it can cause an issue so make sure to test your manually packed version before deploying.
Alternate:
If you feel that your users will rest on your index page for longer than 10 seconds you could pre-load the additional libraries separately using Js Loader Prototype pattern.

Steve Souder's Even Faster Websites is a book you should look into.
Firstly one experience slowdowns because whenever an external script is linked the browser waits for the script to download, parse and then execute. After this only it regains processing rest of the request. So to avoid such slow downs one can look at parallely downloading the scripts. Few techniques are Ajax the scripts if the scripts are in the same domain or use Script Dom element or Script in iframe if the scripts are on external domains
Q1 : For me modularising all the content is a better option with respect to further development if the page content has to be changed constantly. Responsiveness is very important for the end user. A small global.js will help in getting the app up and running.Parallely one can download the tabX.html.
Q2: As the jquery plugins rarely change. The plugins for the tabX.html pages can be downloaded parallely and locally cached so when the tabX.html is loaded the required plugins need not be fetched. SO all the plugins required by the main page should be in one single file and the ones used by the tabX.html's should be in different files.
Q3 : its a personal choice here. Do you want it to be developer friendly or user friendly. I bank on user friendliness. Making responsive and efficient apps is our job !!!. All the advantages of packing everything into a singe files is you will have ease in development. Well ugly code begets beautiful apps :). Users are speed-aholics. For eg. when google changed its 10 results per page to 20 they saw a considerable drop in search queries. So my opinion is not to pack all of them into one and load each parallely
some of the techniques and relevant links on testing each:
XHR eval /ajax : http://stevesouders.com/cuzillion/?ex=10009
XHR Injection : http://stevesouders.com/cuzillion/?ex=10015
Script in Iframe : http://stevesouders.com/cuzillion/?ex=10012
Script DOM element : http://stevesouders.com/cuzillion/?ex=10010

Question 1:
The best practice would be to place all js files in a single "global" file. This minimizes your HTTP Requests. Let's say you have 5 plug-ins, this would me you need to do 5 request, wherein if you combine them as one, you only need to request it once. This might be a little bit heavy on the first load, but the next time around this file will be cached by the browser, so..no worries about the size. HOWEVER, be careful about the sequence of the scripts when combining it. (I.E. : JQuery script should be placed first on the js file before JQuery UI's)
http://articles.sitepoint.com/article/web-site-optimization-steps/4
http://code.google.com/speed/page-speed/docs/rtt.html
Question 2:
You can do it manually or automatically.Dean Edward's Packer is a good choice. If you're using ASP.NET, you can check MB Compression Handler, if you're using APACHE with PHP perhaps you can change the configuration of your htaccess to gzip it
Question 3:
It'd be better if you pack the "global" javascript file as well. This could save up bandwidth and save more time to load. You got the point, combining all the js files you need for the site will save you time from including individual scripts.

How to determine if a javascript was already loaded by other html file

How to determine if a javascript was already loaded by other html file? I want to reduce the redundant loading of the javascript files to decrease the loading time of my webpages.

If your web server is providing correct caching headers this shouldn't be necessary, the browser will cache the javascript file across multiple requests.
You might want to check out the YDN page Best Practices for Speeding Up Your Web Site

If you want to prevent the files from being downloaded twice then this will be automatic provided they are set to be cacheable (most webservers should set these headers sensibly by default).
If you want to make sure that the include tag happens only once when including files in a dynamic language then you will need some sort of manager. ASP.NET provides a scriptmanager class that does this (among other things). I cannot speak for other languages and frameworks

As Rory says the second request will probably be cached, and noting that this is a bit of a design failure if it can happen, you can understand that the cached file is still going to execute with negative effect.
This is horrible, but you could wrap your JS script like this:
if (!document.foo)
{
//your script here
document.foo = true;
}