Cache busting static assets in Zend Framework 1 application - javascript

I'm feeling pretty lost and stupid in the brave new world of "modern" web applications with node.js, module bundlers, task runners, and such.
I have a working Zend Framework (ZF 1) PHP application (which also embeds WordPress multi-site allowing users to create their own blog sites). It is hosted on an Apache server with mod_php. It uses html tables a fair bit for forms and data displays, but thankfully not for entire page layouts, though the css is based on a fixed-width page of 1000px.
The application began development under the notion that javascript should be used only for "progressive enhancement", though eventually we succumbed to requiring that javascript be enabled in order to get correct behavior. We support signup and login using OAuth2 authentication through several providers (Facebook, Google, LinkedIn, Twitter and others), but only via the server flow not the javascript SDKs. We use jQuery and limited amounts of Zend_Dojo javascript libraries plus a handful of homegrown javascript functions (in addition to whatever WordPress uses).
We fairly recently added an Nginx reverse proxy in front of the original Apache webserver. It hosts our ssl certificate and serves static file assets.
Now we're looking to move toward a more responsive design to better accommodate mobile and tablet users, and musing about progressive web apps. So major changes to css and increased use of javascript are in the cards. Although the Nginx serving of static assets gives us eTags, Google Page Speed Insights tells us that we have blocking downloads of javascript and css resources, and that we don't take advantage of browser caching.
It appears from various articles I've seen that the Webpack bundling tool can provide major help in addressing all of these performance bottlenecks. But for the life of me I don't see how it fits in to this ecosystem. My mental model of how our site works is that an http query is analyzed by PHP code, dispatched to a PHP action routine that accesses session data and our MySQL database, and then outputs html via phtml templates (ZF1 view scripts) that contain embedded PHP tags. The phtml templates may contain <script>, <style>, and <img> tags, either directly or by being injected into the html by other PHP functions that manage the overall page layout and the content of its <header> section.
But when I look at Webpack, it seems like it's expecting some sort of top-level javascript file from which it can build a dependency tree of other javascript and css files via import or require directives, or something. And it somehow supports cache-busting by hashing the contents of static asset files, creating new files from them with the hash value embedded in the file name, and editing the references to those files to include the hash value. But for this application, all of the references to javascript, css, and image files appear either in .phtml files (usually within embedded <?php> tags) or in pure .php files. Yet webpack doesn't appear to process php files at all - so I don't see how it can either find the references to javascript, css, and image files, or edit them to include a hash! And the articles I've seen about using webpack in PHP projects don't seem to mention this issue at all. There's an html loader, but not one for PHP. Is there some sort of standard practice about using javascript in an independent modular fashion within PHP sites instead of using <script> tags that I just don't know about?
And finally, different web pages have different requirements for javascript and styles, while webpack seems to want a single javascript main entry point from which all dependencies can be found. Does using webpack in this ecosystem imply making a separate webpack project for each page? I've read lots of articles about webpack, but they all seem to be dealing with web apps that are not structured at all like mine!
I did read this answer here on Stackoverflow which I expected would enlighten me. It came fairly close - it explained that I do need to create a top-level index.js file that requires all of the other javascript files. But since different pages use different javascript, I deduced that I would need to create a different index.js file for each page (and thus treat each page as a different project). Can that be true? Many articles talk about "single page apps", so perhaps that's just the assumption in these kinds of descriptions. Or maybe I need to understand "Code Splitting". Maybe if I keep reading that answer over and over again I'll eventually get the gist. It talks about CSS and style-loader and css-loader, but it's not clear to me how the <style> tags present in my .phtml files get processed by them (not to mention styles enqueued in WordPress code). I've attempted the SurviveJs and Official Webpack documentation, but again, they seem to be talking about a different universe than the one I live in. I'm thinking that a Rosetta stone exists somewhere that would map this new world back to traditional PHP apps! Any pointers?

It's an old question, but I attempt to give some pointers, as I've just went through similar hurdle: trying to integrate Webpack with legacy ZF1 app to do:
Asset bundling
Cache busting via appending version hash to filename
Catering for different bundles for different app routes
My approach:
First I checked in newer versions of Zend_View provide some solutions for versioning front end assets. I found this:
https://docs.zendframework.com/zend-view/helpers/asset/
and really liked the idea of encapsulating versioning concerns in separate config file. Obviously to be able to use this format I either have to use this zend_view helper in legacy app, or simply extend legacy zend_view class and add ->asset method that simply reads in the resource map of this format:
'view_helper_config' => [
'asset' => [
'resource_map' => [
'css/style.css' => 'css/style-3a97ff4ee3.css',
'js/vendor.js' => 'js/vendor-a507086eba.js',
],
],
],
The additional advantage of sticking to this format is that once you'll upgrade your app to newer version of Zend Framework or Zend Expressive, you don't need to change anything, just start using Asset helper of modern Zend_View.
Once we have a map like that we need to make webpack write it. So HtmlWebpackPlugin is not restricted to html files. We can write our own template and have a full control over how templates being written with webpack variables (such as asset name and hash). The big advantage here is that webpack doesn't need to overwrite typically numerous view templates that can turn into a mess and has its own problems (eg. what if we include scripts in controllers via headScript calls?) - it only writes the map. This bit solves the issue #2 - cache busting. Issues #1 and #3 - asset bundling and creating different bundles can now be solved native webpack way - by creating multiple bundles and then writing config file using our custom template:
const path = require('path');
module.exports = {
mode: 'development',
entry: {
'js/vendor.js': './frontend/src/js/vendor.js',
'css/style.css': './frontend/src/css/style.js',
// and so on...
},
output: {
filename: '[name]-[hash].js',
path: path.resolve(__dirname, 'public/js'),
},
plugins: [
new HtmlWebpackPlugin({ // Also generate a test.html
filename: 'view-helper-config.php',
template: 'view-helper-config.tpl'
})
]
};
And the view-helper-config.tpl would be:
'view_helper_config' => [
'asset' => [
'resource_map' => [
<% for (var chunk in htmlWebpackPlugin.files.chunks) { %>
<%chunk%> => <%= htmlWebpackPlugin.files.chunks[chunk].entry %>
<% } %>
],
],
],

Related

Meteor and Bootstrap admin

I would like to use meteor with a bootstrap admin, i.e. a bundle including several bootstrap plugins, script and everything typically made as a kind of framework for developing a web application.
Usually those bundles comes with a lot of dependencies, such as external links for fonts, IE hacks as well as their own shipped file of bootstrap, jQuery and other stuff. If we were in a regular php-like framework it would have been fine.
But in order to make such a template be "native" on meteor, I thought to refactor it in such a way that local dependencies (script and css basically) are stored into folders and not loaded via a <script src="…"></script> tag (otherwise the local path would not be found) but I doubt it is really the best practice, this is why I do consider 3 options:
To use the project/public folder in order to store all the bundle's dependencies (as if it would have been in php for example)
I might refactor the bundle's code by removing any script or style tag aimed to import the js or css into the page and add the corresponding js file aside so that meteor will dynamically load it during at runtime
Like in option 2 but instead of using the bundle's jQuery source files I would install the official jQuery's package for meteor (if existing).
The first (1) option should be the quickest one to get something running but it would not be very meteor native. The advantage however would be to keep the code near to the original one and being able to upgrade once a new version of the bundle would be released.
The 2 other options would be much more elegant (especially the third one) but it would involve a lot of refactoring and induce the risk of introducing bugs I did not expected.
My preference for now is the first option one but I'm afraid of not seing the drawbacks of this approach. Does someone have any experience in importing manually the CSS and JS files the "old fashion way" in meteor ? What is the risk of such an approach compared to using the "place in folder to include" way of meteor ?

Organize DRY template for JavaScript experiments?

I'm building a fairly large-scale JavaScript Backbone.js app with this folder organization:
app
index.html
libs
underscore
jquery
[...]
src
utils
modules
[...]
The index.html file basically loads up all the Backbone.js Routers etc. and instantiates AMD modules etc.
Often however, I find the need to create small applications that basically share dependencies with the Big app.
Suppose I need to create 3 small experiments (separate pages) that all load the same usual suspects (underscore, backbone and a couple of util libraries and modules I've written).
They may though differ in: 1) how they extend these JavaScript libraries, 2) what gets instantiated and 3) markup and interaction.
How do I keep this experimentation DRY?
How do I set up this "extendable Template"?
In my opinion, this is where having a good build system comes in. The more complex your setup, the more useful it is to be able to set up configuration files that can keep your dependency management consolidated in one place. This becomes particularly important when:
You need to load the same sets of dependencies on multiple static pages, but your dependency lists change often during development.
You need to be able to easily create compressed versions of the dependencies for a production version. I find this is pretty important with Backbone, because the uncompressed versions are really big but quite useful during development.
I've generally used Apache Ant for this, but there are lot of build systems out there. What I've done in the past is:
Set up an index.tmpl.html file with the core HTML markup and placeholders for JS scripts, CSS files, and underscore templates.
Make a build.properties file that defines my dependency lists. You can group them in different ways under different property names, e.g. lib.scripts.all or util.scripts.all.
In my build process, create a new index.html file, based on index.tmpl.html, with <script> and other tags to load my dependencies. I have different targets to load the raw files or to compress everything into a production-ready single script file.
You can see an example of this setup in this Github project.
If I understand your requirements, you could set up a similar build file with a few tweaks to allow you to set a) the HTML template to use (your default index or another with experiment-specific markup), b) the output file, c) the specific sets of dependencies to load, d) additional dependencies to load, e.g. experiment-specific modules or initialization scripts. You could set these properties up in a specific target (if you think you'll reuse them a few times) or just specify them on the command line when you invoke ant, via the -D flag.
This would allow you a great deal of flexibility to re-use different portions of your code, and has the added benefit of making it easier to move an "experiment" into your core production code, just by including it permanently in your build process.

Are there ways to improve javascript (Dojo) loading?

I'm starting to use the Dojo toolkit and it has rich features like Dijits and themes which are useful but take forever to load.
I've got a good internet connection but those with slower connections would experience rather slow page loads.
This is also a question about heavy vs light frameworks. If you make heavy use of widgets, what are some techniques to keep page load times down?
Dojo has a build system that will drastically improve load times. Take a look at one of the dojo books or the online docs & look at layered builds. In order to do a build, you need to have the "source" (or "full") version of dojo, which has the build tool included -- you can tell if you have this by the presence of the 'util' directory (which is at the same level as dojo, dijit, dojox). If you don't have the full version, go back to the dojo site & delve down into the download area -- it's not completely obvious perhaps.
Anyway, if you have the right version, you basically just need to make a "build profile" file (or files ... aka a layered build), which is essentially your list of dojo.requires that you would normally have in your html. The build system will jam all the javascript code for all the dijits, dojox stuff, etc. together into a "layered build" (a file) and it will run shrinksafe on it, which sort of minifies the code (removes whitespace, shortens names, etc). It will also do some of this to the css files. Aside from making things much smaller, you get just a single file for all the js code (or a few files if you do more than one layer, but most of the time a single layer suffices).
This will improve your load times at least ten-fold, if not more. It might take you a bit of reading to get down the format of the profiles and the build command itself, but it's not too hard really. Once you create a build file, name it something obvious like "mystuff" and then you can dojo.require the "mystuff" file (which will be in the new build directory that is created when you build, then underneath that & hanging out with the dojo.js file in the dojo directory). Requiring in your built file will satisfy all the dojo.require's you normally do (assuming you have them all listed in the profile to build) and things will load very fast.
Here's a link to the older build docs, which mostly still hold true:
http://www.dojotoolkit.org/book/dojo-book-0-9/part-4-meta-dojo/package-system-and-custom-builds
Here's the updated docs (though perhaps a little incomplete):
docs.dojocampus.org/build/index
It reads harder than it really is ... use the layer.profile file in the profiles directory as a starting point. Just put a couple of things & then do a build & see if you get the release directory created (which should be at the same level as dojo, dijit, etc.) and it will have the entire dojo system in it (all minified) as well as your built (layered) stuff. Much faster.
Dylan Tynan
It's not that big (28k gzipped). Nevertheless you can use Google's hosted version of Dojo. Many of your users will already have it cached.
Once you create a build file, name it something obvious like "mystuff"
and then you can dojo.require the "mystuff" file (which will be in the
new build directory that is created when you build, then underneath
that & hanging out with the dojo.js file in the dojo directory).
Requiring in your built file will satisfy all the dojo.require's you
normally do (assuming you have them all listed in the profile to
build) and things will load very fast
Slight correction -- you don't dojo.require that file, you reference it in an ordinary script tag.
<script type="text/javascript" src="js/dojo/dojo/dojo.js" ></script>
<script type="text/javascript" src="js/dojo/mystuff/mystuff.js"></script>
For the directory layout I put the built file "mystuff.js" into the same directory as my package. So at the same level as dojo, dojox, and dijit, I would have a directory named "mystuff", and within that I have MyClass1.js and MyClass2.js. Then a fragment from the profile.js file for the build looks like:
layers:[
{
name: "../mystuff/mystuff.js",
dependencies: [
"mystuff.MyClass1",
"mystuff.MyClass2"
]
},...
I know this is an old thread. But I'm posting this answer for the benefit of other users like me who may read this.
If you are serving from apache there are other factors too. These settings can make a huge difference - MaxClients and MaxRequestsPerChild. You will need to tweak them based on the resources available to your server/machine serving the files.
Changing this worked really well for me.
Using the google CDN is also a good option although it may not be practical in some situations.
Custom build also has an effect as pointed out in other answers.

YUICompressor or similar in PHP?

I've been using yuicompressor.jar on my test server for on-the-fly minimisation of changed JavaScript files. Now that I have deployed the website to the public server, I noticed that the server's policies forbid the use of exec() or its equivalents, so no more java execution for me.
Is there a decent on-the-fly JS compressor implemented in PHP? The only thing resembling this that I was able to find was Minify, but it's more of a full-blown compression solution with cache and everything. I want to keep the files separate and have the minimised files follow my own naming conventions, so Minify is a bit too complex for this purpose.
The tool, like yuicompressor, should be able to take either a filename or JavaScript as input and should either write to a file or output the compressed JavaScript.
EDIT: To clarify, I'm looking for something that does not have to be used as a standalone (i.e. it can be called from a function, rather than sniffing my GET variables). If I just wanted a compressor, Minify would obviously be a good choice.
EDIT2: A lot has changed in the five years since I asked this question. Today I would strongly recommend separating the front-end workflow from the server code. There are plenty of good tools for JS development around and except for the most trivial jQuery enhancements it's a better idea to have a full workflow with automated bundling, testing and linting in place and just deploy the minified bundles rather than the raw files.
Yes there is, it's called minify.
The only thing in to worry about in the way of complexity is setting up a group, and there's really nothing to it. Edit the groupsConfig.php file if you want multiple JS/CSS in one <script> or <link> statement:
return array(
'js-common' => array('//js/jquery/jquery-1.3.2.min.js', '//js/common.js', '//js/visuals.js',
'//js/jquery/facebox.js'),
'css-common' => array('//css/main.css', '//css/layout.css','//css/facebox.css')
);
To include the above 'js-common' group, do this:
<script type="text/javascript" src="/min/g=js-common"></script>
(i know i was looking for the exact same thing not knowing how to deal directly with the jar file using php - that's how i ended up here so i'm sharing what i found)
Minify is a huge library with tons of functionalities. However the minifying part is a very tiny class : http://code.google.com/p/minify/source/browse/trunk/min/lib/Minify/YUICompressor.php
& very very easy to use :
//set the path to the jar file
Minify_YUIcompressor::$jarFile=_ROOT.'libs/java/yuicompressor.jar';
//set the path to a writable temp folder
Minify_YUIcompressor::$tempDir=_ROOT.'temp/';
//minify
$yourcssminified=Minify_YUIcompressor::minifyCss($yourcssstringnotminified,$youroptions)
same process for js, if you need more functionalities just pick from the library & read the source to see how you can make direct call from your app.
I didn't read the question well, since minify is based on using the jar files, the op can't use it anyway with his server config
Minify also include other minifying methods than yui, for example:
http://code.google.com/p/minify/source/browse/trunk/min/lib/JSMinPlus.php?r=443&spec=svn468
Try Lissa:
Lissa is a generic CSS and JavaScript loading utility. Lissa is an extension of the YUI PHP Loader aimed at solving one of the current loader limitations; combo loading. YUI PHP Loader ships with a combo loader that is capable of reducing HTTP requests and increasing performance by outputting all the YUI JavaScript and/or CSS requirements as a single request per resource type. Meaning even if you needed 8 YUI components which ultimately boil down to say 13 files you would still only make 2 HTTP requests; one for the CSS and another for the JavaScript. That's great, but what about custom non-YUI resources. YUI PHP Loader will load them, but it loads them as separate includes and thus they miss out on benefits of the combo service and the number of HTTP requests for the page increases. Lissa works around this limitation by using the YUI PHP Loader to handle the loading and sort of YUI and/or custom resource dependencies and pairs that functional with Minify.

How do you manage your Javascript files?

Nowadays, we have tons of Javascript libraries per page in addition to the Javascript files we write ourselves. How do you manage them all? How do you minify them in an organized way?
Organization
All of my scripts are maintained in a directory structure that I follow whenever I work on a site. The directory structure normally goes something like this:
+--root
|--javascript
|--lib
|--prototype.js
|--scriptaculous
|--scriptaculous.js
|--effects.js
|--..
|--myOwnScript.js
|--myOwnScript2.js
If, on the off chance, that I'm working on a team uses an inordinate amount of scripts, then I'll normally create a custom directory in which we'll organize scripts by relationship. This doesn't happen terribly often, though.
Compression
Though there are a lot of different compressors and obfuscators out there, I always come back to YUI Compressor.
Inclusion
Unless a site is using some form of a master page, CMS, or something that dictates what can be included on a page beyond my control, I only included the scripts necessarily for the given page just for the small performance sake. If a page doesn't require any script, there will be no script inclusions on that page.
First of all, YUI Compressor.
Keeping them organized is up to you, but most groups that I've seen have just come up with a convention that makes sense for their application.
It's generally optimal to package up your files in such a way that you have a small handful of packages which can be included on any given page for optimal caching.
You also might consider dividing your javascript up into segments that are easy to share across the team.
Cal Henderson (of Flickr fame) wrote Serving JavaScript Fast a while back. It covers asset delivery, not organization, but it might answer some of your questions.
Here are the bullet points:
Yes, you ought to concatenate JavaScript files in production to minimize the number of HTTP requests.
BUT you might not want to concatenate into one giant file; you might want to break it into logical pieces and spread the transfer cost over several pages.
gzip compression is good, but you shouldn't serve gzipped assets to IE <= 6, so you might also want to minify/compress your JavaScript.
I'll add a few bullet points of my own:
You ought to come up with a solution that works for both development and production. In development mode, it should pull in extra JavaScript files on demand; in production it should bundle everything ahead of time. Switching from one behavior to the other should be as easy as setting a flag.
Rails 2.0 handles all this through an asset cache; other web app frameworks might offer similar solutions.
As another answer suggests, placing third-party libraries in a lib directory is a good start. You can also divide your own JS files into sub-directories if it makes sense. Ideally, you'll be able to arrange them in such a way that the files in a given sub-directory can be concatenated into one file.
I will have a folder for all javascript, and a sub folder of that for 3rd party/shared libraries, and sub folders for each component of the site to keep everything organized.
For example:
/
+--/javascript/
+-- lib/
+-- admin/
+-- compnent1/
+-- compnent2/
Then run everything through a minifier/obfuscator during the build process.
I'v been using this lately:
http://code.google.com/apis/ajaxlibs/
And then have a "jscripts" folder where I keep my custom code.
In my last project, we had three kinds of JS files, all of them inside a JS folder.
Library code. A bunch of functions used on most all of the pages, so they were put together in one or a few files.
Classes. These had their own files, organized in folders as needed, but not necessarily so.
Ad hoc JS. Code that was specific to that page. These were saved in files that had the same name as the JSP pages they were supposed to run in.
The biggest effort was in having most of the code on the first two kinds, having custom code only know what to call, and when.
This might be a different approach than what you're looking for, but I've been playing around with the idea of JavaScript templates in our blog engine. In a nutshell, you assign a Javascript template to a page id using the database and it will dynamically include and minify all the JavaScript files associated with that template and create a file in a server-side cache with the template id as a file name. When a page is loaded, it calls the template file which first checks if the file exists in the cache and loads it if it does. If it doesn't exist, it creates it on the fly and includes it. I also use the template file to gzip the conglomerate JavaScript file.
The template idea would work well for site-wide JavaScript (like a JavaScript library), but it doesn't cover page-specific JavaScript. However, you can still use the same approach for page specific JavaScript by including a second file that does the same as above.

Categories