I'm working on a large backbone single page application where we've all the JS files specified in order just before </body> tag. Then, for production, using grunt-usemin we concatenate and make single minified app.min.js file for production. Also all the templates are converted to JS using grunt-contrib-handlebars and concatenated into the same app.min.js file.
As the code is growing, size of app.min.js file has gone up to 1.25MB and still this application has long way to go. There are many major sections yet to be developed.
At the same time, I don't want to load 5 or more JS files and same number of templates when user visits to each screen. Rather, I wanted to have modules and load single module file and user can browse through whole of the module without loading any other JS file.
Also I want those module files to be minified and optimized and revved (for cache busting) for production
I was looking through require.js and it's optimizer. I've just started implementing one of our next module using require.js and when I came to building for deployment, it seems that r.js optimizer creates single output file which brings me back to square.
Is it not possible to do what I'm looking for using require.js, or am I missing something? Or is there any better solution of my problem than require.js?
NOTE: Though there is a benefit of using loader even though single big file is loaded at once, I don't want to load whole code when I just need one module of code.
you can build into multiple files.
inside requirejs:compile:options: do the below
modules: [
{
name: '../build1',
include: [
// 3rd party libs
'backbone', //the reference to these files are present in paths: property
'underscore'
]
},
//build 2
{
name: '../build2',
include: ['build2ReqFile1','build2ReqFile2'],
// Excludes all nested dependencies and built dependencies from "common"
exclude: ['../build1','build2UnReqFile1']
},
//build 3
{
name: '../build3',
include: ['build3ReqFile1','build3ReqFile2'],
// Excludes all nested dependencies and built dependencies from "common"
exclude: ['../build1','../build2','build3UnReqFile1']
},
]
Updating with explanation
yep this is for single page application.
Few things to understand here - whenever you define i.e. define('app/test',['app/load1','app/load2'],function(load1Ref, load2Ref){}), new script tag is added to dom and a data-xyz = name of module i.e. <script data-xyz = 'app/test' src = '' type='text/javascript'> is added. I dont remember what the name of xyz is. The returned value of the define is stored in the context data-xyz. So when ever you do require(['app/test']) it returns the value present in data-xyz of app/test script.
Now when you build, all the require and defines that are referenced are built into one single .js. SO by doing the above you can have multiple builds where in you mention which of the defines each of these individual builds should include or exclude.
So these builds are nothing all the javascript files minified and compiled into one/multiple files. Depending on which files you require you need to include these built files.
If you require 'build2ReqFile1','build2ReqFile2' then you can just include build2.js in your code, if you want backbone and underscore along with 'build2ReqFile1','build2ReqFile2' then you need to include build2.js and build1.js
Is it clear?
Related
Maybe I'm trying to do something silly, but I've got a web application (Angular2+), and I'm trying to build it in an extensible/modular way. In particular, I've got various, well, modules for lack of a better term, that I'd like to be able to include or not, depending on what kind of deployment is desired. These modules include various functionality that is implemented via extending base classes.
To simplify things, imagine there is a GenericModuleDefinition class, and there are two modules - ModuleOne.js and ModuleTwo.js. The first defines a ModuleOneDefinitionClass and instantiate an exported instance ModuleOneDefinition, and then registers it with the ModuleRegistry. The second module does an analogous thing.
(To be clear - it registers the ModuleXXXDefinition object with the ModuleRegistry when the ModuleXXX.js file is run (e.g. because of some other .js file imports one of its exports). If it is not run, then clearly nothing gets registered - and this is the problem I'm having, as I describe below.)
The ModuleRegistry has some methods that will iterate over all the Modules and call their individual methods. In this example, there might be a method called ModuleRegistry.initAllModules(), which then calls the initModule() method on each of the registered Modules.
At startup, my application (say, in index.js) calls ModuleRegistry.initAllModules(). Obviously, because index.js imports the exported ModuleRegistry symbol, this will cause the ModuleRegistry.js code to get pulled in, but since none of the exports from either of the two Module .js files is explicitly referenced, these files will not have been pulled in, and so the ModuleOneDefinition and ModuleTwoDefinition objects will not have been instantiated and registered with the ModuleRegistry - so the call to initAllModules() will be for naught.
Obviously, I could just put meaningless references to each of these ModuleDefinition objects in my index.js, which would force them to be pulled in, so that they were registered by the time I call initAllModules(). But this requires changes to the index.js file depending on whether I want to deploy it with ModuleTwo or without. I was hoping to have the mere existence of the ModuleTwo.js be enough to cause the file to get pulled in and the resulting ModuleTwoDefinition to get registered with the ModuleRegistry.
Is there a standard way to handle this kind of situation? Am I stuck having to edit some global file (either index.js or some other file it references) so that it has information about all the included Modules so that it can then go and load them? Or is there a clever way to cause JavaScript to execute all the .js files in a directory so that merely copying the files it would be enough to get them to load at startup?
a clever way to cause xxJavaScriptxx Node.js to execute all the .js files in a directory:
var fs = require('fs') // node filesystem
var path = require('path') // node path
function hasJsExtension(item) {
return item != 'index.js' && path.extname(item) === '.js'
}
function pathHere(item) {
return path.join('.', item)
}
fs.readdir('./', function(err, list) {
if (err) return err
list.filter(hasJsExtension).map(pathHere).forEach(require) // require them all
})
Angular is pretty different, all the more if it is ng serve who checks if your app needs a module, and if so serves the corresponding js file, at any time needed, not at first load time.
In fact your situation reminds me of C++ with header files Declaration and cpp files with implementation, maybe you just need a defineAllModules function before initAllModules.
Another way could be considering finding out how to exclude those modules from ng-serve, and include them as scripts in your HTML before the others, they would so be defined (if present and so, served), and called by angular if necesary, the only cavehat is the error in the console if one script tag is not fetched, but your app will work anyway, if it supposed to do so.
But anyway, it would be declaring/defining those modules somewhere in ng-serve and also in the HTML.
In your own special case, and not willing to under-evalute ng-serve, but is the total js for your app too heavy to be served at once? (minified and all the ...), since the good-to-go solution may be one of the many tools to build and rebuild your production all.js from your dev js folder at will, or like you said, with a drag&drop in your folder.
Such tool is, again, server-side, but even if you only can push/FTP your javascript, you could use it in your prefered dev environment and just push your new version. To see a list of such tools google 'YourDevEnvironment bundle javascript'.
To do more with angular serve and append static js files under specific conditions, you should use webpack so the first option i see here is eject your webpack configuration and after that you can specify what angular should load or not.
With that said, i will give an example:
With angular cli and ng serve any external javascript files you wanna include, you have to put them inside the scripts array in the angular-cli.json file.However you can not control which file should be included and which one not.
By using webpack configuration you can specify all these thing by passing a flag from your terminal to the webpack config file and do all the process right there.
Example:
var env.commandLineParamater, plugins;
if(env.commandLineParamater == 'production'){
plugins = [
new ScriptsWebpackPlugin({
"name": "scripts",
"sourceMap": true,
"filename": "scripts.bundle.js",
"scripts": [
"D:\\Tutorial\\Angular\\demo-project\\node_moduels\\bootstrap\\dist\\bootstrap.min.js",
"D:\\Tutorial\\Angular\\demo-project\\node_moduels\\jquery\\dist\\jquery.min.js"
],
"basePath": "D:\\Tutorial\\Angular\\demo-project"
}),
]}else{
plugins = [
new ScriptsWebpackPlugin({
"name": "scripts",
"sourceMap": true,
"filename": "scripts.bundle.js",
"scripts": [
"D:\\Tutorial\\Angular\\demo-project\\node_moduels\\bootstrap\\dist\\bootstrap.min.js"
],
"basePath": "D:\\Tutorial\\Angular\\demo-project"
}),
]
}
then:
module.exports = (env) => {
"plugins": plugins,
// other webpack configuration
}
The script.js bundle will be loaded before your main app bundle and so you can control what you load when you run npm run start instead of ng-serve.
To Eject your webpack configuration, use ng eject.
Generally speaking, when you need to control some of angular ng-serve working, you should extract your own webpack config and customize it as you want.
That's something I've struggled with for more than a year now, I don't get how we are supposed to load a JS file that contains several AMD modules at once to avoid making as many HTTP requests as there are JS files.
Since we have to define each module separately in the RequireJS config, how is it possible to load only one merged JS file containing all modules at once?
Here is my RequireJS loader: https://gist.github.com/Vadorequest/9553eaf27ac1f469cf63
In that file, what I'd like to merge are:
The requirejs libs (domReady, text, markdown)
The shared source code between the server and the client (Lang, MessageLang... View)
Because these files will increase progressively and increase the number of HTTP calls.
You will have to do this in a build step. Have you consider using RequireJS Optimizer?
The documentation is pretty solid and you will need to add a couple of parameters to your require config:
{
baseUrl: ".",
paths: {
jquery: "some/other/jquery"
},
name: "main",
out: "scripts.js"
}
This will generate one file (scripts.js) with all your files in it. There is also a bundle option if you like to group some files together requirejs bundles
I personally use gulp to take care of my build process so I actually use gulp-requirejs-bundler but the same principles apply.
I am using RequireJS to modularize my code. The website I am using will have several distinct page categories. For each one of them I want to load general JS file with all the dependencies that exist across all page categories, jQuery being most obvious one. I have managed to do that already.
For the rest of the pages, I want to have separate JS files. Each page has its own module but I am thinking that it would make more sense if I download several pages modules all at once and this way when the user comes to other pages, he will have the JS file already cached and won't have to download.
For now I tried to concatenate these modules which did work rather well. However, I am not sure how to load the specific module from a file. So far I am doing this:
require(['./js/common'], function (common) {
require(['public']);
});
common is the dependencies file and public is my concatenated file with all the dependencies. Inside it has modules for each page but I don't know how to call a specific one. Any ideas?
Take a look at the bundles option for RequireJS' runtime configuration. In your case it would be something like:
bundles: {
public: ['mod1', 'mod2', ...]
}
I've used mod1, mod2 because I don't see the name of the actual modules in your question but what you'd want there are the names of the modules inside the public bundle and that you want to load individually. With this, RequireJS will know that when you want to load such module, it has to get them from public instead of searching for them indivdually.
Then you should be able to change your loading code to:
require(['./js/common'], function (common) {
require(['mod1']);
});
Again, mod1 is for illustration.
Take a look at require-lazy.
It allows you to bundle independent parts of the application into separate bundles. From the developer's point of view, instead of:
define(["module1", "module2"], function(module1, module2) {...});
you write:
define(["lazy!module1", "lazy!module2"], function(module1, module2) {...});
Modules 1 & 2 will not be loaded upfront; they and their dependencies will be loaded lazilly when you call:
module1.get().then(function(realModule1) {
// here you have access to the real module1
});
Check out the examples for a few use cases.
Following the instructions from http://dojotoolkit.org/documentation/tutorials/1.9/dojo_config/, I have been able to create my own custom build.
However, in the dist folder, there are always the dojo and dijit folder beside my app folder. I don't need these and I will serve the content of these two packages from Google CDN.
How would I achieve this without either manually or programmatically deleting them after each build?
Here's my dojoConfig.js
var dojoConfig = {
....
useXDomain: true,
xdWaitSeconds: 10,
async: true,
tlmSiblingOfDojo: false,
packages: [
{name:'dojo',location:'//ajax.googleapis.com/ajax/libs/dojo/1.9.2/dojo/'},
{name:'dijit',location:'//ajax.googleapis.com/ajax/libs/dojo/1.9.2/dijit/'},
'dojotut'
],
parseOnLoad: true,
deps: [ 'dojo/ready', 'dojo/parser', 'dojotut' ],
callback: function (ready, parser, dojotut) {
dojotut.init();
}
};
Best regards!
If you'Re using dojop from CDN you don't need a custom build at all. Main reason for custom builds is to embed your code with the dojo files according to the defined layers in the build configuration made within your profile file (according to tutorial: http://dojotoolkit.org/documentation/tutorials/1.8/build/)
In fact what the dojo build system does is resolve require and define dependencies, group them according to the layers, write them into one file (e.g. modules layered with dojo/text will be added to the dojo/text output file), and minify afterwards.
That doesn't remove the single files as they may be required as well but requiring dojo at the beginning loads all embedded modules as well within just ONE HTTP get for example. Thats reducing latency a lot.
My proposal would be if you just want to distribute your app:
Use the dojo CDN
Copy all your app code together into one minified file (e.g. by using grunt) and load that file as well, resulting in almost as few GET from the browser.
I have a directory like below:
/folder/b.js
/folder/jQuery.js
/folder/a.js
/folder/sub/c.js
I want to minify all these js files in one js file in order:
jQuery.js -> a.js -> b.js -> c.js
Q:
1.How can I do it via grunt-contrib-uglify?(In fact, there are lots of files, it is impractical to specify all source filepaths individually)
2.btw, How can I get unminified files when debug and get minified single file when release and no need to change script tag in html(and how to write the script tag)?
Good questions!
1) Uglify will reorder the functions in the destination file so that function definitions are on top and function execution on bottom but it seems that it will preserve the order of the function executions.
This means that the function jQuery runs to define its global functions will be put first if you make sure jQuery is mentioned first in Uglify's config in the Gruntfile.
I use this config:
uglify: {
options: {
sourceMap: true
},
build: {
files: {
'public/all.min.js': ['public/js/vendor/jquery-1.10.2.min.js', 'public/js/*.js'],
}
}
}
2) I don't think there is one definite way to accomplish this. It depends on what web framework, templating framework and what kind of requirements you have. I use express + jade and in my main jade layout I have:
if process.env.NODE_ENV === 'production'
script(src='/all.min.js')
else
script(src='/js/vendor/jquery-1.10.2.min.js')
script(src='/js/someScript.js')
script(src='/js/otherScript.js')
In my package.json I have:
"scripts": {
"postinstall": "grunt"
},
This means that when I run npm install on deploy (on Heroku) grunt is run to minify/concat files and when the app is started with NODE_ENV=production the minified client side javascript is used. Locally I get served the original client side javascripts for easy debugging.
The two downsides are:
I have to keep the two lists of script files in sync (in the Gruntfile and in the layout.js) I solve this by using *.js in the Gruntfile but this may not suite everyone. You could put the list of javascripts in the Gruntfile and create a jade-template from this but it seems overkill for most projects.
If you don't trust your Grunt config you basically have to test running the application using NODE_ENV=production locally to verify that the minification worked the way you intended.
This can be done using the following Grunt tasks:
https://github.com/gruntjs/grunt-contrib-concat concatenates
files
https://github.com/gruntjs/grunt-contrib-uglify minifies
concatenated files
EDIT
I usually run all my files through a Grunt concatenation task using grunt-contrib-concat. Then I have another task to uglify the concatenated file using grunt-contrib-uglify.
You're probably not going to like this, but the best way is to define your js source files as AMD modules and use Requirejs to manage the order in which they load. The grunt-contrib-requirejs task will recurse your dependency tree and concatenate the js files in the necessary order into one big js file. You will then use uglify (actually r.js has uglify built-in) to minify the big file.
https://github.com/danheberden/yeoman-generator-requirejs has a good example gruntfile and template js files to work from.
EDIT
I've recently started using CommonJS modules instead of AMD since it's much closer to the ES6 module spec. You can achieve the same results (1 big complied+concatenated js file) by running commonjs modules through Browserify. There are plugins for both grunt and gulp to manage the task for you.
EDIT
I'd like to add that if your site is written using ES6 that Rollup is the best new concatenating package. In addition to bundling your files, it will also perform tree shaking, removing parts of libraries you use if included via an import statement. This reduces your codebase to just what you need without the bloat of code you'll never use.
I don't think you can do this with the uglify task alone, but you have a multitude of choices which might lead to your desired outcome.
A possible workflow would be first concatenating (grunt-contrib-concat) the files in order into one single file, and put this concatenated file through uglify. You can either define the order for concat in your Gruntfile, or you use on of those plugins:
First one would be https://github.com/yeoman/grunt-usemin, where you can specify the order in your HTML file, put some comments around your script block. The Google guys made it and it's pretty sweet to use.
Second one would be https://github.com/trek/grunt-neuter, where you can define some dependencies with require, but without the bulk of require.js. It requires changes in your JS code, so might not like it. I'd go with option one.
I ran into the same issue. A quick fix is just to change the filenames - I used 1.jquery.min.js, 2.bootstrap.min.js, etc.
This might be only remotely related to your question but I wanted something similar. Only my order was important in the following way:
I was loading all vendor files (angular, jquery, and their respective related plugins) with a wildcard (['vendor/**/*.js']). But some plugins had names that made them load before angular and jquery. A solution is to manually load them first.
['vendor/angular.js', 'vendor/jquery.js', 'vendor/**/*.js]
Luckily angular and jquery handle being loaded twice well enough. Edit: Although it's not really the best practice to load such large libraries twice, causing your minified file unnecessary bloat. (thanks #Kano for pointing this out!)
Another issue was client-js the order was important in a way that it required the main app file to be loaded last, after all its dependencies have been loaded. Solution to that was to exclude and then include:
['app/**/*.js', '!app/app.js', 'app/app.js']
This prevents app.js from being loaded along with all the other files, and only then includes it at the end.
Looks like the second part of your question is still unanswered. But let me try one by one.
Firstly you can join and uglify a large number of js files into one as explained by the concat answer earlier. It should also be possible to use https://github.com/gruntjs/grunt-contrib-uglify because it does seem to have wildcards. You may have to experiment with 'expand = true' option and wildcards. That takes care of your first question.
For the second part, say you joined and uglified into big-ugly.js
Now in your html you can add following directives:
<!-- build:js:dist big-ugly.js -->
<script src="js1.js"></script>
<script src="js2.js"></script>
<!-- etc etc -->
<script src="js100.js"></script>
<!-- /build -->
And then pass it through the grunt html preprocessor at https://www.npmjs.com/package/grunt-processhtml as part of your grunt jobs.
This preprocessor will replace the entire block with
<script src="big-ugly.js"></script>
Which means that the html file with be semantically equivalent - before and after the grunt jobs; i.e. if the page works correctly in the native form (for debugging) - then the transformed page must work correctly after the grunt - without requiring you to manually change any tags.
This was #1469's answer but he didn't make it clear why this works. Use concat to put all js files into one, this module does this in the order of file names, so I put a prefix to the file names based on orders. I believe it even has other options for ordering.
concat: {
js: {
options: {
block: true,
line: true,
stripBanners: true
},
files: {
'library/dist/js/scripts.js' : 'library/js/*.js',
}
}
},
Then use uglify to create the minified ugly version:
uglify: {
dist: {
files: {
'library/dist/js/scripts.min.js': [
'library/js/scripts.js'
]
},
options: {
}
}
},
If your problem was that you had vendors which needed to be loaded in order (let's say jquery before any jquery plugins). I solved it by putting jquery in its own folder called '!jquery', effectively putting it on top of the stack.
Then I just used concat as you normally would:
concat: {
options: {
separator: ';',
},
build: {
files: [
{
src: ['js/vendor/**/*.js', 'js/main.min.js'],
dest: 'js/global.min.js'
}
]
}
},