Split Webpack CommonsChunkPlugin Output into Multiple Files - javascript

I am using Webpack to bundle my clientside modules and would like to take advantage of parallel asset downloading. I would like html somewhat like this:
<script src="vendor/react.js">
<script src="vendor/underscore.js">
<script src="build/bundle.js">
Where bundle.js contains:
var React = require('react');
var _ = require('underscore');
Note that vendor/react.js and vendor/underscore.js would also be bundled by Webpack.
I know that the Webpack CommonsChunkPlugin can extract all the vendor modules and put them into a single common vendor.js file. Is it possible, though, to split that common output file into two or more output files?

After a bit more digging, I found an answer.
Soundcloud created a webpack plugin specifically for this purpose (see here) though it lacked support for splitting code in the node_modules folder. Another plugin based on it solved that problem and was trivial to set up.

Related

rollup.js how to import additional files without changing source files

my example is based on the rollup.js basic example: https://rollupjs.org/guide/en/#rolluprollup
the project i am working on require additional files to be included and excluded. (i know how to exclude path's using 'rollup-plugin-ignore' plugin.)
how can i tell rollup.js to add files to the import list (same as i was importing them in the source code), without changing the source files. ?
i tried to commonjs plugin
var commonjs = require('rollup-plugin-commonjs');
var include = ["D:\\...abs_path....\\components\\Auth\\FormLogin.jsx"]
commonjs( {"include": include})
but i don't see the 'FormLogin' component in the output file.
is there a more simple way, can you help me please ?
i will appreciate your help.
thank you very much.
Thats a little bit unorthodox because rollup.js uses tree-shaking to analyse your code and only include the used parts and you want something included which isnt used.
the easist way would be concatenation on the command line, like in windows cmd:
type file_from_rollup.js additional.js > bundle.js
second, you could instruct rollup to use two entry points:
rollup main.js libB.js -d ./bundle --format cjs
this will generate two bundled files in the ./bundle folder, both of them tree-shaked
If the FormLogin.jsx is referenced in the source code (seen from the main entry point) try to disable tree shaking like
rollup main.js --no-treeshake --file bundle.js --format cjs
and see if its included. Maybe rollup.js doesnt recognize it corretly: https://rollupjs.org/guide/en/#tree-shaking-doesnt-seem-to-be-working

Using external js file in vue file with Webpack

I am working on a project which is used typescript, vue and webpack together. I have created some components and i can use them by importing. However i have different js files in another root folder like site.js, ruler.js, color.js, speech.js, drware.js and etc. Schema is like below
+|dist
----build.js
+|src
----index.ts
+|main
----Header.vue
----Footer.vue
----Body.vue
+|lib
----site.js
----ruler.js
----drawer.js
----color.js
webpack config is getting index.ts from src folder which is shown above. When I don't use some functions (like jquery plugins or some special funciton) everything is fine. But when i use a functon from site.js webpack fives error like cannot resolve "ruler" from site.js
I have tried to concat by giving second entry in webpack.config.js but it didn' solve my problem. I wonder how to to resolve external js files in vue or ts files using webpack. I alson tried
require(""../src/site.js)
but it didn't work too.
Edit : If i concat the js files manually and give it as script source on html it works without problem but i cannot merge all files like or i don't want to use "gulp" to concat them
Have you tried including a script-loader into your webpack's configuration?
Webpack is a bundler, not a script loader itself. I would recommend you to follow webpack's official instructions to add a script loader.
Good luck!

How to use gulp to build JavaScript bundles?

I want to use gulp to build bundles of JavaScript files.
For example I have the following structure in my project:
/vendor/vendor1/vendor1.js
/vendor/vendor2/vendor2.js
/js/includes/include1.js
/js/includes/include2.js
/js/bundle1.js
/js/bundle2.js
There are vendor includes (1-2), local includes (3-4), and bundle files (5-6).
Vendor includes are just third-party JavaScript libraries installed with bower or composer. They can be CommonJS, AMD or just a plain-old jQuery plugins.
I want to specify dependencies inside of a bundle files like this:
/js/bundle1.js
(function() {
// Vendor includes.
include('vendor1');
include('vendor2');
// Local includes.
include('includes/include1.js');
include('includes/include2.js');
// Some code here.
})();
I want gulp to process this source file and create a final distribution file (bundle) ensuring that all dependencies (includes) are merged together in a single file. So I can include foo.js from my HTML and all dependencies will be available to it.
I want to have a clear and robust system to manage all dependencies inside of a project and build distribution files.
How can I achieve this?
What format should I use for my own scripts (AMD, CommonJS, other)?
How do I specify dependencies in my source bundle files?
How do I build distribution?
Your question is posed as if there's a single answer, but there isn't. The problem you're trying to solve is one that many people have solved in many different ways, and you've identified two of the major options: AMD and CommonJS. There are other ways, but given that you might be new to Javascript dependency management as well as gulp, I'd recommend going with something that's relatively straightforward (even though this subject is inherently not straightforward).
I think the easiest route for you might be:
use CommonJS to express the dependencies
use browserify to resolve them into bundles
in browserify, use the "UMD" method so that you get a single bundle that will work for apps that use either AMD or CommonJS or are not using either of these dependency management systems
The statement in gulp to run browserify as such might look something like:
var browserify = require('gulp-browserify');
gulp.src('bundles/bundle1.js', {read: false})
.pipe(browserify({
'standalone': true
})
.pipe(rename('bundle1Output.js'))
.pipe(gulp.dest('dist'));
That should give you a dist/bundle1Output.js file.
There is a gulp plugin for this:
https://www.npmjs.com/package/gulp-include
It should do what you want, except that in your bundle file instead of this:
(function() {
// Vendor includes.
include('vendor1');
include('vendor2');
// Local includes.
include('includes/include1.js');
include('includes/include2.js');
// Some code here.
})();
You would have to write:
//=require vendor1/**/*.js
//=require vendor2/**/*.js
//=require includes/include1.js
//=require includes/include2.js
// Some code here

Sharing common code across pages with Browserify

I have a fairly large multi-page javascript applications that uses requirejs to organize code. I am researching moving to browserify because I like the simplicity that it offers and I am already used to the node.js module system.
Currently on each page I have javascript that goes like this
<script data-main="/scripts/config/common" src="/scripts/lib/require.js">
<script data-main="/scripts/config/page-specific-js" src="/scripts/lib/require.js">
and I have a common build step and a build for each page. This way the majority of the JS is cached for every page.
Is it possible to do something similar with browserify? Is caching like this even worth it, or is it better to bundle everything into one file across all pages (considering that maybe only one page can depend on a large external library)?
You can use factor-bundle to do exactly this. You will just need to split your code up into different entry points for each file.
Suppose you have 3 pages, /a, /b, and /c. Each page corresponds to an entry point file of /browser/a.js, /browser.b.js, and /browser/c.js. With factor-bundle, you can do:
browserify -p [ factor-bundle -o bundle/a.js -o bundle/b.js -o bundle/c.js ] \
browser/a.js browser/b.js browser/c.js > bundle/common.js
any files used by more than 1 of the entry points will be factored out into bundle/common.js, while all the page-specific code will be located in the page-specific bundle file. Now on each page you can put a script tag for the common bundle and a script tag for the page-specific bundle. For example, for /a you can put:
<script src="bundle/common.js"></script>
<script src="bundle/a.js"></script>
If you don't want to use the command-line version, you can also use factor-bundle from the API:
var browserify = require('browserify');
var fs = require('fs');
var files = [ './files/a.js', './files/b.js', './files/c.js' ];
var b = browserify(files);
b.plugin('factor-bundle', {
outputs: [ 'bundle/a.js', 'bundle/b.js', 'bundle/c.js' ]
});
b.bundle().pipe(fs.createWriteStream('bundle/common.js'));

how to minify js files in order via grunt-contrib-uglify?

I have a directory like below:
/folder/b.js
/folder/jQuery.js
/folder/a.js
/folder/sub/c.js
I want to minify all these js files in one js file in order:
jQuery.js -> a.js -> b.js -> c.js
Q:
1.How can I do it via grunt-contrib-uglify?(In fact, there are lots of files, it is impractical to specify all source filepaths individually)
2.btw, How can I get unminified files when debug and get minified single file when release and no need to change script tag in html(and how to write the script tag)?
Good questions!
1) Uglify will reorder the functions in the destination file so that function definitions are on top and function execution on bottom but it seems that it will preserve the order of the function executions.
This means that the function jQuery runs to define its global functions will be put first if you make sure jQuery is mentioned first in Uglify's config in the Gruntfile.
I use this config:
uglify: {
options: {
sourceMap: true
},
build: {
files: {
'public/all.min.js': ['public/js/vendor/jquery-1.10.2.min.js', 'public/js/*.js'],
}
}
}
2) I don't think there is one definite way to accomplish this. It depends on what web framework, templating framework and what kind of requirements you have. I use express + jade and in my main jade layout I have:
if process.env.NODE_ENV === 'production'
script(src='/all.min.js')
else
script(src='/js/vendor/jquery-1.10.2.min.js')
script(src='/js/someScript.js')
script(src='/js/otherScript.js')
In my package.json I have:
"scripts": {
"postinstall": "grunt"
},
This means that when I run npm install on deploy (on Heroku) grunt is run to minify/concat files and when the app is started with NODE_ENV=production the minified client side javascript is used. Locally I get served the original client side javascripts for easy debugging.
The two downsides are:
I have to keep the two lists of script files in sync (in the Gruntfile and in the layout.js) I solve this by using *.js in the Gruntfile but this may not suite everyone. You could put the list of javascripts in the Gruntfile and create a jade-template from this but it seems overkill for most projects.
If you don't trust your Grunt config you basically have to test running the application using NODE_ENV=production locally to verify that the minification worked the way you intended.
This can be done using the following Grunt tasks:
https://github.com/gruntjs/grunt-contrib-concat concatenates
files
https://github.com/gruntjs/grunt-contrib-uglify minifies
concatenated files
EDIT
I usually run all my files through a Grunt concatenation task using grunt-contrib-concat. Then I have another task to uglify the concatenated file using grunt-contrib-uglify.
You're probably not going to like this, but the best way is to define your js source files as AMD modules and use Requirejs to manage the order in which they load. The grunt-contrib-requirejs task will recurse your dependency tree and concatenate the js files in the necessary order into one big js file. You will then use uglify (actually r.js has uglify built-in) to minify the big file.
https://github.com/danheberden/yeoman-generator-requirejs has a good example gruntfile and template js files to work from.
EDIT
I've recently started using CommonJS modules instead of AMD since it's much closer to the ES6 module spec. You can achieve the same results (1 big complied+concatenated js file) by running commonjs modules through Browserify. There are plugins for both grunt and gulp to manage the task for you.
EDIT
I'd like to add that if your site is written using ES6 that Rollup is the best new concatenating package. In addition to bundling your files, it will also perform tree shaking, removing parts of libraries you use if included via an import statement. This reduces your codebase to just what you need without the bloat of code you'll never use.
I don't think you can do this with the uglify task alone, but you have a multitude of choices which might lead to your desired outcome.
A possible workflow would be first concatenating (grunt-contrib-concat) the files in order into one single file, and put this concatenated file through uglify. You can either define the order for concat in your Gruntfile, or you use on of those plugins:
First one would be https://github.com/yeoman/grunt-usemin, where you can specify the order in your HTML file, put some comments around your script block. The Google guys made it and it's pretty sweet to use.
Second one would be https://github.com/trek/grunt-neuter, where you can define some dependencies with require, but without the bulk of require.js. It requires changes in your JS code, so might not like it. I'd go with option one.
I ran into the same issue. A quick fix is just to change the filenames - I used 1.jquery.min.js, 2.bootstrap.min.js, etc.
This might be only remotely related to your question but I wanted something similar. Only my order was important in the following way:
I was loading all vendor files (angular, jquery, and their respective related plugins) with a wildcard (['vendor/**/*.js']). But some plugins had names that made them load before angular and jquery. A solution is to manually load them first.
['vendor/angular.js', 'vendor/jquery.js', 'vendor/**/*.js]
Luckily angular and jquery handle being loaded twice well enough. Edit: Although it's not really the best practice to load such large libraries twice, causing your minified file unnecessary bloat. (thanks #Kano for pointing this out!)
Another issue was client-js the order was important in a way that it required the main app file to be loaded last, after all its dependencies have been loaded. Solution to that was to exclude and then include:
['app/**/*.js', '!app/app.js', 'app/app.js']
This prevents app.js from being loaded along with all the other files, and only then includes it at the end.
Looks like the second part of your question is still unanswered. But let me try one by one.
Firstly you can join and uglify a large number of js files into one as explained by the concat answer earlier. It should also be possible to use https://github.com/gruntjs/grunt-contrib-uglify because it does seem to have wildcards. You may have to experiment with 'expand = true' option and wildcards. That takes care of your first question.
For the second part, say you joined and uglified into big-ugly.js
Now in your html you can add following directives:
<!-- build:js:dist big-ugly.js -->
<script src="js1.js"></script>
<script src="js2.js"></script>
<!-- etc etc -->
<script src="js100.js"></script>
<!-- /build -->
And then pass it through the grunt html preprocessor at https://www.npmjs.com/package/grunt-processhtml as part of your grunt jobs.
This preprocessor will replace the entire block with
<script src="big-ugly.js"></script>
Which means that the html file with be semantically equivalent - before and after the grunt jobs; i.e. if the page works correctly in the native form (for debugging) - then the transformed page must work correctly after the grunt - without requiring you to manually change any tags.
This was #1469's answer but he didn't make it clear why this works. Use concat to put all js files into one, this module does this in the order of file names, so I put a prefix to the file names based on orders. I believe it even has other options for ordering.
concat: {
js: {
options: {
block: true,
line: true,
stripBanners: true
},
files: {
'library/dist/js/scripts.js' : 'library/js/*.js',
}
}
},
Then use uglify to create the minified ugly version:
uglify: {
dist: {
files: {
'library/dist/js/scripts.min.js': [
'library/js/scripts.js'
]
},
options: {
}
}
},
If your problem was that you had vendors which needed to be loaded in order (let's say jquery before any jquery plugins). I solved it by putting jquery in its own folder called '!jquery', effectively putting it on top of the stack.
Then I just used concat as you normally would:
concat: {
options: {
separator: ';',
},
build: {
files: [
{
src: ['js/vendor/**/*.js', 'js/main.min.js'],
dest: 'js/global.min.js'
}
]
}
},

Categories