In Babel plugins, there's a post hook, which runs after processing a single JS file. However, I want to run some code after all JS files are processed. Is this possible?
For now, I just added a new plugin which runs after the first plugin, but it'll be cleaner if they're one plugin.
The short answer is that Babel does not provide a hook like this because Babel's core transformation system runs one file at a time. The entire API is:
babel.transform("var foo; /* some code */", { filename: "foo.js" });
so it has no way to register something that would run after a group of files has been processed.
The longer answer is that, if you're using Webpack specifically, you can use Babel in combination with a Webpack plugin to collect metadata from each individual file, and then perform some action with all of that metadata, but that is a Webpack/babel-loader-specific feature. An example of one such Webpack plugin is react-intl-webpack-plugin, which is paired with babel-plugin-react-intl.
Related
I have a very old javascript code base and I do not want to use the modern way of compiling all of the javascript files into one using standard webpack because it is not possible due to the way the website code is written.
But I want to write new scripts using modern Javascript (e.g. Promises and Fetch) but still be able to support old browsers like IE11.
I have configured webpack and babel so it gets multiple entry javascript files and for each of them it does the classic transpiling/polyfilling using #babel/preset-env and corejs.
This works and polyfills every script based on the babel target config but it creates one issue. It encapsulates global variables/functions in the script so they are not accessible from other scripts which reference them (yes old javascript). Is there a way to disable this structural modifications?
Also I know I could use only Babel without Webpack for this but the problem is when I try to polyfill e.g. Fetch I have to use https://github.com/github/fetch which cannot be just used with Babel afaik.
Any help appreciated.
I think inevitably your refactorings are modernizing the code, and if you are not careful, one day you can end up bundling everything with webpack;)
The set up you describe, I achieved with with:
module.exports = {
entry: {
messages: "./src/messages",
"hello-world": "./src/hello-world",
},
output: {
library: {
type: "global",
},
filename: "[name].js",
},
};
every export from each file is put directly on window - if you load files in the right order, you can have invisible dependencies maintained across the codebase.
To this setup, you can add babel loader with presets as you stated in your question. Besides, you can tart doing the explicit imports across different files - even if function X is available on global scope, you can migrate some places to import/require it explicitly.
If you want to play with my code yourself, you can find it here:
https://github.com/marcin-wosinek/webpack-legacy/
and here is the example in action:
https://marcin-wosinek.github.io/webpack-legacy/
I'm trying to figure out how to extract some information from babel compilation process.
More specifically, when I run babel (no matter if using Webpack's babel-loader, test frameworks' transformers, Babel's CLI, etc) I'd need to extract some information for each compiled file. Like:
file path
time taken to compile
any other meta data?
What I've tried so far
Speed Measure Plugin for Webpack (link)
Works fine but it provides only Webpack's loaders running time. No info about single compiled files.
Hook into Webpack's compiler/compilation instance
I considered writing a Webpack plugin to hook into the compilation process as described here, but I couldn't find the proper hooks to recognize a file being processed by babel.
Updates
I guess #kidroca pointed out the right direction. More specifically I understand that Babel’s wrapPluginVisitorMethod option is the key for hooking into Babel compilation process.
See babel-minify’s timing plugin.
Related threads:
https://github.com/babel/babel/issues/5340
https://github.com/babel/babel/issues/2206
https://github.com/babel/babel/pull/3659
https://github.com/babel/minify/pull/93
https://github.com/babel/babel/pull/3659
Updates 28/04/18
I eventually tried to wrap a solution into a tool I called babel-timing.
You can use #babel/core and babel.transformSync(code) which will return
Abstract Syntax Tree (AST) information along with some other data. And you can also add some logic to measure the performance of this method
I've setup a minimal repo and played with it myself a little bit: https://github.com/kidroca/babel-meta
Basically you can run npm run analyze-file ./es6-src/es6-module.js or npm run analyze-dir ./es6-src/es6-module.js and checkout the results
This will return:
{
"filename": "/full/path/to/src/file.js",
"cwd": "current/dir",
"ast": "ast information json - lines, comments, and other info",
"executionTime": "execution time in ms",
/* a lot of other info */
}
You can modify the analyze.js file to filter out the info you need
You can modify the .babelrc file to control the transformation and add/remove plugins
I'm trying to build a new project with ES6 modules without bundling. I still want to use babel-7 to translate TypeScript and JSX to JS. I find it hard to figure out how to set up a development-server for it. I couldn't find any kind of "babel-dev-server" that works similar to webpack-dev-server (hot-module-reloading, browser-sync, file-watcher).
One possibility would be to use browser sync as a static server on e.g. dist and run something like babel src --out-dir dist --watch in parallel. But this excludes hot-reloading and seems a bit clumsy to me. Besides, it would still be useful for build- and dev-steps if you could give the JS-files a hash to control caching better. Or can I configure a build-tool like webpack so that it doesn't perform bundling but still performs some transformations (like putting the hashs in the filenames in imports)?
Prototyping way
A very simple way to do this is to see the server and the transpiling as separate steps
You could use a standalone version of babel as the first script that you load, so you can write jsx inside your html document of javascript files without compiling them.
Simply add on of the cdn links from https://cdnjs.com/libraries/babel-standalone/ as a script like so:
<html>
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/babel-standalone/7.0.0-beta.3/babel.min.js"></script>
<script src="/your/jsx/here.js"></script>
<script>
// or here
</script>
</head>
<body>
<div id="application" />
<noscript>This app needs javascript enabled in order to run.</noscript>
</body>
</html>
This would allow you to really quickly prototype things using any webserver that watches files. You can do this using any task runner plugin (i.e. for grunt or gulp) or if you are using visual studio have a look at LiveServer plugin.
When you are moving to production grade you might not want to include the entire babel library. See the other two approaches.
Webpack way
You're asking how to use webpack without bundling. Which can be done using file loader plugin to load every file separately, using a glob pattern. Do make sure whether this is indeed what you need. If all you want is to simply debug your code an relate it back to the original file after compiling, all you need is a standard webpack configuration using bundling and sourcemaps.
Taskrunner way
One way to have even more control over how each file is processed, you can use a taskrunner to do the compile step for you. Below is a simplified example configuration for taskrunner https://gulpjs.com/.
gulpfile.js
const gulp = require('gulp');
const watch = require('gulp-watch');
const webpackStream = require('webpack-stream');
const webpack = require('webpack');
const eslint = require('gulp-eslint');
gulp.task('watch', function() {
return watch('src/**.js', ['compile']);
});
gulp.task('lint', function() {
return gulp.src(['src/*.js', 'src/*/*.js'])
.pipe(eslint({
parser: 'babel-eslint',
parserOptions: {
ecmaFeatures: {
jsx: true
},
sourceType: 'module'
}
}))
.pipe(eslint.format())
.pipe(eslint.failAfterError());
});
gulp.task('compile', ['lint'], function() {
return gulp.src('src/main.js')
.pipe(webpackStream({
output: {
filename: 'main.js',
libraryTarget: 'commonjs2',
sourceMapFilename: 'main.js.map',
},
plugins: [],
module: {
loaders: [
{
test: /\.js$/,
loader: 'babel-loader',
query: {
presets: [
require.resolve('babel-preset-es2015'),
require.resolve('babel-preset-stage-0'),
],
},
},
],
},
}), webpack)
.pipe(gulp.dest('dist/'));
});
This example file can be run, using gulp watch. It'll watch the files for a chance and when it does trigger the other tasks.
I only had an example with webpack, but you can replace it by any other compiler component or even write your own compile step if you want (probably you don't).
This way you have exact control over every step your files go through. Most of which (and more) can also be achieved using the Webpack way. However, it would have the downside of inserting all its boilerplate on top of each processed file, when processing each file as a separate bundle. Ultimately probably something could be done with common chunks plugin.
With the latest release of Snowpack (formerly #pika/web) this should be possible now!
From their website:
TL;DR - With Snowpack you can build modern web apps (using React, Vue, etc.) without a bundler (like Webpack, Parcel, Rollup). No more waiting for your bundler to rebuild your site every time you hit save. Instead, every change is reflected in the browser instantly.
And their "How it Works":
Instead of bundling on every change, just run Snowpack once right after npm install.
Snowpack re-installs your dependencies as single JS files to a new web_modules/ directory. It never touches your source code.
Write code, import those dependencies via an ESM import, and then run it all in the browser.
Skip the bundle step and see your changes reflected in the browser immediately after hitting save.
Keep using your favorite web frameworks and build tools! Babel & TypeScript supported.
check https://www.snowpack.dev/ for more information, they have done a great job with their documentation, it looks really promising!
With webpack and source maps, it shouldn't matter that it changes your code. While this can be a challenge to set up initially, once you get it working you can look at your original source code in the browser debugging tools exactly as they appear to you on disk. The VS Code editor also does a good job of supporting this feature, allowing you to set breakpoints and look at the values of variables directly in your editor without having to use the browser developer tools.
However, if you are still set on trying to get this to work with your original source files then you are right that your ES6 code should just work in most modern browsers
For live reload you could check out the npm livereload package.
Or you could roll your own and figure out how webpack-dev-server does it. They use the chokidar npm package to watch the file system for changes and then they notify the broswer via web sockets. You could probably throw something together that's similar with a little effort.
Here is how webpack-dev-server initiates it:
const watcher = chokidar.watch(watchPath, options);
watcher.on('change', () => {
this.sockWrite(this.sockets, 'content-changed');
});
Obviously there is some JavaScript that runs in the browser waiting on a websocket for that message.
You could use a Webpack plugin like Emit All.
I want to minimize the number of HTTP requests from the client to load scripts in the browser. This is going to be a pretty general question but I still hope I can get some answers because module management in javascript has been a pain so far.
Current situation
Right now, in development, each module is requested individually from the main html template, like this:
<script src="/libraries/jquery.js"></script>
<script src="/controllers/controllername.js"></script>
...
The server runs on Node.js and sends the scripts as they are requested.
Obviously this is the least optimal way of doing so, since all the models, collections, etc. are also separated into their own files which translates into numerous different requests.
As far as research goes
The libraries I have come across (RequireJS using AMD and CommonJS) can request modules from within the main .js file sent to the client, but require a lot of additional work to make each module compliant with each library:
;(function(factory){
if (typeof define === 'function' && define.amd) define([], factory);
else factory();
}(function(){
// Module code
exports = moduleName;
}));
My goal
I'd like to create a single file on the server that 'concatenates' all the modules together. If I can do so without having to add more code to the already existing modules that would be perfect. Then I can simply serve that single file to the client when it is requested.
Is this possible?
Additionally, if I do manage to build a single file, should I include the open source libraries in it (jQuery, Angular.js, etc.) or request them from an external cdn on the client side?
What you are asking to do, from what I can tell, is concat your js files into one file and then in your main.html you would have this
<script src="/pathLocation/allMyJSFiles.js"></script>
If my assumption is correct, then the answer would be to use one of the two following items
GULP link or GRUNT link
I use GULP.
You can either use gulp on a case by case basis, which means calling gulp from the command line to execute gulp code, or use a watch to do it automatically on save.
Besides getting gulp to work and including the gulp files you need to do what you need, I will only provide a little of what I use to get your answer.
In my gulp file I would have something like this
var gulp = require('gulp');
var concat = require('gulp-concat');
...maybe more.
Then I have the file paths I need to be reduced into one file.
var onlyProductionJS = [
'public/application.js',
'public/directives/**/*.js',
'public/controllers/**/*.js',
'public/factories/**/*.js',
'public/filters/**/*.js',
'public/services/**/*.js',
'public/routes.js'
];
and I use this info in a gulp task like the one below
gulp.task('makeOneFileToRuleThemAll', function(){
return gulp.src(onlyProductionJS)
.pipe(concat('weHaveTheRing.js'))
.pipe(gulp.dest('public/'));
});
I then run the task in my command line by calling
gulp makeOneFileToRuleThemAll
This call runs the associated gulp task which uses 'gulp-concat' to get all the files together into one new file called 'weHaveTheRing.js' and creates that file in the destination 'public/'
Then just include that new file into your main.html
<script src="/pathLocation/weHaveTheRing.js"></script>
As for including all your files into one file, including your vendor files, just make sure that your vendor code runs first. It's probably best to keep those separate unless you have a sure fire way of getting your vendor code to load first without any issues.
UPDATE
Here is my gulp watch task.
gulp.task('startTheWatchingEye', function () {
gulp.watch(productionScripts, ['makeOneFileToRuleThemAll']);
});
Then I start up my server like this (yours may differ)
npm start
// in a different terminal window I then type
gulp startTheWatchfuleye
NOTE: you can use ANY movie or show reference you wish! :)
Now just code it up, every time you make a change in the specified files GULP will run your task(s).
If you want to say run Karma for your test runner...
add the following to your gulp file
var karma = require('karma').server;
gulp.task('karma', function(done){
karma.start({
configFile: __dirname + '/karma.conf.js'
}, done);
});
Then add this task karma to your watch I stated above like this...
gulp.task('startTheWatchingEye', function(){
gulp.watch(productionScripts, ['makeOneFileToRuleThemAll', 'karma']);
});
ALSO
Your specific settings may require a few more gulp modules. Usually, you install Gulp globally, as well as each module. Then use them in your various projects. Just make sure that your project's package.json has the gulp modules you need in dev or whatever.
There are different articles on whether to use Gulp or Grunt. Gulp was made after Grunt with a few additions that Grunt was lacking. I don't know if Grunt lacks them anymore. I like Gulp a lot though and find it very useful with a lot of documentation.
Good luck!
I'd like to create a single file on the server that 'concatenates' all the modules together. If I can do so without having to add more code to the already existing modules that would be perfect.
Sure you can. You can use Grunt or Gulp to do that, more specifically grunt-contrib-concat or gulp-concat
Here's an example of a Gruntfile.js configuration to concat every file under a js directory:
grunt.initConfig({
concat: {
dist: {
files: {
'dist/built.js': ['js/**/**.js'],
},
},
},
});
Also, you can minify everything after concatenating, using grunt-contrib-minify.
Both libraries support source maps so, in the case a bug gets to production, you can easily debug.
You can also minify your HTML files using grunt-contrib-htmlmin.
There's also an extremely useful library called grunt-usemin. Usemin let's you use HTML comments to "control" which files get minified (so you don't have to manually add them).
The drawback is that you have to explicitely include them in your HTML via script tags, so no async loading via javascript (with RequireJS for instance).
Additionally, if I do manage to build a single file, should I include the open source libraries in it (jQuery, Angular.js, etc.) or request them from an external cdn on the client side?
That's debatable. Both have pros and cons. Concatenating vendors assures that, if for some reason, the CDN isn't available, your page works as intended. However the file served is bigger so you consume more bandwidth.
In my personal experience, I tend to include vendor libraries that are absolutely essential for the page to run such as AngularJS for instance.
If I understand you correctly, you could use a task runner such as Grunt to concatenate the files for you.
Have a look at the Grunt Concat plugin.
Example configuration from the docs:
// Project configuration.
grunt.initConfig({
concat: {
dist: {
src: ['src/intro.js', 'src/project.js', 'src/outro.js'],
dest: 'dist/built.js',
}
}
});
Otherwise, as you have stated, a 'module loader' system such as Require JS or Browserify may be the way to go.
I have a directory like below:
/folder/b.js
/folder/jQuery.js
/folder/a.js
/folder/sub/c.js
I want to minify all these js files in one js file in order:
jQuery.js -> a.js -> b.js -> c.js
Q:
1.How can I do it via grunt-contrib-uglify?(In fact, there are lots of files, it is impractical to specify all source filepaths individually)
2.btw, How can I get unminified files when debug and get minified single file when release and no need to change script tag in html(and how to write the script tag)?
Good questions!
1) Uglify will reorder the functions in the destination file so that function definitions are on top and function execution on bottom but it seems that it will preserve the order of the function executions.
This means that the function jQuery runs to define its global functions will be put first if you make sure jQuery is mentioned first in Uglify's config in the Gruntfile.
I use this config:
uglify: {
options: {
sourceMap: true
},
build: {
files: {
'public/all.min.js': ['public/js/vendor/jquery-1.10.2.min.js', 'public/js/*.js'],
}
}
}
2) I don't think there is one definite way to accomplish this. It depends on what web framework, templating framework and what kind of requirements you have. I use express + jade and in my main jade layout I have:
if process.env.NODE_ENV === 'production'
script(src='/all.min.js')
else
script(src='/js/vendor/jquery-1.10.2.min.js')
script(src='/js/someScript.js')
script(src='/js/otherScript.js')
In my package.json I have:
"scripts": {
"postinstall": "grunt"
},
This means that when I run npm install on deploy (on Heroku) grunt is run to minify/concat files and when the app is started with NODE_ENV=production the minified client side javascript is used. Locally I get served the original client side javascripts for easy debugging.
The two downsides are:
I have to keep the two lists of script files in sync (in the Gruntfile and in the layout.js) I solve this by using *.js in the Gruntfile but this may not suite everyone. You could put the list of javascripts in the Gruntfile and create a jade-template from this but it seems overkill for most projects.
If you don't trust your Grunt config you basically have to test running the application using NODE_ENV=production locally to verify that the minification worked the way you intended.
This can be done using the following Grunt tasks:
https://github.com/gruntjs/grunt-contrib-concat concatenates
files
https://github.com/gruntjs/grunt-contrib-uglify minifies
concatenated files
EDIT
I usually run all my files through a Grunt concatenation task using grunt-contrib-concat. Then I have another task to uglify the concatenated file using grunt-contrib-uglify.
You're probably not going to like this, but the best way is to define your js source files as AMD modules and use Requirejs to manage the order in which they load. The grunt-contrib-requirejs task will recurse your dependency tree and concatenate the js files in the necessary order into one big js file. You will then use uglify (actually r.js has uglify built-in) to minify the big file.
https://github.com/danheberden/yeoman-generator-requirejs has a good example gruntfile and template js files to work from.
EDIT
I've recently started using CommonJS modules instead of AMD since it's much closer to the ES6 module spec. You can achieve the same results (1 big complied+concatenated js file) by running commonjs modules through Browserify. There are plugins for both grunt and gulp to manage the task for you.
EDIT
I'd like to add that if your site is written using ES6 that Rollup is the best new concatenating package. In addition to bundling your files, it will also perform tree shaking, removing parts of libraries you use if included via an import statement. This reduces your codebase to just what you need without the bloat of code you'll never use.
I don't think you can do this with the uglify task alone, but you have a multitude of choices which might lead to your desired outcome.
A possible workflow would be first concatenating (grunt-contrib-concat) the files in order into one single file, and put this concatenated file through uglify. You can either define the order for concat in your Gruntfile, or you use on of those plugins:
First one would be https://github.com/yeoman/grunt-usemin, where you can specify the order in your HTML file, put some comments around your script block. The Google guys made it and it's pretty sweet to use.
Second one would be https://github.com/trek/grunt-neuter, where you can define some dependencies with require, but without the bulk of require.js. It requires changes in your JS code, so might not like it. I'd go with option one.
I ran into the same issue. A quick fix is just to change the filenames - I used 1.jquery.min.js, 2.bootstrap.min.js, etc.
This might be only remotely related to your question but I wanted something similar. Only my order was important in the following way:
I was loading all vendor files (angular, jquery, and their respective related plugins) with a wildcard (['vendor/**/*.js']). But some plugins had names that made them load before angular and jquery. A solution is to manually load them first.
['vendor/angular.js', 'vendor/jquery.js', 'vendor/**/*.js]
Luckily angular and jquery handle being loaded twice well enough. Edit: Although it's not really the best practice to load such large libraries twice, causing your minified file unnecessary bloat. (thanks #Kano for pointing this out!)
Another issue was client-js the order was important in a way that it required the main app file to be loaded last, after all its dependencies have been loaded. Solution to that was to exclude and then include:
['app/**/*.js', '!app/app.js', 'app/app.js']
This prevents app.js from being loaded along with all the other files, and only then includes it at the end.
Looks like the second part of your question is still unanswered. But let me try one by one.
Firstly you can join and uglify a large number of js files into one as explained by the concat answer earlier. It should also be possible to use https://github.com/gruntjs/grunt-contrib-uglify because it does seem to have wildcards. You may have to experiment with 'expand = true' option and wildcards. That takes care of your first question.
For the second part, say you joined and uglified into big-ugly.js
Now in your html you can add following directives:
<!-- build:js:dist big-ugly.js -->
<script src="js1.js"></script>
<script src="js2.js"></script>
<!-- etc etc -->
<script src="js100.js"></script>
<!-- /build -->
And then pass it through the grunt html preprocessor at https://www.npmjs.com/package/grunt-processhtml as part of your grunt jobs.
This preprocessor will replace the entire block with
<script src="big-ugly.js"></script>
Which means that the html file with be semantically equivalent - before and after the grunt jobs; i.e. if the page works correctly in the native form (for debugging) - then the transformed page must work correctly after the grunt - without requiring you to manually change any tags.
This was #1469's answer but he didn't make it clear why this works. Use concat to put all js files into one, this module does this in the order of file names, so I put a prefix to the file names based on orders. I believe it even has other options for ordering.
concat: {
js: {
options: {
block: true,
line: true,
stripBanners: true
},
files: {
'library/dist/js/scripts.js' : 'library/js/*.js',
}
}
},
Then use uglify to create the minified ugly version:
uglify: {
dist: {
files: {
'library/dist/js/scripts.min.js': [
'library/js/scripts.js'
]
},
options: {
}
}
},
If your problem was that you had vendors which needed to be loaded in order (let's say jquery before any jquery plugins). I solved it by putting jquery in its own folder called '!jquery', effectively putting it on top of the stack.
Then I just used concat as you normally would:
concat: {
options: {
separator: ';',
},
build: {
files: [
{
src: ['js/vendor/**/*.js', 'js/main.min.js'],
dest: 'js/global.min.js'
}
]
}
},