We Keep Coding

Speed up Javascript uglification in Play 2.3.x build for external Webjar sources

I'm using Play framework 2.3.6 and Webjars for web lib dependencies.
That is, my build.sbt contains something like "org.webjars" % "angularjs" % "1.2.26".
To uglify my Javascript sources I added pipelineStages := Seq(rjs, uglify, digest, gzip) to my build.sbt.
Now, when running Play's 'stage' or 'dist' tasks it looks like all Javascript files getting uglified, that is, also files from the Webjar libraries.
[info] Uglify2 file: /target/web/rjs/build/lib/angularjs/angular.js
I would have expected that sources from the external Webjar libraries are left untouched as there already is a minified version.
One problem with that is that the uglify process takes way too long.
How can I speed up the uglification process?

There are two ways to speed up the Javascript building steps:
Install node.js and set export SBT_OPTS="$SBT_OPTS -Dsbt.jse.engineType=Node" before running the activator. Using node.js instead of the default Javascript engine gives a very significant speedup. More details can be found here: Migration to play 2.3, see section about sbt-web
Customize the build steps, e.g.
disable minification by adding to build.sbt: RjsKey.optimize := "none"
limit uglification by adding to build.sbt: e.g. includeFilter in uglify := GlobFilter("myjs/*.js"),
You can find more details about the options on the github site of these plugins:
sbt-uglify
sbt-rjs

Even though that the sbt-uglify documentation says that excludeFilter should exclude webjars and public folder, it doesn't.
Follow Martin's reponse customization part, except that he maid y typo, add an S to RjsKeys:
RjsKeys.optimize := "none"

Jetbrains adding Closure as an external library

I am trying to add Closure as an external library in PyCharm.
These lines demonstrate where I installed closure:
Chriss-MacBook-Pro:closure chris$ pwd
/Users/chris/DevLibrary/closure-library/closure
Chriss-MacBook-Pro:closure chris$ ls
bin css goog known_issues
I've tried adding various directories in the edit libraries dialog such as:
/Users/chris/DevLibrary/closure-library/
/Users/chris/DevLibrary/closure-library/closure
/Users/chris/DevLibrary/closure-library/closure/goog
I've also tried adding the individual files.
But every time my File Watcher runs I still get:
java -jar /Users/chris/Projects/housemaps/compiler.jar
--compilation_level SIMPLE_OPTIMIZATIONS --js housemap.js housemap.js:1: ERROR - required "goog.dom" namespace never provided
goog.require('goog.dom'); ^
1 error(s), 0 warning(s)

The external libraries feature of Jetbrains is for providing your IDE with access to the source code (for code completion, error checking, ect) for Javascript files that are not included with your project, but are going to be hosted externally.
Closure is not designed to be hosted externally as it is supposed to be used with the closure compiler.

Is there a way to use sourcemap sequentially more than 2 times? [duplicate]

Has any one had any success with this?

I think it's more or less an unsolved problem:
https://github.com/jashkenas/coffee-script/issues/2779 . Last meanigingful comment was from jwalton, a month ago.
Still, it doesn't seem rocket science to add support for it, so it will probably come soon.
Michael Ficarra (creator of CoffeeScript Redux) suggested using https://github.com/michaelficarra/commonjs-everywhere .
Two caveats:
It only works for bundling CommonJS modules.
It uses CoffeeScript Redux, which is still in beta (although working quite well it seems), and not 100% compatible with original CoffeeScript compiler.
So this does not work for what you ask for specifically, "concatenation".
Added April 14
You might have luck with these: combine-source-map and/or generate-sourcemap, both by same author.
Added April 26
This looks really simple: https://npmjs.org/package/mapcat . You just have to feed it the individual source map files generated by the coffee compiler.
Added May 16
Mariusz Nowak has just released webmake-coffee. Like CommonJS Everywhere, it requires code to be organized as CommonJS modules. Unlike CommonJS everywhere, it uses regular CoffeeScript.
It also seems the Grunt Coffee-Script plugin has had source-map support for concatenated files for quite a while (two months), effectively proving my original answer to be incorrect.
The upcoming version 2.0 of Snockets will have support for it too.

I ended up going with browserify using coffeeify as the transform option, and enabling browserify's debug option. I bundle up the app on each request for my main.js file, and any runtime errors show up in my original source with pretty decent accuracy.
Sure beats mapping runtime errors in the concatenated/compiled js back to the coffee source with my eyeballs!

I needed to annotate AngularJS code before minification, but grunt-ng-annotate didn't accept input source maps, thus I would not be able to use maps generated by the CoffeeScript compiler.
Apparently, with gulp-sourcemaps this is not an issue:
var gulp = require('gulp');
var $ = require('gulp-load-plugins')(); // loading gulp plugins lazily
// remember to include them in the package.json
gulp.task('appJS', function() {
// concatenate compiled .coffee files and js files into build/app.js
gulp.src(['./app/**/*.js','./app/**/*.coffee'])
.pipe($.sourcemaps.init())
.pipe($['if'](/[.]coffee$/, $.coffee({bare: true}).on('error', $.util.log)))
.pipe($.concat('app.js'))
.pipe($.ngAnnotate())
.pipe($.uglify())
.pipe($.sourcemaps.write())
.pipe(gulp.dest('./build'))
});
The same approach works in other situations, too. In my case, this is the only approach that worked.

I have written a grunt task that does this flawless. Check it out

Use Emscripten with Fortran: LAPACK binding

My goal is to use LAPACK with Emscripten.
My question is: how to port LAPACK to JS? The are two ways I can think of: CLAPACK to JS where my question is: does anybody know an unofficial version that is later than 3.2.1? And the other way to think of is: how to port FORTRAN to JS?
Emscripten is capable of transforming C code to JavaScript. But unfortunately, LAPACK 3.5.0 (http://www.netlib.org/lapack/) is only available in FORTRAN95.
The CLAPACK project (http://www.netlib.org/clapack/) is basically what I want: a C version of LAPACK. But this one is outdated; the latest is 3.2.1.
F2C only works up to FORTRAN 77. LAPACK 3.5.0 was written in FORTRAN 95.
So my question now is: why is there no newer port of LAPACK to C?
The optimal way would be to directly transform the FORTRAN95 code of LAPACK to javascript with clang and emscripten. But I just don't know where to start.
Emscripten currently does not support FORTRAN. But it handles LLVM bitcode, so it should not be a problem to use clang to generate LLVM bc from a FORTRAN file.
For testing purpose, I have this file:
program hello
print *, "Hello World!"
end program hello
It compiles just fine with "clang hello.f -o hello -lgfortran". I am not capable of transforming this into valid bitcode.
clang -c -emit-llvm hello.f
clang -S -emit-llvm hello.f -o hello.bc -lgfortran
None of these approaches works, because emscripten keeps telling me
emcc -c hello.o -o hello.js
hello.o is not valid LLVM bitcode
I am not sure anyways if this would be even possible, because LAPACK obviously needs libgfortran to work. And I can't merge a library into javascript code...
Thanks in advance!
Edit:
I almost managed it to convert BLAS from LAPACK 3.5.0 to JS. I used dragonegg to accomplish this.
gfortran caxpy.f -flto -S -fplugin=/usr/lib/gcc/x86_64-linux-gnu/4.6/plugin/dragonegg.so
gfortran cgerc.f ...
...
After gaining LLVM bitcode from that:
emcc caxpy.s.ll cgerc.s.ll cher.s.ll ... -o blas.js -s EXPORTED_FUNCTIONS="['_caxpy_', ... , '_ztpsv_']"
But emscripten still leaves me with the following errors:
warning: unresolved symbol: _gfortran_st_write
warning: unresolved symbol: _gfortran_string_len_trim
warning: unresolved symbol: _gfortran_transfer_character_write
warning: unresolved symbol: _gfortran_transfer_integer_write
warning: unresolved symbol: _gfortran_st_write_done
warning: unresolved symbol: _gfortran_stop_string
warning: unresolved symbol: cabs
warning: unresolved symbol: cabsf
AssertionError: Did not receive forwarded data in an output - process failed?
The problem is that lgfortran is precompiled I think.

Thank you for your reply!
Indeed I did make progress on this. Finally it is working. I was very close, just follow these steps:
gfortran caxpy.f -S -flto -m32 -fplugin=dragonegg.so
mv caxpy.s caxpy.ll
llvm-as caxpy.ll -o caxpy.o
Note the "m32" flag which I missed earlier.
Warnings like
warning: unresolved symbol: _gfortran_st_write
can be ignored safely. Emscripten creates empty functions in the JavaScript file with this name so if these functions are not called at all there is no problem. If they get called you can easily substitute them with your own functions; the names are somewhat descriptive. Additionally you can have a look at the libgfortran source code (be aware it is GPL).
With this Emscripten source can be extended by hand to support Fortran files. Someday I may publish this on github!

I actually pulled this off recently (https://github.com/harveywi/arpack-js). The Github repo is mostly barren except for the output JS files, but I will be uploading source code, Makefiles, and other instructions soon. After wrangling unsuccessfully with dragonegg for a while, I found a different approach (not as great, but sufficient) which did the trick.
Here is roughly how I did it:
Download the ARPACK source code.
Run f2c on all of the Fortran files to convert them to C.
(This might be the trickiest part): Modify the Makefiles to use Emscripten and LLVM toolchains.
Make the project to produce an LLVM binary.
Use Emscripten again to transpile the LLVM binary to JS.

How to use Google's Closure to compile JavaScript

Google just released Closure, which is a compiler to minify JavaScript.
On the product site, it says "The Closure Compiler has also been integrated with Page Speed".
How do I use Page Speed to compile my web pages JavaScript with Closure?
(Or, is there a web site that I can simply paste in my JavaScript to have closure minify it?

For a single file it's simple
java -jar $path_to_jar/compiler.jar --js input_file.js \
--js_output_file output_file.js
For a multi-file project you can use calcdeps.py in combination with the compiler.jar
#!/bin/sh$
$CALCDEPS_PATH=/path/to_calcdeps #directory containing calcdeps.py
$JAR_PATH=/path/to_jar #directory containing compiler.jar
$CLOSURE_PATH=/path/to_closure #contains directory "closure"
$CALCDEPS_PATH/calcdeps.py --path $CLOSURE_PATH \
--path . \
--compiler_jar $JAR_PATH/compiler.jar \
--input main_project_file.js \
--output_mode compiled \
> compiled_project_file.js
That way compiler gives meaningful information about type errors, etc. Type errors can be caught at compile time because compiler.jar uses certain JSDoc comments for type information.
Extra compiler flags can be passed to calcdeps.py along with -f or --compiler_flags options
If you want to use advanced optimizations set
--compiler_flags "--compilation_level=ADVANCED_OPTIMIZATIONS"
notice the double quotes and the equal sign - had to use that format in bash

The Closure compiler is now available as a JavaScript application. No need for the Java dependency anymore
There are a few ways to integrate with it. I have done it as part of Rollup
ex:
import rollup from 'rollup';
import closure from 'rollup-plugin-closure-compiler-js';
export default {
entry: 'index.js',
dest: 'dist/build.js',
format: 'iife',
plugins: [
closure({
languageIn: 'ECMASCRIPT6',
languageOut: 'ECMASCRIPT5',
compilationLevel: 'ADVANCED',
warningLevel: 'VERBOSE',
externs: [{src:`
var jQuery;
jQuery.fadeIn = function() {};
var ko;
ko.applyBindings = function(vm) {};
ko.computed = function(a,b) {};
ko.observable = function(a) {};
`}],
})
]
}
More info here:
http://www.syntaxsuccess.com/viewarticle/using-the-closure-compiler---advanced_optimizations

"Page Speed 1.4 Beta integrates the Closure Compiler to minify JavaScript files automatically. However, you will need to download and install the Page Speed Beta and Closure Compiler separately."
http://code.google.com/speed/page-speed/download.html
I haven't installed this version yet, but I'm fairly certain that Page Speed will present you with compiled code in its optimization recommendations.

It seems that Closure Compiler is integrated with Page Speed only for Windows.

Use the closure compiler with PHP (hosted via CURL or local via command line tool)
http://bohuco.net/blog/2009/11/google-closure-compiler-with-php/

If you need to compile multiple js files or if you would like to simplify compilation process, you may use kjscompiler: https://github.com/knyga/kjscompiler (based on google closure compiler)