We Keep Coding

Make pdf.js 1.5 and require.js play nice together

In my project I have long used require.js together with the pdf.js library. Pdf.js have until recently been putting itself on the global object. I could still use it in my requirejs config by using a shim. The pdfjs library will in turn load another library called pdf.worker. In order to find this module the solution was to add a property to the global PDFJS object called workerSrc and point to the file on disk. This could be done before or after loading the pdfjs library.
The pdfjs library uses the pdf.worker to start a WebWorker and to do so it needs the path to a source file.
When I tried to update the pdfjs library in my project to a new version (1.5.314) the way to load and include the library have changed to use UMD modules and now everything get's a bit tricky.
The pdfjs library checks if the environment is using requirejs and so it defines itself as a module named "pdfjs-dist/build/pdf". When this module loads it checks for a module named "pdfjs-dist/build/pdf.worker". Since I have another folder structure I have added them to my requirejs config object with a new path:
paths: {
"pdfjs-dist/build/pdf": "vendor/pdfjs/build/pdf",
"pdfjs-dist/build/pdf.worker": "vendor/pdfjs/build/pdf.worker"
}
This is to make the module loader to find the modules at all. In development this works great. When I try to use the requirejs optimizer in my grunt build step however, it will put all of my project files into one single file. This step will try to include the pdf.worker module as well and this generates an error:
Error: Cannot uglify2 file: vendor/pdfjs/build/pdf.worker.js. Skipping
it. Error is: RangeError: Maximum call stack size exceeded
Since the worker source needs to be in a single file on disk I don't want this module to be included.
So I've tried two different config-settings in the requirejs config.
The first attempt was to override the paths property in my grunt build options:
paths: {
"pdfjs-dist/build/pdf.worker": "empty:"
}
The second thing to test is to exclude it from my module:
modules: [{
name: "core/app",
exclude: [
"pdfjs-dist/build/pdf.worker"
]
}]
Both techniques should tell the optimizer not to include the module but both attempts ended up with the same error as before. The requirejs optimizer still tries to include the module into the build and the attempt to uglify it ends up with a RangeError.
One could argue that since the uglify step fails it will not be included and I can go about my bussiness, but if the uglify step should happen to start working at a new update of pdfjs - what then?
Can anyone help me figure out why the requirejs config won't just exclude it in the build step and how to make it do so.

I found out what the core of my problem was and now I have a way to solve the problem and make my build process to work. My build step in grunt is using grunt-contrib-requirejs and I needed to override some options in the config for this job.
I didn't want the pdf.worker module to be included in my concatenated and minified production code.
I didn't want r.js to minify it only to later exclude it from the concatenated file.
I tried to solve the first problem thinking that it would mean that the second problem also should be solved. When I figured out the two were separate I finally found a solution.
In the r.js example on github there is a property named fileExclusionRegExp. This is what I now use to tell r.js not to copy the file over to the build folder.
fileExclusionRegExp: /pdf.worker.js/
Second, I need to tell the optimizer to not include this module in the concatenated file. This is done by overriding the paths property for this module to the value of "empty:".
paths: {
"pdfjs-dist/build/pdf.worker": "empty:"
}
Now my grunt build step will work without errors and all is well.
Thanks to async5 for informing me about the bug with uglify and the pdf.worker. The workaround is applied in another grunt task that uglify the worker and copies it into the build-folder separately. The options object for the grunt-contrib-uglify task will need this property in order to not break the pdf.worker file:
compress: {
sequences: false
}
Now my project works great when built for production.

How to read and parse gruntfile programatically so I can modify it and save again?

I am creating a complex Yeoman Generator and I need to read an existing gruntfile and modify it.
Any javascript way of parsing a gruntfile is enought.
Any help will be appreciated.

imo i dont think you can achieve that by parsing js files and generating a new gruntfile.
instead of doing that you can try creating template files ex. .jade (or another template engine) so , you can produce the final gruntfile, you can look into grunt configuration object grunt.config and eventually read information, interpolate data into your template(s) and produce the final gruntfile.js.

Here's a modular approach to grunt that you can use without having to modify gruntfile.js every time you add new tasks, plugin or custom.
Add load-grunt-tasks npm module as a dev dependency: npm install load-grunt-tasks --save-dev. This plugin allows you to load all your grunt dependencies using one line of code.
Specify a folder in your project to hold all your modular configuration files. This example uses the path grunt/configs/. Each json file's name should match the it's property name in the grunt config. For example, the config for grunt-copy-config should be grunt/configs/copy.json
Copy the gruntfile.js I've posted as a public gist (stack overflow wasn't formatting the code properly). This gruntfile will automatically load your plugins and compile a config object using the project's package.json file and every .json file found in grunt/configs/. The gruntfile also loads any custom tasks I've saved into grunt/tasks/
As an alternate option - you can host your modular config objects in JavaScript files instead of json files (that's what I do). This provides me with the capability to dynamically compile my config objects if I need to. If that's something that interests you, I can share that code as well.

Thank you guys for the tips, great ideas, with your guidance I came up using a json file for each configuration:
module.exports = function(grunt) {
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
browserify: grunt.file.readJSON('browserify.json'),
uglify: grunt.file.readJSON('uglify.json'),
sass: grunt.file.readJSON('sass.json'),
cssmin: grunt.file.readJSON('cssmin.json'),
});
grunt.loadNpmTasks('grunt-browserify');
grunt.loadNpmTasks('grunt-contrib-uglify');
grunt.loadNpmTasks('grunt-contrib-sass');
grunt.loadNpmTasks('grunt-contrib-cssmin');
grunt.registerTask('default', ['browserify', 'sass']);
grunt.registerTask('production', ['browserify', 'sass', 'uglify', 'cssmin']);
};
Now I can easily modify the configuration

Loading and parsing .ini file inside or from a Gruntfile

Gruntfile.js is located at project root/Gruntfile.js
asset_compress.ini is located at project root/app/Config/asset_compress.ini
Tests are written using Jasmine, and the specs all locate in project root/tests/**/*
The following is a stripped-down Gruntfile. If you need more, feel free to ask:
module.exports = function(grunt) {
'use strict';
grunt.initConfig({
jasmine: {
test: {
src: [
'node_modules/jasmine-expect/dist/jasmine-matchers.js',
'app/webroot/js/libraries/jquery-2.0.js',
'app/webroot/js/api/ClassUtility.js',
'app/webroot/js/api/**/*.js'
],
options: {
log: true,
specs: [
'tests/app/webroot/js/api/ClassUtility.spec.js',
'tests/**/*.spec.js'
]
}
}
}
});
};
ClassUtility (and its specification) need to be loaded before any other part of the API, because it contains everything all other "classes" rely on. This is why it's declared specifically above all other api classes.
However, I have many(!) dependencies and many other files that I need, and they're all present in a (rather large) asset_compress.ini. Ideally, I would want to keep that single ini file as the only list and have my Gruntfile read from that list to know what source files it should load.
TLDR:
How would I configure my Gruntfile with the content of my ini file?

As bfred.it suggested, there were plenty of node packages that can parse .ini files. However, none of them worked the way I wanted them to work so I decided to create my own.
The source code, documentation, instructions and anything else you may desire can be found here: https://bitbucket.org/skelware/node-file-parser/
Feel free to request features on its issue tracker!

The easy way to load and parse an ini file in node:
First install on command line:
npm install parse-ini
Then in the code:
var iniParser = require('parse-ini');
var parsedIni = iniParser.parse('yourfile.ini');
// job done, you can use results in parsedIni:
console.log(parsedIni.sectionName.variableName);
console.log(parsedIni.variableWithoutSectionName);

Running tasks configured across multiple grunt.js files

I have a node app that includes multiple unpublished modules. My app's package.json includes a few git dependencies:
"module-a": "git+ssh://git#github.com:me/module-a.git",
"module-b": "git+ssh://git#github.com:me/module-b.git"
and each of those have their own grunt config. Eg in node_modules/module-a/grunt.js:
module.exports = function(grunt) {
grunt.initConfig({
lint: {
files: ['server/**/*.js', 'test/**/*.js']
},
jshint: {
options: require('./lint-ci')
}
});
grunt.registerTask('default', 'lint');
};
(they also run tests, etc, but I'm keeping it simple here)
Is there a built-in way to do this with grunt? Note that I want to keep the dependent grunt.js files for convenience when I've only changed something within that dependency.
The only solutions I have found are
build up my main grunt.js programmatically (eg, iterating over my dependencies in package.json to build the lint and test config)
call grunt multiple times using --config node_modules/module-a/grunt.js
Neither seems ideal. Is there a better way?

Just a thought but have you looked at grunt-hub?
https://github.com/shama/grunt-hub

Working project structure that uses grunt.js to combine JavaScript files using RequireJS?

I have some projects that use RequireJS to load individual JavaScript modules in the browser, but I haven't optimized them yet. In both development and production, the app makes a separate request for each JavaScript file, and now I would like to fix that using Grunt.
I have tried to put together a simple project structure to no avail, so I'm wondering if someone can provide a working example for me. My goals are the following:
In development mode, everything works in the browser by issuing a separate request for each required module. No grunt tasks or concatenation are required in development mode.
When I'm ready, I can run a grunt task to optimize (combine) all of the JavaScript files using r.js and test that out locally. Once I'm convinced the optimized application runs correctly, I can deploy it.
Here's a sample structure for the sake of this conversation:
grunt-requirejs-example/
grunt.js
main.js (application entry point)
index.html (references main.js)
lib/ (stuff that main.js depends on)
a.js
b.js
requirejs/
require.js
text.js
build/ (optimized app goes here)
node_modules/ (necessary grunt tasks live here)
Specifically, I'm looking for a working project structure that I can start from. My main questions are:
If this project structure is flawed, what do you recommend?
What exactly needs to be in my grunt.js file, especially to get the r.js optimizer working?
If all of this isn't worth the work and there's a way to use the grunt watch task to automatically build everything in development mode every time I save a file, then I'm all ears. I want to avoid anything that slows down the loop from making a change to seeing it in the browser.

I use the grunt-contrib-requirejs task to build project based on require.js. Install it inside your project directory with:
npm install grunt-contrib-requirejs --save-dev
BTW: --save-dev will add the package to your development dependencies in your package.json. If you're not using a package.json in your project, ignore it.
Load the task in your grunt file with:
grunt.loadNpmTasks('grunt-contrib-requirejs');
And add the configuration to your grunt.initConfig
requirejs: {
production: {
options: {
baseUrl: "path/to/base",
mainConfigFile: "path/to/config.js",
out: "path/to/optimized.js"
}
}
}
Now you're able to build your require.js stuff into a single file that will be minimized with uglifyjs by running grunt requirejs
You can bundle a set of different tasks into some sort of main task, by adding this to your grunt file
grunt.registerTask('default', ['lint', 'requirejs']);
With this, you can simply type grunt and grunt will automatically run the default task with the two 'subtasks': lint and requirejs.
If you need a special production task: define it like the above
grunt.registerTask('production', ['lint', 'requirejs', 'less', 'copy']);
and run it with
grunt production
If you need different behaviors for 'production' and 'development' inside i.e. the requirejs task, you can use so called targets. In the configuration example above it's already defined as production. You can add another target if you need (BTW, you can define a global config for all targets by adding a options object on the same level)
requirejs: {
// global config
options: {
baseUrl: "path/to/base",
mainConfigFile: "path/to/config.js"
},
production: {
// overwrites the default config above
options: {
out: "path/to/production.js"
}
},
development: {
// overwrites the default config above
options: {
out: "path/to/development.js",
optimize: none // no minification
}
}
}
Now you can run them both at the same time with grunt requirejs or individually with grunt requirejs:production, or you define them in the different tasks with:
grunt.registerTask('production', ['lint', 'requirejs:production']);
grunt.registerTask('development', ['lint', 'requirejs:development']);
Now to answer your questions:
I would definitely use a subfolder in your project. In my case I use a 'src' folder for development that is build into a 'htdocs' folder for production. The project layout I prefere is:
project/
src/
js/
libs/
jquery.js
...
appname/
a.js
b.js
...
main.js // require.js starter
index.html
...
build/
... //some tmp folder for the build process
htdocs/
... // production build
node_modules/
...
.gitignore
grunt.js
package.json
see above
You can do so, but I wouldn't recommend to add requirejs to the watch task, it's a resource hungry task and it will slow down your machine noticeable.
Last but not least: Be very cautious when playing around with r.js. Especially when you want to optimize the whole project with r.js by adding a modules directive to your config. R.js will delete the output directory without asking. If it happens that it is accidentally configured to be your system root, r.js will erase your HDD. Be warned, I erased my whole htdocs folder permanently some time ago while setting up my grunt task... Always add keepBuildDir:true to your options when playing around with the r.js config.