Recursively compress directory of javascript files into a single file - javascript

I would like be able to run a single command in my project folder to concatenate and compress all of my javascript files (perhaps with YUI Compressor) into a single output file.
If possible I would like to partially specify the order in which they are concatenated together but not have to keep track of every single javascript file. Perhaps a config file could be built which looks like this:
application.js
excanvas.js
json2.js
jquery*.js
flot/*
backbone*.js
app/screen-*.js
app/main.js
app/crud-*.js
app/*
*
Does anyone know of either an existing tool to do something like this, could whip together a bash/ruby/node/perl script, or even a better methodology? I'm building a Single Page App with heavy JS usage (~40 files) to be consumed by people with low bandwidth.
I would need the solution to be executable on my OS X development machine.

find . -iname "*.js" -exec cat "{}" \; > singlefile.js
[JS compressor] singlefile.js
First concatenate the files, then compress them.
If you really care, though, you may want a real JS optimizer like the RequireJS optimizer.

given a folder of javascript files:
geee: ~/src/bash/js-files
$ find .
.
./application.js
./jquery-ui.js
./all-scripts.js
./cp.js
./excanvas.js
./backbone-worldwide.js
./jquery-plugin.js
./.found
./app
./app/crud-sel.js
./app/screen-detach.js
./app/aligator.js
./app/crud-in.js
./app/giraffe.js
./app/screen-attach.js
./app/main.js
./app/crud-del.js
./app/mouse.js
./app/monkey.js
./app/screen-shot.js
./backbone-national.js
./backbone23.js
./ummap.js
./CONFIG
./backbone-ibm.js
./ieee754.js
./flot
./flot/cow.js
./flot/moo.js
./flot/cat.js
./flot/bull.js
./flot/dog.js
./flot/sheep.js
./lines
./droiddraw-r1b21
./droiddraw-r1b21/._readme.txt
./droiddraw-r1b21/readme.js
./droiddraw-r1b21/LICENSE.js
./jquery-1.7.js
./ole.js
./touch
./json2.js
./xls2txt.js
./DO.sh
./backbone-isp.js
with a slightly modified configuration file:
geee: ~/src/bash/js-files
$ cat CONFIG
application.js
excanvas.js
json2.js
jquery*.js
flot/*
backbone*.js
app/screen-*.js
app/main.js
app/crud-*.js
app/*js
*js
and this bash script:
$ cat DO.sh
PROJECT=/home/jaroslav/src/bash/js-files # top folder of the web-app
SUPERJS=${PROJECT}/all-scripts.js
CONFIG=${PROJECT}/CONFIG # your the priority file (notice *js)
FOUND=${PROJECT}/.found # where to save results
JSMIN=$HOME/bin/jsmin # change to /usr/local/bin/jsmin or some other tool
echo > $FOUND # remove results from previous run
if [ ! -x $JSMIN ]
then
TMPJSMIN=/tmp/jsmin.c
wget -q https://github.com/douglascrockford/JSMin/raw/master/jsmin.c -O $TMPJSMIN & FOR=$?
echo "fetching jsmin (by Douglas Crockford) from github"
wait $FOR
gcc -o $JSMIN $TMPJSMIN
fi
cat $CONFIG | \
while read priority
do
eval "find $priority|sort -n" | \
while read amatch;
do
grep -q $amatch $FOUND || echo $amatch >> $FOUND
done
done
echo minifying:
cat $FOUND
cat `cat $FOUND` | $JSMIN > $SUPERJS
you will find the "merged" script in all-scripts after runing the script:
geee: ~/src/bash/js-files
$ . DO.sh
fetching jsmin (by Douglas Crockford) from github
[1]+ Done wget -q https://github.com/douglascrockford/JSMin/raw/master/jsmin.c -O $TMPJSMIN
minifying:
application.js
excanvas.js
json2.js
jquery-1.7.js
jquery-plugin.js
jquery-ui.js
flot/bull.js
flot/cat.js
flot/cow.js
flot/dog.js
flot/moo.js
flot/sheep.js
backbone23.js
backbone-ibm.js
backbone-isp.js
backbone-national.js
backbone-worldwide.js
app/screen-attach.js
app/screen-detach.js
app/screen-shot.js
app/main.js
app/crud-del.js
app/crud-in.js
app/crud-sel.js
app/aligator.js
app/giraffe.js
app/monkey.js
app/mouse.js
all-scripts.js
cp.js
ieee754.js
ole.js
ummap.js
xls2txt.js
Let me know if you need me to explain the script or if it fails on OS X.

The following script will follow the order of your config file and use the patterns given
#!/bin/bash
shopt -s nullglob;
while read config; do
cat $config >> out.js
done < /path/to/config/file

I ended up building a solution which uses a json file to list all of the files required by the app. On the dev environment the files are individually loaded by the browser. On the production server, the big compiled file is loaded. On my dev machine, I manually run a command to iterate over each file, appending it to a big JS file and running YUI Compressor.
It's a little hacky, but here it is:
https://github.com/renownedmedia/js-compressor

Related

Remove unused keys from JSON file

Hello I'm looking for the best way to automatize my problem:
I am working on a web application in which I use JSON translation files that have this form:
{ "unique_key" : "value"}
I have several files, one for each supported language, which all have the same number of items.
Ex :
i18n_en.json
{ "greeting" : "Hello"}
i18n_fr.json
{ "greeting" : "Bonjour"}
I have very badly managed the evolution of these files, and I end up with keys that are no longer used (I easily think 30% of the ~500 keys), the problem being that I don't know which ones. And that I would have to manually search through the entire architecture of my application to find those that are used and redo a clean file.
My idea to automate this process being:
Open one of the JSON files (no matter which one, they all have the
same number of keys).
Loop for each key
For each key, browse the entire architecture of my project ( by
looking only in the *.html or *.js files)
If we find an occurrence of this key, create an entry in a new clean
json with the key + value of it.
I don't really know which language to use that would be optimized for this kind of task, thank you for guiding me!
Maybe something like i18next-scanner will be useful. You are not the first with such a problem.
I ended up creating my own shell script :
path_to_project=path/to/my/project
RED='\033[0;31m'
GREEN='\033[0;32m'
rm temp.json
rm final.json
touch temp.json
touch final.json
echo "{" >> temp.json
while IFS=, read -r key value
do
if grep -r -q --include=\*.{js,html} --exclude-dir={node_modules,bower_components} $key $path_to_project; then
# write in new json
echo "\"$key\":\"$value\"," >> temp.json
echo -e "${GREEN} $key was found !"
else
echo -e "${RED} $key not found"
fi
done < data.csv
echo "}" >> temp.json
#remove new lines
cat temp.json | tr -d '\r' >> final.json
For this to work, I had to convert my json file to csv (data.csv).
The final json file needs to be reworked a little manually after the script, but really nothing overwhelming.

Arrays using commander

Hei guys!
I need help with the commander node.js library. I need create this CLI which accepts 3 flags, --input, --output and --pattern, like:
commander
.version('3.0.0')
.usage('[options] <file ...>')
.option('-i, --input', 'Array of files to be extracted')
.option('-o, --output', 'Output file name')
.option('-p, --pattern', 'Pattern name to be used in the extraction')
.parse(process.argv);
My problem is with the input flag. I need send several files, for that i need an array data type.
The problem is: I just can't figure it out how to make this:
node ./bin/extract -i ../files/*.PDF
become an array with all my files that are inside my files directory. I already try to run every sample in the documentation, and i didn't find the solution for my problem. Also, i searched in the issues and didn't find either... what is strange, maybe i am doing something wrong and you guys could help??
Thanks!
You can use Coercion to achieve it:
function scanDir(val) {
files = fs.readdirSync(val);
return files;
}
program
.version('0.0.1')
.option('-s, --scan [value]', '', scanDir)
.parse(process.argv);
console.log(' scan: %j', program.scan);
And call it like:
node app.js -s /foo

How to have a the docpad grunt skeleton min vendor js files with live reload

I'm using skeleton #2, HTML5BP + Grunt. The first time I docpad run the following happens:
info: LiveReload listening to new socket on channel /docpad-livereload
Performing writeFiles (postparing) at 0/1 0% [...] Running "min:js" (min) task
File "../out/scripts/all.min.js" created.
Uncompressed size: 298495 bytes.
Compressed size: 38257 bytes gzipped (106756 bytes minified).
Which is as is supposed to be. However using the livereload plugin if I change a template or document file, I get:
--Running "min:js" (min) task
File "../out/scripts/all.min.js" created.
Uncompressed size: 0 bytes.
Editing my script.js throws it into the mix, but none of my vendor js files are rendered with it, which is just as useless. grunt-cssmin renders all scss/css files grunt-config.json regardless, which works fine. Moving my js from /files/vendor to /documents/scripts didn't change this behavior.
I've done a little poking around, but I'm new to grunt and nothing jumped out at me.
It'd be nice if I could either:
a) have all JS files in grunt-config.json minned and zipped each time
b) not have grunt min js files in development environment
As is if I want to make any changes to something regarding javascript, I need to ctrl-c docpad and then run it again, which is meh.
Not ideal, but effective enough:
events:
# Write After
# Used to minify our assets with grunt
writeAfter: (opts,next) ->
# Prepare
docpad = #docpad
rootPath = docpad.config.rootPath
balUtil = require 'bal-util'
_ = require 'underscore'
# Make sure to register a grunt `default` task
command = ["#{rootPath}/node_modules/.bin/grunt", 'default']
# Execute
balUtil.spawn command, {cwd:rootPath,output:true}, ->
src = []
gruntConfig = require './grunt-config.json'
_.each gruntConfig, (value, key) ->
src = src.concat _.flatten _.pluck value, 'src'
#_.each src, (value) ->
# balUtil.spawn ['rm', value], {cwd:rootPath, output:false}, ->
#balUtil.spawn ['find', '.', '-type', 'd', '-empty', '-exec', 'rmdir', '{}', '\;'], {cwd:rootPath+'/out', output:false}, ->
next()
# Chain
#
The three lines around "balUtil" which perform find/rm commands were commented out.
Not ideal since the "uncompressed" files are left around -- but that's not really the end of the world. Live-reloading to empty pages was a tad more frustrating, ultimately.
There could be a way to further enhance this to detect a live reload (development) vs generating a build for production, but I haven't grokked that yet.

Django's support for translations in Javascript files

I read and followed the instructions in here, but can't seem to see the string in the javascript in the po file.
structure of my project is:
cb/
cb_app
cb
static_files
templates
First I copied these into my url.py:
js_info_dict = {
'packages': ('cb_app',),
}
urlpatterns = patterns('',
(r'^jsi18n/$', 'django.views.i18n.javascript_catalog', js_info_dict),
)
Then I added this script to my html:
<script type="text/javascript" src="{% url django.views.i18n.javascript_catalog %}"></script>
The actual script where I would like to get the translation, is as simple as that:
$(document).ready(function () {
$('#id_sales_item').chosen({no_results_text: gettext('No results match')});
});
...and is utilized in the same html.
So is there anything more I need to do?
All I did then was to run the line below both from cb/cb and from cb/cb_app.
django-admin.py makemessages -l en_GB
But still no sign of 'No results match' in either cb/cb/locale nor in cb/cb_app/locale
Any tips?
I have finally found the problem.
The documentation suggests creating the messages once from the Django project and once from Django app. That way you end up with two locale directory. And in neither of those would the javascript translations be picked up anyway. This is quite a mess.
The cleanest solution I have found is to go to settings.py and insert this line (see also my project hierarchy above):
LOCALE_PATHS = ( '/home/kave/projects/cb/locale',)
Then create a directory called locale in the project-root-directory (see the path above)
Don't forget applying the entries into url.py and html as well (see above).
Finally now that the local's are unified into one place, go to project-root-directory: /home/kave/projects/cb and run these two commands:
django-admin.py makemessages -l en_GB
django-admin.py makemessages -d djangojs -l en_GB
The first command get the translation texts from both project and app subfolders.
The second gets the javascript translation into a second po file.
Thats it.

have make build/minimize JS files that have changed

I'm trying to learn Make and building a Makefile into my app to help me with building and minimizing my .js files for use of a combo loader server application later on.
What I'm trying to accomplish is that when I run make, it'll copy over to the build directory only the .js files that have changed since the last run, and then minify that file and generate a -min.js copy. Finally I need to always make sure I generate a new meta.js file.
I've pasted what I have working below, the trouble with this is that it's not picking only the changed .js files, but each file on each run. I'm missing something in how to get Make to pick only changed files in this instance.
BOOKIE_JS = bookie/static/js/bookie
JS_BUILD_PATH = bookie/static/js/build
JS_META_SCRIPT = scripts/js/generate_meta.py
jsbuild: $(JS_BUILD_PATH)/bookie/meta.js
clean_js:
rm -rf $(JS_BUILD_PATH)/*
$(JS_BUILD_PATH)/bookie/meta.js: $(BOOKIE_JS)/y*-min.js
$(JS_META_SCRIPT) -n YUI_MODULES -s $(BOOKIE_JS)/y* -o $(JS_BUILD_PATH)/bookie/meta.js
$(BOOKIE_JS)/y*-min.js: $(BOOKIE_JS)/y*.js
scripts/js/jsmin_all.py $(JS_BUILD_PATH)/bookie
# this is the part that runs for each .js file and I'd like it to only run for the *modified* files
$(BOOKIE_JS)/y*.js: $(JS_BUILD_PATH)/bookie
cp $# $(JS_BUILD_PATH)/bookie/
$(JS_BUILD_PATH)/bookie:
mkdir $(JS_BUILD_PATH)/bookie
clean: clean_js
.PHONE: clean clean_js
Current output:
cp bookie/static/js/bookie/yapi.js bookie/static/js/build/bookie/
cp bookie/static/js/bookie/ymodel.js bookie/static/js/build/bookie/
cp bookie/static/js/bookie/ytagcontrol.js bookie/static/js/build/bookie/
cp bookie/static/js/bookie/yview.js bookie/static/js/build/bookie/
scripts/js/jsmin_all.py bookie/static/js/build/bookie
scripts/js/generate_meta.py -n YUI_MODULES -s bookie/static/js/bookie/y* -o bookie/static/js/build/bookie/meta.js
I'd like to see only the cp of the changed files.
I think you intended to make a pattern rule but used the wrong syntax. For example, this:
$(BOOKIE_JS)/y*-min.js: $(BOOKIE_JS)/y*.js
scripts/js/jsmin_all.py $(JS_BUILD_PATH)/bookie
means each of the $(BOOKIE_JS)/y*-min.js files depends on the $(BOOKIE_JS)/y*.js files -- all of them, not just the one with a similar name. If you do this:
$(BOOKIE_JS)/y%-min.js: $(BOOKIE_JS)/y%.js
scripts/js/jsmin_all.py $(JS_BUILD_PATH)/bookie
then the % must be replaced with the same string on each side, so for example $(BOOKIE_JS)/yapi-min.js depends only on $(BOOKIE_JS)/yapi.js

Categories