Hello I'm looking for the best way to automatize my problem:
I am working on a web application in which I use JSON translation files that have this form:
{ "unique_key" : "value"}
I have several files, one for each supported language, which all have the same number of items.
Ex :
i18n_en.json
{ "greeting" : "Hello"}
i18n_fr.json
{ "greeting" : "Bonjour"}
I have very badly managed the evolution of these files, and I end up with keys that are no longer used (I easily think 30% of the ~500 keys), the problem being that I don't know which ones. And that I would have to manually search through the entire architecture of my application to find those that are used and redo a clean file.
My idea to automate this process being:
Open one of the JSON files (no matter which one, they all have the
same number of keys).
Loop for each key
For each key, browse the entire architecture of my project ( by
looking only in the *.html or *.js files)
If we find an occurrence of this key, create an entry in a new clean
json with the key + value of it.
I don't really know which language to use that would be optimized for this kind of task, thank you for guiding me!
Maybe something like i18next-scanner will be useful. You are not the first with such a problem.
I ended up creating my own shell script :
path_to_project=path/to/my/project
RED='\033[0;31m'
GREEN='\033[0;32m'
rm temp.json
rm final.json
touch temp.json
touch final.json
echo "{" >> temp.json
while IFS=, read -r key value
do
if grep -r -q --include=\*.{js,html} --exclude-dir={node_modules,bower_components} $key $path_to_project; then
# write in new json
echo "\"$key\":\"$value\"," >> temp.json
echo -e "${GREEN} $key was found !"
else
echo -e "${RED} $key not found"
fi
done < data.csv
echo "}" >> temp.json
#remove new lines
cat temp.json | tr -d '\r' >> final.json
For this to work, I had to convert my json file to csv (data.csv).
The final json file needs to be reworked a little manually after the script, but really nothing overwhelming.
Related
I am making a pot file where I want the file to scan for gettext keywords in a JS jQuery file. It scans the php perfectly, but the .js file seems not be working. I am wondering if I have actually the extractor setup correct for the command's language. This is what I have:
Language: JS
List of extensions: *.js
Command: xgettext --language=C --force-po -o %o %C %K %F
It scans the file, but nothing is added. What is the language for jQuery?
To give more information:
This is an example of the JS file:
jQuery(document).ready(function() {
if(location.href.indexOf('/<?php esc_html_e('url_page_discount', 'custom-translation-strings' ?>/') > -1){ //rewards
jQuery('.rs_custom_fblike_button').attr('value', '<?php esc_attr_e('I Like', 'custom-translation-strings' ?>');
if (jQuery('#menu-item-10159').length){
jQuery('#rs_friend_subject').attr('value', '<?php esc_attr_e('Discount for an interesting store!', 'custom-translation-strings' ?>');
}
}
});
I have covered the esc_attr_e() in the sources keywords:
I am wondering if I have actually the extractor setup correct for the command's language
You don't. The very fact that you're even configuring extraction suggests that you're either using a very old version of Poedit or you're adding custom configurations. In either case, don't: update to the latest, delete or disable all your custom extractors, and let Poedit do its thing with defaults.
Command: xgettext —language=C --force-po -o %o %C %K %F
You're asking Poedit to treat JavaScript files as being written in C. The results predictably follow the GIGO principle.
This is an example of the JS file:
You're putting (even PHP!) code into string literals. That's a) not going to work and b) cannot have any translatable strings extracted from.
Hei guys!
I need help with the commander node.js library. I need create this CLI which accepts 3 flags, --input, --output and --pattern, like:
commander
.version('3.0.0')
.usage('[options] <file ...>')
.option('-i, --input', 'Array of files to be extracted')
.option('-o, --output', 'Output file name')
.option('-p, --pattern', 'Pattern name to be used in the extraction')
.parse(process.argv);
My problem is with the input flag. I need send several files, for that i need an array data type.
The problem is: I just can't figure it out how to make this:
node ./bin/extract -i ../files/*.PDF
become an array with all my files that are inside my files directory. I already try to run every sample in the documentation, and i didn't find the solution for my problem. Also, i searched in the issues and didn't find either... what is strange, maybe i am doing something wrong and you guys could help??
Thanks!
You can use Coercion to achieve it:
function scanDir(val) {
files = fs.readdirSync(val);
return files;
}
program
.version('0.0.1')
.option('-s, --scan [value]', '', scanDir)
.parse(process.argv);
console.log(' scan: %j', program.scan);
And call it like:
node app.js -s /foo
Specifically, I want to print the results of a mongodb find() to a file. The JSON object is too large so I'm unable to view the entire object with the shell window size.
The shell provides some nice but hidden features because it's an interactive environment.
When you run commands from a javascript file via mongo commands.js you won't get quite identical behavior.
There are two ways around this.
(1) fake out the shell and make it think you are in interactive mode
$ mongo dbname << EOF > output.json
db.collection.find().pretty()
EOF
or
(2) use Javascript to translate the result of a find() into a printable JSON
mongo dbname command.js > output.json
where command.js contains this (or its equivalent):
printjson( db.collection.find().toArray() )
This will pretty print the array of results, including [ ] - if you don't want that you can iterate over the array and printjson() each element.
By the way if you are running just a single Javascript statement you don't have to put it in a file and instead you can use:
$ mongo --quiet dbname --eval 'printjson(db.collection.find().toArray())' > output.json
Since you are doing this on a terminal and just want to inspect a record in a sane way, you can use a trick like this:
mongo | tee somefile
Use the session as normal - db.collection.find().pretty() or whatever you need to do, ignore the long output, and exit. A transcript of your session will be in the file tee wrote to.
Be mindful that the output might contain escape sequences and other garbage due to the mongo shell expecting an interactive session. less handles these gracefully.
Just put the commands you want to run into a file, then pass it to the shell along with the database name and redirect the output to a file. So, if your find command is in find.js and your database is foo, it would look like this:
./mongo foo find.js >> out.json
Put your query (e.g. db.someCollection.find().pretty()) to a javascript file, let's say query.js. Then run it in your operating system's shell using command:
mongo yourDb < query.js > outputFile
Query result will be in the file named 'outputFile'.
By default Mongo prints out first 20 documents IIRC. If you want more you can define new value to batch size in Mongo shell, e.g.
DBQuery.shellBatchSize = 100.
Using print and JSON.stringify you can simply produce a valid JSON result.
Use --quiet flag to filter shell noise from the output.
Use --norc flag to avoid .mongorc.js evaluation. (I had to do it because of a pretty-formatter that I use, which produces invalid JSON output)
Use DBQuery.shellBatchSize = ? replacing ? with the limit of the actual result to avoid paging.
And finally, use tee to pipe the terminal output to a file:
// Shell:
mongo --quiet --norc ./query.js | tee ~/my_output.json
// query.js:
DBQuery.shellBatchSize = 2000;
function toPrint(data) {
print(JSON.stringify(data, null, 2));
}
toPrint(
db.getCollection('myCollection').find().toArray()
);
Hope this helps!
I managed to save result with writeFile() function.
> writeFile("/home/pahan/output.txt", tojson(db.myCollection.find().toArray()))
Mongo shell version was 4.0.9
In the new mongodb shell 5.0+ mongosh, it integrate the Node.js fs module, so you can simply do below in the new mongosh shell for pretty print the output:
fs.writeFileSync('output.json', JSON.stringify(db.test.find().toArray(), null, 2))
Without any problems such as the ObjectId has been stripped, etc., which is better than the printjson or .pretty().
The above code can work as the description denotes:
The MongoDB Shell, mongosh, is a fully functional JavaScript and Node.js 14.x REPL environment for interacting with MongoDB deployments. You can use the MongoDB Shell to test queries and operations directly with your database.
The old mongo shell also marked as Legacy, so you should move to this new way.
Using this answer from Asya Kamsky, I wrote a one-line bat script for Windows. The line looks like this:
mongo --quiet %1 --eval "printjson(db.%2.find().toArray())" > output.json
Then one can run it:
exportToJson.bat DbName CollectionName
Also there is mongoexport for that, but I'm not sure since which version it is available.
Example:
mongoexport -d dbname -c collection --jsonArray --pretty --quiet --out output.json
As answer by Neodan mongoexport is quite useful with -q option for query. It also convert ObjectId to standard format of JSON "$oid". E.g:
mongoexport -d yourdb -c yourcol --jsonArray --pretty -q '{"field": "filter value"}' -o output.json
you can use this command to acheive it:
mongo admin -u <userName> -p <password> --quiet --eval "cursor = rs.status(); printjson(cursor)" > output.json
It is stated here that Babel can extract gettext messages for Python and Javascript files.
Babel comes with a few builtin extractors: python (which extracts
messages from Python source files), javascript, and ignore (which
extracts nothing).
The command line extractor is documented here - but with no examples on usage.
Also in the same pointer above, there is some mention of a config file to be used with extraction, but not much expanded on.
When I run the basic command for the extractor on a dir with js files, I get only the .PO header generated but no messages.
$ pybabel extract /path/to/js-dir
# Translations template for PROJECT.
# Copyright (C) 2012 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL#ADDRESS>, 2012.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL#ADDRESS\n"
"POT-Creation-Date: 2012-04-22 19:39+1000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL#ADDRESS>\n"
"Language-Team: LANGUAGE <LL#li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 0.9.6\n"
$
Here is a sample segment from a js file I'm trying to extract messages for:
else if(data.status == "1"){
var follow_html = gettext('Follow');
object.attr("class", 'button follow');
object.html(follow_html);
var fav = getFavoriteNumber();
fav.removeClass("my-favorite-number");
if(data.count === 0){
data.count = '';
fav.text('');
}else{
var fmts = ngettext('%s follower', '%s followers', data.count);
fav.text(interpolate(fmts, [data.count]));
}
}
I would appreciate it if someone can provide exact CLI options and config settings to make the extraction work, or a pointer to such.
Create a file (babel.cfg) with the following content:
[javascript:*.js]
encoding = utf-8
Then, do:
pybabel extract -F babel.cfg /path/to/js-dir
That should be enough for you to have some message strings.
BTW, you can consult the help for the extract command by doing:
pybabel extract --help
I had a similar issue and was able to get around it by disabling default keywords with babel.
pybabel extract -k __ -F babel.cfg --no-default-keywords /path/to/js-dir
You must specify at least one keyword in the command when you disable the defaults (-k [keyword]). I chose -k __ because "__" was a pattern I was looking for.
Just use this command and replace the "__" after -k with one from your babel.cfg file.
Edit: this allows you to use your own keywords rather than gettext()
You can create an object in as flask global and translate it with gettext
g.i18n = {
'Casa' : lazy_gettext('Home'),
'Auto' : lazy_gettext('Car'),
'Persona' : lazy_gettext('Person')
}
Then add it as a variable
<script>
var i18n = {{ g.i18n | tojson }}
</script>
and use it in JS:
var labelTranslate = {
Casa: i18n.Casa,
Persona: i18n.Persona,
Auto: i18n.Auto
};
You can actually use gettext directly in Javascript.
See: jsgettext. It allows you to use the standard *gettext functions, including the one using contexts and/or plural forms.
It can read PO/MO files or you can import custom made JSON files instead.
See this file of this project for a complete example.
I would like be able to run a single command in my project folder to concatenate and compress all of my javascript files (perhaps with YUI Compressor) into a single output file.
If possible I would like to partially specify the order in which they are concatenated together but not have to keep track of every single javascript file. Perhaps a config file could be built which looks like this:
application.js
excanvas.js
json2.js
jquery*.js
flot/*
backbone*.js
app/screen-*.js
app/main.js
app/crud-*.js
app/*
*
Does anyone know of either an existing tool to do something like this, could whip together a bash/ruby/node/perl script, or even a better methodology? I'm building a Single Page App with heavy JS usage (~40 files) to be consumed by people with low bandwidth.
I would need the solution to be executable on my OS X development machine.
find . -iname "*.js" -exec cat "{}" \; > singlefile.js
[JS compressor] singlefile.js
First concatenate the files, then compress them.
If you really care, though, you may want a real JS optimizer like the RequireJS optimizer.
given a folder of javascript files:
geee: ~/src/bash/js-files
$ find .
.
./application.js
./jquery-ui.js
./all-scripts.js
./cp.js
./excanvas.js
./backbone-worldwide.js
./jquery-plugin.js
./.found
./app
./app/crud-sel.js
./app/screen-detach.js
./app/aligator.js
./app/crud-in.js
./app/giraffe.js
./app/screen-attach.js
./app/main.js
./app/crud-del.js
./app/mouse.js
./app/monkey.js
./app/screen-shot.js
./backbone-national.js
./backbone23.js
./ummap.js
./CONFIG
./backbone-ibm.js
./ieee754.js
./flot
./flot/cow.js
./flot/moo.js
./flot/cat.js
./flot/bull.js
./flot/dog.js
./flot/sheep.js
./lines
./droiddraw-r1b21
./droiddraw-r1b21/._readme.txt
./droiddraw-r1b21/readme.js
./droiddraw-r1b21/LICENSE.js
./jquery-1.7.js
./ole.js
./touch
./json2.js
./xls2txt.js
./DO.sh
./backbone-isp.js
with a slightly modified configuration file:
geee: ~/src/bash/js-files
$ cat CONFIG
application.js
excanvas.js
json2.js
jquery*.js
flot/*
backbone*.js
app/screen-*.js
app/main.js
app/crud-*.js
app/*js
*js
and this bash script:
$ cat DO.sh
PROJECT=/home/jaroslav/src/bash/js-files # top folder of the web-app
SUPERJS=${PROJECT}/all-scripts.js
CONFIG=${PROJECT}/CONFIG # your the priority file (notice *js)
FOUND=${PROJECT}/.found # where to save results
JSMIN=$HOME/bin/jsmin # change to /usr/local/bin/jsmin or some other tool
echo > $FOUND # remove results from previous run
if [ ! -x $JSMIN ]
then
TMPJSMIN=/tmp/jsmin.c
wget -q https://github.com/douglascrockford/JSMin/raw/master/jsmin.c -O $TMPJSMIN & FOR=$?
echo "fetching jsmin (by Douglas Crockford) from github"
wait $FOR
gcc -o $JSMIN $TMPJSMIN
fi
cat $CONFIG | \
while read priority
do
eval "find $priority|sort -n" | \
while read amatch;
do
grep -q $amatch $FOUND || echo $amatch >> $FOUND
done
done
echo minifying:
cat $FOUND
cat `cat $FOUND` | $JSMIN > $SUPERJS
you will find the "merged" script in all-scripts after runing the script:
geee: ~/src/bash/js-files
$ . DO.sh
fetching jsmin (by Douglas Crockford) from github
[1]+ Done wget -q https://github.com/douglascrockford/JSMin/raw/master/jsmin.c -O $TMPJSMIN
minifying:
application.js
excanvas.js
json2.js
jquery-1.7.js
jquery-plugin.js
jquery-ui.js
flot/bull.js
flot/cat.js
flot/cow.js
flot/dog.js
flot/moo.js
flot/sheep.js
backbone23.js
backbone-ibm.js
backbone-isp.js
backbone-national.js
backbone-worldwide.js
app/screen-attach.js
app/screen-detach.js
app/screen-shot.js
app/main.js
app/crud-del.js
app/crud-in.js
app/crud-sel.js
app/aligator.js
app/giraffe.js
app/monkey.js
app/mouse.js
all-scripts.js
cp.js
ieee754.js
ole.js
ummap.js
xls2txt.js
Let me know if you need me to explain the script or if it fails on OS X.
The following script will follow the order of your config file and use the patterns given
#!/bin/bash
shopt -s nullglob;
while read config; do
cat $config >> out.js
done < /path/to/config/file
I ended up building a solution which uses a json file to list all of the files required by the app. On the dev environment the files are individually loaded by the browser. On the production server, the big compiled file is loaded. On my dev machine, I manually run a command to iterate over each file, appending it to a big JS file and running YUI Compressor.
It's a little hacky, but here it is:
https://github.com/renownedmedia/js-compressor