Grunt - mongoimpot --headerlines? - javascript

I am trying to upload a CSV file using the mongoimport plugin for grunt. I am receiving the following error when running grunt:
Running "mongoimport" task
2017-03-29T18:43:57.463-0700 error validating settings: must specify --fields, --fieldFile or --headerline to import th
is file type
2017-03-29T18:43:57.464-0700 try 'mongoimport --help' for more information
The error message is pretty obvious so I adjusted my grunt task to include headerline: true
This doesn't appear to be correct as the error remains the same. After looking at their documentation for any mentioning of the --headerline property, the only thing I see is this:
collection.fields
Specify a comma separated list of field names when importing csv or tsv files that do not have field names in the first (i.e. header) line of the file.
- https://github.com/andrewkeig/grunt-mongoimport
What is the correct way using grunt to import mongodb data from a csv?
gruntfile.ts
module.exports = (grunt) => {
grunt.initConfig({
mongoimport: {
options: {
db: 'mynewdatabase',
collections: [
{
name: 'somethings',
type: 'csv',
file: 'data/test.csv',
headerline: true // <----does not work
}
]
}
}
});
//Load Plugins
grunt.loadNpmTasks('grunt-mongoimport');
//Run Tasks
grunt.registerTask('default', ['mongoimport']);
}
The csv file does have headlines and if I run the below mongo command in a terminal, it works fine:
use mynewdatabase
mongoimport --db mynewdatabase --collection somethings --type csv --headerline --file test.csv
Looking at the mongimport source code, they have the following logic:
if (collection.type === 'csv' || collection.type === 'tsv') {
if (collection.headerLine) args.push('--headerline');
}
If i'm passing the type of csv then it looks like --hearline should be getting added already so the error message doesn't seem to be aligning with what i'm seeing the code/task...

I was looking over the documentation. In their source code they have the following
if (collection.type === 'csv' || collection.type === 'tsv') {
if (collection.headerLine) args.push('--headerline');
}
This means if you pass headerLine: true and your type: 'csv' then you mongoimport will include the --headerline
module.exports = (grunt) => {
grunt.initConfig({
mongoimport: {
options: {
db: 'mynewdatabase',
collections: [
{
name: 'somethings',
type: 'csv',
file: 'data/test.csv',
headerLine: true // <----works
}
]
}
}
});
//Load Plugins
grunt.loadNpmTasks('grunt-mongoimport');
//Run Tasks
grunt.registerTask('default', ['mongoimport']);
}

Related

Modify JS files using another JS file

How can I edit a JS file (containing a JSON object) using another JS file?
Example:
JS file to be edited (eslintrc.js):
module.exports = {
root: true,
parserOptions: {
parser: 'babel-eslint',
sourceType: 'module'
},
env: {
browser: true
}
}
And I want to let's say add additional configuration lets say in:
env: {
browser: true,
node: true
}
I tried doing something like:
fs = require('fs');
var m = JSON.parse(fs.readFileSync('.eslintrc.js'));
console.log(m)
But of course, as expected, I am getting errors.
undefined:1
module.exports = {
^
SyntaxError: Unexpected token m in JSON at position 0
at JSON.parse (<anonymous>)
To also add, this is not the only file I need to update. There are several others that look similar in nature but not exactly the same such as:
module.exports = function (ctx) {
return {
// --> boot files are part of "main.js"
boot: [
],
css: [
'custom.scss'
]
}
}
How can I open the JS file and be able to edit the JSON object inside without distoring the actual JS file?
Thank you!
You must eliminate 'module.exports =' from eslintrc.js in order to parse it with JSON.parse function.

Terser does not give minified file

I am trying to minify an angularjs application using grunt and terser. I first used uglifiy-es but then read that it has some issues. So I tried terser. But the output does not give me minified files.
The gruntfile.js
module.exports = function(grunt) {
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
//grunt task configuration will go here
ngAnnotate: {
options: {
singleQuotes: true
},
app: {
files: {
'./public/min-safe/js/_config_min.js': ['./controllers/_config.js'],
'./public/min-safe/js/ctrl_accountingdashboard.js': ['./controllers/ctrl_accountingdashboard.js'],
}
}
},
concat: {
js: { //target
src: ['./public/min/app.js', './public/min-safe/js/*.js'],
dest: './public/min/app.js'
}
},
terser: {
options: {},
files: {
'./public/min/app.js': ['./public/min/app.js'],
},
}
});
//load grunt tasks
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-terser');
grunt.loadNpmTasks('grunt-ng-annotate');
//register grunt default task
grunt.registerTask('default', ['ngAnnotate', 'concat', 'terser']);
}
I had the same problem. According to the documentation, this should work but it didn't for me. Wrapping the "files" setting in a custom target works for me:
terser: {
options: {},
main: {
files: {
'./public/min/app.js': ['./public/min/app.js'],
}
}
}
To add to #Tim's great answer:
Here is an example that allows to run grunt-terser with path / file wildcard patterns (globbing) – which it does not support out of the box.
Please note the helper properties _src and _dest in the terser config which are not read by grunt-terser itself but by the task terser_all. This task expands the globbing pattern(s) in _src and builds the real config in the files property. When done it runs terser with that updated config.
module.exports = function (grunt) {
grunt.initConfig({
terser: {
dist: {
options: {
compress: {
drop_console: true // remove console.log, console.info, ...
}
},
files: {
// FILLED through terser_all task below!
// Examples config:
// "dist/example.js": [ "path/to/files/example.js" ]
// "dist/example_joined.js": [ "path/to/files/*.js" ]
},
// -----
// HELPER PROPERTIES to build the files prop (see above) in the terser_all task below.
_src: [
"path/to/files/*.js"
],
_dest: "dist/"
}
}
});
grunt.registerTask('terser_all', function () {
// work on this target in terser config
var terser_target_name = "dist";
// read the terser config
var terser_config = grunt.config.get('terser') || {};
var terser_target_config = terser_config[terser_target_name] || {};
// get the destination dir
var destDir = terser_target_config._dest;
// loop through all source files and create an entry in the terser config for each of it
var files = grunt.file.expand(terser_target_config._src);
for (const [i, file] of files.entries()) {
grunt.log.writeln(file);
// add this file to the terser config as: dest_file: src_file
terser_target_config.files[destDir + file] = file;
}
// show new config on CLI
grunt.log.writeflags(terser_target_config);
// write back config and run task
grunt.config.set('terser', terser_config);
grunt.task.run('terser');
});
grunt.loadNpmTasks('grunt-terser');
grunt.registerTask('build', ['terser_all']);
grunt.registerTask('default', ['build']);
};
Just a note:
If you try to "disable" some options by renaming this disables the whole process. At least this was my result with grunt-terser. I was left with the original js file.
{
mangleX: {
reserved: [/* ... */]
}
}

Dynamically set script tag through Grunt based on ENV

During my Grunt build process, I would like to set the script tag in my index.html which references the Google Maps API dynamically based on process.env.NODE_ENV.
Something like:
let googleMapsUrl;
if (process.env.NODE_ENV === 'development') {
googleMapsUrl = '//maps.googleaps.com/maps/api/js?v=3.exp&libraries=visualization';
} else {
googleMapsUrl = `//maps.googleaps.com/maps/api/js?key=${process.env.GOOGLE_MAPS_API_KEY}v=3.exp&libraries=visualization`;
}
My question is how do I then insert googleMapsUrl into a script tag in index.html like so: <script src=googleMapsUrl></script>
There are many options that you could use. From simple, like grunt-replace to more advanced ones like grunt-processhtml. I will describe the first one as it’s a good option for simple tasks and requires just a simple configuration.
Using grunt-replace
grunt-replace will search for simple variable definitions like ##foo in your source files, and replace those variables with your supplied value or the returning value from a callback.
First install the plugin using npm install grunt-replace --save-dev
Then configure the task as follows:
replace: {
dist: {
options: {
patterns: [
{
match: 'gmaps',
replacement: function() {
var googleMapsUrl;
if(process.env.NODE_ENV === 'development') {
googleMapsUrl = '//maps.googleaps.com/maps/api/js?v=3.exp&libraries=visualization';
} else {
googleMapsUrl = '//maps.googleaps.com/maps/api/js?key=' + process.env.GOOGLE_MAPS_API_KEY + 'v=3.exp&libraries=visualization';
}
return googleMapsUrl;
}
}
]
},
files: [
{
expand: true,
flatten: true,
src: ['path/to/your/source/file.html'], dest: 'destination/folder/'
}
]
}
}
Load the plugin using grunt.loadNpmTasks('grunt-replace') and then add it to your build process.
On your source files just add the defined variable as your script source:
<script src="##gmaps"></script>
The task will replace ##gmaps with the returning value from the defined callback.
Hope it helps.
More about plugin can be found here.

Grunt run curl on changed file only

I'm trying to upload a file to a server on change using grunt-run and curl. I can get it to work if I hard code the file name into the actual task, but I'm trying to run it based on the file that changed...here's my grunt file so far (stripped down to the parts related to this question).
module.exports = function(grunt) {
var config = require('./config.json');
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
watch: {
less : {
files : ['front-end/less/**/*.less'],
tasks : ['newer:less','sync:css','run:deploy_less_files']
},
},
less: {
development: {
options: {
paths: ['front-end/_builds/css'],
sourceMap : true,
},
files: [{
cwd: "front-end/less",
expand : true,
src : [ '**/*.less', '!_settings.less'],
dest : "front-end/_builds/css",
ext: ".css"
}]
},
},
sync : {
target: {},
css : {
expand: true,
flatten :true,
verbose: true,
cwd : "front-end/_builds/css/",
src : "**/*.css",
dest : "target/css/"
},
},
run : {
deploy_less_files : {}
}
});
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-contrib-less');
grunt.loadNpmTasks('grunt-sync');
grunt.loadNpmTasks('grunt-run');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-clean');
grunt.loadNpmTasks('grunt-newer');
grunt.registerTask('default', ['watch']);
grunt.event.on('watch', function(action, filepath, target) {
if (target == "less") {
grunt.config.set('run.deploy_less_files.exec','curl -u ' + config.credentials.user + ':' + config.credentials.pw + ' -T ' + filepath + ' http://localhost:8080/assets/less/');
grunt.task.run("run:deploy_less_files");
}
});
}
Here's what I am trying to do in order:
Watch all LESS files in /front-end/less
If a file changes, compile it to css and place in front-end/_builds/css directory
Sync the contents of front-end/_builds/css with my target/css directory.
Upload the file via curl to my localhost.
Ideally, I'd like to just grab the css file from either my target or the _builds directory and upload it to my localhost, but I can sort that out if I can get this part working.
Any help would be appreciated. Thanks.
See this comment in the grunt-contrib-watch github repo. Excerpts from it read:
"Don't use grunt.event.on('watch') to run your tasks."
and...
"The watch event is not intended for running tasks."
However, you can utilize the grunt.event.on('watch', ...) listener to configure the exec property.
The following gist shows how to meet your requirement:
Gruntfile.js
module.exports = function(grunt) {
var config = require('./config.json');
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
watch: {
less : {
files : ['front-end/less/**/*.less'],
tasks : ['newer:less','sync:css'/*,'run:deploy_less_files'*/]
},
// 1. Added new target to watch the directory in which the resultant
// .css files are saved. Set the `task` to `run:deploy_css_files`.
// The `nospawn` option must be set to true.
generatedCss : {
files : ['target/css/**/*'],
tasks : ['run:deploy_css_files'],
options: {
nospawn: true
}
}
},
less: {
development: {
options: {
paths: ['front-end/_builds/css'],
sourceMap : true,
},
files: [{
cwd: "front-end/less",
expand : true,
src : [ '**/*.less', '!_settings.less'],
dest : "front-end/_builds/css",
ext: ".css"
}]
},
},
sync : {
target: {},
css : {
expand: true,
flatten :true,
verbose: true,
cwd : "front-end/_builds/css/",
src : "**/*.css",
dest : "target/css/"
},
},
run : {
deploy_css_files : {// 2. Renamed target and added `exec` key. The
exec: '' // `exec` value is intentionally empty and will
} // be configured via the `watch` event listener.
}
});
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-contrib-less');
grunt.loadNpmTasks('grunt-sync');
grunt.loadNpmTasks('grunt-run');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-clean');
grunt.loadNpmTasks('grunt-newer');
// 3. 'less' and 'sync' targets are aliased before the `watch`
// task to ensure the path defined in the `files` property of the
// `watch:generatedCss` task exists before `watch`ing.
grunt.registerTask('default', ['less', 'sync', 'watch']);
// 4. Listen for changes via the `watch:generatedCss` target only.
// Configures the `exec` property of the `run:deploy_css_files`
// target to `curl` the most recently created `.css` file.
grunt.event.on('watch', function(action, filepath, target) {
if (target === 'generatedCss') {
var cmd = 'curl -u ' + config.credentials.user + ':' +
config.credentials.pw + ' -T ' + filepath +
' http://localhost:8080/assets/less/';
grunt.config('run.deploy_css_files.exec', cmd);
}
});
}
Further explanation
Firstly, in the Gruntfile.js above, you'll notice that run:deploy_less_files alias has been omitted from your original watch.less.tasks array. Then the following changes were made:
Added a new target named generatedCss to the watch Task. Its files value specifies the path to the directory in which the resultant .css files are saved. The value in the task property array is set to run:deploy_css_files. The nospawn option is set to true.
Note As you mentioned the following in your question:
"Ideally, I'd like to just grab the css file from either my target or the _builds directory and upload it to my localhost,.."
I chose to name the target generatedCss and renamed the task to run deploy_css_files (instead of deploy_less_files) as this better reflects the actual intent.
The files that ultimately get uploaded via curl to your localhost will be from the target/css/ directory, simply because that's the directory we're watching for changes.
Replaced your original run task with the following instead:
run : {
deploy_css_files : {
exec: ''
}
}
Note The target has been renamed and the exec property added. Its value is intentionally empty as this will be configured via the watch event listener.
less and sync targets are aliased before the watch task of the default registered task. This ensures that, (when initially running grunt via your CLI), the path defined in the files property of the watch:generatedCss task exists before watching begins.
Finally, in the grunt.event.on('watch', ...) listener we listen for changes via the watch:generatedCss target only and configure the exec property of the run:deploy_css_files target to curl the most recently created .css file.
Running
When running the grunt command via your CLI any changes made to the .less files, (i.e. those residing in the /front-end/less directory), will trigger the tasks in the correct order (as per your listed points 1-4).
Caveat: I didn't actually test running the curl command, however the file path for the most recent generated .css file is assigned to the filepath variable in the grunt.event.on('watch', ...) listener, thus it can be referenced when configuring the run.deploy_css_files.exec task/target.
Note: you'll need to ensure the server supports POST requests for your curl command to succceed (i.e. it's not something like SimpleHTTPServer).

Calling grunt.config.set inside grunt.util.spawn doesn't seem to have any effect

I'm trying to set the current Git SHA in my project's Grunt configuration, but when I try to access it from another task it isn't available, What am I missing?
grunt.registerTask('sha', function () {
var done = this.async();
grunt.util.spawn({
cmd: 'git',
args: ['rev-parse', '--short', 'HEAD']
}, function (err, res) {
if (err) {
grunt.fail.fatal(err);
} else {
grunt.config.set('git', {sha: res.stdout});
if (grunt.option('debug') || grunt.option('verbose')) {
console.log("[sha]:", res.stdout);
}
}
done();
});
});
After running the task, I expect the config to be available in another task configuration:
requirejs: {
dist: {
...
out: '<%= app.dist %>/scripts/module_name.<%= git.sha %>.js'
...
}
}
So... What's the problem?
The problem is that Require JS is writing to the file public/scripts/module_name..js, the SHA is not available in the configuration (when the filename should be public/scripts/module_name.d34dc0d3.js).
UPDATE:
The problem is that I'm running requirejs tasks with grunt-concurrent, so the Grunt configuration is not available for requirejs.
grunt.registerTask('build', [
...
'getsha',
'concurrent:dist',
...
]);
And the concurrent task, looks like:
concurrent: {
dist: [
...
'requirejs',
...
]
}
Since grunt-concurrent will spawn tasks in child processes, they do not have access to the context of the parent process. Which is why doing grunt.config.set() within the parent context is not available in the config of the child context.
Some of the solutions to make the change available in the child context are:
Write the data to the file system
Write the data to a temporary file with grunt.file.write('./tmp/gitsha', res.stdout) and then have the task being ran in a child process read the temporary file:
out: (function() {
var out = grunt.config('app.dist') + '/scripts/module_name.';
if (grunt.file.exists('./tmp/gitsha')) {
out += grunt.file.read('./tmp/gitsha');
} else {
out += 'unknown';
}
return out + '.js';
}())
Use a socket
This is a very convoluted solution but a solution nonetheless. See the net node docs: http://nodejs.org/api/net.html#net_net_createserver_options_connectionlistener on creating a server on the parent process then have the child process connect to the socket for the data.
Or check out https://github.com/shama/blackbox for a library that makes this method a bit simpler.
Fork the parent process instead of spawn/exec
Another method is to use fork: http://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options instead of grunt-concurrent. Fork lets you send messages to child processes with child.send('gitsha') and receive them in the child with process.on('message', function(gitsha) {})
This method also can get very convoluted.
Use a proxy task
Have your sha task set the config as you're currently doing:
grunt.registerTask('sha', function() {
grunt.config.set('git', { sha: '1234' });
});
Change your concurrent config to call a proxy task with the sha:
grunt.initConfig({
concurrent: {
dist: [
'proxy:requirejs:<%= git.sha %>'
]
}
});
Then create a proxy task that runs a task with setting the passed value first:
grunt.registerTask('proxy', function(task, gitsha) {
grunt.config.set('git', { sha: gitsha });
grunt.task.run(task);
});
The above can be simplified to set values specifically on requirejs but just shown here as a generic example that can be applied with any task.

Categories