We Keep Coding

Passing sensitive information in the req object Node.js server

I am an ex-PHP Laravel developer and I am trying to explore other options right now, currently using Node.js, Sequelize, Koa and Next.js to build a small app.
When sequelize is initialized, a models folder gets created automatically and it has along the lines the following code.
...
fs
.readdirSync(__dirname)
.filter(file => {
console.log('Filer file: ', file)
return (file.indexOf('.') !== 0) && (file !== basename) && (file.slice(-3) === '.js');
})
...
Which is working fine when imported in the main server.js file.
I am trying to figure out what is the best way to include this in any of the other files, in my case I need to have access to it via the newly release API routes from Next.js /api/{endpoint} (https://nextjs.org/docs/api-routes/introduction).
If I import the file there __dirname (from the main sequelize models file) is the root directory of my machine which makes me think that if this code is automatically generated code by their CLI tool I am probably doing something wrong.
One way I found to make this work is by passing the db object via the req object in the server.js file.
router.get('/api/*', async (ctx) => {
/**
* There has to be a better way
*/
ctx.req.db = db
await handle(ctx.req, ctx.res);
ctx.respond = false;
ctx.res.statusCode = 200;
});
Is there better way to handle all of this?

File path in Meteor

I'm having trouble identifying the path to a file in the public directory c:\TEMP\todos\.meteor\local\build\programs\server\public\main.py. Meteor complains the file or directory doesn't exist. Already searched the other postings about the similar issue (e.g., Reading files from a directory inside a meteor app) but didn't help.
Here is the error message.
=> Your application has errors. Waiting for file change.
=> Modified -- restarting.
=> Meteor server restarted
W20151206-04:05:57.893(-5)? (STDERR) Error inside the Async.runSync: ENOENT, no such file or directory 'c:\TEMP\todos\.meteor\local\build\programs\server\public'
Client code
Meteor.call('runPython', function(err, response) {
if(err){
} else {
console.log(response);
}
})
Server code
Meteor.startup( function (){
Meteor.methods({
runPython: function (){
var PythonShell = Meteor.npmRequire('python-shell');
var fs = Meteor.npmRequire('fs');
var runPython = Async.runSync(function (done){
var files = fs.readdirSync('./public/');
// PythonShell.run('main.py', function ... was tried first but Meteor complained that "main.py doesn't exist". So below is a different attempt.
var py = _(files).reject(function(fileName){
return fileName.indexOf('.py') <0;
})
PythonShell.run(py, function (err) {
// PythonShell.run(path.join(py,"main.py") ... was also tried but resulted in the same error
if (err) throw err;
console.log('script running failed');
});
})
return "Success";
}
})
})

All files inside the public folder should be read using '/':
var files = fs.readdirSync('/');
More here: http://docs.meteor.com/#/full/structuringyourapp
For server-side only (might be your case and probably a better solution) you can put everything under the private/ folder and access them by using the Assets API: http://docs.meteor.com/#/full/assets_getText

Clearly I was overthinking it. Specifying a full path to the file was all I needed to do.
PythonShell.run('c:\\project\\public\\main.py', function ...

If your application allows moving the Python script to /private instead of /public, you can take advantage of Meteor's Assets:
Assets allows server code in a Meteor application to access static server assets, which are located in the private subdirectory of an application’s tree. Assets are not processed as source files and are copied directly into your application’s bundle.
e.g. If you move your script to /private/scripts/script.py you can generate the absolute path the Meteor way by doing Assets.absoluteFilePath('scripts/script.py').

How to zip a folder in node js application & after zip downloading start?

I tried with npm package adm-zip 0.4.4 because the latest one 0.4.7 doesn't work, adm-zip 0.4.4 works on Windows but not on Mac & Linux. Another problem is that I only want zip_folder to be zipped but it zipps the whole directory structure staring from folder_1. This is the code:
var zip = new admZip();
zip.addLocalFolder("./folder_1/folder_2/folder_3/**zip_folder**");
zip.writeZip("./folder_1/folder_2/folder_3/download_folder/zip_folder.zip");
All this happens on the server side. I have searched a lot and tried many npm packages to zip a folder or directory. Any suggestions or any other good approach to solve my problem?

You could also use node-archiver, which was very helpful when I was using it. First you need create an instance of the archiver as follows:
var fs = require('fs');
var archiver = require('archiver');
var archive = archiver.create('zip', {});
var output = fs.createWriteStream(__dirname + '/zip_folder.zip');
This way you tell the archiver to zip the files or folders based on the format you pass along with the method. In this case it's zip. In addition, we create a writeStream which will be piped to the archiver as its output. Also, we use the directory method to append a directory and its files, recursively, given its dirpath:
archive.pipe(output);
archive
.directory(__dirname + '/folder_1/folder_2/folder_3/download_folder/zip_folder')
.finalize();
At the end, we need to finalize the instance which prevents further appending to the archive structure.
Another option is to use the bulk method like so:
archive.bulk([{
expand: true, cwd: './folder_1/folder_2/folder_3/download_folder/zip_folder/',
src: ['**/*']
}]).finalize();
Update 1
A little explanation for the [**/*] syntax: This will recursively include all folders ** and files *.

Try to use the system's zip function:
var execFile = require('child_process').execFile;
execFile('zip', ['-r', '-j', zipName, path], function(err, stdout) {
if(err){
console.log(err);
throw err;
}
console.log('success');
});
Replace zipName and path with what you need.

Protractor won't take screenshot in Jenkins

I am using the following code to take screenshots (in after each) when a test fails in Protractor:
function failScreenshot() {
var fs = require('fs');
var spec = jasmine.getEnv().currentSpec;
var specName = spec.description.split(' ').join('_');
if (spec.results().passed()) {
return;
} else {
browser.takeScreenshot().then(
function(png) {
var stream = fs.createWriteStream('screenshots/' + specName + '.png');
stream.write(new Buffer(png, 'base64'));
stream.end();
});
}
}
When I am running the tests locally, the screenshot works just as expected. When running the tests via Jenkins, the tests will stop at the first fail and the screenshot is not created. Also, the folders and paths are correct, I have checked them over and over again. My Jenkins version is 1532.1
Any ideeas on how could I solve this issue?

After further documentation I have found the answer. It was a problem with the path. It seems like NODE JS does not read the path as I thought.
The ./ returns the current directory, except in the require() function. When using require(), it reads ./ to the directory of the file in which it was called (obviously, the mistake was here). __dirname is always the directory of the file in which is used.
The code to be used for my path is the following:
__dirname + '/screenshots/' + specName + '.png'

You can also take the screenshots in jenkins by using the mocha-proshot reporter.
It is a npm package which can be downloaded easily and is very easy to setup.

How to wait until a file is available in a Jake build (Node.js)?

Is there a way in a Node.js Jake build to wait until a certain file has been copied, and advance to do some operation only after the destination file can be found? I think this question pretty much comes down to "is there a way to copy files synchronously in Node.js/Jake?" (Perhaps something else than writing something from scratch, using the combination of fs.readSync and fs.writeSync.)
Background:
I'm developing a web app that is run on Node.js (with Express) during development, but will be deployed on a Java server in production. (We use Jade and Stylus in the client and Express enables us to run the app without generating all the HTML files etc. and deploying it after every change.)
I use Jake for making the build, i.e. generating HTML files from Jade files and CSS from Stylus files etc. Now I'm also trying to concatenate all of the app's JavaScript files into one minimized file and change all the HTML files to use that instead of all the separate JS files that are used in "raw" form during development.
However, I now have a problem with that last step. My idea was to copy all of my Jade files into a temporary directory for the deployment build and replace the reference (in a Jade file used as a header on all HTML pages) to a list of all separate JS files to the one that has just been generated by concatenating and minimizing the whole bunch. But as I first copy all of the Jade files to another location (which happens asynchronously) and try to edit one of the files, opening the file always fails since the copy operation hasn't really finished yet.
This is what I have now (in a simplified form) in my jakefile:
var fs = require('fs');
var fse = require('fs-extra');
var path = require('path');
var glob = require('glob');
var Snockets = require('snockets');
var snockets = new Snockets();
// generating the minimized JS file
snockets.getConcatenation(baseDir + '/scripts/all.js', { minify: true }, function(err, allJs) {
if (err) {
throw err;
}
fs.writeFileSync(generatedJsFileName, allJs);
});
// copying all the Jade files to a temp dir
glob.sync('**/*.*', {
cwd : srcDir
}).forEach(function(file) {
var loadPath = srcDir + '/' + file;
var savePath = targetDir + '/' + file;
fse.mkdirsSync(path.dirname(savePath));
fse.copy(loadPath, savePath);
});
// trying to read one of the copied files (which fails, since the file cannot be found yet)
fs.readFile(targetDir + '/views/includes/head.jade', 'utf8', function(err, data) {
...
});
This might be a stupid question, and a stupid way to try to solve the problem in the first place. So, also suggestions for a better approach are very welcome.
Update:
I also tried using Parseq, putting each operation (creating the JS file, copying the Jade files, reading one file) in its own function, but even that gives me the same error. If I run the script several times without deleting the target directory of the copy operation in between, the file can be found. So e.g. the path is correct and the problem really seems to be about timing.

I didn't really find an answer to the main question so I don't know if this helps anyone else facing the same problem. But I did find a way to get around the problem.
I ended using the same original Jade files for the two different conversions, but in the second conversion I use a custom js function to change the script tag reference to point to the minified file.
I.e.
var data = jade.compile(str, { filename: file, pretty: true })({
css: function(path) {
return '<link rel="stylesheet" href="/styles/' + path + '.css" />';
},
js: function(path) {
var name = '<script src="/scripts/';
if (path == 'all') {
name += generatedJsFileName;
}
else {
name += path + '.js';
}
name += '"></script>';
return name;
}
});
It might not be the prettiest workaround but it works.