We Keep Coding

how can I loop through files in a folder and subfolders using electron?

I have a desktop app that I am making that requires me to loop through a folder and all of it's subfolders to find files with a certain character in their name. is this possible using electron? I can't figure out how to do it

this is what am currenlty using in an electron app am building
const iterateDir = () => {
const files = [],
dirs = [];
return function dirIt(directory) {
try {
let dirContent = fs.readdirSync(directory);
dirContent.forEach( path => {
const fullPath = join(directory,path);
if ( fs.statSync(fullPath).isFile() )
files.push(fullPath);
else
dirs.push(fullPath);
});
if ( dirs.length !== 0 )
dirIt(dirs.pop());
return files;
} catch(ex) {
console.log(ex);
return false;
}
};
};

I was able to go through folders and subfolders by using the npm package, glob.
in this demonstration, only files are sent to the console, but if you were to send the variable 'res' to the console, you would get an array with all of the folders and files in the selected folder.
as an aside, normally chrome can't view folder paths, that's why I used the function '.path'. it is created by electron to serve that purpose. in the html file, I use
<input id="directoryVal" type="file"
webkitdirectory />
so that I may browse for a folder.
var getDirectories = function (src, callback) {
glob(src + '/**/*', callback);
};
listdir.addEventListener('click', function (event) {
var directoryVal = document.getElementById('directoryVal').files[0].path
getDirectories(directoryVal, function (err, res) {
if (err) {
console.log('Error', err);
} else {
for (i=0; i< res.length; i++){
if (fs.lstatSync(res[i]).isDirectory() == false){
console.log(res[i]);
}
}
}
});
})

Accessing typescript file variable values using gulp

I have several typescript files, some of them export a const named APIS.
I'm trying to access those exports (I want to concatenated all of them to a single file), but it doesn't seem to work. I'm obviously doing something wrong, but I'm not sure what.
For example, I have a folder named services, with 2 files: service1.ts, service2.ts.
service1.ts:
...
export const APIS = [ { "field1" : "blabla" } ];
service2.ts: does not contain the APIS var.
This is my gulpfile.js:
var gulp = require('gulp');
var concat = require('gulp-concat');
var map = require('gulp-map');
gulp.task('default', function() {
return gulp.src('.../services/*.ts')
.pipe(map(function(file) {
return file.APIS;
}))
.pipe(concat('all.js'))
.pipe(gulp.dest('./test/'));
});
When I run this task, I get nothing. When I added console.log(file.APIS); to the map function, I get undefined for all the values (although it is defined in service1.ts!).
This is following to: Extracting typescript exports to json file using gulp
EDIT: OK, so I tried saving the exports in a .js file instead of a .ts file, and now I can access those vars using require:
gulp.task('default', function() {
return gulp.src('./**/*.service.export.js')
.pipe(map(function(file) {
var fileObj = require(file.path);
...
}))
Now if I try console.log(fileObj.APIS); I get the correct values. What I'm still confused about is how I can pass these value on, and create a single file out of all these vars. Is it possible to push them into an array?

This will not work as you think it would work. Gulp itself knows nothing about typescript files, that file is a vinyl-file and has no knowledge about the typescript code within its content.
Edit
Based on your example, you can do something like this:
var gulp = require('gulp');
var concat = require('gulp-concat');
var map = require('gulp-map');
var fs = require('fs');
gulp.task('test', function ()
{
var allConstants = [];
var stream = gulp.src('./**/*.service.export.js')
.pipe(map(function(file)
{
var obj = require(file.path);
if (obj.APIS != null)
allConstants = allConstants.concat(obj.APIS);
return file;
}));
stream.on("end", function (cb)
{
// Do your own formatting here
var content = allConstants.map(function (constants)
{
return Object.keys(constants).reduce(function (aggregatedString, key)
{
return aggregatedString + key + " : " + constants[key];
}, "");
}).join(", ");
fs.writeFile('filename.txt', content, cb);
});
return stream;
});

Suggestion
If you want to collect multiple variables into a single file i.e. a common variables file I suggest gulp-replace.
Steps
Create a file, require it and use tags within that file to place your variables.
Advice
If you are already using services don't create an array. Instead create an object (JSON) where every property is a constant. i.e.
var constants = {
const_1: 0,
const_2: 1,
const_3: 2,
}

gulp-concat : from unknown source to the same destination

I recently started to use gulp to keep my dev project organized and I've run into a little something that I cant figure out. So this is my task :
gulp.task('jsassemble', function () {
return gulp
.src('vendor/proj/**/**/src/assets/js/*.js')
.pipe(concat('all.js'))
.pipe(gulp.dest('public/js'));
});
As you can see, it'll fetch every js file in vendor/proj/anyFolder/anySubFolder/src/assets/js, put them together, rename the newly created js 'all.js' and then put it in public/js. The problem is that I would like to have gulp to keep the folder hierarchy, for example :
Source = vendor/proj/anyFolder1/anySubFolder1/src/assets/js/*.js
Destination = public/js/anyFolder1/anySubFolder1/src/assets/js/all.js
Source = vendor/proj/anyFolder1/anySubFolder2/src/assets/js/*.js
Destination = public/js/anyFolder1/anySubFolder2/src/assets/js/all.js
Instead of simply having everything on those folder into a 1 and only public/js/all.js
Is there anyway to do it ? I've tried to google it first but I wasn't able to properly formulate my question in a few words and was given not-wanted results :/

You could create the function which keep your folder hierarchy. In this page (http://www.jamescrowley.co.uk/2014/02/17/using-gulp-packaging-files-by-folder/) you find the solution.
var fs = require('fs');
var path = require('path');
var es = require('event-stream');
var gulp = require('gulp');
var concat = require('gulp-concat');
var rename = require('gulp-rename');
var uglify = require('gulp-uglify');
var scriptsPath = './src/scripts/';
function getFolders(dir){
return fs.readdirSync(dir)
.filter(function(file){
return fs.statSync(path.join(dir, file)).isDirectory();
});
}
gulp.task('scripts', function() {
var folders = getFolders(scriptsPath);
var tasks = folders.map(function(folder) {
return gulp.src(path.join(scriptsPath, folder, '/*.js'))
.pipe(concat(folder + '.js'))
.pipe(gulp.dest(scriptsPath))
.pipe(uglify())
.pipe(rename(folder + '.min.js'))
.pipe(gulp.dest(scriptsPath));
});
return es.concat.apply(null, tasks);
});

Thanks to #caballerog who puts my on the right path, here's the explained code :
//get every folder from a 'pathTo/Something'
function getFolders(dir){
return fs.readdirSync(dir)
return fs.statSync(path.join(dir, file)).isDirectory();
}
var projectsRoot = 'vendor/proj/';
var pathToJsFiles = '/src/assets/js/';
var pathToPublic = 'public/js/';
gulp.task('scripts', function() {
var sites = [];
var pathToProjects = [];
// Fetching every folders in vendor/proj
projects = getFolders(projectsRoot);
// Fetching every subfolder in vendor/proj/something
for(index in projects){
sites.push(getFolders(projectsRoot + '/' + projects[index]));
}
// Pushing every projects/site that exists into an array
for(var i=0;i<projects.length;i++){
for(var j=0; j<sites.length; j++)
if(sites[i][j] != null)
pathToProjects.push(projects[i] + '/' + sites[i][j]);
}
// Fetching every JS on vendor/proj/pathToAProject/pathToJsFiles
// concatenate them together
// and sending them to pathToPublic/pathToAProject/all.js
var tasks = pathToProjects.map(function(pathToAProject) {
return gulp.src( projectsRoot + pathToAProject + pathToJsFiles + '/*.js')
.pipe(concat('all.js'))
.pipe(gulp.dest(pathToPublic + pathToAProject));
});
return es.concat.apply(null, tasks);
});
TL:DR = get every JS file in pathToPublic/someFolder/someFolder/PathToJS.

Syncchronously and recursive filesystem file finding

I just have a quick question there:
I am using Node.JS to write a commandline tool that validates JSON Files with JSON Schemas. So, now I have a problem that when wanting to get all the schemas, that I always get "undefined" for using a async function but otherwise only sync functions.
For this commandline tool async is NOT needed.
Could someone help me out and give me a hand on how to make it work just fine?
var getJSONSchemaFiles = function (dir) {
results2 = [];
var recursive = require('recursive-readdir');
recursive(dir, function (err, files) {
// Files is an array of filename
// console.log(files);
files.forEach(function (entry) {
if (entry.indexOf(".schema.json") > -1) {
results2.push(entry);
}
});
console.log(results2);
});
return results2;
};
I am using the npm "recursive-readdir" but I think that I do not even need a npm for this kind of thing?

Ok, this enumerates all files under the given path synchronously:
var fs = require('fs');
function recursiveReaddir(path) {
var stat = fs.lstatSync(path);
if(stat.isFile())
return [path];
if(!stat.isDirectory())
return [];
return [].concat.apply([], fs.readdirSync(path).map(function(fname) {
return recursiveReaddir(path + '/' + fname);
}));
}

Use glob module https://github.com/isaacs/node-glob. There is async and Sync methods like: glob.sync(pattern, [options]); and glob(pattern, [options], cb);
Example from their docs:
var glob = require("glob")
// options is optional
glob("**/*.js", options, function (er, files) {
// files is an array of filenames.
// If the `nonull` option is set, and nothing
// was found, then files is ["**/*.js"]
// er is an error object or null.
})

How to use promises to return the final result of an array?

I am currently trying to return a request of all the file names (in each existing folder) on a particular website. My web application is using NodeJS, Express, Cheerio, and Request to web scrape. My code is first getting a list of all the folder names. After retrieving a list of folder names, it then goes inside each folder name to get a list of file names and store them in the 'files' array. Finally, the 'files' array is what will be sent to the client-side.
Right now I am having a big issue with asynchronous stuff because my request would always return an empty list of 'files'. I have the Q node module installed and have tried using promises, but have had no luck getting the results I want. I am still new to nodeJS and would love it if someone can help me out.. :)
exports.getAllImages = function(req, res) {
var folders = [];
var files = [];
//Step 1: Get folder names and store all of them in the 'folders' array
var foldersUrl = 'http://students.washington.edu/jmzhwng/Images/';
request(foldersUrl, function(error, response, html){
if(!error){
var $ = cheerio.load(html);
$("a:contains('-')").filter(function(){
var data = $(this)[0].attribs.href;
folders.push(data);
})
//Step 2: Using the 'folders' array, get file names in each folder and store all of them in the 'files' array
for (var i=0; i < folders.length; i++) {
var imagesUrl = 'http://students.washington.edu/jmzhwng/Images/' + folders[i];
request(imagesUrl, function(error, response, html){
if(!error){
var $ = cheerio.load(html);
$("a:contains('.')").filter(function(){
var data = $(this)[0].attribs.href;
files.push(data);
})
}
})
}
//Step 3: Return all file names to client-side
res.json({
images: files
}, 200);
console.log('GET ALL IMAGES - ' + JSON.stringify(files));
}
})
For better readability or support, you can view the JSFiddle I created here: http://jsfiddle.net/fKGrm/

You don’t necessarily need promises for this—you’re 95% of the way there already without them. The main issue, as I think you’re aware, is that your response is being sent before the image requests come back. You just need to wait for those to finish before you send the response.
The most basic way is to count the number of callbacks you receive in your Step 2. When it equals the folders.length, then send your response.
Here’s a simplified version of that:
var request = require('request'),
cheerio = require('cheerio');
var baseUrl = 'http://students.washington.edu/jmzhwng/Images/';
var files = [];
request(baseUrl, function (error, res, body) {
var folders = folderLinks(cheerio.load(body));
count = 0;
folders.forEach(function (folder) {
request(baseUrl + folder, function (error, res, body) {
files.push.apply(files, fileLinks(cheerio.load(body)));
if (++count == folders.length) {
console.log(files);
}
});
});
});
function folderLinks ($) {
return $('a:contains(-)').get().map(function (a) {
return a.attribs.href;
});
}
function fileLinks ($) {
return $('a:contains(.)').get().map(function (a) {
return a.attribs.href;
});
}