I am attempting to do a file search based off array of file names and root directory. I found some file search snips online that seem to work when I am finding just 1 single file, but it will not work when there is more than 1 file specified. Below is the snippet:
const fs = require('fs');
const path = require('path');
var dir = '<SOME ROOT DIRECTORY>';
var file = 'Hello_World.txt'
var fileArr = ['Hello_World.txt', 'Hello_World2.txt', 'Hello_World3.txt'];
const findFile = function (dir, pattern) {
var results = [];
fs.readdirSync(dir).forEach(function (dirInner) {
dirInner = path.resolve(dir, dirInner);
var stat = fs.statSync(dirInner);
if (stat.isDirectory()) {
results = results.concat(findFile(dirInner, pattern));
}
if (stat.isFile() && dirInner.endsWith(pattern)) {
results.push(dirInner);
}
});
return results;
};
console.log(findFile(dir, file));
Any thoughts on how I can get this working with an array rather than just a single file string?
Doing the following seems to be working, but didn't know if there were other ways to do this which may be simpler:
newFileArr = [];
fileArr.forEach((file => {
//findFile(dir, file).toString();
console.log(findFile(dir, file).toString());
}));
The only thing that needs to change is the condition to determine if an individual filepath meets the search criteria. In the code you've posted, it looks like dirInner.endsWith(pattern), which says "does the filepath end with the given pattern?"
We want to change that to say "does the filepath end with any of the given patterns?" And closer to how the code will look, we can rephrase that as "Can we find a given pattern such that the filepath ends with that pattern?"
Let's rename pattern to patterns. Then we can simple replace dirInner.endsWith(patterns) with patterns.some(pattern => dirInner.endsWith(pattern))
Related
How can I loop through the folder that has subfolder and retrieve all files with extension '.element.ts' ?
const fs = require('fs');
const filesDir = fs.readdirSync('packages/web-components/src');
// the json result that will be generated
let content = [];
files.forEach(file => {
if (fileName === '???')
content.push(file);
});
const fs=require('fs');
function getAllFiles (dir, allFilesList = []){
const files = fs.readdirSync(dir);
files.map(file => {
const name = dir + '/' + file;
if (fs.statSync(name).isDirectory()) { // check if subdirectory is present
getAllFiles(name, allFilesList); // do recursive execution for subdirectory
} else {
allFilesList.push(name); // push filename into the array
}
})
return allFilesList;
}
const allFiles = getAllFiles('./testfolder');
const fileEndsWith = allFiles.filter(file => file.endsWith('.element.ts'))
console.log(fileEndsWith);
Easy way: use glob package.
const glob = require("glob");
const pattern = "packages/web-components/src/*.element.ts"
const elementTsFilenames = glob.sync(pattern);
Manual way:
const dir = "packages/web-components/src";
const extension = ".element.ts";
const elementTsFilenames = fs.readdirSync(dir).filter(fn => fn.endsWith(extension));
This is easy enough that the manual way is as easy or easier; if you have more complex requirements (e.g. recursively searching subdirectories), a library approach is nice.
I'll show you how to recursively get all the files in a directory (even those located in a subdirectory).
To do this, we need to create a recursive function that can call itself when dealing with sub-directories. And we also need the function to go through each of the sub-directories and add any new files it encounters. Then we also need to check if the filename contains a specific string. When the function is finished , it should return an array with all the files it encountered.
Here's what the recursive function looks like:
const fs = require("fs")
const path = require("path")
const getAllFiles = function(dirPath, extension, arrayOfFiles) {
files = fs.readdirSync(dirPath);
arrayOfFiles = arrayOfFiles || [];
files.forEach(function(file) {
if (fs.statSync(dirPath + "/" + file).isDirectory()) {
arrayOfFiles = getAllFiles(dirPath + "/" + file, arrayOfFiles);
} else if (file.endsWith(extension)){
arrayOfFiles.push(path.join(__dirname, dirPath, "/", file));
}
});
return arrayOfFiles;
}
First, we require() the Node.js path module. Since this is
included with Node.js, you don't need to install anything for it to
work. This module will help us easily create full file paths for our
files.
The getAllFiles variable holds the recursive function that will go
through each subdirectory and return an array of filenames. It takes a
directory file path, a specified character and an optional arrayOfFiles as arguments.
Inside the getAllFiles function, we first use the readdirSync()
function to get all of the files and directories inside the given
dirPath supplied to the function.
Then, we create an arrayOfFiles that will hold all the filenames
that will be returned when the function is done running.
Next, we loop over each item (file or directory) found by the
readdirSync() function. If the item is a directory, we have the
function recursively call itself to get all of the files and
sub-directories inside the given directory.
And if the item is a file, we simply append the file path to the
arrayOfFiles array. (When the end of the file name is confirmed to
contain characters)
When the forEach loop has finished, we return the arrayOfFiles
array.
Here is how you use the function in your code:
const result = getAllFiles("packages/web-components/src", ".element.ts");
I don't think you need an npm package for this: It's not too hard to walk the file system using an async iterator and filter the results based on something like a regular expression.
Another bonus of an async technique is that it doesn't block your thread while it iterates the files (other work can be done in between each result while it's searching), especially if you have a lot of sub-directories/files to search through.
If you want to reduce your project's dependencies, you can do something like this:
example.mjs:
import * as path from 'node:path';
import {readdir} from 'node:fs/promises';
/** Search all subdirectories, yielding matching file entries */
export async function* findFiles (dir, regexpFilter) {
for (const entry of await readdir(dir, {withFileTypes: true})) {
const fPath = path.resolve(dir, entry.name);
if (entry.isDirectory()) {
yield* findFiles(fPath, regexpFilter);
continue;
}
if (regexpFilter && !regexpFilter.test(entry.name)) continue;
yield Object.assign(entry, {path: fPath});
}
}
async function main () {
const dir = 'packages/web-components/src';
// Regular expression which means: ends with '.element.ts'
const filter = /\.element\.ts$/;
for await (const entry of findFiles(dir, filter)) {
// ^^^^^^
// If you don't include a filter argument, then all files will be iterated
console.log(entry.name); // just the file name
console.log(entry.path); // the full file path
}
}
main();
I'm in another pickle I've realized over the past week that my images are not loading due to the fact the links have expired so I wanna find out how to go about using a file directory in the code.
Here's what I've tried:
});
client.on('message', message => {
if (message.content.startsWith('L!hug')) {
var fs = require('fs');
var files = fs.readdirSync('C:\Users\nevbw\Desktop\games\FBIBot\images\hugs')
/* now files is an Array of the name of the files in the folder and you can pick a random name inside of that array */
let chosenFile = files[Math.floor(Math.random() * files.length)]
}
});
and
});
client.on('message', message => {
if (message.content.startsWith('L!hug')) {
const path = 'C:\Users\nevbw\Desktop\games\FBIBot\images\hugs';
const fs = require('fs');
fs.readdirSync(path).forEach(file => {
ranfile = Math.floor(Math.random() * file.length);
message.channel.sendFile(ranfile);
})
}
});
found out through searching and searching but found a answer the modified it to this, i hope people use this in future reference!
const num = (Math.floor(Math.random()* 5)+1).toString(); message.channel.send({files: [`./slap/slap${num}.gif`]})
Using fs.readdirSync('./images/') instead of fs.readFileSync('./images/') works easier, but then you will have to create the folder inside of VSC and put the images in the folder, you can also drag and drop the images into the solution and use:
var files = fs.readdirSync(`./images/`).filter(file => file.endsWith('.png'))
so that when it looks for an image, it doesn't select anything else. hope it helps for some people.
Happy to help.
You're using FS the wrong way. This Is What It Should Look Like :D Also Here Is Some Documentation on It ( https://nodejs.org/dist/latest-v13.x/docs/api/fs.html ).
-- Code --
Also Just As A Tip! I See You Are Using Full Directories, That's Quite Innificeng (E.g if You Change Your Username, Drive ID, etc.) so in fs provided the image is in the same folder you can just do ./(ImageName), or if it is in the same folder but under another say /FBIBot/Images you can do ./Images/(ImageName). ^^
--
What The Error Was: (I Unfortunately Cannot Test it But I Am Like 99% Sure).
You Were Using fs.readdirSync(path).forEach(file => { When You Were Meant To Be Using fs.readfilesync(path).forEach(file => {.
-- First Code --
});
client.on('message', message => {
if (message.content.startsWith('L!hug')) {
var fs = require('fs');
var files = fs.readfileSync('C:\Users\nevbw\Desktop\games\FBIBot\images\hugs')
/* now files is an Array of the name of the files in the folder and you can pick a random name inside of that array */
let chosenFile = files[Math.floor(Math.random() * files.length)]
}
});
-- Second Code --
});
client.on('message', message => {
if (message.content.startsWith('L!hug')) {
var fs = require('fs');
var files = fs.readFileSync('C:\Users\nevbw\Desktop\games\FBIBot\images\hugs')
/* now files is an Array of the name of the files in the folder and you can pick a random name inside of that array */
let chosenFile = files[Math.floor(Math.random() * files.length)]
}
});
^^
I am running a script which looks into a directory and lists files, then checks for the file type to process, if the extension matches then the file is read and each line of the file (.col which is just a txt file renamed) is inserted into an array.
Now after the file is read and the array is populated I would like to use the array and do some further processing, e.g create a db record. I am missing something really basic here because on each console log I do as below I always get the full items (in my array) of the contents of all files.
So to make it a bit simpler:
array is empty.
Then file is read and processed and array now has
array[0]=line 0 of file
array[0]=line 1 of file etc
const fs = require('fs');
const readline =require('readline');
var files = fs.readdirSync('/home/proj/data');
var path = require('path');
var model=[];
var lineReader=[];
for(var i=0; i<files.length; i++) {
if(path.extname(files[i]) === ".col") {
lineReader[i] = readline.createInterface({
input: require('fs').createReadStream(files[i])
});
lineReader[i].on('line', function (line) {
model.push(line);
}).on('close', async function() {
console.log(model);
});
}
}
Instead the script is run and array[] holds all lines of all files that match the extension.
Your help is greatly appreciated and anyone is allowed to scorch my JS as I am pretty sure I am missing something basic here.
So, you want to read the files in parallel (because that's what your program does) and put it in an array of arrays?
You can make the reading file mechanism a promise and use it using Promise.all. Here is an example to get you started.
const fs = require('fs');
const readline = require('readline');
var files = fs.readdirSync('./');
var path = require('path');
function readFile(fileName) {
return new Promise(resolve => {
const array = [];
const lineReader = readline.createInterface({
input: fs.createReadStream(files[i])
});
lineReader.on('line', function (line) {
array.push(line);
}).on('close', async function () {
//do some proc
console.log(array);
resolve(array);
});
});
}
const readFilePromises = [];
for (var i = 0; i < files.length; i++) {
if (path.extname(files[i]) === ".js") {
readFilePromises.push(readFile(files[i]));
}
}
Promise.all(readFilePromises) //or await Promise.all([..])
.then(data => {
console.log(data);//will be array of arrays
})
If you want a single Array you can always flatten the result using data.flat()
If your files are not very big and sync methods are OK, you can simplify the code this way:
'use strict';
const fs = require('fs');
const path = require('path');
const model = [];
fs.readdirSync('/home/proj/data')
.filter(name => path.extname(name) === '.col')
.forEach((name) => {
model.push(...fs.readFileSync(name, 'utf8').split('\n'));
});
console.log(model);
I'm trying to get the current clicked folder from an array of my app's root folder and read it with readdirSync and I can't find anything helpful.
So far, what I got is only the index number on click, but I can't read it because it is not a string.
Here is what I have now :
const fs = require('fs');
$('li').click(function() {
file = $('li').index(this);
fs.readdirSync(file);
console.log(file);
})
My app
So, to read my root folder I have : let files = fs.readdirSync('.') and a forEach function. I want to click on "js" folder for example, and display what is inside it but I don't know how.
I'm new to electron and node.js :)
Thanks!
This bit of code will be used in the code that follows after:
const fileSystem = require("fs");
const getDirectoryContents = path => fileSystem.readdirSync(path);
Assuming your ul has an id called main, this is what I'd do:
$("#main li").on("click", function clickHandler() {
const $clickedElement = $(this);
const elementName = $clickedElement.text(); // gets the text that was clicked
const fsEntry = fileSystem.statSync(elementName);
if (fsEntry.isDirectory()) { // retrieve the contents only if the clicked text represents a folder
const files = ['<ul>'].concat(
getDirectoryContents(elementName).map(entryName => `<li>${entryName}</li>`),
'</ul>'
).join('');
$clickedElement.append(files); // not exactly the best way to do it, but does the job
} else {
// do something if the entry is a file
}
});
Hope this helps.
i'm trying to make an app that searches for all files
contains a specified string under the current directory/subdirectory.
as i understand it means i need to create a read stream, loop it, load the read data to an array, if the word found give __filename, dirname and if ! not found message.
unfortunately, i could not make it work...
any clue?
var path = require('path'),
fs=require('fs');
function fromDir(startPath,filter,ext){
if (!fs.existsSync(startPath)){
console.log("no dir ",startPath);
return;
};
var files=fs.readdirSync(startPath);
let found = files.find((file) => {
let thisFilename = path.join(startPath, file);
let stat = fs.lstatSync(thisFilename);
var readStream = fs.createReadStream(fs);
var readline = require('readline');
if (stat.isDirectory()) {
fromDir(thisFilename, filename,readline, ext);
} else {
if (path.extname(createReadStream) === ext && path.basename(thisFilename, ext) === filename) {
return true;
}
}
});
console.log('-- your word has found on : ',filename,__dirname);
}
if (!found) {
console.log("Sorry, we didn't find your term");
}
}
fromDir('./', process.argv[3], process.argv[2]);
Because not everything was included in the question, I made an assumption:
We are looking for full words (if that's not the case, replace the regex with a simple indexOf()).
Now, I've split the code into two functions - to make it both more readable and easier to recursively find the files.
Synchronous version:
const path = require('path');
const fs = require('fs');
function searchFilesInDirectory(dir, filter, ext) {
if (!fs.existsSync(dir)) {
console.log(`Specified directory: ${dir} does not exist`);
return;
}
const files = getFilesInDirectory(dir, ext);
files.forEach(file => {
const fileContent = fs.readFileSync(file);
// We want full words, so we use full word boundary in regex.
const regex = new RegExp('\\b' + filter + '\\b');
if (regex.test(fileContent)) {
console.log(`Your word was found in file: ${file}`);
}
});
}
// Using recursion, we find every file with the desired extention, even if its deeply nested in subfolders.
function getFilesInDirectory(dir, ext) {
if (!fs.existsSync(dir)) {
console.log(`Specified directory: ${dir} does not exist`);
return;
}
let files = [];
fs.readdirSync(dir).forEach(file => {
const filePath = path.join(dir, file);
const stat = fs.lstatSync(filePath);
// If we hit a directory, apply our function to that dir. If we hit a file, add it to the array of files.
if (stat.isDirectory()) {
const nestedFiles = getFilesInDirectory(filePath, ext);
files = files.concat(nestedFiles);
} else {
if (path.extname(file) === ext) {
files.push(filePath);
}
}
});
return files;
}
Async version - because async is cool:
const path = require('path');
const fs = require('fs');
const util = require('util');
const fsReaddir = util.promisify(fs.readdir);
const fsReadFile = util.promisify(fs.readFile);
const fsLstat = util.promisify(fs.lstat);
async function searchFilesInDirectoryAsync(dir, filter, ext) {
const found = await getFilesInDirectoryAsync(dir, ext);
for (file of found) {
const fileContent = await fsReadFile(file);
// We want full words, so we use full word boundary in regex.
const regex = new RegExp('\\b' + filter + '\\b');
if (regex.test(fileContent)) {
console.log(`Your word was found in file: ${file}`);
}
};
}
// Using recursion, we find every file with the desired extention, even if its deeply nested in subfolders.
async function getFilesInDirectoryAsync(dir, ext) {
let files = [];
const filesFromDirectory = await fsReaddir(dir).catch(err => {
throw new Error(err.message);
});
for (let file of filesFromDirectory) {
const filePath = path.join(dir, file);
const stat = await fsLstat(filePath);
// If we hit a directory, apply our function to that dir. If we hit a file, add it to the array of files.
if (stat.isDirectory()) {
const nestedFiles = await getFilesInDirectoryAsync(filePath, ext);
files = files.concat(nestedFiles);
} else {
if (path.extname(file) === ext) {
files.push(filePath);
}
}
};
return files;
}
If you have not worked with / understand async/await yet, it is a great step to take and learn it as soon as possible. Trust me, you will love not seeing those ugly callbacks again!
UPDATE:
As you pointed in comments, you want it to execute the function after running node process on the file. You also want to pass the function parameters as node's arguments.
To do that, at the end of your file, you need to add:
searchFilesInDirectory(process.argv[2], process.argv[3], process.argv[4]);
This extracts our arguments and passes them to the function.
With that, you can call our process like so (example arguments):
node yourscriptname.js ./ james .txt
Personally, if I were to write this, I would leverage the beauty of asynchronous code, and Node.js's async / await.
As a very side note:
You can easily improve readability of your code, if you add proper formatting to it. Don't get me wrong, it's not terrible - but it can be improved:
Use spaces OR newlines after commas.
Use spaces around equality operators and arithmetic operators.
As long as you are consistent with formatting, everything looks much better.