I'm trying to write a script that allows me to search for a string in a number of folders and then return the output. I've managed to find the readdirSync function which does what I want, however I am unable to print out the total number of strings found because it is ran before the function is complete. Please see example below.
var totalNumberOfStringFound = 0;
function numberofStringsInFolder(dir, search) {
var fs = require('fs');
var results = [];
var searchTerm = search;
fs.readdirSync(dir).forEach(function (file) {
file = dir + '/' + file;
var stat = fs.statSync(file);
if (stat && stat.isDirectory()) {
results = results.concat(numberOfTagsInFolder(file));
} else {
fs.readFile(file, bar);
function bar(err, data) {
err ? Function('error', 'throw error')(err) : (fileContent = data.toString('utf8'));
var count = (fileContent.match(new RegExp(searchTerm, 'gi')) || []).length;
totalNumberOfStringFound = totalNumberOfStringFound + count;
console.log('NUMBER OF STRINGS FOUND: ' + totalNumberOfStringFound);
}
// Holds the list of files found
results.push(file);
}
});
console.log("totalNumberOfStringFound: " + totalNumberOfStringFound);
return results;
}
numberofStringsInFolder('./folder', 'HELLO');
Output
totalNumberOfStringFound: 0
NUMBER OF STRINGS FOUND: 2
NUMBER OF STRINGS FOUND: 3
The output (totalNumberOfStringFound) should be 3, but because totalNumberOfStringFound is called before the function has finished, it is showing as 0. I've looked online and some people use the timeout function, but I don't want to use that because I don't know how long it will take to complete. I would really appreciate it if someone can help. Thank you
Related
So, I am making this discord bot and I wanted to make it so if someone's message gets deleted, It can be stored onto a .txt file, And if someone wanted, They could just type in a command and the bot will show the list of deleted messages.
So, when the command is fired, it will need a number parameter and that parameter will be the number of messages that will be shown.
Let's call that number numberOfSnipes
The code will get the .txt file and turn it into an array by splitting each line and putting it in one array.
As so,
fs.readFile('deletedMsgs/' + message.guildId + '.txt', function (err, data) {
messagesArray = data.toString().split('\n');
});
As the code says, It will be called messagesArray
Now the problem comes here, We will need to iterate through the array and get the number of messages needed, This will be put into another array which will be called messagesToShow
So, I tried coding that and failed,
Code:
messagesToShow = []
for (let i = messagesArray.length; i < messagesArray.length - numberOfSnipes; i--) {
console.log(i)
messagesToShow.push(messagesArray[i])
}
FYI, console.log(i) did not log anything.
I tried to log messagesArray, It logged an empty array
Keep in mind that numberOfSnipes and messagesArray were both logged and they returned the right information.
Since, This can be more easier for you all. To solve, I will provide the whole code:
if (message.content.toLowerCase().startsWith(config.prefix + 'snipelist')) {
numberOfSnipes = parseInt(message.content.split(' ')[1]);
if (fs.existsSync('deletedMsgs/' + message.guildId + '.txt')) {
fs.readFile(
'deletedMsgs/' + message.guildId + '.txt',
function (err, data) {
messagesArray = data.toString().split('\n');
console.log(messagesArray);
if (numberOfSnipes > messagesArray.length) {
message.reply('Unable to return messages.');
} else {
messagesToShow = [];
for (
let i = messagesArray.length;
i < messagesArray.length - numberOfSnipes;
i--
) {
console.log(i);
messagesToShow.push(messagesArray[i]);
}
console.log(messagesToShow);
finalMessage = ' ';
for (let letter in messagesToShow.toString()) {
if (letter != ',') {
finalMessage += letter;
} else if (letter == ',') {
finalMessage += '\n';
}
}
message.reply(finalMessage);
}
}
);
}
}
Any help is appreciated.
I've been scouring similar problems but haven't seem to have found a solution that quite works on my end. So I'm working on a Discord bot that takes data from a MongoDB database and displays said data in the form of a discord embedded message using Mongoose. For the most part, everything is working fine, however one little section of my code is giving me trouble.
So I need to import an array of both all available users and the "time" data of each of those users. Here is the block of code I use to import said data:
for (i = 0;i < totalObj; i++){
timeArray[i] = await getData('time', i);
userArray[i] = await getData('user', i);
}
Now this for loop references a function I made called getData which obtains the data from MongoDB by this method:
async function getData(field, value){
var data;
await stats.find({}, function(err, result){
if(err){
result.send(err);
}else{
data = result[value];
}
});
if(field == "user"){
return data.user;
}else if (field == "time"){
return data.time;
}else{
return 0;
}
So that for loop is where my errors currently lie. When I try to run this code and display my data through a discord message, I get this error and the message does not get sent:
(node:13936) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'time' of undefined
Now the strange thing is, this error does not happen every time. If I continue calling the command that triggers this code from my discord server, it's almost like a 50/50 shot if the command actually shows the message or instead gives this error. It is very inconsistent.
This error is confounding me, as the undefined part does not make sense to me. The objects that are being searched for in the mongoDB collection are definitely defined, and the for loop never exceeds the number of objects present. My only conclusion is that I'm doing something wrong with my asynchronous function design. I have tried altering code to use the getData function less often, or to not use awaits or asynchronous design at all, however this leaves my final discord message with several undefined variables and an eventual crash.
If anyone has any advice or suggestions, that would be very much appreciated. Just for reference, here is the full function that receives the data, sorts it, and prepares a string to be displayed on the discord server (though the error only seems to occur in the first for loop):
async function buildString(){
var string = "";
var totalObj;
var timeArray = [];
var userArray = [];
var stopSort = false;
await stats.find({}, function(err, result){
if(err){
result.send(err);
}else{
totalObj = result.length;
}
});
for (i = 0;i < totalObj; i++){
timeArray[i] = await getData('time', i);
userArray[i] = await getData('user', i);
}
while(!stopSort){
var keepSorting = false;
for(i = 0; i < totalObj ; i++){
var target = await convertTime(timeArray[i]);
for(j = i + 1 ; j < totalObj ; j++){
var comparison = await convertTime(timeArray[j]);
if(target > comparison){
//Switch target time with comparison time so that the lower time is up front
var temp = timeArray[i];
timeArray[i] = timeArray[j];
timeArray[j] = temp;
//Then switch the users around so that the user always corresponds with their time
var userTemp = userArray[i];
userArray[i] = userArray[j];
userArray[j] = userTemp;
//The loop will continue if even a single switch is made
keepSorting = true;
}
}
}
if(!keepSorting){
stopSort = true;
}
}
//String building starts here
var placeArray = [':first_place: **1st', ':second_place: **2nd', ':third_place: **3rd', '**4th', '**5th', '**6th', '**7th', '**8th', '**9th', '**10th'];
for(i = 0; i < totalObj; i++){
string = await string.concat(placeArray[i] + ": " + userArray[i] + "** - " + timeArray[i] + " \n\n");
console.log('butt');
}
console.log("This String:" + string);
return string;
}
I think problem is you are trying to await function with callback, it will not work => access to data.time may run before data = result[value]. If you need await callback, you can use custom Promise (or use util.promisify, more info here)
Promise:
function findStats(options) {
return new Promise((resolve, reject) => {
return stats.find(options, function (err, result) {
if (err) {
return reject(err)
}
return resolve(result)
})
})
}
utils.promisify
const util = require('util');
const findStats = util.promisify(stats.find);
Now you can use await in your function
async function getData(field, value) {
try {
const result = await findStats({})
const data = result.value
if (field === 'user') {
return data.user
}
if (field === 'time') {
return data.time
}
return 0
} catch (error) {
// here process error the way you like
// or remove try-catch block and sanitize error in your wrap function
}
}
I have a file with 65,000,000 lines, that is about 2gb in size.
I want to read this file in N lines at a time, perform a db insert operation, and then read the next N, with N being, say, 1000 in this case. Insert order doesn't matter, so synchronous is fine.
What's the best way of doing this? I've only found was to either load in 1 line at a time, or methods that read the whole file into memory. Sample code below, that I've been using to read the file one line at a time. :
var singleFileParser = (file, insertIntoDB) => {
var lr = new LineByLineReader(file);
lr.on('error', function(err) {
// 'err' contains error object
console.error(err);
console.error("Error reading file!");
});
lr.on('line', function(line) {
insertIntoDB(line);
// 'line' contains the current line without the trailing newline character.
});
lr.on('end', function() {
// All lines are read, file is closed now.
});
};
Lines can only be parsed one at a time by someone. So, if you want 10 at once, then you just collect them one at a time until you have collected 10 and then process the 10.
I did not think Jarek's code quite worked right so here's a different version that collects 10 lines into an array and then calls dbInsert():
var tenLines = [];
lr.on('line', function(line) {
tenLines.push(line);
if (tenLines.length === 10) {
lr.pause();
dbInsert(<yourSQL>, function(error, returnVal){
if (error) {
// some sort of error handling here
}
tenLines = [];
lr.resume();
});
}
});
// process last set of lines in the tenLines buffer (if any)
lr.on('end', function() {
if (tenLines.length !== 0) {
// process last set of lines
dbInsert(...);
}
});
Jarek's version seems to call dbInsert() on every line event rather than only every 10th line event and did not process any left over lines at the end of the file if they weren't a perfect multiple of 10 lines long.
Something like this should do
var cnt = 0;
var tenLines = [];
lr.on('line', function(line) {
tenLines.push(line);
if (++cnt >= 10) {
lr.pause();
// prepare your SQL statements from tenLines
dbInsert(<yourSQL>, function(error, returnVal){
cnt = 0;
tenLines = [];
lr.resume();
});
}
});
This is my solution inside an async function:
let multipleLines = [];
const filepath = '<file>';
const numberLines = 50;
const lineReader = require('readline').createInterface({
input: require('fs').createReadStream(filepath)
});
// process lines by numberLines
for await (const line of lineReader) {
multipleLines.push(line);
if (multipleLines.length === numberLines) {
await dbInsert();
multipleLines = [];
}
}
// process last set of lines (if any)
if (multipleLines.length !== 0) {
await dbInsert();
}
This is my solution using built modules rather then NPM's line-by-line package
var lineReader = require('readline').createInterface({
input: require('fs').createReadStream(fileToRead)
});
lines = []
lineReader.on('line', function (line) {
lines.push(line.split(","));
if (lines.length == 10) {
makeCall()
lines = []
}
});
lineReader.on('close', function () {
if (lines.length !== 0) {
makeCall()
lines = []
}
});
function makeCall() {
//Do something with lines
}
I'm trying to make it possible to choose different SSID's to switch the Wlan you are connected to from Browser.
var sys = require('sys');
var exec = require('child_process').exec;
app.get(prefix + '/wlan', function(req, res){
child = exec("iwlist wlan0 scan | grep ESSID", function(error, stdout, stderr){
if(error !== null){
console.log('Exec error ' + error);
}
else {
res.send(stdout);
}
});
});
This is my code so far to get a SSID list..
The Output is like that:
ESSID:"WLAN-GUEST" ESSID:"WLAN1" ESSID:"WLAN-GUEST" ESSID:"WLAN1" ESSID:"WLAN2"
I have no idea why two ESSID's are listed twice but my main question is, how can I parse this to JSON or how can I access each entry like an array (wlanlist[0])?
Edit:
I tried to stdout.replace(" ",", "); and JSON.parse but as it's async it's sent without changes. (Not sure if that would work as sync)
Edit2: Trying to access the data like that:
$(document).ready(function() {
$.get(prefix + '/wlan', function(wlanlist){
document.getElementById("wlanoptions").options[0] = new Option("Select your WLAN:","");
document.getElementById("wlanoptions").options[1] = new Option(wlanlist[0],wlanlist[0])
});
});
Final Result:
var wlanlistarray = stdout.split("ESSID:");
res.send(wlanlistarray);
In addition:
//extract ssid and remove quotes
var wlanlist = new Array;
var step1 = stdout.split("ESSID:");
for(i = 1; i < step1.length; i++){
var arr = new Array;
arr = step1[i].split('"');
//if exists in array -> continue; else create new entry in wlanlist
if(wlanlist.indexOf(arr[1]) === -1){wlanlist.push(arr[1]);}
else{continue;}
}
res.send(wlanlist);
This should return an array of SSIDs:
stdout.split("ESSID:")
Now clean up the " and you are all done.
I have a config.js file which I believe is JSON which is called when the application first starts:
var config={};
config.user = [
{id:'JSMITH', priceModify:'true'},
{id:'JBLOGGS', priceModify:'false'},
]
config.price = [
{id:"price01", name:"priceName01", primary:"57.25", secondary:"34.54"},
{id:"price02", name:"priceName02", primary:"98.26", secondary:"139.45"},
{id:"price03", name:"priceName03", primary:"13.87", secondary:"29.13"}
]
To pull / push data I just use the following:
// Read
var curPrice = config.price[0].primary;
// Write
config.price[0].primary = "98.24";
How do I go about exporting the config file with the new value so that it will load next time the application is opened? I can use the file system object to write the file, I just don't understand how I would export everything (and preferably keep the same format).
I originally thought about reading the whole config file into a variable, cycling through to find the required block, id, and key and replacing the value, then writing the whole thing back, but I can't seem to figure out how to replace that specific value only.
Any help would be greatly appreciated
Edit Apologies, I forgot to mention that this application is completely offline and uses local directories
Solution
I stumbled across a few solutions to different issues which, when combined, gave me the perfect solution. First we cycle the Javascript object, building an array of the detail and then converting the array to a string:
vMethod.convertToText = function(obj) {
var string = [];
var output = '';
var count= 0;
var countTotal = 0;
if (typeof(obj) == "object" && (obj.join == undefined)) {
count= 0;
countTotal = 0;
string.push("{");
for (prop in obj) {
countTotal++;
}
for (prop in obj) {
if(count==countTotal - 1) {
string.push(prop, ": ", vMethod.convertToText(obj[prop]),'}\r\n');
} else {
string.push(prop, ": ", vMethod.convertToText(obj[prop]), ",");
}
count++;
};
} else if (typeof(obj) == "object" && !(obj.join == undefined)) {
count= 0;
countTotal = 0;
string.push("[\r\n")
for (prop in obj) {
countTotal++;
}
for(prop in obj) {
if(count==countTotal - 1) {
string.push(vMethod.convertToText(obj[prop]),'];\r\n');
} else {
string.push(vMethod.convertToText(obj[prop]), ",");
}
count++;
}
} else if (typeof(obj) == "function") {
string.push(obj.toString())
} else {
string.push(JSON.stringify(obj))
}
output = string.join("").toString();
//output = output.slice(1, -1);
return output;
}
Then we clean the array (neccessary for me to remove excess characters)
vMethod.cleanConfigText = function() {
var outputText = vMethod.convertToText(config);
outputText = outputText.slice(1, -1);
outputText = 'var config = {};\r\n'+outputText;
outputText = outputText.replace('user:','config.user =');
outputText = outputText.replace(',price:','config.price =');
outputText = outputText.slice(0, -2);
outputText = outputText.replace(/"/g, "'")
return outputText;
}
Finally a function to export the object into my config.js file:
vMethod.writeToConfig = function() {
vObject.fileSystem = new ActiveXObject('Scripting.FileSystemObject');
vObject.fileSystemFile = vObject.fileSystem.CreateTextFile('source\\js\\config.js',true);
vObject.fileSystemFile.Write(vMethod.cleanConfigText());
vObject.fileSystemFile.Close();
delete vObject.fileSystemFile;
delete vObject.fileSystem;
}
So when I want to export a change in the config, I just call:
vMethod.writeToConfig();
The only difference in the file format is that the commas appear at the start of a trailing line rather than the end of a preceding line but I can live with that!
Edit Turns out I'm anally retentive and the commas were bugging me
Added these to the clean up function and now the config is identical to before but without the indent
outputText = outputText.replace(/[\n\r]/g, '_');
outputText = outputText.replace(/__,/g, ',\r\n');
outputText = outputText.replace(/__/g, '\r\n');
Thank you to those that looked at the question and tried to help, very much appreciated.
Edit
DO NOT READ THE SOLUTION ABOVE, IT IS IN THE WRONG PLACE AND THERFORE IS NOT A VALID ANSWER. YOU'VE BEEN WARNED.
You can use a very popular npm package: https://www.npmjs.com/package/jsonfile . There are many but I've choosen this one.
Usually config stuff should be in json or .env files.
Now, all you have to do is use jsonfile's API to read/write JSON and parse (the package does the serialization/deserialization) it at the beginning when the application starts.
Example:
var jsonfile = require('jsonfile');
var util = require('util');
var config = null;
var file = './config.json';
// Reading
jsonfile.readFile(file, function(err, obj) {
config = obj;
});
// Writing
// Edit your config blah blah
config.user = [
{id:'JSMITH', priceModify:'true'},
{id:'JBLOGGS', priceModify:'false'},
];
config.price = [
{id:"price01", name:"priceName01", primary:"57.25", secondary:"34.54"},
{id:"price02", name:"priceName02", primary:"98.26", secondary:"139.45"},
{id:"price03", name:"priceName03", primary:"13.87", secondary:"29.13"}
];
jsonfile.writeFile(file, config, function (err) {
if(err) return err;
console.log('Config saved to file!');
});