We Keep Coding

Writing on stdin when using dockerode

How can I write to the container's stdin, when using dockerode library? I tried doing it in a multiple ways, but nothing seems to work.
My current code that is not able to write to stdin:
export async function nameToStdName(
pluginName: string,
pluginDescription: string,
pluginId: number,
numberOfDuplicates: number
) {
const docker = new Docker();
const input = `${pluginName}; ${pluginDescription}`;
// Run the docker container and pass input from a string
const dockerImageName = 'name-to-stdname';
const dockerCmd = ['python', '/app/main.py', '-i', pluginId.toString(), '-v', numberOfDuplicates.toString()];
const options = {
cmd: dockerCmd,
AttachStdin: true,
AttachStdout: true,
Tty: false,
};
const container = await docker.createContainer({
Image: dockerImageName,
...options,
});
await container.start();
const stream = await container.attach({
stream: true,
stdin: true,
stdout: true,
});
// Handle output from container's stdout
let name = "";
stream.on('data', (data: Stream) => {
console.log(`Received output: ${data.toString()}`);
name += data.toString();
});
// Pass input to container's stdin
stream.write(input);
await container.wait();
return name;
}

How to call an hardhat plugin in a hardhat task?

I'm trying to create an hardhat task calling an hardhat extension, i'm using nodejs spawn but It doesn't work :/
How can I get this working ?
./tasks/checkk.js
const fs = require('fs');
const { spawn } = require('node:child_process')
task("checkk", "check contract's code on etherscan")
.addPositionalParam("myContract")
.setAction(async (taskArgs, hre) => {
const argsFile = `../args/${taskArgs.myContract}.js`
// after this line this is pseudo code only, nothing works...
const address = fs.readFile(`./addresses/${taskArgs.myContract}_${hre.network.name}_ADDRESS.txt`, 'utf-8')
const myCommand = `npx hardhat verify --network ${hre.network.name} ${address} --constructor-args ${argsFile}`
const command = spawn('myCommand', {
stdio: 'inherit',
shell: true
})
command.stdout.on('data', (output) => {
console.log("Output: ", output.toString())
})
})

Ok, hre.run is what I needed.
const fs = require('fs');
task("checkk", "check contract's code on etherscan : checkk MyContract --network MyNetwork")
.addPositionalParam("myContract")
.setAction(async (taskArgs, hre) => {
const CONTRACT_ARGUMENTS = require(`../args/${taskArgs.myContract}.js`)
const address = fs.readFileSync(`./addresses/${taskArgs.myContract}_${hre.network.name}_ADDRESS.txt`, "utf8")
await hre.run("verify:verify", {
address: address,
constructorArguments: CONTRACT_ARGUMENTS,
});
})

How to access Winston output from within Jest test

UPDATE: I believe I have found a (probably hacky) way to solve both problems, will post my updated code and solutions tomorrow for anyone interested.
I am trying to create a CLI application and decided I wanted to format the log output for a better user experience, including adding a verbose mode. I thus installed winston and managed to get it working how I wanted to. I then installed jest as I the app is getting more complex so I wanted to automate testing. I decided to make my first test for the logger as it was the last thing I was working on and I have immediately run into problems. My plan was to count the lines of output from the console log and check they were equal to ten: this would then mean I wouldn't have to hardcode in any error specific error messages which may change if I decide to change the log formatting at a later date. I would also like a test that changes the environment variable from dev to prod to ensure that it works correctly in both environments: does anyone know if this is possible? I am currently using dotenv for managing my environment variables and have added the relevant code to my jest config file so it will read the variables correctly. I have been reading through various stack overflow posts and the jest docs about mock functions in an attempt to solve these problems but it is all flying over my head. I don't strictly need these test to work to get on with my app as I'm pretty confident it is all working fine but it is frustrating to not be able to solve these problems and it would be useful to know in the future in case I do need to make a test that relies on the log output. Can anyone help?
set-log-to.js
const { createLogger, format, transports } = require('winston');
const { combine, printf, errors } = format;
const nodeEnvironment = process.env.NODE_ENV;
const cc = require('../constants/chalk-classes');
const {
fatal,
caveat,
victory,
error,
warn,
attempt,
success,
info,
debug,
plain,
} = cc;
const config = {
levels: {
fatal: 0,
caveat: 0,
victory: 0,
error: 1,
warn: 2,
attempt: 3,
success: 3,
info: 3,
verbose: 4,
debug: 5,
},
};
const formatting = combine(
errors({ stack: true }),
printf((infoObj) => {
const { level, message } = infoObj;
switch (level) {
case 'fatal':
return `${fatal(`${level}:`)} ${plain(message)}`;
case 'caveat':
return `${caveat(`${level}:`)} ${plain(message)}`;
case 'victory':
return `${victory(`${level}:`)} ${plain(message)}`;
case 'error':
return `${error(`${level}: ${message}`)}`;
case 'warn':
return `${warn(`${level}: ${message}`)}`;
case 'attempt':
return `${attempt(message)}`;
case 'success':
return `${success(message)}`;
case 'info':
return `${info(message)}`;
case 'verbose':
return `${plain(message)}`;
case 'debug':
return `${debug(level)}: ${plain(message)}`;
}
})
);
function setLevel(level) {
if (!level) {
if (nodeEnvironment === 'dev') {
return (level = 'debug');
} else {
return (level = 'warn');
}
} else {
return level;
}
}
function setLogTo(level) {
level = setLevel(level);
const log = createLogger({
levels: config.levels,
level,
transports: [
new transports.Console({
format: formatting,
}),
],
});
return log;
}
module.exports = setLogTo;
set-log-to.test.js
const setLogTo = require('../set-log-to');
test('All log levels function correctly', () => {
let log = setLogTo('debug');
log.fatal('This is fatal');
log.caveat('This is a caveat');
log.victory('This is a victory');
log.error('This is an error');
log.warn('This is a warning');
log.attempt('This is an attempt');
log.success('This is a success');
log.info('This is some info');
log.verbose('This is verbose');
log.debug('This is a debug');
expect(???).toEqual(10);
});
test('Logger does not print debug as standard', () => {
let log = setLogTo();
log.warn('This is a warning');
log.verbose('This is a verbose statement');
log.debug('This is a debug statement');
expect(???).toEqual(1);
});
test('Logger does not print info when set to error', () => {
let log = setLogTo('error');
log.info('This is an info statement');
log.error('This is an error')
expect(???).toEqual(1);
});
test('Dotenv works correctly', () => {
let log = setLogTo();
log.debug('This is a debug');
nodeEnvironment = 'prod';
log.debug('This us a debug');
expect(???).toEqual(1)
});

Ok, so Winston doesn't send messages to the console, or if it does it's the global console. Either way I figured that the best way around this would be to add a file logger to the console logger, subsequently reading the lines from the file rather than the console and then deleting the file after ever test. This then caused problems as standard Jest behaviour is to run its test simultaneously, so each test was attempting to access the same temporary file at the same time. This was easy to fix by passing the --runInBand option to jest in my package.json file which makes Jest run the tests sequentially.
To solve the environment variable problem, it turns out that Jest sets the NODE_ENV variable to 'test' when it starts up. I added a jest.resetModules() to the beforeEach statement allowing me to manually declare the environment at the start of each test. I stored the jest test environment in a constant at the start of the file so I could use this for the majority of the tests that did not require explicit environment variables to be set. I also created an afterAll() function to make sure that process.env was set back to the test environment for any subsequent tests.
Code:
package.json:
"logtest": "jest --runInBand ./src/helper/tests/set-log-to.test.js",
set-log-to.test.js:
const fs = require('fs');
const path = require('path');
const winston = require('winston');
let setLogTo = require('../set-log-to');
const { snipconDir } = require('../../constants/core-dirs');
const logTests = path.join(snipconDir, './temp/log-tests.txt');
const testEnv = process.env;
function createTestLog(level) {
let log = setLogTo(level);
log.add(
new winston.transports.File({
filename: logTests,
})
);
return log;
}
function allLogLevels(log) {
log.fatal('This is fatal');
log.caveat('This is a caveat');
log.victory('This is a victory');
log.error('This is an error');
log.warn('This is a warning');
log.attempt('This is an attempt');
log.success('This is a success');
log.info('This is some info');
log.verbose('This is verbose');
log.debug('This is a debug');
}
async function getLines() {
let data = await fs.promises.readFile(logTests);
let lines = data.toString().split('\n');
lines.pop();
return lines;
}
beforeEach(async (done) => {
jest.resetModules();
try {
await fs.promises.writeFile(logTests, '');
done();
} catch (err) {
console.log(err);
}
});
afterEach(async (done) => {
try {
await fs.promises.unlink(logTests);
done();
} catch (err) {
console.error('Failed to delete log-tests.js');
}
});
afterAll(() => {
process.env = testEnv;
});
test('When level is explicitly set to debug, all messages show', async () => {
process.env = testEnv;
let log = createTestLog('debug');
allLogLevels(log);
let lines = await getLines();
expect(lines.length).toEqual(10);
});
test('Standard behaviour shows messages from level 2 and below', async () => {
process.env = testEnv;
let log = createTestLog();
allLogLevels(log);
let lines = await getLines();
expect(lines.length).toEqual(5);
});
test('When explicitly set to error level, loggger displays all level 0 messages', async () => {
process.env = testEnv;
let log = createTestLog('error');
allLogLevels(log);
let lines = await getLines();
expect(lines.length).toEqual(3);
});
test('Verbose level displays messages from level 4 and below', async () => {
process.env = testEnv;
let log = createTestLog('verbose');
allLogLevels(log);
let lines = await getLines();
expect(lines.length).toEqual(9);
});
test('Development environment displays all messages', async () => {
process.env.NODE_ENV = 'dev';
let log = createTestLog();
allLogLevels(log);
let lines = await getLines();
expect(lines.length).toEqual(10);
});
test('Production environment displays messages from level 2 and below', async () => {
process.env.NODE_ENV = 'prod';
let log = createTestLog();
allLogLevels(log);
let lines = await getLines();
expect(lines.length).toEqual(5);
});
set-log-to.js:
const { createLogger, format, transports } = require('winston');
const { combine, printf, errors } = format;
const cc = require('../constants/chalk-classes');
const {
fatal,
caveat,
victory,
error,
warn,
attempt,
success,
info,
debug,
plain,
} = cc;
const config = {
levels: {
fatal: 0,
caveat: 0,
victory: 0,
error: 1,
warn: 2,
attempt: 3,
success: 3,
info: 3,
verbose: 4,
debug: 5,
},
};
const formatting = combine(
errors({ stack: true }),
printf((infoObj) => {
const { level, message } = infoObj;
switch (level) {
case 'fatal':
return `${fatal(`${level}:`)} ${plain(message)}`;
case 'caveat':
return `${caveat(`${level}:`)} ${plain(message)}`;
case 'victory':
return `${victory(`${level}:`)} ${plain(message)}`;
case 'error':
return `${error(`${level}: ${message}`)}`;
case 'warn':
return `${warn(`${level}: ${message}`)}`;
case 'attempt':
return `${attempt(message)}`;
case 'success':
return `${success(message)}`;
case 'info':
return `${info(message)}`;
case 'verbose':
return `${plain(message)}`;
case 'debug':
return `${debug(level)}: ${plain(message)}`;
}
})
);
function setLevel(level) {
if (!level) {
if (process.env.NODE_ENV === 'dev') {
return (level = 'debug');
} else {
return (level = 'warn');
}
} else {
return level;
}
}
function setLogTo(level) {
level = setLevel(level);
const log = createLogger({
levels: config.levels,
level,
transports: [
new transports.Console({
format: formatting,
}),
],
});
return log;
}
module.exports = setLogTo;

Nodejs how to check independently a process is running by PID?

I'm using child_process to spawn a child process and get return PID from it. I need to manage this child process by its PID. Below is my code:
const childProcess = require('child_process');
let options = ['/c', arg1, arg2];
const myProcess = childProcess.spawn('cmd.exe', options, {
detached: false,
shell: false
});
let pid = myProcess.pid;
During run time, I want to use PID to validate independently from outside if the process is running or not (finished / killed). I want to know how to do this and what is the best way to make this validation in Nodejs?. I'm running application in Windows environment.
Any suggestion is appreciated, thanks.

I found out a solution as suggestion of is-running module. But I don't want to install new module into my project just for this purpose, so I created my own checkRunning() function as below:
// Return true if process following pid is running
checkRunning(pid) {
try {
return process.kill(pid, 0);
} catch (error) {
console.error(error);
return error.code === 'EPERM';
}
}
Following the Nodejs document about process.kill(pid[, signal]), I can use process.kill() to check the existence of process with specific signal argument is value 0 (not killing process as function name).
I make a copy of the document said:
As a special case, a signal of 0 can be used to test for the existence of a process

Probably this will help, npm module called is-running https://npmjs.org/package/is-running as was mentioned here -
https://stackoverflow.com/a/14884949/7927724

If you want to know when a child process exits, you can check the exit event
const { spawn } = require('child_process');
const bat = spawn('cmd.exe', ['/c', 'my.bat']);
bat.stdout.on('data', (data) => {
console.log(data.toString());
});
bat.stderr.on('data', (data) => {
console.log(data.toString());
});
bat.on('exit', (code) => {
console.log(`Child exited with code ${code}`);
});

Here is a code fragment as a reference
win32 (cond) {
return new Promise((resolve, reject) => {
const cmd = 'WMIC path win32_process get Name,Processid,ParentProcessId,Commandline,ExecutablePath'
const lines = []
const proc = utils.spawn('cmd', ['/c', cmd], { detached: false, windowsHide: true })
proc.stdout.on('data', data => {
lines.push(data.toString())
})
proc.on('close', code => {
if (code !== 0) {
return reject(new Error('Command \'' + cmd + '\' terminated with code: ' + code))
}
let list = utils.parseTable(lines.join('\n'))
.filter(row => {
if ('pid' in cond) {
return row.ProcessId === String(cond.pid)
} else if (cond.name) {
if (cond.strict) {
return row.Name === cond.name || (row.Name.endsWith('.exe') && row.Name.slice(0, -4) === cond.name)
} else {
// fix #9
return matchName(row.CommandLine || row.Name, cond.name)
}
} else {
return true
}
})
.map(row => ({
pid: parseInt(row.ProcessId, 10),
ppid: parseInt(row.ParentProcessId, 10),
// uid: void 0,
// gid: void 0,
bin: row.ExecutablePath,
name: row.Name,
cmd: row.CommandLine
}))
resolve(list)
})
})
},
which come from https://github.com/yibn2008/find-process/blob/master/lib/find_process.js

For the related subject here the best packages
Check only by pid
https://www.npmjs.com/package/is-running
Search by pid, name patterns and different means packages:
https://www.npmjs.com/package/find-process
https://www.npmjs.com/package/ps-node
https://github.com/sindresorhus/process-exists

Write to a CSV in Node.js

I am struggling to find a way to write data to a CSV in Node.js.
There are several CSV plugins available however they only 'write' to stdout.
Ideally I want to write on a row-by-row basis using a loop.

You can use fs (https://nodejs.org/api/fs.html#fs_fs_writefile_file_data_options_callback):
var dataToWrite;
var fs = require('fs');
fs.writeFile('form-tracking/formList.csv', dataToWrite, 'utf8', function (err) {
if (err) {
console.log('Some error occured - file either not saved or corrupted file saved.');
} else{
console.log('It\'s saved!');
}
});

The docs for node-csv-parser (npm install csv) specifically state that it can be used with streams (see fromStream, toStream). So it's not hard-coded to use stdout.
Several other CSV parsers also come up when you npm search csv -- you might want to look at them too.

Here is a simple example using csv-stringify to write a dataset that fits in memory to a csv file using fs.writeFile.
import stringify from 'csv-stringify';
import fs from 'fs';
let data = [];
let columns = {
id: 'id',
name: 'Name'
};
for (var i = 0; i < 10; i++) {
data.push([i, 'Name ' + i]);
}
stringify(data, { header: true, columns: columns }, (err, output) => {
if (err) throw err;
fs.writeFile('my.csv', output, (err) => {
if (err) throw err;
console.log('my.csv saved.');
});
});

If you want to use a loop as you say you can do something like this with Node fs:
let fs = require("fs")
let writeStream = fs.createWriteStream('/path/filename.csv')
someArrayOfObjects.forEach((someObject, index) => {
let newLine = []
newLine.push(someObject.stringPropertyOne)
newLine.push(someObject.stringPropertyTwo)
....
writeStream.write(newLine.join(',')+ '\n', () => {
// a line was written to stream
})
})
writeStream.end()
writeStream.on('finish', () => {
console.log('finish write stream, moving along')
}).on('error', (err) => {
console.log(err)
})

In case you don't wanna use any library besides fs, you can do it manually.
let fileString = ""
let separator = ","
let fileType = "csv"
let file = `fileExample.${fileType}`
Object.keys(jsonObject[0]).forEach(value=>fileString += `${value}${separator}`)
fileString = fileString.slice(0, -1)
fileString += "\n"
jsonObject.forEach(transaction=>{
Object.values(transaction).forEach(value=>fileString += `${value}${separator}`)
fileString = fileString.slice(0, -1)
fileString += "\n"
})
fs.writeFileSync(file, fileString, 'utf8')

For those who prefer fast-csv:
const { writeToPath } = require('#fast-csv/format');
const path = `${__dirname}/people.csv`;
const data = [{ name: 'Stevie', id: 10 }, { name: 'Ray', id: 20 }];
const options = { headers: true, quoteColumns: true };
writeToPath(path, data, options)
.on('error', err => console.error(err))
.on('finish', () => console.log('Done writing.'));

**In case you don't wanna use any library besides fs, you can do it manually. More over you can filter the data as you want to write to CSV file
**
router.get('/apiname', (req, res) => {
const data = arrayOfObject; // you will get from somewhere
/*
// Modify old data (New Key Names)
let modifiedData = data.map(({ oldKey1: newKey1, oldKey2: newKey2, ...rest }) => ({ newKey1, newKey2, ...rest }));
*/
const path = './test'
writeToFile(path, data, (result) => {
// get the result from callback and process
console.log(result) // success or error
});
});
writeToFile = (path, data, callback) => {
fs.writeFile(path, JSON.stringify(data, null, 2), (err) => { // JSON.stringify(data, null, 2) help you to write the data line by line
if (!err) {
callback('success');
// successfull
}
else {
callback('error');
// some error (catch this error)
}
});
}

this is the code that worked for me in nest js
import { Parser } from "json2csv";
const csv = require('csvtojson');
const csvFilePath = process.cwd() + '/' + file.path;
let csv data = await csv().fromFile(csvFilePath); /// read data from csv into an array of json
/// * from here how to write data into csv *
data.push({
label: value,
.......
})
}
const fields = [
'field1','field2', ...
]
const parser = new Parser({ fields, header:false }); /// if dont want header else remove header: false
const csv = parser.parse(data);
appendFileSync('./filename.csv',`${csv}\n`); // remove /n if you dont want new line at the end