We Keep Coding

Pipe spawned process stdout to function on flush

My goal is to spawn another binary file in a child process, then handle the stdout on a line-by-line basis (And do some processing against that line). For testing I'm using Node. To do this I tried a readable & writable stream but it a type error is throwed saying "The argument 'stdio' is invalid. Received Writable"
const rs = new stream.Readable();
const ws = new stream.Writable();
const child = cp.spawn("node", [], {
stdio: [process.stdin, ws, process.stderr]
})
let count = 0;
ws.on("data", (data) => {
console.log(data, count)
});
Anyone have any ideas?

One way is to use the stdio streams returned by spawn and manually pipe them:
const child = cp.spawn("node", [])
child.stdin.pipe(process.stdin)
child.stdout.pipe(ws)
child.stderr.pipe(process.stderr)
let count = 0;
ws.on("data", (data) => {
console.log(data, count)
});

Using child process output globally

For a project I need to incorporate a backend Python function with Javascript (main code for a chatbot). Using Child processes, it seems to work (when using 'node script.js'). However, I need to access the data from the called python function. Right now, all I am getting is the the output.I tried to store it in the global variable, but it's showing as 'undefined'. Is there a way to actually access the data so I can use it outside the stdout.on?
This is the Javascript code for running the pythonscript:
// Give a path to the QR scanner Python file
const qrScannerPath = "python/qrCodeScanner.py"
const base64Arg = "base64_2.txt"
// Provide the '.exe' python file. If python is available as an 'environment varaible', then simply refer to it as 'python'"
const pythonExe = "python"
// Function to convert a utf-8 array to a string
const utfConverter = function (data) {
return String.fromCharCode.apply(String,(data))
}
// let's us handle python scripts
const spawn = require("child_process").spawn
const scriptExe = spawn(pythonExe, [qrScannerPath, base64Arg])
// If all goes well, the program should execute the Python code
let counterpartyData = {}
scriptExe.stdout.on("data", function (data) {
console.log("getting the Python script data...")
let cp = JSON.parse(utfConverter(data))
counterpartyData = {... cp} //should store the data to the variable
});
console.log(counterpartyData) // shows 'undefinied"
// In case our python script cannot be run, we'll get an error
scriptExe.stderr.on("data", (data) => {
console.error("error : " + data.toString())
});
//logs error message
scriptExe.on('error', (error) => {
console.error('error: ', error.message);
});
// Logs a message saying that our code worked perfectly fine
scriptExe.on("exit", (code) => {
console.log("Process quit with code : " + code)
})
If I run this code with node, the output of 'counterpartyData' is undefined. However, inside the stdout, it actually prints out the data I want.
Furthermore, I get python import errors when running the app on Heroku :(.
Thank you in advance!!
Happy New Year and joyful greetings <3

Python to NodeJS communication not working through spawn

I'm trying to replicate this process to establish communication be nodejs and python through stdin and stdout : https://healeycodes.com/javascript/python/beginners/webdev/2019/04/11/talking-between-languages.html
Context:
Sender - which is giving output to stdout
Listener - one who's reading it
Now, when Python is sender and NodeJS is Listener isn't working. NodeJS gives no output.
On further digging my issue is very similar to Not receiving stdout from nodejs spawned process except I don't need unbuffered output as such. Tried sol from last ques.. didn't work.
Here are files:
sensor.py
import random, time
import sys
time.sleep(random.random() * 5) # wait 0 to 5 seconds
temperature = (random.random() * 20) - 5 # -5 to 15
print(temperature, flush=True, end='')
sys.stdout.flush()
listener.js
const { spawn } = require('child_process');
//spawn()
const child = spawn('python', ['path-to-sensor.py']);
console.log('Here');
child.stdout.on('data', function(data) {
console.log('Got the data')
console.log(data)
});
child.on('error', function () {
console.log("Failed to start child.");
});
child.on('close', function (code) {
console.log('Child process exited with code ' + code);
});
child.stdout.on('end', function () {
console.log('Finished collecting data chunks.');
});
Reason: Event listener - child.stdout.on('data', callback) is never called
What I want the output to be:
Store the stdout from sensor.py in a variable
Can't figure out what else I can do to fix this

In sensor.py you are not using the right syntax in your code:
print(temperature, flush=True, end='')
The right syntax is:
print(object(s), sep=separator, end=end, file=file, flush=flush)
Source
If you left the code in sensor.py as:
print(temperature)
And in listener.js as:
child.stdout.on('data', function(data) {
console.log('Got the data')
//console.log(data)
console.log(data.toString('utf8'))
});
It works ok

Communicating between NodeJS and Python: Passing back multiple arguments

As of right now I am using the built in child_process to start up the Python script and listen for any data passed back via stdout.on('data', (data)) like in line 6 of the first JS code. But from the Google searches I have done I only see examples of one thing being passed back or a group of things being passed back all clumped together. I was wondering if it was possible to send back more than just one argument. Below is my code:
JS:
const spawn = require('child_process').spawn;
pythonProcess = spawn('python', ["/path/to/python/file"]);
pythonProcess.stdout.on('data', (data) => {
console.log(data);
});
Python:
import sys
var thing1 = "Cold";
var thing2 = "Hot";
var thing3 = "Warm";
print(thing1);
print(thing2);
print(thing3);
sys.stdout.flush();
But what I want to happen is maybe pass back something like an array which is filled with the things I want to send back so that I can access them in the JS file like so:
const spawn = require('child_process').spawn;
pythonProcess = spawn('python', ["/path/to/python/file"]);
pythonProcess.stdout.on('data', (data) => {
thing1 = data[0];
thing2 = data[1];
thing3 = data[2];
})
console.log('thing1: ' + thing1);
console.log('thing2: ' + thing2);
console.log('thing3: ' + thing3);
Which would output:
thing1: Hot
thing2: Cold
thing3: Warm
How would I do this?
Thanks in advance!

There isn't an interface that communicate directly between Node.js and Python, so you can't pass custom arguments, what you're doing is just executing a python program using child_process, so you don't send arguments, anything that is received on 'data' its what is printed to stdout from python.
So what you need to do, is serialize the data, and then deserialize it in Node, you can use JSON for this.
From your python script, output the following JSON object:
{
"thing1": "Hot",
"thing2": "Cold",
"thing3": "Warm"
}
And in your Node.js script:
const spawn = require('child_process').spawn;
const pythonProcess = spawn('python', ["/path/to/python/file"]);
const chunks = [];
pythonProcess.stdout.on('data', chunk => chunks.push(chunk));
pythonProcess.stdout.on('end', () => {
try {
// If JSON handle the data
const data = JSON.parse(Buffer.concat(chunks).toString());
console.log(data);
// {
// "thing1": "Hot",
// "thing2": "Cold",
// "thing3": "Warm"
// }
} catch (e) {
// Handle the error
console.log(result);
}
});
Have in mind that data is chunked, so will have to wait until the end event is emitted before parsing the JSON, otherwise a SyntaxError will be triggered. (Sending JSON from Python to Node via child_process gets truncated if too long, how to fix?)
You can use any type of serialization you feel comfortable with, JSON is the easiest since we're in javascript.
Note that stdout is a stream, so it's asyncrhonous, that's why your example would never work.
pythonProcess.stdout.on('data', (data) => {
thing1 = data[0];
thing2 = data[1];
thing3 = data[2];
})
// Things do not exist here yet
console.log('thing1: ' + thing1);
console.log('thing2: ' + thing2);
console.log('thing3: ' + thing3);

Store output of shell command in sqlite

If I execute a certain shell command in node js, the output is on the console. Is there a way I can save it in a variable so it can be POST to Sqlite database.
const shell = require('shelljs');
shell.exec('arp -a');
In this scenario, I want to store the IP address of a specific MAC/Physical address into the database. How can this be done?
Any help would be much appreciated. Thank you

You need to get the output of the command you're passing to exec. To do that, just call stdout, like this:
const shell = require('shelljs');
const stdout = shell.exec('arp -a').stdout;
Then just parse that output to get your ipaddress:
const entries = stdout.split('\r\n');
// entries sample
[ '',
'Interface: 10.17.60.53 --- 0xd',
' Internet Address Physical Address Type',
' 10.11.10.52 6c-4b-90-1d-97-b8 dynamic ',
' 10.10.11.254 xx-yy-53-2e-98-44 dynamic ']
Then you can filter your wanted address with some more manipulation.
EDIT:
To get the ip address, you could do:
let ipAddr = null;
for (let i = 0; i < entries.length; i++) {
if (entries[i].indexOf('6c-4b-90-1d-97-b8') > -1) {
ipAddr = entries[i].match(/\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b/)[0];
break;
}
}
console.log(ipAddr); // '10.11.10.52'

I'm merely copy pasting from the docs. You should research more.
You need to add a listener to stdout
var child = exec('arp -a', {async:true});
child.stdout.on('data', function(data) {
/* ... do something with data ... */
});
Or adding the callback directly when calling exec
exec('some_long_running_process', function(code, stdout, stderr) {
console.log('Exit code:', code);
console.log('Program output:', stdout);
console.log('Program stderr:', stderr);
});

You can access the result of the command run using shell.exec with the .output property. Try the code below.
var shell = require('shelljs');
var result = shell.exec('arp -a').output;
If you don't want the result in the console, you can specify the silent option.
var result = shell.exec('arp -a', {silent: true}).output;
Now, you can use regular expressions to extract ip and mac address from the result.
I am getting the result of the command like below:
? (xxx.xxx.xxx.xxx) at xx:xx:xx:xx:xx:xx [ether] on eth0
? (yyy.yyy.yyy.yyy) at yy:yy:yy:yy:yy:yy [ether] on eth0
You can use the following code to extract ip and mac.
var res = result.split("\n").map(function(item){
return item.match(/\((\d+\.\d+\.\d+\.\d+)\) at (..:..:..:..:..:..)/);
});
console.log(res[0][1]); //IP of first interface
console.log(res[0][2]); //MAC of first interface
console.log(res[1][1]); //IP of second interface
console.log(res[1][2]); //MAC of second interface
NOTE
I was not able to find the .output property in the documentation but trying the shell.exec function in the node console revealed it.
The .stdout property or the exec function mentioned in other answers doesn't work for me. They are giving undefined errors.