We Keep Coding

Communicating between NodeJS and Python: Passing back multiple arguments

As of right now I am using the built in child_process to start up the Python script and listen for any data passed back via stdout.on('data', (data)) like in line 6 of the first JS code. But from the Google searches I have done I only see examples of one thing being passed back or a group of things being passed back all clumped together. I was wondering if it was possible to send back more than just one argument. Below is my code:
JS:
const spawn = require('child_process').spawn;
pythonProcess = spawn('python', ["/path/to/python/file"]);
pythonProcess.stdout.on('data', (data) => {
console.log(data);
});
Python:
import sys
var thing1 = "Cold";
var thing2 = "Hot";
var thing3 = "Warm";
print(thing1);
print(thing2);
print(thing3);
sys.stdout.flush();
But what I want to happen is maybe pass back something like an array which is filled with the things I want to send back so that I can access them in the JS file like so:
const spawn = require('child_process').spawn;
pythonProcess = spawn('python', ["/path/to/python/file"]);
pythonProcess.stdout.on('data', (data) => {
thing1 = data[0];
thing2 = data[1];
thing3 = data[2];
})
console.log('thing1: ' + thing1);
console.log('thing2: ' + thing2);
console.log('thing3: ' + thing3);
Which would output:
thing1: Hot
thing2: Cold
thing3: Warm
How would I do this?
Thanks in advance!

There isn't an interface that communicate directly between Node.js and Python, so you can't pass custom arguments, what you're doing is just executing a python program using child_process, so you don't send arguments, anything that is received on 'data' its what is printed to stdout from python.
So what you need to do, is serialize the data, and then deserialize it in Node, you can use JSON for this.
From your python script, output the following JSON object:
{
"thing1": "Hot",
"thing2": "Cold",
"thing3": "Warm"
}
And in your Node.js script:
const spawn = require('child_process').spawn;
const pythonProcess = spawn('python', ["/path/to/python/file"]);
const chunks = [];
pythonProcess.stdout.on('data', chunk => chunks.push(chunk));
pythonProcess.stdout.on('end', () => {
try {
// If JSON handle the data
const data = JSON.parse(Buffer.concat(chunks).toString());
console.log(data);
// {
// "thing1": "Hot",
// "thing2": "Cold",
// "thing3": "Warm"
// }
} catch (e) {
// Handle the error
console.log(result);
}
});
Have in mind that data is chunked, so will have to wait until the end event is emitted before parsing the JSON, otherwise a SyntaxError will be triggered. (Sending JSON from Python to Node via child_process gets truncated if too long, how to fix?)
You can use any type of serialization you feel comfortable with, JSON is the easiest since we're in javascript.
Note that stdout is a stream, so it's asyncrhonous, that's why your example would never work.
pythonProcess.stdout.on('data', (data) => {
thing1 = data[0];
thing2 = data[1];
thing3 = data[2];
})
// Things do not exist here yet
console.log('thing1: ' + thing1);
console.log('thing2: ' + thing2);
console.log('thing3: ' + thing3);

How to get filename and line number of where a function is called in Node?

When working in Python I always have this simple utility function which returns the file name and line number from where the function is called:
from inspect import getframeinfo, stack
def d():
""" d stands for Debug. It returns the file name and line number from where this function is called."""
caller = getframeinfo(stack()[1][0])
return "%s:%d -" % (caller.filename, caller.lineno)
So in my code I simply put a couple debug lines like this to see how far we get before some error occurs:
print d()
# Some buggy code here
print d()
# More buggy code here
print d(), 'here is the result of some var: ', someVar
This works really well for me because it really helps debugging quickly.
I'm now looking for the equivalent in a node backend script. I was searching around but I can't find anything useful (maybe I'm looking for the wrong words?).
Does anybody know how I can create a Javascript/nodejs function which outputs the file name and line number from where the function is called? All tips are welcome!

You can create an Error to get where the Error is, and its stack trace. Then you can put that into a function, to get the line where it is.
function thisLine() {
const e = new Error();
const regex = /\((.*):(\d+):(\d+)\)$/
const match = regex.exec(e.stack.split("\n")[2]);
return {
filepath: match[1],
line: match[2],
column: match[3]
};
}
console.log(thisLine());
This works for me in Google Chrome.
And also in node.
Note to #j08691's comment:
Both this and this seem to be using lineNumber, which is not present (as far as I could test) in NodeJS.

Printing line number with custom string
const moment = require('moment');
const log = console.log;
const path = require('path');
function getTime(time) { return moment().format('YYYY-MM-DD HH:mm:ss') };
function line(num = 2) {
const e = new Error();
const regex = /\((.*):(\d+):(\d+)\)$/
const match = regex.exec(e.stack.split("\n")[num]);
const filepath = match[1];
const fileName = path.basename(filepath);
const line = match[2];
const column = match[3];
return {
filepath,
fileName,
line,
column,
str: `${getTime()} - ${fileName}:${line}:${column}`
};
}
log(line().str, "mylog1");
log(line().str, "mylog2");
log(line().str, "mylog3");
OUTPUT
2021-11-22 13:07:15 - test.js:44:5 mylog1
2021-11-22 13:07:15 - test.js:45:5 mylog2
2021-11-22 13:07:15 - test.js:46:5 mylog3

You can use this gulp plugin gulp-log-line . It Logs file and line number without the extra cost of reading the stack.
you just have to install gulp and gulp-log-line using the
npm install gulp --save and npm install gulp-log-line command. after that you need to create and write the below code in gulpfile.js and run
gulp log-line to create a duplicate file in the build folder :-
var gulp = require('gulp');
var logLine = require('gulp-log-line');
gulp.task('line-log', function() {
return gulp.src("file.js", {buffer : true})
//Write here the loggers you use.
.pipe(logLine(['console.log', 'winston.info']))
.pipe(gulp.dest('./build'))
})
gulp.task('default', ['line-log'])
Example
file.js :-
console.log('First log')
var someVariable
console.log('Second log')
Becomes
console.log('file.js:1', 'First log')
var someVariable
console.log('file.js:3', 'Second log')

The only way I've found to get anything relating to line numbers is to trap the window.onerror function, and when there's an error that will get passed the error message, the file URL and the line number:
window.onerror = function(msg, url, line) {
alert(msg + "\n" + url + ":" + line);
};
This works for me on Chrome - I don't know about other browsers.
EDIT when this answer was given in Feb' 15 there was no mention of NodeJS in the question. That was only added in November '17.

Node.js - child_process.exec and output redirection

I'm trying to write a file templating script using Node.js. I have a JSON file called template.json which stores template information. The idea behind my script is that, if I type something like:
tmpl.js java Joe
it will do the following:
Call touch Joe.java
Read template.json to get the template for Java files
Use its information to replace all the placeholders with Joe
Write the result to Joe.java
Execute emacsclient Joe.java
Now, I wrote this script as follows:
#!/usr/local/bin/node --harmony
var templates = require('./config/template.json'),
args = process.argv;
if (args.length < 4) {
console.log("Not enough arguments!");
}
else {
var type = args[2],
name = args[3];
if (type in templates) {
var tmpl = templates[type],
contents = make_output(tmpl["format"],name),
file_name = name + tmpl["extension"],
command = "touch " + file_name + " && echo -e '" + contents +
"' &> " + file_name + " && emacsclient " + file_name;
invoke(command);
}
else {
console.log("No template for %s", type);
}
}
//Helpers
//Invokes comm with args in the terminal, returns all output
//Does not play nice with command redirection
function invoke(comm) {
var exec = require('child_process').exec,
child = exec(comm,
function (error, stdout, stderr) {
if (error !== null) {
console.log(stderr);
}
});
}
//If template is a format string, processes it with x as the
//replacement. Otherwise, just evaluates.
//Limited to one replacement at most.
function make_output(template, x) {
if(/.*\%s.*/i.test(template)) {
var util = require('util');
return util.format(template,x);
}
else {
return template;
}
}
Basically, the command it ends up building is something like:
touch Joe.java && echo -e `bunch of template stuffs` &> Joe.java && emacsclient Joe.java
Now, the problem I am getting is that the above command relies on output redirection, which my invoke command doesn't deal with very well - specifically, everything executes, but I get an empty file! Is there a way I can change either invoke or what I'm constructing to be invoked to avoid this problem?

The issue is that Node's child_process.exec starts sh but you are using features that are peculiar to bash. The &> is interpreted as & > in sh (two operators: a control operator and a redirection operator) and echo -e will use sh's builtin implementation of echo, which does not understand -e.
It would probably be possible to work around the issues above but using the shell like you do is fragile. For instance if your template contains single quotes ('), these quotes may interfere with the single quotes you use in your command. A more robust way to do it would be to change the main part of your code to use fs.writeFileSync rather than using shell commands to write to your file:
var templates = require('./config/template.json'),
fs = require("fs"),
args = process.argv;
if (args.length < 4) {
console.log("Not enough arguments!");
}
else {
var type = args[2],
name = args[3];
if (type in templates) {
var tmpl = templates[type],
contents = make_output(tmpl["format"],name),
file_name = name + tmpl["extension"],
command = "emacsclient " + file_name;
fs.writeFileSync(file_name, contents);
invoke(command);
}
else {
console.log("No template for %s", type);
}
}
You'd also want to modify make_output to perform the transformations that echo -e would have done for you.

python to node.js confusion

So I have this python code that I'm trying to convert to node.js, but I am not sure how.
import urllib.request, re
def getDef(word):
link = "http://www.merriam-webster.com/dictionary/%s" % word
data = urllib.request.urlopen(link).read().decode()
try:
return re.search("<p>: (.*?)</p><p>", data).group(1)
except:
return "No match"
class newDefinition:
def __init__(self, word):
self.definition = getDef(word);
>>> definition = newDefintion("color")
>>> print(definition.definition)
a quality such as red, blue, green, yellow, etc., that you see when you look at something
In node.js however though it I can seem to return it like in python because of it's callback way of doing things, or at least I can't seem to return it which is why I'm asking how would I do the node.js equivalent or is their no equivalent? Here is what I have so far maybe you can spot what I'm doing wrong and how to fix it
var urllib = require("urllib"); // installed with npm
var getDef = function(word){
var link = "http://www.merriam-webster.com/dictionary/" + word;
var urlData = urllib.request(link, {}, function(err, data, res){
var re = new RegExp("<p>: (.*?)</p><p>");
var results = data.toString();
var match = re.exec(results)[1];
return match; // Expected it to give urlData the definition
});
return urlData;
}
var Definition = function(word){
this.definition = getDef(word);
}
definition = new Definition("color");
console.log(definition.definition); // this won't give the definition but the information of the urllib itself rather.
So in general trying to figure out is how to use asynchronous code so I can return things that I need, but I am not use to this concept either so is there an equivalent to this in python? Also if you can point me to some good documentation on asynchronous code that would be great also.

Since return will actually just exit your function instead of returning a value, you need to use a callback. It would look like this:
var urllib = require("urllib");
var getDef = function(word, callback){
var link = 'http://www.merriam-webster.com/dictionary/' + word;
urllib.request(link, {}, function(err, data, res) {
var re = new RegExp('<p>: (.*?)</p><p>');
var results = data.toString();
var match = re.exec(results)[1];
callback(match);
});
};
Then you would pass a callback while calling the function:
getDef('color', function(definition) {
console.log(definition);
});
Edit: Setting an object's property has the same idea. It might look like this instead:
var Definition = function(word) {
var self = this;
getDef(world, function(definition, callback) {
self.definition = definition;
callback.call(self);
});
};
And would be called like so:
var definition = new Definition('color', function() {
console.log(definition.definition);
});

Here is my two cent worth suggestion.
Never ever use regular expressions to parse HTML (Refer here for more details), instead use the XPath like library to parse the document. You can use libraries like cheerio or phantomjs.
Here is a clean solution.
var request = require('request'),
when = require('when'),
cheerio = require('cheerio');
var URL = 'http://www.merriam-webster.com/dictionary/';
/**
* #param word: Word to search the dictionary
* #returns
* Promise object which resolves to array of
* definitions of the word
*/
var getDef = function(word){
var defer = when.defer();
request(URL + word, function(err, res, body){
if (err || res.statusCode !== 200){
defer.reject();
}
var defs = [];
var $ = cheerio.load(body);
$('.wordclick .headword:first-child p').each(function(i,ele){
var definition = $(ele).text();
defs.push(definition);
});
defer.resolve(defs);
});
return defer.promise;
}
getDef('happy').then(function(words){
console.log(words);
});
Note: Here I am using when (a Promise+ library) instead of the Node's standard CPS style.

How to write multiline javascript code to file with node?

I generate some jS code on the front end and i want to write it to a js file by making an api call to a node server and i'm having some trouble with 1) ensuring the code isnt written to file as a literal string and 2) ensuring that multiple lines are preserved.
Here's my code for how ive attempted this:
//client
var bar = "mongolia"; //this value changes hence pulling it into variable
var jsCode = "function(){ \n
baz = foo(" + bar + "); \n
return baz*2;"
var data = {
code: jsCode
}
$http.post('api/code', data).success(function(savedCode){
console.log("successful and the saved code is", savedCode);
});
//server
'use strict';
var jf = require('jsonfile'),
util = require('util');
var codeFile = './code/js/autoGen.js';
exports.addCode = function(req, res) {
var newCode = req.body.code;
//write to file
return jf.writeFile(codeFile, newCode, function(err) {
if(!err) {
return res.json({newCode: newCode});
}
else {
return res.send(err);
}
});
};
When i run this, I get a syntax error in my jsCode definition.

JavaScript syntax for multiline strings is as follows:
var jsCode = "function(){ \
baz = foo(" + bar + "); \
return baz*2;"

I got around both issues by
1) Using coffeescript for multiline strings and the #{} interpolation pattern for including the variable portions
2) Using fs for writing to file (Thanks for the tip #verybadalloc)