I'm using node.js to spawn multiple phantomjs workers through childProcess.exec and writing the buffer output to DB. I wanted to know the max number of processes that can be spawned via node before node crashes. Phantomjs script is complex which includes logging in and doing stuff, and takes close to 5seconds to return an output.
code looks like this:
childProcess.exec('./phantomsjs testScript.js', function(err, stdout, stderr){}
Related
There is a server, that I have an access to, but do not have ownership on. It serves a node js / express application on a default port 3000. There are several scripts, that are usually run either manually from the terminal or by cron job. What I want to do is to have a button on the client-side and make an ajax request to a certain route and execute a node js command with inline arguments. For example:
node script.js 123
All routes are set and working. I have a CliController file that handles requests and has to run the command above. Currently I am using the following code:
cp.exec(`node script.js ${ip}`, function (err, stdout, stderr) {
if (err) {
console.log(err);
}
console.log(stdout);
console.log(stderr);
});
The script.js file is in root folder of the project, but the project itself was built by using express-generator and is being served using node bin/www command. There is a service/process on the server that runs nodemon to restart this project if it fails as well. Therefore I do not have access to output of that particular process.
If I run the command above in the terminal (from the root folder of the project, to be precise), it works fine and I see the output of the script. But if I press the button on the webpage to make a request, I am pretty sure that the script does not execute, because it has to make an update to database and I do not see any changes. I also tried to use child_process.spawn and child_process.fork and failed to get it working.
I also tried to kill nodemon and quickly start the project again to the see console output. If I do this, everything works.
What am I doing wrong ?
The process invoked may be in a blocking state, hence the parent script is simply waiting for the children process to terminate, or return something.
We can avoid this behaviour right into the shell command, by adding & (ampersand control operator) at the end.
This makes a command running in the background. (Notice, you can still control the children(s) process using the PID's and POSIX signals, this is another subject, but very related and you might find it very handy pretty soon).
Also notice that killing/stopping the parent script will also kill the children(s). This can be avoided using nohup.
This is not linked to JavaScript or node.js, but to bash, and can be used with anything in the shell.
cp.exec(`node script.js ${ip} &`, function (err, stdout, stderr) {
if (err) {
console.log(err);
}
console.log(stdout);
console.log(stderr);
});
Bash reference manual
I have a route on my NodeJS server which will spawn a python script like so:
const pythonProcess = spawn("python3",["/var/www/ip/uploads/script.py", project_num])
I then listen for a stdout flush like so:
pythonProcess.stdout.on('data', (data) => {
})
If the route is called by 2 different clients at roughly the same time, will they interfere with each other? I have seen some cases of the 2 clients getting the same data returned by stdout, when it should have been different. This might be caused by my python script, but I also have a hunch that getting data from stdout can be a bit shoddy.
In an online training video I am watching to learn Node, the narrator says that "spawn is better for longer processes involving large amounts of data, whereas execute is better for short bits of data."
Why is this? What is the difference between the child_process spawn and execute functions in Node.js, and when do I know which one to use?
The main difference is that spawn is more suitable for long-running processes with huge output. That's because spawn streams input/output with a child process. On the other hand, exec buffers output in a small (by default 200K) buffer. exec first spawns a subshell, and then tries to execute your process. To cut a long story short, use spawn in case you need a lot of data streamed from a child process and exec if you need features like shell pipes, redirects or even more than one program at a time.
Some useful links - DZone Hacksparrow
child process created by spawn()
does not spawn a shell
streams the data returned by the child process (data flow is constant)
has no data transfer size limit
child process created by exec()
does spawn a shell in which the passed command is executed
buffers the data (waits till the process closes and transfers the data in on chunk)
maximum data transfer up to Node.js v.12.x was 200kb (by default), but since Node.js v.12x was increased to 1MB (by default)
-main.js (file)
var {spawn, exec} = require('child_process');
// 'node' is an executable command (can be executed without a shell)
// uses streams to transfer data (spawn.stout)
var spawn = spawn('node', ['module.js']);
spawn.stdout.on('data', function(msg){
console.log(msg.toString())
});
// the 'node module.js' runs in the spawned shell
// transfered data is handled in the callback function
var exec = exec('node module.js', function(err, stdout, stderr){
console.log(stdout);
});
-module.js (basically returns a message every second for 5 seconds than exits)
var interval;
interval = setInterval(function(){
console.log( 'module data' );
if(interval._idleStart > 5000) clearInterval(interval);
}, 1000);
the spawn() child process returns the message module data every 1 second for 5 seconds, because the data is 'streamed'
the exec() child process returns one message only module data module data module data module data module data after 5 seconds (when the process is closed) this is because the data is 'buffered'
NOTE that neither the spawn() nor the exec() child processes are designed for running node modules, this demo is just for showing the difference, (if you want to run node modules as child processes use the fork() method instead)
A good place to start is the NodeJS documentation.
For 'spawn' the documentation state:
The child_process.spawn() method spawns a new process using the given command, with command line arguments in args. If omitted, args defaults to an empty array.
While for 'exec':
Spawns a shell then executes the command within that shell, buffering any generated output. The command string passed to the exec function is processed directly by the shell and special characters (vary based on shell) need to be dealt with accordingly.
The main thing appears to be whether you need handle the output of the command or not, which I imagine could be the factor impacting performance (I haven't compared). If you care only about process completion then 'exec' would be your choice. Spawn opens streams for stdout and stderr with ondata events, exec just returns a buffer with stdout and stderr as strings.
A quote from the official docs:
For convenience, the child_process module provides a handful of synchronous and asynchronous alternatives to child_process.spawn() and child_process.spawnSync(). Each of these alternatives are implemented on top of child_process.spawn() or child_process.spawnSync().
I'm learning nodejs and I want to send some file to the printing queue.
I tried elctron-printer and node-printer modules but actually they do not work proprly (can't detect printer with printer.list command for example). Now I'm trying to make it with child_process module and I want to know is there any posibility to start file with its associated application with "print" argument like a python can do it?
For example, this is a code sample of file execution with nodejs:
var childProcess = require('child_process');
childProcess.exec('start printme.txt', function (err, stdout, stderr) {
if (err) {
console.error(err);
return;
}
console.log(stdout);
process.exit(0);// exit process once it is opened
})
Unfortunately it seems that "print" argument is invalid for this code.
And this is a code sample for python and it works fine on windows:
import os
os.startfile('printme.txt', 'print')
All in all I hope that there is possibility to emulate system commands with nodejs.
Otherwise I will have to execute python script via nodejs just for file printing, something like this:
let python = spawn('python', [path.join(app.getAppPath(), '..', 'python_scripts/print_file.py'])
But it is terrible way to do it.
I know Windows 8 'apps' can be developed using web technologies but I haven't been able to find out if terminal commands can be run in the background using web technologies as an interface. I basically have mongoDB on my computer and it takes two terminal windows open to run it. I thought it might be a neat project to see if I could write a little app that is nothing more than a button that launches both commands behind the scenes saving me the hassle of going to the directories and running the commands manually for both terminal windows.
If you plan to launch apps via server-side JavaScript (e.g. node.js), use the child_process module..
The workflow would be that in the windows 8 gui side, it will just issue a request to your own local server in node.js, then it would execute those commands.
Example:
var exec = require('child_process').exec;
var child = exec("insert command here", function(err, stdout, stderr) { });
See examples exec and spawn for more examples.
======
Another thing you can do is create a batch (.bat) file that contains those two commands needed for your mongodb instance and put that as a shortcut in the Windows 8 Start Screen.
It depends on what kinds of commands you need to execute, and when and where. If you plan to execute commands remotely, I'd assume server-side JS would be appropriate, but if you plan to execute commands locally, I think all you need is just batch scripting.