We Keep Coding

Node spawn child process doesn't execute the command after exec child process in aws node 10 lambda

I am attempting to run 2 child processes, but one seems to be blocked and eventually times out the node lambda.
Environment:
AWS node 10 lambda running in a docker container.
Accesses ffmpeg and ffprobe via a lambda layer in the /opt/bin directory.
child_process.exec
I am running ffprobe in a child_process.exec to get the file format of an audio file. I am using exec because the output is a small json response (which shouldn't consume much memory).
child_process.spawn
Shortly after I run ffmpeg to convert the audio file to mp3 using child_process.spawn.
The problem is the FFMPEG child_process.spawn command doesn't run after ffprobe (even though ffprobe successfully completes). If I don't run the ffprobe command the FFMPEG command runs perfectly.
Which leads me to believing this is an issue with how I am dealing with child processes in node.
Is it possible the child_process.exec ffprobe command is somehow still running/ blocking the new ffmpeg (child_process.spawn) command from running - if so how do I check this?
When I access the running processes in the docker container only the new ffmpeg command seems to be running, although it consumes no memory and just hangs - seemingly doing nothing. I even tried launching the ffmpeg command from the docker cli (avoiding using the node env) and this works fine and runs as expected.

So it seems my issue wasn't really between exec and spawn, I am not 100% sure but I think it could be that the child process was preserved in the container and resumed in the next invocation of the lambda.
Changing to child_process.spawnSync waits until the child process exits and keeps things cleaner and I haven't encountered this problem since using this.
A more thorough explanation from someone else would be really appreciated.

Node.js Spawn vs. Execute

In an online training video I am watching to learn Node, the narrator says that "spawn is better for longer processes involving large amounts of data, whereas execute is better for short bits of data."
Why is this? What is the difference between the child_process spawn and execute functions in Node.js, and when do I know which one to use?

The main difference is that spawn is more suitable for long-running processes with huge output. That's because spawn streams input/output with a child process. On the other hand, exec buffers output in a small (by default 200K) buffer. exec first spawns a subshell, and then tries to execute your process. To cut a long story short, use spawn in case you need a lot of data streamed from a child process and exec if you need features like shell pipes, redirects or even more than one program at a time.
Some useful links - DZone Hacksparrow

child process created by spawn()
does not spawn a shell
streams the data returned by the child process (data flow is constant)
has no data transfer size limit
child process created by exec()
does spawn a shell in which the passed command is executed
buffers the data (waits till the process closes and transfers the data in on chunk)
maximum data transfer up to Node.js v.12.x was 200kb (by default), but since Node.js v.12x was increased to 1MB (by default)
-main.js (file)
var {spawn, exec} = require('child_process');
// 'node' is an executable command (can be executed without a shell)
// uses streams to transfer data (spawn.stout)
var spawn = spawn('node', ['module.js']);
spawn.stdout.on('data', function(msg){
console.log(msg.toString())
});
// the 'node module.js' runs in the spawned shell
// transfered data is handled in the callback function
var exec = exec('node module.js', function(err, stdout, stderr){
console.log(stdout);
});
-module.js (basically returns a message every second for 5 seconds than exits)
var interval;
interval = setInterval(function(){
console.log( 'module data' );
if(interval._idleStart > 5000) clearInterval(interval);
}, 1000);
the spawn() child process returns the message module data every 1 second for 5 seconds, because the data is 'streamed'
the exec() child process returns one message only module data module data module data module data module data after 5 seconds (when the process is closed) this is because the data is 'buffered'
NOTE that neither the spawn() nor the exec() child processes are designed for running node modules, this demo is just for showing the difference, (if you want to run node modules as child processes use the fork() method instead)

A good place to start is the NodeJS documentation.
For 'spawn' the documentation state:
The child_process.spawn() method spawns a new process using the given command, with command line arguments in args. If omitted, args defaults to an empty array.
While for 'exec':
Spawns a shell then executes the command within that shell, buffering any generated output. The command string passed to the exec function is processed directly by the shell and special characters (vary based on shell) need to be dealt with accordingly.
The main thing appears to be whether you need handle the output of the command or not, which I imagine could be the factor impacting performance (I haven't compared). If you care only about process completion then 'exec' would be your choice. Spawn opens streams for stdout and stderr with ondata events, exec just returns a buffer with stdout and stderr as strings.

A quote from the official docs:
For convenience, the child_process module provides a handful of synchronous and asynchronous alternatives to child_process.spawn() and child_process.spawnSync(). Each of these alternatives are implemented on top of child_process.spawn() or child_process.spawnSync().

sails debug command not working in Sails.js

I am creating my first sails.js app. When I tried
sails debug
I'm getting the following error on my command prompt
Debugger listening on port 5858
info: Starting app...
error: Grunt :: Error: listen EADDRINUSE
at exports._errnoException (util.js:746:11)
at Agent.Server._listen2 (net.js:1129:14)
at listen (net.js:1155:10)
at Agent.Server.listen (net.js:1240:5)
at Object.start (_debugger_agent.js:20:9)
at startup (node.js:86:9)
at node.js:814:3
To get the PID of the process using port:5858, I tried running
C:\Windows\system32>netstat -a -n -o
but unfortunately there is no process bound to port 5858. Am I missing something here?
I'm using Windows 8.1 with node.js v0.12.0 and sails.js 0.11.0

My server uses node 0.10.38 with sails because of some weird unfixed grunt thing with 11+. Haven't pulled up this issue in a while, but it looks like there's new activity... check out this comment in particular, which explains the issue and a possible fix (direct quote):
Possible Solution:
Looking at the options for child_process.fork, the --debug flag is being passed down to the child upon exiting the womb i.e. running
sails debug :
// ./node_modules/sails/bin/sails-debug.js
// Spin up child process for Sails
Womb.spawn('node', ['--debug', pathToSails, 'lift'], {
stdio: 'inherit'
});
setting options.execArgv to an empty array removes the flag and allows the process to continue:
// ./node_modules/sails/lib/hooks/grunt/index.js
var child = ChildProcess.fork(
path.join(__dirname, 'grunt-wrapper.js'),
[
taskName,
'--pathToSails='+pathToSails,
'--gdsrc='+ pathToSails + '/node_modules'
],
{
silent: true,
stdio: 'pipe',
execArgv: []
}
);

It seems like a bug in Sails. You can apply the fix your self by replacing your Sails' file:
./node_modules/sails/lib/hooks/grunt/index.js
with the contents of the following:
https://raw.githubusercontent.com/balderdashy/sails/88ffc0ed9949f8c74ea390efb5610b0e378fa02c/lib/hooks/grunt/index.js
This is the file that will be in the Sails' release v12.

Did you try to run in debug like simple node.js?
node --debug app.js

Node.js why does a child process not start right away?

I'm trying to write a global node command line program that will take any (windows or unix) console command I give to it and execute it in a new console window. I also want the program to exit after it has spawned its process so the console I'm using isn't blocked by a node script that has a child process running.
This is a simple version of what I have so far:
myScript.js:
var exec = require('child_process').exec;
exec("start startScript.cmd"); // windows start command opens a new cmd window
process.exit(0);
startScript.cmd:
mkdir test
I have also tried this (But this doesn't work even without the process.exit):
myScript.js:
var spawn = require('child_process').spawn;
var child = spawn('start', ['startScript.cmd'], { detached: true, stdio: ['ignore', 'ignore', 'ignore']});
child.unref();
process.exit(0);
The problem is, calling process.exit() seems to prevent the child process from fully starting and so nothing happens unless I do some setTimeout shenanigans. However the behavior seems random. On a different computer it behaves like I want it to. Both computers have the same version of node (v0.10.33).
The directory test is never made unless I remove the process.exit line or use a setTimeout on it.
Any idea why this happens or how to get around it? Keep in mind I don't want to wait until the child process is finished. I want to be able to return to my command line immediately.
Thanks!

I figured it out.
The answer is: don't use exec and don't use start or call or any other weird windows command in the spawn call. Set the CWD if need be in the spawn options and either spawn what you want directly or make a OS specific script that calls your command if it still isn't working. One example of a windows script is literally just:
%*
That way, start or any other weird windows command will be executed correctly (but do you really need them?).

Node.js Detached child process is invoked as interactive foreground process

I am invoking a detached child_process (in windows) like this
var stdout = fs.openSync('./out.log', 'a');
var stderr = fs.openSync('./out.log', 'a');
var child = child_process.spawn('node', ['./main.js'], {
detached: true,
stdio: [ 'ignore', stdout, stderr ],
env: process.env
});
child.unref();
The spawned process, i.e. main.js in turn forks some workers using child_process.fork.
Normally the child and sub-children processes should run in background, correct me if I am wrong. But in when I try, the sub-children processes are running as interactively in foreground. I don't know what is causing them to behave like this. Could anyone point what could be the problem?
If I set detached to false, it works fine, but then the original parent can't exit without having all children exited.

I think this may be a problem with node on windows.
I tried both your code and the sample code on the node documentation you are for sure referencing yourself at http://nodejs.org/api/child_process.html#child_process_child_process_spawn_command_args_options on my Ubuntu virtual machine and it works just fine. Process is detached and keeps running and there is no output anywhere but to out.log because it is running successfully in the background.
I ran both a simple ping command and also ran a node instance on a javascript file like you are trying to do and both worked just fine.
I would recommend you do the same with a ping command in windows. . .
var child = child_process.spawn('ping', ['-n','25','localhost'],. . .
If that works like you would expect, there may be something inside your main.js that is causing the issue. If it doesn't work either, I would say you are up against a bug in node on windows and should report it.
Hope that helps some!