I'd like to start off by saying that I'm a complete NodeJS noobie.
So, that being said, I'm trying to get a better understanding of the behavior of parent and child processes when using the Cluster API of NodeJS along with process.exit. I also realize that all of these doubts relate to the behavior of Unix itself.
I know that process.exit() causes the current process to exit with the "success" code, 0.
For my test, I have 2 files: index.js and indexWorker.js.
Here's what index.js looks like:
var cluster = require('cluster');
if (cluster.isMaster) {
// Setup the required fork behavior.
cluster.setupMaster({
exec: 'indexWorker.js'
});
var cpuCount = require('os').cpus().length;
// Create a worker for each logical core.
for (var i = 0; i < cpuCount; ++i) {
cluster.fork({'NODE_ENV': 'development'});
}
console.log('Calling process.exit on parent.');
process.exit();
}
And here's what indexWorker.js looks like:
var cluster = require('cluster');
console.log("Started worker having ID: " + cluster.worker.id);
Here are my questions/doubts:
Doubt 1
Now, since I'm using process.exit on the parent, my understanding is that all the worker process ought to become orphan process. However, firstly, the control doesn't return to my terminal window until I do a ^C, despite the fact that the parent (index.js) is exiting right away. Also, ps, ps -ax, ps -elf | grep <pattern> don't seem to think that there are any node programs running.
So my doubt is: why isn't the control returning to the terminal and why isn't ps showing me anything?
Doubt 2
If I remove process.exit from index.js, my understanding is that the control doesn't return to my terminal because despite the parent having finished it's task, the child processes are running and thus the parent hasn't exited. Is that a correct understanding or am I missing something?
Doubt 3
If I remove the cluster.fork call and process.exit() call from index.js, the script completes and returns control to my terminal window immediately, as index.js isn't doing much in that case.
However, as you can see, indexWorker.js isn't doing any long operations either. It's simply printing a console message. Why, in that case, are the processes still running? Shouldn't the behavior be the same as the parent's, i.e. exit on script completion and thereby eventually return control to the terminal window?
I've simply been running node index.js for all these tests; never putting any processes in the background. My tests are being run on a Mac.
That's all the doubts I have, for now. :)
I've basically been experimenting by putting process.exit in different places and running ps, hence my doubts around these points. If there are any more things that you'd like to point out regarding the parent/child process behavior that aren't covered in my doubts, I'd greatly appreciate it! It'll definitely help me and the community in some way.
So, thanks in advance to the SO community!
Cheers!
Related
I have taken over an Electron project from another developer.
The problem I am facing is that the project does not show any errors. Even including something like throw "this is an error" does not produce any output on the main process or render process consoles or any sort of standard error popup.
I have checked to confirm that electron-unhandled is not in use and that nothing registers 'uncaughtException'.
What am I missing that could cause this behavior?
Search for: unhandledRejection
unhandledRejection : This will catch any thrown errors, or non fatal errors you have successfully handled via throw.
uncaughtException : This only catches fatal errors or errors that would crash your node instance
WebWorkers : There will be yet another console for webworkers if your using those.
package.json : Take a look in here at the script executed to start electron or however your starting it... Make sure the console is not being sent to a remote console. This feature would allow for debugging the application via Chrome/Firefox vs the standard console. Pretty common for electron apps. If is done via the startup command.
May look something like this:
process.on('unhandledRejection', function (err) {
});
Also, make sure you include any modules in your searching for suppressors as the issue may exist somewhere in the node_modules directory and many IDE's (mine does by default) exclude that directory in indexing/searches.
Another possible reason could be stdout and/or stderr redirection, the problem is this could be achieved by several ways so it's hard to suggest you what to check...
If there is some child_process call to launch a sub-process you could check the stdio array used, or you can check if some low level operation is performed against file descriptors 1 and 2...
Hope this helps.
Are you facing the problem as mentioned in this official thread. You may disable the original event listeners and manage the ELECTRON_BROWSER_WINDOW_ALERT event by my event listener.
Here is the solution
ipcMain.removeAllListeners("ELECTRON_BROWSER_WINDOW_ALERT")
ipcMain.on("ELECTRON_BROWSER_WINDOW_ALERT", (event, message, title)=>{
console.warn(`[Alert] ** ${title} ** ${message}`)
event.returnValue = 0 // **IMPORTANT!**
})
I've got a little sandbox project I've been playing around with for the last few weeks to learn the in's and out's of implementing a TestCafe runner.
I've managed to solve all my problems except one and at this point I've tried everything I can think of.
Reviewed the following similar questions:
How to close testcafe runner
How to get the testCafe exit code
But still my problem remains.
I've toyed around with my argv.json file.
I've toyed around with my CICDtestBranches.json file.
I've toyed around with my package.json file.
I've tested the same branch that has the problem on multiple
machines.
I've tested with multiple browsers (Firefox & Chrome) -
both produce the same problem.
I've tried to re-arrange the code, see
below
I've tried add multiple tests in a fixture and added a page
navigation to each one.
I've tried to remove code that is processing
irrelevant options like video logs & concurrency (parallel execution)
I also talked with some coworkers around the office who have done similar projects and asked them what they did to fix the problem. I tried their recommendations, and even re-arranging things according to what they tried and still no joy.
I've read through the TestCafe documentation on how to implement a test runner several times and still I haven't been able to find any specific information about how to solve a problem with the browser not closing at the end of the test/fixture/script run.
I did find a few bugs that describe similar behavior, but all of those bugs have been fixed and the remaining bugs are specific to either Firefox or Safari. In my case the problem is with both Chrome & Firefox. I am running TestCafe 1.4.2. I don't want to file a bug with TestCafe unless it really is a confirmed bug and there is nothing else that can be done to solve it.
So I know others have had this same problem since my coworker said he faced the same problem with his implementation.
Since I know I am out of options at this point, I'm posting the question here in the hopes that someone will have a solution. Thank you for taking the time to look over my problem.
When executing the below code, after the return returnData; is executed, the .then statement is never executed so the TestCafe command and browser window are never terminated.
FYI the following code is CommonJS implemented with pure NodeJS NOT ES6 since this is the code that starts TestCafe (app.js) and not the script code.
...**Boiler Plate testcafe.createRunner() Code**...
console.log('Starting test');
var returnData = tcRunner.run(runOptions);
console.log('Done running tests');
return returnData;
})
.then(failed => {
console.log(`Test finished with ${failed} failures`);
exitCode = failed;
if (argv.upload) return upload(jsonReporterName);
else return 0;
testcafe.close();
process.exit(exitCode);
})
.then(() => {
console.log('Killing TestCafe');
testcafe.close();
process.exit(exitCode);
});
I've tried to swap around the two final .then statements to try and see if having one before the other will cause it to close. I copied the testcafe.close() and process.exit() and put them after the if-else statement in the then-failed block, although I know they might-should not get called because of the if-else return statements just before that.
I've tried moving those close and exit statements before the if-else returns just to see if that might solve it.
I know there are a lot of other factors that could play into this scenario, like I said I played around with the runOptions:
const runOptions = {
// Testcafe run options, see: https://devexpress.github.io/testcafe/documentation/using-testcafe/programming-interface/runner.html#run
skipJSErrors: true,
quarantineMode: true,
selectorTimeout: 50000,
assertionTimeout: 7000,
speed: 0.01
};
Best way I can say to access this problem and project and all of the code would be to clone the git lab repo:
> git clone "https://github.com/SethEden/CAFfeinated.git"
Then checkout the branch that I have been working this problem with: master
You will need to create an environment variable on your system to tell the framework what sub-path it should work with for the test site configuration system.
CAFFEINATED_TEST_SITE_NAME value: SethEden
You'll need to do a few other commands:
> npm install
> npm link
Then execute the command to run all the tests (just 1 for now)
> CAFfeinated
The output should look something like this:
$ CAFfeinated
Starting test
Done running tests
Running tests in:
- Chrome 76.0.3809 / Windows 10.0.0
LodPage
Got into the setup Test
Got to the end of the test1, see if it gets here and then the test is still running?
√ LodPage
At this point the browser will still be spinning, and the command line is still busy. You can see from the console output above that the "Done running tests" console log has been output and the test/fixture should be done since the "Got to the end of the test1,..." console log has also been executed, that is run as part of the test.after(...). So the next thing to execute should be in the app.js with the .then(()) call.....but it's not. What gives? Any ideas?
I'm looking for what specifically will solve this problem, not just so that I can solve it, but so others don't run into the same pitfall in the future. There must be some magic sauce that I am missing that is probably very obvious to others, but not so obvious to me or others who are relatively new to JavaScript & NodeJS & ES6 & TestCafe.
The problem occurs because you specified the wrong value for the runner.src() method.
The cause of the issue is in your custom reporter. I removed your reporter and now it works correctly. Please try this approach and recheck your reporter.
I have a Node.js script that runs to do some database operations. I create a couple of child processes using the spawn functionality. The strange thing is, I stop getting carriage returns in my output. For example:
running...
more output.
more output.
complete.
The setup is as follows:
Main parent process: kicks off child piped to tee
Main child process: does majority of db transactions
Child subprocess 1: opens a VPN tunnel and stays running through entire execution
Child subprocess 2: Runs a brief script then terminates
I'm spawning the child processes in this manner:
var spawn = require('child_process').spawn;
STATE.childProcesses.logged = spawn('sh',['-c', cmd],{
stdio: [0]
});
STATE.childProcesses.logged.stdout.on('data', function (data) {
process.stdout.write(data.toString().replace(/\n/g, "\r\n"));
});
STATE.childProcesses.logged.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});
STATE.childProcesses.logged.on('exit', function (code) {
console.log('main child process exited with code ' + code);
});
I was able to work around the issue by doing a .replace(/\n/g, "\r\n") on the child process's stdout data as seen above.
However, now when the main parent process terminates, the bash shell it was running in is suffering from a similar issue: it outputs neither carriage returns nor newlines. Also text that I type doesn't show up. If I close the Terminal window it's running in (Mac OSX) and reopen the issue goes away.
I can verify that when the parent process terminates, all child processes are also terminated. At least, I call .kill('SIGINT') on all of them and I'm able to close Terminal without any warnings about child processes getting terminated.
I'm wondering if it has something to do with the routing of stdio for the spawned processes, but can't figure it out. Any tips greatly appreciated!
I was able to solve the issue of bash getting screwed up post execution by being careful about child process termination. Turns out I wasn't terminating one of the child processes properly and once I made sure it was dead before ending the parent process the bash issue went away.
I never did figure out why I needed to have the .replace(/\n/g, "\r\n") though.
Fatal Windows exception, code 0xc0000005.
PhantomJS has crashed. Please read the bug reporting guide at
<http://phantomjs.org/bug-reporting.html> and file a bug report.
This error pops up randomly when I am recursively opening a new page using casperJS. It's not all that random, it shows up after a couple of iterations (maybe around 50).
I don't think it's a memory leak, because I am monitoring the memory usage while running the script, and have closed all the heavy memory applications.
I've seen people reporting this issue on github, but I don't think it has a fix yet.
Is there a way I can let my machine to rerun Casper script after it detects a crash? (i.e. auto run $casperjs run.js after it detects the crash)
I was thinking to use shell script to do this, but not sure exactly how to detect the crash.
Of course, any ideas on fixing this crash would be good too.
Not sure if this helps, but I am putting my Casper code here too:
var runCasper = function(){
casper.start('https://www.example.com', function() {
// Do something
});
casper.then(function() {
// Do something
runCasper();
});
casper.run();
}
One thing you can try is keep checking if the casper/phantom process is running. you can do this by writing a status file at the start and end of each run. Check out https://stackoverflow.com/a/15283576/2231632 for using filesystem within casper. Code snippted from that answer:
var fs = require('fs');
var utils = require('utils');
var data = fs.read('testdata.dat');
utils.dump(data);
And then add a shell script that frequently reads this file and if a particular process is in 'running' phase for over 10 minutes or so, you can re-run casper from the shell script.
Ideally, every recursion should have a exit condition and you can't keep on recursing forever. What is it that you want to achieve by recursively opening the same page again and again?
I wrote a module in node.js that performs some network operation. I wrote a small script that uses this module (the variable check below). It looks like this:
check(obj, function (err, results) {
// ...
console.log("Check completed");
});
Now here is the interesting thing. When this code executes as part of a mocha test, the test exits as expected. I see the log statement printed and the process exits.
When the code is executed as a standalone node script, the log statement gets printed, but the process just hangs.
When I try to debug it and I start the program using --debug-brk and use node-inspector, it exits early! I see that process.on 'exit' is called. It exits while some internal callbacks within the module weren't called yet. So the log statement above isn't printed either.
I am stuck now and am not sure why this is happening. Has anyone seen similar behaviour?
When you run it as a script and it hangs when "done", it means node still has callbacks registered waiting for events. Node doesn't know that those events won't fire anymore. You can either just call process.exit() if you know it's time to exit, or you can explicitly close/unbind/disconnect everything (network connections, db connections, etc). If you properly close everything, node should then exit.
The module wtfnode (mentioned by Nathan Arthur) or why-is-node-running can be really helpful tracking this down.
If the program exits unexpectedly, it can be because the event loop becomes empty and there is nothing else to do (because some code forgot to emit an error or do something else to keep the event loop going). In this case Node exits with code 0 and you won't get any error messages whatsoever, so it can be really confusing.
See https://github.com/archiverjs/node-archiver/issues/457 for an example of this happening.