I'm trying to start a server daemon from within my js code, then to access it from the same program. However, when I use the execFile() method from the child_process module, this blocks the entire program: the execFile() call never stops, and I am unable to access the server. However, I know the daemon is started since I see a process of the same name start up from my activity monitor (macos equivalent of task manager).
I've also tried exec() and spawn() from the same module, and it gave the same results.
What I'd want to be able to do is to start the daemon as a separate process, forget about it, then stop it when I'm done using it. Is there any way I could do at least the first two?
Here is my code (the runArduino function is where I start the daemon, and the main() function is where I access it):
const grpcLib = require('grpc');
const protoLoader = require('#grpc/proto-loader');
const pathLib = require("path");
const utilLib = require('util');
const exec = utilLib.promisify(require('child_process').execFile);
const RPC_PATH = pathLib.join(__dirname, "arduino-cli/rpc")
var PROTO_PATH = pathLib.join(RPC_PATH, "/commands/commands.proto");
const options = {
keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true,
includeDirs:
[
RPC_PATH
]
}
const packageDefinition = protoLoader.loadSync(PROTO_PATH, options);
const arduinoCli = grpcLib.loadPackageDefinition(packageDefinition).cc.arduino.cli.commands;
function runArduino()
{
exec(__dirname+"/arduino-cli_macos", ['daemon'],function(err, data)
{
console.log(err);
console.log(data);
});
}
function main()
{
var client = new arduinoCli.ArduinoCore('localhost:50051', grpcLib.credentials.createInsecure());
client.Version({}, function(err, response){
console.log("Running version: ", response); //returns a version number
});
}
runArduino();
main();
The first time I run it, this is what I get (execution doesn't stop):
Running version: undefined
Once the daemon is up and I run it, I get this (I am able to access the server now and execution does end):
Running version: { version: '0.11.0' }
Error: Command failed: /Users/Herve/Desktop/MyStuff/ArduinoX/ArduinoX/arduino-cli_macos daemon
Failed to listen on TCP port: 50051. Address already in use.
at ChildProcess.exithandler (child_process.js:303:12)
at ChildProcess.emit (events.js:315:20)
at maybeClose (internal/child_process.js:1021:16)
at Socket.<anonymous> (internal/child_process.js:443:11)
at Socket.emit (events.js:315:20)
at Pipe.<anonymous> (net.js:674:12) {
killed: false,
code: 5,
signal: null,
cmd: '/Users/Herve/Desktop/MyStuff/ArduinoX/ArduinoX/arduino-cli_macos daemon'
}
I believe you should either await the exec or run main() in exec's callback. Right now your main() executes before child process is started.
exec(__dirname+"/arduino-cli_macos", ['daemon'],function(err, data)
{
console.log(err);
console.log(data);
});
This is why at the first run you're getting undefined. Child process is not killed automatically, I guess, this is why at second run you can do the RPC, but can't really start the child process again, as it's already running (and is occupying the 50051 port).
If your application is starting the child process, I believe it has to take care of killing it as well:
var childProcessHandler = exec(__dirname+"/arduino-cli_macos", ['daemon'],function(err, data)
{
console.log(err);
console.log(data);
});
// ... and later in your code:
childProcessHandler.kill()
This way you could start/stop the application without having to worry about cleaning the processes. Only thing which you have to consider is taking care of cleanup in case an exception occurs.
Edit okay, it appears that to start process as a daemon, you'd have to use spaw with detached option:
const { spawn } = require('child_process');
const child = spawn(__dirname+"/arduino-cli_macos", {
detached: true
});
child.unref();
Related
When initializing Nextjs via node -r and node --require the application takes 4-5 minutes to load. The telemetry script loads within the first 5 seconds so this issue is likely related to Nextjs or node. This contrasts calling without node require module's 30 second load time.
Without node require module:
"dev": "env-cmd -f environments/.env.development next dev",
With node require module:
"dev": "env-cmd -f environments/.env.development node --require ./tracing.js ./node_modules/next/dist/bin/next dev",
This implementation is based on ross-hagan's blog about instrument-nextjs-opentelemetry
Alternative to a custom server
I originally started off with a completely separate tracing.js script
with the contents of our start.js script without the startServer call.
This separates the telemetry SDK startup from the server. You can then
keep the Next.js built-in startup behaviours by using the node
--require (-r) to load in a module before starting the Next app.
In your npm run dev script in your package.json this looks like:
node -r tracing.js ./node_modules/.bin/next dev
I switched away from this after frustration getting the node command
to run in a Dockerfile as this was destined for a Google Kubernetes
Engine runtime. Also, some concern about use of the --require flag.
See if it works for you to do it this way, as a Next.js custom server
comes with some consequences documented in their docs!
I've tried two separate tracing.js without success in reducing load times.
tracing.js provided by open telemetry:
/* tracing.js */
// Require dependencies
const opentelemetry = require("#opentelemetry/sdk-node");
const { getNodeAutoInstrumentations } = require("#opentelemetry/auto-instrumentations-node");
const { diag, DiagConsoleLogger, DiagLogLevel } = require('#opentelemetry/api');
// For troubleshooting, set the log level to DiagLogLevel.DEBUG
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);
const sdk = new opentelemetry.NodeSDK({
traceExporter: new opentelemetry.tracing.ConsoleSpanExporter(),
instrumentations: [getNodeAutoInstrumentations()]
});
sdk.start()
As well as the customized tracing.js for jaeger:
const process = require('process');
const opentelemetry = require('#opentelemetry/sdk-node');
const {
getNodeAutoInstrumentations,
} = require('#opentelemetry/auto-instrumentations-node');
const { Resource } = require('#opentelemetry/resources');
const {
SemanticResourceAttributes,
} = require('#opentelemetry/semantic-conventions');
const { JaegerExporter } = require('#opentelemetry/exporter-jaeger');
const hostName = process.env.OTEL_TRACE_HOST || 'localhost';
const options = {
tags: [],
endpoint: `http://${hostName}:1234/api/traces`,
};
const traceExporter = new JaegerExporter(options);
// configure the SDK to export telemetry data to the console
// enable all auto-instrumentations from the meta package
const sdk = new opentelemetry.NodeSDK({
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: 'my_app',
}),
traceExporter,
instrumentations: [
getNodeAutoInstrumentations({
// Each of the auto-instrumentations
// can have config set here or you can
// npm install each individually and not use the auto-instruments
'#opentelemetry/instrumentation-http': {
ignoreIncomingPaths: [
// Pattern match to filter endpoints
// that you really want to stop altogether
'/ping',
// You can filter conditionally
// Next.js gets a little too chatty
// if you trace all the incoming requests
...(process.env.NODE_ENV !== 'production'
? [/^\/_next\/static.*/]
: []),
],
// This gives your request spans a more meaningful name
// than `HTTP GET`
requestHook: (span, request) => {
span.setAttributes({
name: `${request.method} ${request.url || request.path}`,
});
},
// Re-assign the root span's attributes
startIncomingSpanHook: (request) => {
return {
name: `${request.method} ${request.url || request.path}`,
'request.path': request.url || request.path,
};
},
},
}),
],
});
// initialize the SDK and register with the OpenTelemetry API
// this enables the API to record telemetry
sdk
.start()
.then(() => console.log('Tracing initialized'))
.catch((error) =>
console.log('Error initializing tracing and starting server', error)
);
// gracefully shut down the SDK on process exit
process.on('SIGTERM', () => {
sdk
.shutdown()
.then(() => console.log('Tracing terminated'))
.catch((error) => console.log('Error terminating tracing', error))
.finally(() => process.exit(0));
});
Separately, building and then serving does not speed up the load times either.
Check to see if you were experiencing the issue that was reported here [#opentelemetry/instrumentation] require performance grows linearly with each instrumentation plugin. Within this issue, there was a bug that caused instrumentation to be layered on top of itself repeatedly. They have since fixed the issue.
See also this answer.
I'm working with a temp file that's downloaded from a server. but when I ran it on macOS, it ran fine. and did what it was supposed to do. but when I ran it on windows, it keeps giving me an EBUSY error when trying to spawn the child.
I tried delaying the start of the file. tried to remove the chmod so it just runs on Linux and macOS. But still getting the Ebusy error. Am I doing something wrong?
Note:
I can launch the binary from another node instance outside. Like from a cmd. but launching it from the node instances that created it leads to a Ebusy error.
temp.open('', (err, info) => {
if (err) throw err;
console.log('File: ', info.path);
console.log('Filedescriptor: ', info.fd);
var data = fs.createWriteStream(info.path);
res.data.pipe(data);
data.on('close', async () => {
fs.chmodSync(info.path, 0o755);
await delay(1000);
var child = cp.spawn(info.path, [key, jwt], { stdio: ['inherit', 'inherit', 'inherit', 'ipc'] });
Error:
Error: spawn EBUSY
at ChildProcess.spawn (node:internal/child_process:415:11)
at Object.spawn (node:child_process:707:9)
at WriteStream.<anonymous>
at WriteStream.emit (node:events:394:28)
at emitCloseNT (node:internal/streams/destroy:138:10)
at processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -4082,
code: 'EBUSY',
syscall:
Edit:
I created a new module to try to spawn the child that way. it will be forked in the main process. but I'm still getting the same error in the fork. still the same error.
const cp = require('child_process')
const childpath = process.argv[2]
var argv = [];
for (let index = 3; index < process.argv.length; index++) {
const element = process.argv[index];
argv.push(element)
}
console.log(argv)
var child = cp.spawn(childpath, argv, {
stdio: ['inherit', 'inherit', 'inherit', 'ipc']
})
child.on('message', (msg) => {
if (process.send) process.send(msg)
else process.emit('message', msg)
})
child.on('error', (msg) => {
console.log(msg)
process.emit('error', msg)
})
child.on('close', (msg) => {
process.exit(msg)
})
Update:
I've noticed that I cannot run the file until the process the created it is ended. meaning that the process that needs to use it is using it. but I cannot do the thing I want to do with it.EI spawn it
Update 2:
The next thing I tried was to create a symbolic link with the node. And still nothing but I noticed that the file doesn't become runnable until the main process is ended. which means that the process that I'm running has something to do with it. So I need to be able to unlike it from the process that's running. it seems like windows need to do some initialization after the file is made. and because it's still connected to the main process in some way it's not able to. Im guess this is why when I ended the process, the node icon showed up on the symlink and that's why I'm able to run it manually.
Final Update:
I'm gonna work on a file system module that acts like temp files but is just regular files. that the process is tracking. giving the function of temp files but not really. this will allow me to use the function of temp files but without the file being unable to be executed. It seems like making it a temp file made it so it could not be executed by the same process that created it. it seems like a windows file system thing and how they handle temp file permissions.
The way Temp files are handled in windows is different than macOS. to my best estimate, it seems like Temp files in windows are linked to the process that created them and aren't able to be changed or accessed by that process. The problem lies in trying to execute these temp files which windows has assigned to a process already making it unassignable. NodeJs needs to be able to access the file to be able to launch it and windows permission will not allow it until the main process is killed or ended. this unlinks the file from that process making it accessible to the rest of the system.
I have a node.js server set up and would like to send something to the command prompt on start up (because I couldn't figure out how to do it when the server was already running).
Do I do this from the server.js file? And what would the code look like?
I currently have this argument, which works when I normally access the windows command prompt on my own:
C:\Users\path\python.exe C:\Users\path\test.py
Server.js:
const exec= require("child_process").exec;
const pythonProcess =
exec("C:/Users/myPath/python.exe",
["C:/Users/myPath/test.py"]);
Everything compiles, and the server starts, but there is no evidence that the python file has been run. (When it normally runs, there is a document saved on my desktop) Am I doing something wrong?
Thank You.
It sounds like you're wanting to spawn a python process from a node process. You probably want to use Node's child_process.spawn() instead of child_process.exec(). Node's child_process.exec() is
asynchronous, and
buffer stdout and stderr, so you won't see any of their output until the callback on the exec() is invoked when the child process exits... and since you haven't given it a callback, you'll never see it.
Try something like this:
const childProcess = require('child_process') ;
const python = '.../python.exe' ;
const argv = [
'.../test.py',
] ;
const child = childProcess.spawn( python, argv, {
detached: false,
stdio: [
'inherit', // inherit STDIN from the parent process
'inherit', // inherit STDOUT from the parent process
'inherit', // inherit STDERR from the parent process
],
shell: false,
}) ;
child.on('error', ( err ) => {
console.log(`${python}: ${err.stack}`) ;
}) ;
child.on('exit', (code, signal) => {
console.log(`${python}: exited with exit code ${code}, signal '${signal}'`) ;
}) ;
But if all you're trying to do here is kick off a python program... why don't you do that via a shell script (on Windows, either a "batch" file (*.cmd) file or a Powershell script (*.ps)?
I have a gulp task that's hanging after completion.
Running --verbose on my command tells me that it completed the run, but it doesnt exit the process
// ./gulpfile.js/precon/index.js
const { preconMoveFile, preconExcelToMysql } = require('../../src/lib/preCon')
module.exports = async done => {
try {
await preconMoveFile()
await preconExcelToMysql()
done()
// return <-- doesnt work either per comment
} catch (e) {
throw e
}
}
below is the command line output, pardon my debug log output
C:\Users\ALilland\Documents\macros\experiments\forecast-scraper>set NODE_ENV=development& gulp precon --verbose
[12:33:57] Using gulpfile ~\Documents\macros\experiments\forecast-scraper\gulpfile.js
[12:33:57] Starting 'precon'...
[2019-07-30T19:33:57.264Z] DEBUG - [preCon.preconMoveFile] existing ./src/tmp/precon.xlsx removed
[2019-07-30T19:33:57.333Z] DEBUG - [preCon.preconMoveFile] copied new file to ./src/tmp/precon.xlsx
[2019-07-30T19:33:58.965Z] INFO - [initialize.db.caePreconForecast] created caePreconForecast
[2019-07-30T19:33:59.012Z] DEBUG - [preCon.preconExcelToMysql] added rows to db
[12:34:00] Finished 'precon' after 3.24 s
I tried several of the options shown in the docs with no success, and also tried several of the options here with no success
I have also tried calling process.exit(0) but that produces the Did you forget to signal async completion? warning that the docs mention
Edit:
I should also note my root index.js file that requires the precon task
// ./gulpfile.js/index.js
exports.default = require('./default')
exports.precon = require('./precon')
The hang does not result from gulp, there is a background process still running preventing gulp from closing
the culprit was an open mysql pool connection
const mysql = require('mysql')
const params = require('../config/mysql')
const db = mysql.createPool(params)
db.end() // <-- running this after each task resolves the issue
I try to start and stop a Tomcat instance on Windows using Electron.
I have modified the electron-quick-start project to stop my Tomcat instance with a batch file which calls Tomcat's shutdown.bat when all Electron windows are closed or before my application exits.
However, when I close my application, there is no output from the shutdownTomcat.on ("data", ...) and shutdownTomcat.on ("exit", ...) listeners. The only output is from a console.log ("Hello world") from my app.on ("before-quit", ...).
I chose this approach because I am new to Electron and want to test NodeJS' spawn's behaviour.
When I use my code outside of the app.on () listeners, the output is shown but my Tomcat instance is not being stopped. However, my startTomcat.bat file, which calls Tomcat's startup.bat, works without any problems.
I already read NodeJS' childProcess' documentation ("Spawning .bat and .cmd files on Windows"), but I cannot get it to work; which leads to my question, namely where the problem is and what I'm doing wrong.
My Main.js file I use for the main process:
const { app, BrowserWindow } = require('electron');
const { spawn } = require('child_process');
const path = require('path');
const start = path.resolve("./start.bat");
const startTomcat = spawn('cmd.exe', ['/c', start], {
cwd: process.cwd(),
detached: true,
});
// ...
app.on('before-quit',()=> {
const shutdownTomcat = spawn('cmd.exe', ['/c', stop], {
detached: true
// stdio: 'ignore'
});
shutdownTomcat.stdout.on('data', (data) => {
console.log("This data won't show up...", data);
});
shutdownTomcat.stderr.on('data', (data) => {
console.log(data.toString());
});
shutdownTomcat.on('exit', (code) => {
console.log(`Child Shutdown Tomcat exited with code ${code}`);
});
console.log("Hello World");
});
And finally, the batch file (stop.bat) I'm using to call Tomcat's shutdown.bat:
cd /D "%~dp0"
cd "..\\apache-tomcat\\bin"
call shutdown.bat
Most probably your electron application is already terminated by the time your events would have fired. Therefore there is no longer reference to your spawned process and listeners.
You can try event.preventDefault() to cancel the app quitting. Then you can explicitly app.quit() once you are done (but beware of infinitely looping through your before-quit listener; you may remove it or app.exit(0)).