Detach node child process after receiving a first message - javascript

I'm building a system with node.js where the user is able to spawn worker processes from a web interface, and I'm having trouble combining these two criteria for the system:
The web server process and the worker process should run independently, so that if the web server crashes, the worker process isn't affected.
The mother process should be able to receive at least one initial message from the worker process. After that message, I want to log the output from the worker process to a file.
I've been able to accomplish both criteria by themselves, but not together. Here's what the relevant parts of my code currently looks like:
function spawnWorker (id, callback) {
var worker = spawn("node", [require.resolve("worker"), id], {
detached: true,
stdio: ["ignore", "pipe", "pipe"]
});
worker.unref();
var logfile = fs.createWriteStream("foo.log");
worker.stdout.once("data", function (data) {
worker.stdout.pipe(logfile);
worker.stderr.pipe(logfile);
callback(undefined, data.toString());
});
}
This meets criteria 2 (ie. it receives the first message from the worker and redirects all following output to a file), but if I kill the mother process, the worker process is killed as well, even though the output has been redirected.
My guess is that this is because the worker's stdout and stderr are piped to the mother process before they're sent to the file stream. If I were to pass in a stream when spawning the process, according to the docs, node would instead duplicate that stream in the worker process:
The stream's underlying file descriptor is duplicated in the child process[...]
Is there a way I can remove the connection between the mother process and the worker process after having received the first message? Or should I use some type of RPC to accomplish this, and if so, what would be a good choice for this one message once communication?

I solved this using node's net module to spawn a server on a random port that is passed to the worker when it's started (so it knows where to send a response when it's ready).
Here's an example of an implementation of my solution, to clarify, this meets the requirements of:
saving the log output of the worker process,
keeping the worker process detached from the mother process (so that if the latter crashes, it doesn't affect the worker)
server.js
var net = require("net"),
fs = require("fs"),
spawn = require("child_process").spawn,
randomPort = require("get-port");
randomPort(function (port) {
var server = net.createServer(function (connection) {
connection.on("data", function (data) {
server.close();
/**
* Response received from worker, this is where you would put
* your logic that returns a response to the client that initiated
* the request to the server.
*/
});
});
server.listen(port);
var logfile = fs.createWriteStream("log");
var worker = spawn("node", ["worker.js", port], {
detached: true,
stdio: ["ignore", logfile, logfile]
});
worker.unref();
worker.on("close", function (status, message) {
console.log("worker closed", status, message);
});
});
worker.js
var net = require("net");
var options = process.argv.slice(2),
rpcPort = Number(options[1]);
var client = net.connect({port: rpcPort}, function () {
client.end(JSON.stringify({started: true}}));
});

Related

SSE/Redis - how to recover messages sent when SSE goes offline

On a website I have a very simple Live chat setup that uses SSE/Redis and pub/sub structure.
The basic setup (without going into details) is:
Client-side using EventSource
Opens SSE connection and subscribes to live events sent by SSE daemon. Sends messages to an API endpoint
connect(hash, eventListener) {
const url = `${url}?client=$hash=${hash}`;
sseSource = new EventSource(url);
sseSource.onopen = function(e) {
reconnectFrequencySeconds = 1;
}
sseSource.onerror = err => {
this.closeSSEStream();
this.reconnectSSEStream(hash, eventListener);
};
sseSource.addEventListener('messages', event => {
const messages = JSON.parse(event.data);
eventListener(messages);
});
},
API endpoint
That stores message in the database and pushes it to a Redis channel.
Redis DB
That keeps and serves the messages.
Server-side SSE daemon
Subscribes client to a channel in a Redis DB and forwards messages to the subscribers using SSE stream.
const subscriber = redis.createClient();
subscriber.select(config.redisDatabase);
subscriber.on('message', function (channel, message) {
log(connectionId, 'Redis: new msg on channel: ' + channel, message);
let event = {
event: 'messages',
data: message
};
currentClient.connection.write(event);
});
The whole thing works pretty well, however, it is one tweak away from perfection.
During deploy we restart our workers (including SSE daemon) and while it goes offline users do not receive LIVE updates. It reconnects just fine but messages that have been sent during down time are lost (as daemon starts listening to messages on reconnect only).
My only idea for a workaround involves an overengineered solution where "lost" messages are collected with a separate API endpoint on reconnect and displayed to the user.
Is there an out-of-the-box way to receive messages that have been stored to Redis BEFORE subscribing to a channel? E.g. "pop" unprocessed messages or something like that?
when you have reconnected send request to check if you are new msg with time of last msg
and if you are newer msg send it in result msg to avoid new request

Socket.io disconnected unexpectedly

I have node.js service and angular client using socket.io to transport some message during long time http request.
Service:
export const socketArray: SocketIO.Socket[] = [];
export let socketMapping: {[socketId: string]: number} = {};
const socketRegister: hapi.Plugin<any> = {
register: (server) => {
const io: SocketIO.Server = socket(server.listener);
// Whenever a session connected to socket, create a socket object and add it to socket array
io.on("connection", (socket) => {
console.log(`socket ${socket.id} connected`);
logger.info(`socket ${socket.id} connected`);
// Only put socket object into array if init message received
socket.on("init", msg => {
logger.info(`socket ${socket.id} initialized`);
socketArray.push(socket);
socketMapping[socket.id] = msg;
});
// Remove socket object from socket array when disconnected
socket.on("disconnect", (reason) => {
console.log(`socket ${socket.id} disconnected because: ${reason}`)
logger.info(`socket ${socket.id} disconnected because: ${reason}`);
for(let i = 0; i < socketArray.length; i ++) {
if(socketArray[i] === socket) {
socketArray.splice(i, 1);
return;
}
}
});
});
},
name: "socketRegister",
version: "1.0"
}
export const socketSender = async (socketId: string, channel: string, content: SocketMessage) => {
try {
// Add message to db here
// await storeMessage(socketMapping[socketId], content);
// Find corresponding socket and send message
logger.info(`trying sending message to ${socketId}`);
for (let i = 0; i < socketArray.length; i ++) {
if (socketArray[i].id === socketId) {
socketArray[i].emit(channel, JSON.stringify(content));
logger.info(`socket ${socketId} send message to ${channel}`);
if (content.isFinal == true) {
// TODO: delete all messages of the process if isFinal is true
await deleteProcess(content.processId);
}
return;
}
}
} catch (err) {
logger.error("Socket sender error: ", err.message);
}
};
Client:
connectSocket() {
if (!this.socket) {
try {
this.socket = io(socketUrl);
this.socket.emit('init', 'some-data');
} catch (err) {
console.log(err);
}
} else if (this.socket.disconnected) {
this.socket.connect();
this.socket.emit('init', 'some-data');
}
this.socket.on('some-channel', (data) => {
// Do something
});
this.socket.on('disconnect', (data) => {
console.log(data);
});
}
They usually work fine but produce disconnection error randomly. From my log file, we can see this:
2018-07-21T00:20:28.209Z[x]INFO: socket 8jBh7YC4A1btDTo_AAAN connected
2018-07-21T00:20:28.324Z[x]INFO: socket 8jBh7YC4A1btDTo_AAAN initialized
2018-07-21T00:21:48.314Z[x]INFO: socket 8jBh7YC4A1btDTo_AAAN disconnected because: ping timeout
2018-07-21T00:21:50.849Z[x]INFO: socket C6O7Vq38ygNiwGHcAAAO connected
2018-07-21T00:23:09.345Z[x]INFO: trying sending message to C6O7Vq38ygNiwGHcAAAO
And at the same time of disconnect message, front-end also noticed a disconnect event which saying transport close.
From the log, we can get the work flow is this:
Front-end started a socket connection and sent an init message to back-end. It also save the socket.
Back-end detected the connection and received init message
Back-end put the socket to the array so that it can be used anytime anywhere
The first socket was disconnected unexpectedly and another connection is published without front-end's awareness so front-end never send a message to initialize it.
Since front-end's saved socket is not changed, it used the old socket id when made http request. As a result, back-end sent message with the old socket which was already removed from socket array.
The situation doesn't happen frequently. Does anyone know what could cause the disconnect and unknown connect issue?
It really depends what "long time http request" is doing. node.js runs your Javascript as a single thread. That means it can literally only do one thing at a time. But, since many things that servers do are I/O related (read from a database, get data from a file, get data from another server, etc...) and node.js uses event-driven asynchronous I/O, it can often have many balls in the air at the same time so it appears to be working on lots of requests at once.
But, if your complex http request is CPU-intensive, using lots of CPU, then it's hogging the single Javascript thread and nothing else can get done while it is hogging the CPU. That means that all incoming HTTP or socket.io requests have to wait in a queue until the one node.js Javascript thread is free so it can grab the next event from the event queue and start to process that incoming request.
We could only really help you more specifically if we could see the code for this "very complex http request".
The usual way around CPU-hogging things in node.js is to offload CPU-intensive stuff to other processes. If it's mostly just this one piece of code that causes the problem, you can spin up several child processes (perhaps as many as the number of CPUs you have in your server) and then feed them the CPU-intensive work and leave your main node.js process free to handle incoming (non-CPU-intensive) requests with very low latency.
If you have multiple operations that might hog the CPU, then you either have to farm them all out to child processes (probably via some sort of work queue) or you can deploy clustering. The challenge with clustering is that a given socket.io connection will be to one particular server in your cluster and if it's that process that just happens to be executing a CPU-hogging operation, then all the socket.io connections assigned to that server would have bad latency. So, regular clustering is probably not so good for this type of issue. The work-queue and multiple specialized child processes to handle CPU-intensive work are probably better because those processes won't have any outside socket.io connections that they are responsible for.
Also, you should know that if you're using synchronous file I/O, that blocks the entire node.js Javascript thread. node.js can not run any other Javascript during a synchronous file I/O operation. node.js gets its scalability and its ability to have many operations in flight at the same from its asynchronous I/O model. If you use synchronous I/O, you completely break that and ruin scalability and responsiveness.
Synchronous file I/O belongs only in server startup code or in a single purpose script (not a server). It should never be used while processing a request in a server.
Two ways to make asynchronous file I/O a little more tolerable are by using streams or by using async/await with promisified fs methods.

Close mubsub client after emitting with socket.io-mongodb-emitter

I'm trying to emit some message into a socket.io room using socket.io-mongodb-emitter. My problem is that, my script never exists, because it keeps a connection alive to mongodb.
Please check my examples below, and give me some advice, how can I make the script close mongodb connection and let itself exit.
emitter.js this script emits the message, the client gets it, but the process doesn't exit.
var io = require('socket.io-mongodb-emitter')("mongodb://localhost:27017/test");
io.in('someRoom').emit('someMessage');
emitter-2.js this script exits, but the client never gets the message.
var io = require('socket.io-mongodb-emitter')("mongodb://localhost:27017/test");
io.in('someRoom').emit('someMessage');
io.client.close();
emitter-3.js this one works perfectly, the client gets the message, and the process exits. But setTimeout is an extremely bad solution, there must be some proper way to let this process exit by itself.
var io = require('socket.io-mongodb-emitter')("mongodb://localhost:27017/test");
io.in('someRoom').emit('someMessage');
setTimeout(function () {
io.client.close();
},100);
Maybe use a callback to get an acknowledgement that the client received the message, then call the close function
server:
var message = "hello";
io.in('someRoom').emit('someMessage', message, function(data){
console.log(data);
io.client.close();
});
client:
socket.on('someMessage', function(message, callback){
// do something with message here
callback('message received');
});
You need to use the .emit's ability to receive an acknowledgement that the message was sent.
Do like so:
io.in('someRoom').emit('someMessage',function(){
console.log('message was sent');
io.client.close();
);
On Server you need to call the function when it is received.
socket.on('someMessage', function (name, fn) {
fn('received');
});
Documentation here

Access grpc stream variable for long-running process in Node

I am using Node.js to connect to a server using gRPC that performs a long running task.
The server sends a unidirectional stream to the client (the Node.js app) while the job is in progress. I need to implement a Stop button and am told that closing the gRPC stream will stop the job in progress.
This is currently my code:
let express = require('express'),
router = express.Router(),
grpc = require('grpc'),
srv = grpc.load(__dirname + '/job_handler.proto').ns;
let startJob = (jobID, parameters) => srv.createJob(jobID, parameters);
router.post('/jobs', (req, res) => {
let lengthyOperation = startJob(jobID, parameters);
lengthyOperation.on('data', (data) => {
console.log(`Data from lengthy operation: ${data}`);
});
lengthyOperation.on('end', () =>
console.log('Lengthy operation completed');
});
res.setHeader('Location', `/jobs/${jobID}`);
res.status(202).send();
});
As you can see, I send an HTTP 202 response to the client upon creating the job and it continues asynchronously in the background.
Questions:
How do I close the stream?
How do I access the lengthyOperation variable to do so?
The lengthyOperation object has a cancel method that cancels the call. So, when you want to stop the stream, just call lengthyOperation.cancel().
Note that when you do this, it will cause the call to end with an error. I would recommend adding a lengthyOperation.on('error', ...) handler to handle that error.

Send message to specific client with socket.io and node.js

I'm working with socket.io and node.js and until now it seems pretty good, but I don't know how to send a message from the server to an specific client, something like this:
client.send(message, receiverSessionId)
But neither the .send() nor the .broadcast() methods seem to supply my need.
What I have found as a possible solution, is that the .broadcast() method accepts as a second parameter an array of SessionIds to which not send the message, so I could pass an array with all the SessionIds connected at that moment to the server, except the one I wish send the message, but I feel there must be a better solution.
Any ideas?
Ivo Wetzel's answer doesn't seem to be valid in Socket.io 0.9 anymore.
In short you must now save the socket.id and use io.sockets.socket(savedSocketId).emit(...) to send messages to it.
This is how I got this working in clustered Node.js server:
First you need to set Redis store as the store so that messages can go cross processes:
var express = require("express");
var redis = require("redis");
var sio = require("socket.io");
var client = redis.createClient()
var app = express.createServer();
var io = sio.listen(app);
io.set("store", new sio.RedisStore);
// In this example we have one master client socket
// that receives messages from others.
io.sockets.on('connection', function(socket) {
// Promote this socket as master
socket.on("I'm the master", function() {
// Save the socket id to Redis so that all processes can access it.
client.set("mastersocket", socket.id, function(err) {
if (err) throw err;
console.log("Master socket is now" + socket.id);
});
});
socket.on("message to master", function(msg) {
// Fetch the socket id from Redis
client.get("mastersocket", function(err, socketId) {
if (err) throw err;
io.sockets.socket(socketId).emit(msg);
});
});
});
I omitted the clustering code here, because it makes this more cluttered, but it's trivial to add. Just add everything to the worker code. More docs here http://nodejs.org/api/cluster.html
each socket joins a room with a socket id for a name, so you can just
io.to('socket#id').emit('hey')
docs: http://socket.io/docs/rooms-and-namespaces/#default-room
The simplest, most elegant way
verified working with socket.io v3.1.1
It's as easy as:
client.emit("your message");
And that's it. Ok, but how does it work?
Minimal working example
Here's an example of a simple client-server interaction where each client regularly receives a message containing a sequence number. There is a unique sequence for each client and that's where the "I need to send a message to a particular client" comes into play.
Server
server.js
const
{Server} = require("socket.io"),
server = new Server(8000);
let
sequenceNumberByClient = new Map();
// event fired every time a new client connects:
server.on("connection", (socket) => {
console.info(`Client connected [id=${socket.id}]`);
// initialize this client's sequence number
sequenceNumberByClient.set(socket, 1);
// when socket disconnects, remove it from the list:
socket.on("disconnect", () => {
sequenceNumberByClient.delete(socket);
console.info(`Client gone [id=${socket.id}]`);
});
});
// sends each client its current sequence number
setInterval(() => {
for (const [client, sequenceNumber] of sequenceNumberByClient.entries()) {
client.emit("seq-num", sequenceNumber);
sequenceNumberByClient.set(client, sequenceNumber + 1);
}
}, 1000);
The server starts listening on port 8000 for incoming connections. As soon as a new connection is established, that client is added to a map that keeps track of its sequence number. The server also listens for the disconnect event to remove the client from the map when it leaves.
Each and every second, a timer is fired. When it does, the server walks through the map and sends a message to every client with their current sequence number, incrementing it right after. That's all that is to it. Easy peasy.
Client
The client part is even simpler. It just connects to the server and listens for the seq-num message, printing it to the console every time it arrives.
client.js
const
io = require("socket.io-client"),
ioClient = io.connect("http://localhost:8000");
ioClient.on("seq-num", (msg) => console.info(msg));
Running the example
Install the required libraries:
npm install socket.io#3.1.1 socket.io-client#3.1.1
Run the server:
node server
Open other terminal windows and spawn as many clients as you want by running:
node client
I have also prepared a gist with the full code here.
Well you have to grab the client for that (surprise), you can either go the simple way:
var io = io.listen(server);
io.clients[sessionID].send()
Which may break, I doubt it, but it's always a possibility that io.clients might get changed, so use the above with caution
Or you keep track of the clients yourself, therefore you add them to your own clients object in the connection listener and remove them in the disconnect listener.
I would use the latter one, since depending on your application you might want to have more state on the clients anyway, so something like clients[id] = {conn: clientConnect, data: {...}} might do the job.
You can use
//send message only to sender-client
socket.emit('message', 'check this');
//or you can send to all listeners including the sender
io.emit('message', 'check this');
//send to all listeners except the sender
socket.broadcast.emit('message', 'this is a message');
//or you can send it to a room
socket.broadcast.to('chatroom').emit('message', 'this is the message to all');
In 1.0 you should use:
io.sockets.connected[socketid].emit();
Whatever version we are using if we just console.log() the "io" object that we use in our server side nodejs code, [e.g. io.on('connection', function(socket) {...});], we can see that "io" is just an json object and there are many child objects where the socket id and socket objects are stored.
I am using socket.io version 1.3.5, btw.
If we look in the io object, it contains,
sockets:
{ name: '/',
server: [Circular],
sockets: [ [Object], [Object] ],
connected:
{ B5AC9w0sYmOGWe4fAAAA: [Object],
'hWzf97fmU-TIwwzWAAAB': [Object] },
here we can see the socketids "B5AC9w0sYmOGWe4fAAAA" etc. So, we can do,
io.sockets.connected[socketid].emit();
Again, on further inspection we can see segments like,
eio:
{ clients:
{ B5AC9w0sYmOGWe4fAAAA: [Object],
'hWzf97fmU-TIwwzWAAAB': [Object] },
So, we can retrieve a socket from here by doing
io.eio.clients[socketid].emit();
Also, under engine we have,
engine:
{ clients:
{ B5AC9w0sYmOGWe4fAAAA: [Object],
'hWzf97fmU-TIwwzWAAAB': [Object] },
So, we can also write,
io.engine.clients[socketid].emit();
So, I guess we can achieve our goal in any of the 3 ways I listed above,
io.sockets.connected[socketid].emit();
OR
io.eio.clients[socketid].emit();
OR
io.engine.clients[socketid].emit();
You can do this
On server.
global.io=require("socket.io")(server);
io.on("connection",function(client){
console.log("client is ",client.id);
//This is handle by current connected client
client.emit('messages',{hello:'world'})
//This is handle by every client
io.sockets.emit("data",{data:"This is handle by every client"})
app1.saveSession(client.id)
client.on("disconnect",function(){
app1.deleteSession(client.id)
console.log("client disconnected",client.id);
})
})
//And this is handle by particular client
var socketId=req.query.id
if(io.sockets.connected[socketId]!=null) {
io.sockets.connected[socketId].emit('particular User', {data: "Event response by particular user "});
}
And on client, it is very easy to handle.
var socket=io.connect("http://localhost:8080/")
socket.on("messages",function(data){
console.log("message is ",data);
//alert(data)
})
socket.on("data",function(data){
console.log("data is ",data);
//alert(data)
})
socket.on("particular User",function(data){
console.log("data from server ",data);
//alert(data)
})
As of version 1.4.5, be sure you provide a properly prefixed socketId in io.to().
I was taking the socketId the Client logged to debug and it was without prefix so I ended up searching forever till I found out! So you might have to do it like this if the Id you have is not prefixed:
io.to('/#' + socketId).emit('myevent', {foo: 'bar'});
io.sockets.sockets[socket.id].emit(...) worked for me in v0.9
Also you can keep clients refferences. But this makes your memmory busy.
Create an empty object and set your clients into it.
const myClientList = {};
server.on("connection", (socket) => {
console.info(`Client connected [id=${socket.id}]`);
myClientList[socket.id] = socket;
});
socket.on("disconnect", (socket) => {
delete myClientList[socket.id];
});
then call your specific client by id from the object
myClientList[specificId].emit("blabla","somedata");
Socket.IO allows you to “namespace” your sockets, which essentially means assigning different endpoints or paths.
This might help:
http://socket.io/docs/rooms-and-namespaces/

Categories