On a website I have a very simple Live chat setup that uses SSE/Redis and pub/sub structure.
The basic setup (without going into details) is:
Client-side using EventSource
Opens SSE connection and subscribes to live events sent by SSE daemon. Sends messages to an API endpoint
connect(hash, eventListener) {
const url = `${url}?client=$hash=${hash}`;
sseSource = new EventSource(url);
sseSource.onopen = function(e) {
reconnectFrequencySeconds = 1;
}
sseSource.onerror = err => {
this.closeSSEStream();
this.reconnectSSEStream(hash, eventListener);
};
sseSource.addEventListener('messages', event => {
const messages = JSON.parse(event.data);
eventListener(messages);
});
},
API endpoint
That stores message in the database and pushes it to a Redis channel.
Redis DB
That keeps and serves the messages.
Server-side SSE daemon
Subscribes client to a channel in a Redis DB and forwards messages to the subscribers using SSE stream.
const subscriber = redis.createClient();
subscriber.select(config.redisDatabase);
subscriber.on('message', function (channel, message) {
log(connectionId, 'Redis: new msg on channel: ' + channel, message);
let event = {
event: 'messages',
data: message
};
currentClient.connection.write(event);
});
The whole thing works pretty well, however, it is one tweak away from perfection.
During deploy we restart our workers (including SSE daemon) and while it goes offline users do not receive LIVE updates. It reconnects just fine but messages that have been sent during down time are lost (as daemon starts listening to messages on reconnect only).
My only idea for a workaround involves an overengineered solution where "lost" messages are collected with a separate API endpoint on reconnect and displayed to the user.
Is there an out-of-the-box way to receive messages that have been stored to Redis BEFORE subscribing to a channel? E.g. "pop" unprocessed messages or something like that?
when you have reconnected send request to check if you are new msg with time of last msg
and if you are newer msg send it in result msg to avoid new request
I created an observable for my websocket connection using WebSocketSubject from rxjs. So far so good the server-client communication is working. Now the thing is: I can't distinguish between the origins of the message in my client. I send messages by calling next() on the subject. All subscriptions on the client get those messages too. How can I send messages only to the server instead?
The implementation mainly stems from this article: https://medium.com/factory-mind/angular-websocket-node-31f421c753ff
My code:
socket$: WebSocketSubject<any>;
constructor() {
this.socket$ = WebSocketSubject.create(SOCKET_URL);
this.socket$.subscribe(
(message) => console.log('<-- ' + message),
(err) => console.error('Error on WebSocket:', err),
() => console.warn('Completed!')
);
}
send(message: SocketMessage) {
const tmp: any = {};
tmp.type = message.type;
tmp.payload = message.payload;
// This will be received by the server but also by client subscriptions = bad
this.socket$.next(JSON.stringify(tmp));
}
I found a solution trying to have your described behavior :
RxJS6's WebSocket's documentation states that
Calling next does not affect subscribers of WebSocketSubject - they have no information that something was sent to the server (unless of course the server responds somehow to a message).
Thus, by using RxJS 6 instead of RxJS 5 you should not have the described behavior.
I send JSON requests one by one to the nodejs server. After 6th request, server can't reply to the client immediately and then it takes a little while(15 seconds or little bit more and send back to me answer 200 ok) It occurs a writing json value into MongoDB and time is important option for me in terms with REST call. How can I find the error in this case? (which tool or script code can help me?) My server side code is like that
var controlPathDatabaseSave = "/save";
app.use('/', function(req, res) {
console.log("req body app use", req.body);
var str= req.path;
if(str.localeCompare(controlPathDatabaseSave) == 0)
{
console.log("controlPathDatabaseSave");
mongoDbHandleSave(req.body);
res.setHeader('Content-Type', 'application/json');
res.write('Message taken: \n');
res.write('Everything all right with database saving');
res.send("OK");
console.log("response body", res.body);
}
});
My client side code as below:
function saveDatabaseData()
{
console.log("saveDatabaseData");
var oReq = new XMLHttpRequest();
oReq.open("POST", "http://192.168.80.143:2800/save", true);
oReq.setRequestHeader("Content-type", "application/json;charset=UTF-8");
oReq.onreadystatechange = function() {//Call a function when the state changes.
if(oReq.readyState == 4 && oReq.status == 200) {
console.log("http responseText", oReq.responseText);
}
}
oReq.send(JSON.stringify({links: links, nodes: nodes}));
}
--Mongodb save code
function mongoDbHandleSave(reqParam){
//Connect to the db
MongoClient.connect(MongoDBURL, function(err, db)
{
if(!err)
{
console.log("We are connected in accordance with saving");
} else
{
return console.dir(err);
}
/*
db.createCollection('user', {strict:true}, function(err, collection) {
if(err)
return console.dir(err);
});
*/
var collection = db.collection('user');
//when saving into database only use req.body. Skip JSON.stringify() function
var doc = reqParam;
collection.update(doc, doc, {upsert:true});
});
}
You can see my REST call in google chrome developer editor. (First six call has 200 ok. Last one is in pending state)
--Client output
--Server output
Thanks in advance,
Since it looks like these are Ajax requests from a browser, each browser has a limit on the number of simultaneous connections it will allow to the same host. Browsers have varied that setting over time, but it is likely in the 4-6 range. So, if you are trying to run 6 simultaneous ajax calls to the same host, then you may be running into that limit. What the browser does is hold off on sending the latest ones until the first ones finish (thus avoiding sending too many at once).
The general idea here is to protect servers from getting beat up too much by one single client and thus allow the load to be shared across many clients more fairly. Of course, if your server has nothing else to do, it doesn't really need protecting from a few more connections, but this isn't an interactive system, it's just hard-wired to a limit.
If there are any other requests in process (loading images or scripts or CSS stylesheets) to the same origin, those will count to the limit too.
If you run this in Chrome and you open the network tab of the debugger, you could actually see on the timline exactly when a given request was sent and when its response was received. This should show you immediately whether the later requests are being held up at the browser or at the server.
Here's an article on the topic: Maximum concurrent connections to the same domain for browsers.
Also, keep in mind that, depending upon what your requests do on the server and how the server is structured, there may be a maximum number of server requests that can efficiently processed at once. For example, if you had a blocking, threaded server that was configured with one thread for each of four CPUs, then once the server has four requests going at once, it may have to queue the fifth request until the first one is done causing it to be delayed more than the others.
I'm building a system with node.js where the user is able to spawn worker processes from a web interface, and I'm having trouble combining these two criteria for the system:
The web server process and the worker process should run independently, so that if the web server crashes, the worker process isn't affected.
The mother process should be able to receive at least one initial message from the worker process. After that message, I want to log the output from the worker process to a file.
I've been able to accomplish both criteria by themselves, but not together. Here's what the relevant parts of my code currently looks like:
function spawnWorker (id, callback) {
var worker = spawn("node", [require.resolve("worker"), id], {
detached: true,
stdio: ["ignore", "pipe", "pipe"]
});
worker.unref();
var logfile = fs.createWriteStream("foo.log");
worker.stdout.once("data", function (data) {
worker.stdout.pipe(logfile);
worker.stderr.pipe(logfile);
callback(undefined, data.toString());
});
}
This meets criteria 2 (ie. it receives the first message from the worker and redirects all following output to a file), but if I kill the mother process, the worker process is killed as well, even though the output has been redirected.
My guess is that this is because the worker's stdout and stderr are piped to the mother process before they're sent to the file stream. If I were to pass in a stream when spawning the process, according to the docs, node would instead duplicate that stream in the worker process:
The stream's underlying file descriptor is duplicated in the child process[...]
Is there a way I can remove the connection between the mother process and the worker process after having received the first message? Or should I use some type of RPC to accomplish this, and if so, what would be a good choice for this one message once communication?
I solved this using node's net module to spawn a server on a random port that is passed to the worker when it's started (so it knows where to send a response when it's ready).
Here's an example of an implementation of my solution, to clarify, this meets the requirements of:
saving the log output of the worker process,
keeping the worker process detached from the mother process (so that if the latter crashes, it doesn't affect the worker)
server.js
var net = require("net"),
fs = require("fs"),
spawn = require("child_process").spawn,
randomPort = require("get-port");
randomPort(function (port) {
var server = net.createServer(function (connection) {
connection.on("data", function (data) {
server.close();
/**
* Response received from worker, this is where you would put
* your logic that returns a response to the client that initiated
* the request to the server.
*/
});
});
server.listen(port);
var logfile = fs.createWriteStream("log");
var worker = spawn("node", ["worker.js", port], {
detached: true,
stdio: ["ignore", logfile, logfile]
});
worker.unref();
worker.on("close", function (status, message) {
console.log("worker closed", status, message);
});
});
worker.js
var net = require("net");
var options = process.argv.slice(2),
rpcPort = Number(options[1]);
var client = net.connect({port: rpcPort}, function () {
client.end(JSON.stringify({started: true}}));
});
I'm working with socket.io and node.js and until now it seems pretty good, but I don't know how to send a message from the server to an specific client, something like this:
client.send(message, receiverSessionId)
But neither the .send() nor the .broadcast() methods seem to supply my need.
What I have found as a possible solution, is that the .broadcast() method accepts as a second parameter an array of SessionIds to which not send the message, so I could pass an array with all the SessionIds connected at that moment to the server, except the one I wish send the message, but I feel there must be a better solution.
Any ideas?
Ivo Wetzel's answer doesn't seem to be valid in Socket.io 0.9 anymore.
In short you must now save the socket.id and use io.sockets.socket(savedSocketId).emit(...) to send messages to it.
This is how I got this working in clustered Node.js server:
First you need to set Redis store as the store so that messages can go cross processes:
var express = require("express");
var redis = require("redis");
var sio = require("socket.io");
var client = redis.createClient()
var app = express.createServer();
var io = sio.listen(app);
io.set("store", new sio.RedisStore);
// In this example we have one master client socket
// that receives messages from others.
io.sockets.on('connection', function(socket) {
// Promote this socket as master
socket.on("I'm the master", function() {
// Save the socket id to Redis so that all processes can access it.
client.set("mastersocket", socket.id, function(err) {
if (err) throw err;
console.log("Master socket is now" + socket.id);
});
});
socket.on("message to master", function(msg) {
// Fetch the socket id from Redis
client.get("mastersocket", function(err, socketId) {
if (err) throw err;
io.sockets.socket(socketId).emit(msg);
});
});
});
I omitted the clustering code here, because it makes this more cluttered, but it's trivial to add. Just add everything to the worker code. More docs here http://nodejs.org/api/cluster.html
each socket joins a room with a socket id for a name, so you can just
io.to('socket#id').emit('hey')
docs: http://socket.io/docs/rooms-and-namespaces/#default-room
The simplest, most elegant way
verified working with socket.io v3.1.1
It's as easy as:
client.emit("your message");
And that's it. Ok, but how does it work?
Minimal working example
Here's an example of a simple client-server interaction where each client regularly receives a message containing a sequence number. There is a unique sequence for each client and that's where the "I need to send a message to a particular client" comes into play.
Server
server.js
const
{Server} = require("socket.io"),
server = new Server(8000);
let
sequenceNumberByClient = new Map();
// event fired every time a new client connects:
server.on("connection", (socket) => {
console.info(`Client connected [id=${socket.id}]`);
// initialize this client's sequence number
sequenceNumberByClient.set(socket, 1);
// when socket disconnects, remove it from the list:
socket.on("disconnect", () => {
sequenceNumberByClient.delete(socket);
console.info(`Client gone [id=${socket.id}]`);
});
});
// sends each client its current sequence number
setInterval(() => {
for (const [client, sequenceNumber] of sequenceNumberByClient.entries()) {
client.emit("seq-num", sequenceNumber);
sequenceNumberByClient.set(client, sequenceNumber + 1);
}
}, 1000);
The server starts listening on port 8000 for incoming connections. As soon as a new connection is established, that client is added to a map that keeps track of its sequence number. The server also listens for the disconnect event to remove the client from the map when it leaves.
Each and every second, a timer is fired. When it does, the server walks through the map and sends a message to every client with their current sequence number, incrementing it right after. That's all that is to it. Easy peasy.
Client
The client part is even simpler. It just connects to the server and listens for the seq-num message, printing it to the console every time it arrives.
client.js
const
io = require("socket.io-client"),
ioClient = io.connect("http://localhost:8000");
ioClient.on("seq-num", (msg) => console.info(msg));
Running the example
Install the required libraries:
npm install socket.io#3.1.1 socket.io-client#3.1.1
Run the server:
node server
Open other terminal windows and spawn as many clients as you want by running:
node client
I have also prepared a gist with the full code here.
Well you have to grab the client for that (surprise), you can either go the simple way:
var io = io.listen(server);
io.clients[sessionID].send()
Which may break, I doubt it, but it's always a possibility that io.clients might get changed, so use the above with caution
Or you keep track of the clients yourself, therefore you add them to your own clients object in the connection listener and remove them in the disconnect listener.
I would use the latter one, since depending on your application you might want to have more state on the clients anyway, so something like clients[id] = {conn: clientConnect, data: {...}} might do the job.
You can use
//send message only to sender-client
socket.emit('message', 'check this');
//or you can send to all listeners including the sender
io.emit('message', 'check this');
//send to all listeners except the sender
socket.broadcast.emit('message', 'this is a message');
//or you can send it to a room
socket.broadcast.to('chatroom').emit('message', 'this is the message to all');
In 1.0 you should use:
io.sockets.connected[socketid].emit();
Whatever version we are using if we just console.log() the "io" object that we use in our server side nodejs code, [e.g. io.on('connection', function(socket) {...});], we can see that "io" is just an json object and there are many child objects where the socket id and socket objects are stored.
I am using socket.io version 1.3.5, btw.
If we look in the io object, it contains,
sockets:
{ name: '/',
server: [Circular],
sockets: [ [Object], [Object] ],
connected:
{ B5AC9w0sYmOGWe4fAAAA: [Object],
'hWzf97fmU-TIwwzWAAAB': [Object] },
here we can see the socketids "B5AC9w0sYmOGWe4fAAAA" etc. So, we can do,
io.sockets.connected[socketid].emit();
Again, on further inspection we can see segments like,
eio:
{ clients:
{ B5AC9w0sYmOGWe4fAAAA: [Object],
'hWzf97fmU-TIwwzWAAAB': [Object] },
So, we can retrieve a socket from here by doing
io.eio.clients[socketid].emit();
Also, under engine we have,
engine:
{ clients:
{ B5AC9w0sYmOGWe4fAAAA: [Object],
'hWzf97fmU-TIwwzWAAAB': [Object] },
So, we can also write,
io.engine.clients[socketid].emit();
So, I guess we can achieve our goal in any of the 3 ways I listed above,
io.sockets.connected[socketid].emit();
OR
io.eio.clients[socketid].emit();
OR
io.engine.clients[socketid].emit();
You can do this
On server.
global.io=require("socket.io")(server);
io.on("connection",function(client){
console.log("client is ",client.id);
//This is handle by current connected client
client.emit('messages',{hello:'world'})
//This is handle by every client
io.sockets.emit("data",{data:"This is handle by every client"})
app1.saveSession(client.id)
client.on("disconnect",function(){
app1.deleteSession(client.id)
console.log("client disconnected",client.id);
})
})
//And this is handle by particular client
var socketId=req.query.id
if(io.sockets.connected[socketId]!=null) {
io.sockets.connected[socketId].emit('particular User', {data: "Event response by particular user "});
}
And on client, it is very easy to handle.
var socket=io.connect("http://localhost:8080/")
socket.on("messages",function(data){
console.log("message is ",data);
//alert(data)
})
socket.on("data",function(data){
console.log("data is ",data);
//alert(data)
})
socket.on("particular User",function(data){
console.log("data from server ",data);
//alert(data)
})
As of version 1.4.5, be sure you provide a properly prefixed socketId in io.to().
I was taking the socketId the Client logged to debug and it was without prefix so I ended up searching forever till I found out! So you might have to do it like this if the Id you have is not prefixed:
io.to('/#' + socketId).emit('myevent', {foo: 'bar'});
io.sockets.sockets[socket.id].emit(...) worked for me in v0.9
Also you can keep clients refferences. But this makes your memmory busy.
Create an empty object and set your clients into it.
const myClientList = {};
server.on("connection", (socket) => {
console.info(`Client connected [id=${socket.id}]`);
myClientList[socket.id] = socket;
});
socket.on("disconnect", (socket) => {
delete myClientList[socket.id];
});
then call your specific client by id from the object
myClientList[specificId].emit("blabla","somedata");
Socket.IO allows you to “namespace” your sockets, which essentially means assigning different endpoints or paths.
This might help:
http://socket.io/docs/rooms-and-namespaces/