I am using the library node_redis as the client for a micro-service message client that I am writing. Clients get messages from their application in their outbox that they need to send to other services. Everything is working great but I am trying to build some resilience on the part of the application that uses the redis client to communicate with the redis-server.
My idea is that the redis client-server connection status should be highly available to the clients. I mean that if a connection goes down, I would like to know within the second instead of the default timeout of 300 seconds. Currently I am using the free redislabs tier hosted on AWS, but I should be moving this to run in its own container on my kubernetes cluster.
I need to know the state of client connections in the network because I would like to not send messages when the network conditions are not right and not rely on the error handling to handle this sort of event. Knowing how often and when these high latency events occur will also help me diagnose and improve my network and my microservices.
Note: I wanted to set the connect_timeout value in the client options but this is listed as deprecated.
Something like that?
var redis = require('redis');
var client = redis.createClient();
var reconnectAfter = 15000;
client.on( 'error', function () {
console.log( (new Date()) + " Redis: disconnect");
setTimeout( connect, reconnectAfter);
});
connect = function(){
client = redis.createClient();
}
Related
In a Node JS app, I want my server to be able to send a notification to clients, in some specific cases. Searching the net on the subject, I find Service Workers which seems to be what I need; but I am not totally sure.
Here is why. When I look at various tutorials and videos, it seems like the notifications always somewhat originates from some client. But this is not what I want. It should be the server deciding when and what to send as a notification, not some client. So comes my questions:
Am I misunderstanding the way to use Service Workers?
Is this the right path to have my server send a notification to clients?
If the answer to the last question is NO. Then what is the proper way?
Like Dimitar wrote earlier, you could use socket.io to send messages from the server to the client and vice versa. This can be done as simple as that:
//Server sent message:
io.on('connection', (socket) => {
socket.emit('notificationToClient', 'Message'); //message sent from server to client
});
});
//Client receives the message:
const socket = io.connect('http://localhost:3000');
socket.on('notificationToClient', (data) => { //received message
console.log(data);
});
The websocket API is a little less confusing than socket.io but IMO, I'd go with socket.io since it already deals with a lot of the heavy lifting like re-connecting, custom namespaces, rooms etc. There are several fantastic udemy courses on socket.io.
See the socket.io documentation here: https://socket.io/docs/
Update 5/9/2020:
Michel - Since you asked about how to implement Socket.io in your code I'll give you an example of how I implemented this technology in my most recent project. I was working on a web app that connects to ELK Alarm systems and is able to arm/disarm these systems as well as receive sensor input (chime signals) at any remote location. The backend is Node.js and the UI is React.
//client side code
import React, { Component, Suspense } from 'react';
import panels from "./panels.js"
const socketConnection = io("http://localhost:5000"); //Socket.io
function App() {
function handleClick(panel) {
socketConnection.emit("panelData", panel); //Socket.io - sends the connection data for the ELK alarm panel to the express server
socketConnection.on('data', (data) => { //Socket.io - receives data from the server.
//does something with the data
})
}
}
//server side code
const express = require('express');
const elkClient = require('elk-client');
const app = express();
const server = app.listen('5000', () => {
console.log('------Server Running on 5000---------');
});
let io = new sio(server);
io.on("connection", (socket) => { //boilerplate code - establishes the "handshake" with the client.
socket.on("panelData", async (msg) => { //receives the ELK connection parameters from the client. See client side code
const client = new ElkClient({
connection: {
site: msg.name,
host: msg.host,
}
})
await client.connect();
client.on("message", (elkMessage) => { // This is NOT Socket.io - server is listening for emitted data from the alarm system - beam breaks, alarms, arm/disarm etc. [See event emitter][2]
if(elkMessage.messageType == 'A' && elkMessage.subMessageType == 'S') {
const armingStatusReport = {elkMessage};
socket.emit("data", armingStatusReport); //Socket.io - emits the received data to the client.
}
})
I tried to simplify the code above to hit the point. The server is using the connection parameters that it receives from the client to connect to a remote alarm system. The server then waits and listens for incoming data with client.on()(event emitter - not socket.io). As the data is received, I'm using Socket.io to send the received data back to the client with socket.emit(); Due to the nature of how alarm systems work, data is sent event driven so the fetch api wouldn't really fit the bill but socket.io (or websockets) does.
Let me know if I can help with anything else. With this recent project, I have spent the last few months EXTENSIVELY dealing with socket.io, including namespaces, rooms etc. I also had to create a real-time monitoring app for performance management for my server utilizing socket.io since I implemented clustering. Feel free to contact me anytime!
The answer is WebSockets.
Check out Socket.IO. It will give you the possibility of receiving live notifications on the client and you'll have complete server control on them.
What about using twilio library to manage your notification job .twilio library
I have a vb.net application that opens a socket and listens on it.
I need to communicate via this socket to that application using a javascript running on a browser. That is i need to send some data on this socket so that the app which is listening on this socket can take that data, do some stuff using some remote calls and get some more data and put it back on the socket that my javascript needs to read and print it in the browser.
Ive tried, socket.io, websockify but none have proved to be useful.
Hence the question, is what i am trying even possible? Is there a way that a javascript running in a browser can connect to a tcp socket and send some data and listen on it for some more data response on the socket and print it to the browser.
If this is possible can some one point me in the right direction as to which would help me establish the goal.
As for your problem, currently you will have to depend on XHR or websockets for this.
Currently no popular browser has implemented any such raw sockets api for javascript that lets you create and access raw sockets, but a draft for the implementation of raw sockets api in JavaScript is under-way. Have a look at these links:
http://www.w3.org/TR/raw-sockets/
https://developer.mozilla.org/en-US/docs/Web/API/TCPSocket
Chrome now has support for raw TCP and UDP sockets in its ‘experimental’ APIs. These features are only available for chrome apps and, although documented, are hidden for the moment. Having said that, some developers are already creating interesting projects using it, such as this IRC client.
To access this API, you’ll need to enable the experimental flag in your extension’s manifest. Using sockets is pretty straightforward, for example:
chrome.experimental.socket.create('tcp', '127.0.0.1', 8080, function(socketInfo) {
chrome.experimental.socket.connect(socketInfo.socketId, function (result) {
chrome.experimental.socket.write(socketInfo.socketId, "Hello, world!");
});
});
This will be possible via the navigator interface as shown below:
navigator.tcpPermission.requestPermission({remoteAddress:"127.0.0.1", remotePort:6789}).then(
() => {
// Permission was granted
// Create a new TCP client socket and connect to remote host
var mySocket = new TCPSocket("127.0.0.1", 6789);
// Send data to server
mySocket.writeable.write("Hello World").then(
() => {
// Data sent sucessfully, wait for response
console.log("Data has been sent to server");
mySocket.readable.getReader().read().then(
({ value, done }) => {
if (!done) {
// Response received, log it:
console.log("Data received from server:" + value);
}
// Close the TCP connection
mySocket.close();
}
);
},
e => console.error("Sending error: ", e)
);
}
);
More details are outlined in the w3.org tcp-udp-sockets documentation.
http://raw-sockets.sysapps.org/#interface-tcpsocket
https://www.w3.org/TR/tcp-udp-sockets/
Another alternative is to use Chrome Sockets
Creating connections
chrome.sockets.tcp.create({}, function(createInfo) {
chrome.sockets.tcp.connect(createInfo.socketId,
IP, PORT, onConnectedCallback);
});
Sending data
chrome.sockets.tcp.send(socketId, arrayBuffer, onSentCallback);
Receiving data
chrome.sockets.tcp.onReceive.addListener(function(info) {
if (info.socketId != socketId)
return;
// info.data is an arrayBuffer.
});
You can use also attempt to use HTML5 Web Sockets (Although this is not direct TCP communication):
var connection = new WebSocket('ws://IPAddress:Port');
connection.onopen = function () {
connection.send('Ping'); // Send the message 'Ping' to the server
};
http://www.html5rocks.com/en/tutorials/websockets/basics/
Your server must also be listening with a WebSocket server such as pywebsocket, alternatively you can write your own as outlined at Mozilla
ws2s project is aimed at bring socket to browser-side js. It is a websocket server which transform websocket to socket.
ws2s schematic diagram
code sample:
var socket = new WS2S("wss://ws2s.feling.io/").newSocket()
socket.onReady = () => {
socket.connect("feling.io", 80)
socket.send("GET / HTTP/1.1\r\nHost: feling.io\r\nConnection: close\r\n\r\n")
}
socket.onRecv = (data) => {
console.log('onRecv', data)
}
See jsocket. Haven't used it myself. Been more than 3 years since last update (as of 26/6/2014).
* Uses flash :(
From the documentation:
<script type='text/javascript'>
// Host we are connecting to
var host = 'localhost';
// Port we are connecting on
var port = 3000;
var socket = new jSocket();
// When the socket is added the to document
socket.onReady = function(){
socket.connect(host, port);
}
// Connection attempt finished
socket.onConnect = function(success, msg){
if(success){
// Send something to the socket
socket.write('Hello world');
}else{
alert('Connection to the server could not be estabilished: ' + msg);
}
}
socket.onData = function(data){
alert('Received from socket: '+data);
}
// Setup our socket in the div with the id="socket"
socket.setup('mySocket');
</script>
In order to achieve what you want, you would have to write two applications (in either Java or Python, for example):
Bridge app that sits on the client's machine and can deal with both TCP/IP sockets and WebSockets. It will interact with the TCP/IP socket in question.
Server-side app (such as a JSP/Servlet WAR) that can talk WebSockets. It includes at least one HTML page (including server-side processing code if need be) to be accessed by a browser.
It should work like this
The Bridge will open a WS connection to the web app (because a server can't connect to a client).
The Web app will ask the client to identify itself
The bridge client sends some ID information to the server, which stores it in order to identify the bridge.
The browser-viewable page connects to the WS server using JS.
Repeat step 3, but for the JS-based page
The JS-based page sends a command to the server, including to which bridge it must go.
The server forwards the command to the bridge.
The bridge opens a TCP/IP socket and interacts with it (sends a message, gets a response).
The Bridge sends a response to the server through the WS
The WS forwards the response to the browser-viewable page
The JS processes the response and reacts accordingly
Repeat until either client disconnects/unloads
Note 1: The above steps are a vast simplification and do not include information about error handling and keepAlive requests, in the event that either client disconnects prematurely or the server needs to inform clients that it is shutting down/restarting.
Note 2: Depending on your needs, it might be possible to merge these components into one if the TCP/IP socket server in question (to which the bridge talks) is on the same machine as the server app.
The solution you are really looking for is web sockets. However, the chromium project has developed some new technologies that are direct TCP connections TCP chromium
I'm looking for a Node.js module (or a suggestion/idea) so that my server can create/handle a 2-way communication between specific clients
In other words, I want to be able to transmit data coming from client X to client Y and vice versa.
I was looking into socket.io but it seems to work as a traditional web socket server that receives messages from all clients and then sends the same message to all clients. I might be wrong as I have no experience in this field.
In my case, I do want my server to be able to receive messages from different clients but I want the server to then forward each of those messages to a specific client instead.
e.g.
forward msg coming from client X to client Y only (and vice versa)
forward msg coming from client A to client B only (and vice versa)
etc.
I have zero experience with web sockets (I'm not even sure whether web sockets is what I'm looking for) but I do have experience with basic Node.js servers.
Thanks for any help!
In a Node.js WebSocket server, since you're coding the server, you have complete control over who gets what messages. You can do something like this...
When a client makes a WebSocket connection to the server and you store their connection object in an array, you can treat their connection's index in the array as their ID. If you want, you can broadcast this ID (and their IP address) to all other clients at that point to let them know who has connected. Then when a client wants to send a message to another client or clients, have the client doing the messaging include the destination client(s) along with the message. Here's the relevant parts of the server:
var client_list = [];
ws_server.on('request', function(request) {
var connection = request.accept(null, request.origin);
//let all other clients know that a new client has arrived
//send the new client's ID and IP address
for(var i = 0; i < client_list.length; i++) {
client_list[i].send(JSON.stringify({
type: "new client",
data: {
ID: client_list.length,
IP: connection.remoteAddress
}
}));
}
//store the new connection
client_list.push(connection);
//let the new client know what their ID is
connection.send(JSON.stringify({
type: "assignment",
data: {
ID: client_list.length - 1
}
}));
//when you receive a message from a client of type "relay",
//send it to the intended targets
connection.on("message", function(message) {
var parsed_message = JSON.parse(message.utf8Data);
if(parsed_message.type === "relay") {
for(var i = 0; i < parsed_message.targets.length; i++) {
client_list[parsed_message.targets[i]].send(parsed_message.data);
}
}
});
});
An outgoing message from a client would look like this:
{
type: "relay",
targets: [1, 33, 7],
data: {
content: "Hey guy!",
origin: my_id
}
}
I didn't test that, so let me know if it gives you trouble.
I am new to NodeJs too. Brief answer to your question
Socket.io is the way forward. You can isolate your client communication by connecting them on different ports.
client X to client Y only - port 1234
client A to client B only - port 1256
Regards
Socket.io will help you set up websockets between multiple clients and a server, so that you may synchronize data in real time between the connected instances. This is a client-server communication.
What I think you want is a WebRTC library, such as PeerJS. This one in particular sets up a handshake server using Node.js to bridge the connection between the two clients and let them communicate on their own from that point, for as long as the websocket lasts (i.e: a refresh will break the connection). This library can help you achieve client-client real time communication.
This Stack Overflow thread may help you further with a longer list of WebRTC libraries.
Today, I integrated Redis into my node.js application and am using it as a session store. Basically, upon successful authentication, I store the corresponding user object in Redis.
When I receive http requests after authentication, I attempt to retrieve the user object from Redis using a hash. If the retrieval was successful, that means the user is logged in and the request can be fulfilled.
The act of storing the user object in Redis and the retrieval happen in two different files, so I have one Redis client in each file.
Question 1:
Is it ok having two Redis clients, one in each file? Or should I instantiate only one client and use it across all areas of the application?
Question 2:
Does the node-redis library provide a method to show a list of connected clients? If it does, I will be able to iterate through the list, and call client.quit() for each of them when the server is shutting down.
By the way, this is how I'm implementing the "graceful shutdown" of the server:
//Gracefully shutdown and perform clean-up when kill signal is received
process.on('SIGINT', cleanup);
process.on('SIGTERM', cleanup);
function cleanup() {
server.stop(function() {
//todo: quit all connected redis clients
console.log('Server stopped.');
//exit the process
process.exit();
});
};
In terms of design and performance, it's best to create one client and use it across your application. This is pretty easy to do in node. I'm assuming you're using the redis npm package.
First, create a file named redis.js with the following contents:
const redis = require('redis');
const RedisClient = (function() {
return redis.createClient();
})();
module.exports = RedisClient
Then, say in a file set.js, you would use it as so:
const client = require('./redis');
client.set('key', 'value');
Then, in your index.js file, you can import it and close the connection on exit:
const client = require('./redis');
process.on('SIGINT', cleanup);
process.on('SIGTERM', cleanup);
function cleanup() {
client.quit(function() {
console.log('Redis client stopped.');
server.stop(function() {
console.log('Server stopped.');
process.exit();
});
});
};
Using multiple connections may be required by how the application uses Redis.
For instance, as soon as a connection is used the purpose of listening to a pub/sub channel, then it can only be used for this and nothing else. Per the documentation on SUBSCRIBE:
Once the client enters the subscribed state it is not supposed to issue any other commands, except for additional SUBSCRIBE, PSUBSCRIBE, UNSUBSCRIBE and PUNSUBSCRIBE commands.
So if your application needs to subscribe to channels and use Redis as general value cache, then it needs two clients at a minimum: one for subscribing to channels and one for using Redis as a cache.
There are also Redis commands that are blocking like BLPOP. A busy web server normally replies to multiple requests at once. Suppose that for answering request A the server uses its Redis client to issue a blocking command. Then request B comes and the server needs to answer Redis with a non-blocking command but the client is still waiting for the blocking command issued for request A to finish. Now the response to request B is delayed by another request. This can be avoided by using a different client for the second request.
If you do not use any of the facilities that require more than one connection, then you can and should use just one connection.
If the way you use Redis is such that you need more than one connection, and you just need a list of connections but no sophisticated connection management, you could just create your own factory function: it would call redis.createClient() and save the client before returning it. Then at shutdown time, you could go over the list of saved clients and close them. Unfortunately, node-redis does not provide such functionality built-in.
If you need more sophisticated client management than the factory function described above, then the typical way to manage the multiple connections created is to use a connection pool but node-redis does not provide one. I usually access Redis through Python code so I don't have a recommendation for Node.js libraries, but an npm search shows quite a few candidates.
I have a vb.net application that opens a socket and listens on it.
I need to communicate via this socket to that application using a javascript running on a browser. That is i need to send some data on this socket so that the app which is listening on this socket can take that data, do some stuff using some remote calls and get some more data and put it back on the socket that my javascript needs to read and print it in the browser.
Ive tried, socket.io, websockify but none have proved to be useful.
Hence the question, is what i am trying even possible? Is there a way that a javascript running in a browser can connect to a tcp socket and send some data and listen on it for some more data response on the socket and print it to the browser.
If this is possible can some one point me in the right direction as to which would help me establish the goal.
As for your problem, currently you will have to depend on XHR or websockets for this.
Currently no popular browser has implemented any such raw sockets api for javascript that lets you create and access raw sockets, but a draft for the implementation of raw sockets api in JavaScript is under-way. Have a look at these links:
http://www.w3.org/TR/raw-sockets/
https://developer.mozilla.org/en-US/docs/Web/API/TCPSocket
Chrome now has support for raw TCP and UDP sockets in its ‘experimental’ APIs. These features are only available for chrome apps and, although documented, are hidden for the moment. Having said that, some developers are already creating interesting projects using it, such as this IRC client.
To access this API, you’ll need to enable the experimental flag in your extension’s manifest. Using sockets is pretty straightforward, for example:
chrome.experimental.socket.create('tcp', '127.0.0.1', 8080, function(socketInfo) {
chrome.experimental.socket.connect(socketInfo.socketId, function (result) {
chrome.experimental.socket.write(socketInfo.socketId, "Hello, world!");
});
});
This will be possible via the navigator interface as shown below:
navigator.tcpPermission.requestPermission({remoteAddress:"127.0.0.1", remotePort:6789}).then(
() => {
// Permission was granted
// Create a new TCP client socket and connect to remote host
var mySocket = new TCPSocket("127.0.0.1", 6789);
// Send data to server
mySocket.writeable.write("Hello World").then(
() => {
// Data sent sucessfully, wait for response
console.log("Data has been sent to server");
mySocket.readable.getReader().read().then(
({ value, done }) => {
if (!done) {
// Response received, log it:
console.log("Data received from server:" + value);
}
// Close the TCP connection
mySocket.close();
}
);
},
e => console.error("Sending error: ", e)
);
}
);
More details are outlined in the w3.org tcp-udp-sockets documentation.
http://raw-sockets.sysapps.org/#interface-tcpsocket
https://www.w3.org/TR/tcp-udp-sockets/
Another alternative is to use Chrome Sockets
Creating connections
chrome.sockets.tcp.create({}, function(createInfo) {
chrome.sockets.tcp.connect(createInfo.socketId,
IP, PORT, onConnectedCallback);
});
Sending data
chrome.sockets.tcp.send(socketId, arrayBuffer, onSentCallback);
Receiving data
chrome.sockets.tcp.onReceive.addListener(function(info) {
if (info.socketId != socketId)
return;
// info.data is an arrayBuffer.
});
You can use also attempt to use HTML5 Web Sockets (Although this is not direct TCP communication):
var connection = new WebSocket('ws://IPAddress:Port');
connection.onopen = function () {
connection.send('Ping'); // Send the message 'Ping' to the server
};
http://www.html5rocks.com/en/tutorials/websockets/basics/
Your server must also be listening with a WebSocket server such as pywebsocket, alternatively you can write your own as outlined at Mozilla
ws2s project is aimed at bring socket to browser-side js. It is a websocket server which transform websocket to socket.
ws2s schematic diagram
code sample:
var socket = new WS2S("wss://ws2s.feling.io/").newSocket()
socket.onReady = () => {
socket.connect("feling.io", 80)
socket.send("GET / HTTP/1.1\r\nHost: feling.io\r\nConnection: close\r\n\r\n")
}
socket.onRecv = (data) => {
console.log('onRecv', data)
}
See jsocket. Haven't used it myself. Been more than 3 years since last update (as of 26/6/2014).
* Uses flash :(
From the documentation:
<script type='text/javascript'>
// Host we are connecting to
var host = 'localhost';
// Port we are connecting on
var port = 3000;
var socket = new jSocket();
// When the socket is added the to document
socket.onReady = function(){
socket.connect(host, port);
}
// Connection attempt finished
socket.onConnect = function(success, msg){
if(success){
// Send something to the socket
socket.write('Hello world');
}else{
alert('Connection to the server could not be estabilished: ' + msg);
}
}
socket.onData = function(data){
alert('Received from socket: '+data);
}
// Setup our socket in the div with the id="socket"
socket.setup('mySocket');
</script>
In order to achieve what you want, you would have to write two applications (in either Java or Python, for example):
Bridge app that sits on the client's machine and can deal with both TCP/IP sockets and WebSockets. It will interact with the TCP/IP socket in question.
Server-side app (such as a JSP/Servlet WAR) that can talk WebSockets. It includes at least one HTML page (including server-side processing code if need be) to be accessed by a browser.
It should work like this
The Bridge will open a WS connection to the web app (because a server can't connect to a client).
The Web app will ask the client to identify itself
The bridge client sends some ID information to the server, which stores it in order to identify the bridge.
The browser-viewable page connects to the WS server using JS.
Repeat step 3, but for the JS-based page
The JS-based page sends a command to the server, including to which bridge it must go.
The server forwards the command to the bridge.
The bridge opens a TCP/IP socket and interacts with it (sends a message, gets a response).
The Bridge sends a response to the server through the WS
The WS forwards the response to the browser-viewable page
The JS processes the response and reacts accordingly
Repeat until either client disconnects/unloads
Note 1: The above steps are a vast simplification and do not include information about error handling and keepAlive requests, in the event that either client disconnects prematurely or the server needs to inform clients that it is shutting down/restarting.
Note 2: Depending on your needs, it might be possible to merge these components into one if the TCP/IP socket server in question (to which the bridge talks) is on the same machine as the server app.
The solution you are really looking for is web sockets. However, the chromium project has developed some new technologies that are direct TCP connections TCP chromium