Related
In a Node JS app, I want my server to be able to send a notification to clients, in some specific cases. Searching the net on the subject, I find Service Workers which seems to be what I need; but I am not totally sure.
Here is why. When I look at various tutorials and videos, it seems like the notifications always somewhat originates from some client. But this is not what I want. It should be the server deciding when and what to send as a notification, not some client. So comes my questions:
Am I misunderstanding the way to use Service Workers?
Is this the right path to have my server send a notification to clients?
If the answer to the last question is NO. Then what is the proper way?
Like Dimitar wrote earlier, you could use socket.io to send messages from the server to the client and vice versa. This can be done as simple as that:
//Server sent message:
io.on('connection', (socket) => {
socket.emit('notificationToClient', 'Message'); //message sent from server to client
});
});
//Client receives the message:
const socket = io.connect('http://localhost:3000');
socket.on('notificationToClient', (data) => { //received message
console.log(data);
});
The websocket API is a little less confusing than socket.io but IMO, I'd go with socket.io since it already deals with a lot of the heavy lifting like re-connecting, custom namespaces, rooms etc. There are several fantastic udemy courses on socket.io.
See the socket.io documentation here: https://socket.io/docs/
Update 5/9/2020:
Michel - Since you asked about how to implement Socket.io in your code I'll give you an example of how I implemented this technology in my most recent project. I was working on a web app that connects to ELK Alarm systems and is able to arm/disarm these systems as well as receive sensor input (chime signals) at any remote location. The backend is Node.js and the UI is React.
//client side code
import React, { Component, Suspense } from 'react';
import panels from "./panels.js"
const socketConnection = io("http://localhost:5000"); //Socket.io
function App() {
function handleClick(panel) {
socketConnection.emit("panelData", panel); //Socket.io - sends the connection data for the ELK alarm panel to the express server
socketConnection.on('data', (data) => { //Socket.io - receives data from the server.
//does something with the data
})
}
}
//server side code
const express = require('express');
const elkClient = require('elk-client');
const app = express();
const server = app.listen('5000', () => {
console.log('------Server Running on 5000---------');
});
let io = new sio(server);
io.on("connection", (socket) => { //boilerplate code - establishes the "handshake" with the client.
socket.on("panelData", async (msg) => { //receives the ELK connection parameters from the client. See client side code
const client = new ElkClient({
connection: {
site: msg.name,
host: msg.host,
}
})
await client.connect();
client.on("message", (elkMessage) => { // This is NOT Socket.io - server is listening for emitted data from the alarm system - beam breaks, alarms, arm/disarm etc. [See event emitter][2]
if(elkMessage.messageType == 'A' && elkMessage.subMessageType == 'S') {
const armingStatusReport = {elkMessage};
socket.emit("data", armingStatusReport); //Socket.io - emits the received data to the client.
}
})
I tried to simplify the code above to hit the point. The server is using the connection parameters that it receives from the client to connect to a remote alarm system. The server then waits and listens for incoming data with client.on()(event emitter - not socket.io). As the data is received, I'm using Socket.io to send the received data back to the client with socket.emit(); Due to the nature of how alarm systems work, data is sent event driven so the fetch api wouldn't really fit the bill but socket.io (or websockets) does.
Let me know if I can help with anything else. With this recent project, I have spent the last few months EXTENSIVELY dealing with socket.io, including namespaces, rooms etc. I also had to create a real-time monitoring app for performance management for my server utilizing socket.io since I implemented clustering. Feel free to contact me anytime!
The answer is WebSockets.
Check out Socket.IO. It will give you the possibility of receiving live notifications on the client and you'll have complete server control on them.
What about using twilio library to manage your notification job .twilio library
I have a vb.net application that opens a socket and listens on it.
I need to communicate via this socket to that application using a javascript running on a browser. That is i need to send some data on this socket so that the app which is listening on this socket can take that data, do some stuff using some remote calls and get some more data and put it back on the socket that my javascript needs to read and print it in the browser.
Ive tried, socket.io, websockify but none have proved to be useful.
Hence the question, is what i am trying even possible? Is there a way that a javascript running in a browser can connect to a tcp socket and send some data and listen on it for some more data response on the socket and print it to the browser.
If this is possible can some one point me in the right direction as to which would help me establish the goal.
As for your problem, currently you will have to depend on XHR or websockets for this.
Currently no popular browser has implemented any such raw sockets api for javascript that lets you create and access raw sockets, but a draft for the implementation of raw sockets api in JavaScript is under-way. Have a look at these links:
http://www.w3.org/TR/raw-sockets/
https://developer.mozilla.org/en-US/docs/Web/API/TCPSocket
Chrome now has support for raw TCP and UDP sockets in its ‘experimental’ APIs. These features are only available for chrome apps and, although documented, are hidden for the moment. Having said that, some developers are already creating interesting projects using it, such as this IRC client.
To access this API, you’ll need to enable the experimental flag in your extension’s manifest. Using sockets is pretty straightforward, for example:
chrome.experimental.socket.create('tcp', '127.0.0.1', 8080, function(socketInfo) {
chrome.experimental.socket.connect(socketInfo.socketId, function (result) {
chrome.experimental.socket.write(socketInfo.socketId, "Hello, world!");
});
});
This will be possible via the navigator interface as shown below:
navigator.tcpPermission.requestPermission({remoteAddress:"127.0.0.1", remotePort:6789}).then(
() => {
// Permission was granted
// Create a new TCP client socket and connect to remote host
var mySocket = new TCPSocket("127.0.0.1", 6789);
// Send data to server
mySocket.writeable.write("Hello World").then(
() => {
// Data sent sucessfully, wait for response
console.log("Data has been sent to server");
mySocket.readable.getReader().read().then(
({ value, done }) => {
if (!done) {
// Response received, log it:
console.log("Data received from server:" + value);
}
// Close the TCP connection
mySocket.close();
}
);
},
e => console.error("Sending error: ", e)
);
}
);
More details are outlined in the w3.org tcp-udp-sockets documentation.
http://raw-sockets.sysapps.org/#interface-tcpsocket
https://www.w3.org/TR/tcp-udp-sockets/
Another alternative is to use Chrome Sockets
Creating connections
chrome.sockets.tcp.create({}, function(createInfo) {
chrome.sockets.tcp.connect(createInfo.socketId,
IP, PORT, onConnectedCallback);
});
Sending data
chrome.sockets.tcp.send(socketId, arrayBuffer, onSentCallback);
Receiving data
chrome.sockets.tcp.onReceive.addListener(function(info) {
if (info.socketId != socketId)
return;
// info.data is an arrayBuffer.
});
You can use also attempt to use HTML5 Web Sockets (Although this is not direct TCP communication):
var connection = new WebSocket('ws://IPAddress:Port');
connection.onopen = function () {
connection.send('Ping'); // Send the message 'Ping' to the server
};
http://www.html5rocks.com/en/tutorials/websockets/basics/
Your server must also be listening with a WebSocket server such as pywebsocket, alternatively you can write your own as outlined at Mozilla
ws2s project is aimed at bring socket to browser-side js. It is a websocket server which transform websocket to socket.
ws2s schematic diagram
code sample:
var socket = new WS2S("wss://ws2s.feling.io/").newSocket()
socket.onReady = () => {
socket.connect("feling.io", 80)
socket.send("GET / HTTP/1.1\r\nHost: feling.io\r\nConnection: close\r\n\r\n")
}
socket.onRecv = (data) => {
console.log('onRecv', data)
}
See jsocket. Haven't used it myself. Been more than 3 years since last update (as of 26/6/2014).
* Uses flash :(
From the documentation:
<script type='text/javascript'>
// Host we are connecting to
var host = 'localhost';
// Port we are connecting on
var port = 3000;
var socket = new jSocket();
// When the socket is added the to document
socket.onReady = function(){
socket.connect(host, port);
}
// Connection attempt finished
socket.onConnect = function(success, msg){
if(success){
// Send something to the socket
socket.write('Hello world');
}else{
alert('Connection to the server could not be estabilished: ' + msg);
}
}
socket.onData = function(data){
alert('Received from socket: '+data);
}
// Setup our socket in the div with the id="socket"
socket.setup('mySocket');
</script>
In order to achieve what you want, you would have to write two applications (in either Java or Python, for example):
Bridge app that sits on the client's machine and can deal with both TCP/IP sockets and WebSockets. It will interact with the TCP/IP socket in question.
Server-side app (such as a JSP/Servlet WAR) that can talk WebSockets. It includes at least one HTML page (including server-side processing code if need be) to be accessed by a browser.
It should work like this
The Bridge will open a WS connection to the web app (because a server can't connect to a client).
The Web app will ask the client to identify itself
The bridge client sends some ID information to the server, which stores it in order to identify the bridge.
The browser-viewable page connects to the WS server using JS.
Repeat step 3, but for the JS-based page
The JS-based page sends a command to the server, including to which bridge it must go.
The server forwards the command to the bridge.
The bridge opens a TCP/IP socket and interacts with it (sends a message, gets a response).
The Bridge sends a response to the server through the WS
The WS forwards the response to the browser-viewable page
The JS processes the response and reacts accordingly
Repeat until either client disconnects/unloads
Note 1: The above steps are a vast simplification and do not include information about error handling and keepAlive requests, in the event that either client disconnects prematurely or the server needs to inform clients that it is shutting down/restarting.
Note 2: Depending on your needs, it might be possible to merge these components into one if the TCP/IP socket server in question (to which the bridge talks) is on the same machine as the server app.
The solution you are really looking for is web sockets. However, the chromium project has developed some new technologies that are direct TCP connections TCP chromium
I have a MongoDB on a web server and I have a Mean stack web application. Within the MongoDB, I have a bunch of data that I want to pass to my web application. I have looked into options, but I'm a little unsure as to which route to take to make this happen. I have considered using socket.io to set up a listener on my web server to open up a connection with my DB, and then making an API call or direct DB connection to return my data.
I should add that my app is based on the mean.io boilerplate code.
My question is where within my stack do I need to put the socket.io code?
Would it be wise to include the socket.io code at the root of my application or only in the packages/modules that require the data to be passed from my web server to my application?
And finally, is socket.io even the preferred route to take? Is there another method to achieve this that makes more sense such as express.js?
There's no need to use socket.io unless you want real-time constant, persistant streaming of information from your database to your front-end.
First in your express app, connect to the db with https://github.com/mongodb/node-mongodb-native. Then, you can just set up a JSON endpoint route for the data to get sent to. Then the client webpage that needs the data simply sends an XHR request to the JSON page.
EDIT: OP does want to use socket.io.
If you want to use socket.io, you have to include it on both the server and the client.
Server-side:
const express = require('express');
const app = express();
const server = require('http').Server(app);
// socket.io
const io = require('socket.io').listen(server);
// handling socket.io requests
io.on('connection', (socket) => {
log.info('new connection.');
socket.emit('data', {/** whatever data you need to send **/});
socket.on('disconnect', (e) => {
log.info('user disconnected.');
});
});
Client-side:
Be sure to include socket.io in the html file (get it from the website). Then, in your client-side js:
const socket = io();
socket.on('connect', () => {
console.log('connected to server.');
});
socket.on('disconnect', () => {
console.log('disconnected.');
});
socket.on('data', (data) => {
// this is where you receive the data
// do something with your data
});
You should create a route in Express and then, request that route from your angular SPA with an http module, like $http ou Restangular.
Take a look at this article https://scotch.io/tutorials/setting-up-a-mean-stack-single-page-application
I have a vb.net application that opens a socket and listens on it.
I need to communicate via this socket to that application using a javascript running on a browser. That is i need to send some data on this socket so that the app which is listening on this socket can take that data, do some stuff using some remote calls and get some more data and put it back on the socket that my javascript needs to read and print it in the browser.
Ive tried, socket.io, websockify but none have proved to be useful.
Hence the question, is what i am trying even possible? Is there a way that a javascript running in a browser can connect to a tcp socket and send some data and listen on it for some more data response on the socket and print it to the browser.
If this is possible can some one point me in the right direction as to which would help me establish the goal.
As for your problem, currently you will have to depend on XHR or websockets for this.
Currently no popular browser has implemented any such raw sockets api for javascript that lets you create and access raw sockets, but a draft for the implementation of raw sockets api in JavaScript is under-way. Have a look at these links:
http://www.w3.org/TR/raw-sockets/
https://developer.mozilla.org/en-US/docs/Web/API/TCPSocket
Chrome now has support for raw TCP and UDP sockets in its ‘experimental’ APIs. These features are only available for chrome apps and, although documented, are hidden for the moment. Having said that, some developers are already creating interesting projects using it, such as this IRC client.
To access this API, you’ll need to enable the experimental flag in your extension’s manifest. Using sockets is pretty straightforward, for example:
chrome.experimental.socket.create('tcp', '127.0.0.1', 8080, function(socketInfo) {
chrome.experimental.socket.connect(socketInfo.socketId, function (result) {
chrome.experimental.socket.write(socketInfo.socketId, "Hello, world!");
});
});
This will be possible via the navigator interface as shown below:
navigator.tcpPermission.requestPermission({remoteAddress:"127.0.0.1", remotePort:6789}).then(
() => {
// Permission was granted
// Create a new TCP client socket and connect to remote host
var mySocket = new TCPSocket("127.0.0.1", 6789);
// Send data to server
mySocket.writeable.write("Hello World").then(
() => {
// Data sent sucessfully, wait for response
console.log("Data has been sent to server");
mySocket.readable.getReader().read().then(
({ value, done }) => {
if (!done) {
// Response received, log it:
console.log("Data received from server:" + value);
}
// Close the TCP connection
mySocket.close();
}
);
},
e => console.error("Sending error: ", e)
);
}
);
More details are outlined in the w3.org tcp-udp-sockets documentation.
http://raw-sockets.sysapps.org/#interface-tcpsocket
https://www.w3.org/TR/tcp-udp-sockets/
Another alternative is to use Chrome Sockets
Creating connections
chrome.sockets.tcp.create({}, function(createInfo) {
chrome.sockets.tcp.connect(createInfo.socketId,
IP, PORT, onConnectedCallback);
});
Sending data
chrome.sockets.tcp.send(socketId, arrayBuffer, onSentCallback);
Receiving data
chrome.sockets.tcp.onReceive.addListener(function(info) {
if (info.socketId != socketId)
return;
// info.data is an arrayBuffer.
});
You can use also attempt to use HTML5 Web Sockets (Although this is not direct TCP communication):
var connection = new WebSocket('ws://IPAddress:Port');
connection.onopen = function () {
connection.send('Ping'); // Send the message 'Ping' to the server
};
http://www.html5rocks.com/en/tutorials/websockets/basics/
Your server must also be listening with a WebSocket server such as pywebsocket, alternatively you can write your own as outlined at Mozilla
ws2s project is aimed at bring socket to browser-side js. It is a websocket server which transform websocket to socket.
ws2s schematic diagram
code sample:
var socket = new WS2S("wss://ws2s.feling.io/").newSocket()
socket.onReady = () => {
socket.connect("feling.io", 80)
socket.send("GET / HTTP/1.1\r\nHost: feling.io\r\nConnection: close\r\n\r\n")
}
socket.onRecv = (data) => {
console.log('onRecv', data)
}
See jsocket. Haven't used it myself. Been more than 3 years since last update (as of 26/6/2014).
* Uses flash :(
From the documentation:
<script type='text/javascript'>
// Host we are connecting to
var host = 'localhost';
// Port we are connecting on
var port = 3000;
var socket = new jSocket();
// When the socket is added the to document
socket.onReady = function(){
socket.connect(host, port);
}
// Connection attempt finished
socket.onConnect = function(success, msg){
if(success){
// Send something to the socket
socket.write('Hello world');
}else{
alert('Connection to the server could not be estabilished: ' + msg);
}
}
socket.onData = function(data){
alert('Received from socket: '+data);
}
// Setup our socket in the div with the id="socket"
socket.setup('mySocket');
</script>
In order to achieve what you want, you would have to write two applications (in either Java or Python, for example):
Bridge app that sits on the client's machine and can deal with both TCP/IP sockets and WebSockets. It will interact with the TCP/IP socket in question.
Server-side app (such as a JSP/Servlet WAR) that can talk WebSockets. It includes at least one HTML page (including server-side processing code if need be) to be accessed by a browser.
It should work like this
The Bridge will open a WS connection to the web app (because a server can't connect to a client).
The Web app will ask the client to identify itself
The bridge client sends some ID information to the server, which stores it in order to identify the bridge.
The browser-viewable page connects to the WS server using JS.
Repeat step 3, but for the JS-based page
The JS-based page sends a command to the server, including to which bridge it must go.
The server forwards the command to the bridge.
The bridge opens a TCP/IP socket and interacts with it (sends a message, gets a response).
The Bridge sends a response to the server through the WS
The WS forwards the response to the browser-viewable page
The JS processes the response and reacts accordingly
Repeat until either client disconnects/unloads
Note 1: The above steps are a vast simplification and do not include information about error handling and keepAlive requests, in the event that either client disconnects prematurely or the server needs to inform clients that it is shutting down/restarting.
Note 2: Depending on your needs, it might be possible to merge these components into one if the TCP/IP socket server in question (to which the bridge talks) is on the same machine as the server app.
The solution you are really looking for is web sockets. However, the chromium project has developed some new technologies that are direct TCP connections TCP chromium
I'm trying to evaluate using IndexedDB to solve the offline issue. It would be populated with data currently stored in a MongoDB database (as is).
Once data is stored in IndexedDB, it may be changed on the MongoDB server and I need to propagate those changes. Is there any existing framework or Library to do somehting like this for Mongo. I already know about CouchDB/PouchDB and am not exploring those two.
[Sync solution for 2021]
I know the question asked was for MongoDB specifically, but since this is an old thread I thought readers might be looking for other solutions for new apps or rebuilds. I can really recommend to check out AceBase because it does exactly what you were looking for back then.
AceBase is a free and open source realtime database that enables easy storage and synchronization between browser and server databases. It uses IndexedDB in the browser, its own binary db / SQL Server / SQLite storage on the server. Offline edits are synced upon reconnect and clients are notified of remote database changes in realtime through a websocket (FAST!).
On top of this, AceBase has a unique feature called "live data proxies" that allow you to have all changes to in-memory objects to be persisted and synced to local and server databases, and remote changes to automatically update your in-memory objects. This means you can forget about database coding altogether, and code as if you're only using local objects. No matter whether you're online or offline.
The following example shows how to create a local IndexedDB database in the browser, how to connect to a remote database server that syncs with the local database, and how to create a live data proxy that eliminates further database coding. AceBase supports authentication and authorization as well, but I left it out for simplicity.
const { AceBaseClient } = require('acebase-client');
const { AceBase } = require('acebase');
// Create local database with IndexedDB storage:
const cacheDb = AceBase.WithIndexedDB('mydb-local');
// Connect to server database, use local db for offline storage:
const db = new AceBaseClient({ dbname: 'mydb', host: 'db.myproject.com', port: 443, https: true, cache: { db: cacheDb } });
// Wait for remote database to be connected, or ready to use when offline:
db.ready(async () => {
// Create live data proxy for a chat:
const emptyChat = { title: 'New chat', messages: {} };
const proxy = await db.ref('chats/chatid1').proxy(emptyChat); // Use emptyChat if chat node doesn't exist
// Get object reference containing live data:
const chat = proxy.value;
// Update chat's properties to save to local database,
// sync to server AND all other clients monitoring this chat in realtime:
chat.title = `Changing the title`;
chat.messages.push({
from: 'ewout',
sent: new Date(),
text: `Sending a message that is stored in the database and synced automatically was never this easy!` +
`This message might have been sent while we were offline. Who knows!`
});
// To monitor and handle realtime changes to the chat:
chat.onChanged((val, prev, isRemoteChange, context) => {
if (val.title !== prev.title) {
alert(`Chat title changed to ${val.title} by ${isRemoteChange ? 'us' : 'someone else'}`);
}
});
});
For more examples and documentation, see AceBase realtime database engine at npmjs.com
Open up a changeStream with the resumeToken. There's no guarantee of causal consistency however since we're talking multiple disparate databases.
I haven't worked with IndexDB, but the design problem isn't that uncommon. My understanding of your app is that when the client makes the connection to MongoDB, you pull a set of documents down for local storage and disconnect. The client then can do things locally (not connected to the data server), and then push up the changes.
The way I see it you've got to handle two general cases:
when the MongoDB server is updated and breaks continuity with the client, the client will have to
poll for the data (timer?) or
keep a websocket open to let notifications free-flow over the pipe
when the user needs to push changed data back up the pipe
you can reconnect asynchronously, check for state changes, (resolving conflicts according to your business rules)
have a server side (light) interface for handling conflicts (depending on complexity of your app, comparing time stamps of state changes in MongoDB to IndexedDB updates should suffice)