Context
Not long ago, I discovered a great service called Serveo. It allows me to expose my local apps to the Internet using reverse SSH tunneling.
e.g. Connections to https://abc.serveo.net get forwarded to http://localhost:3000 on my machine.
To do this, they require no client installation, and I can just type this in the command line:
ssh -R 80:localhost:3000 serveo.net
where 80 is the remote port on serveo.net to which I want to bind, and localhost:3000 is the local address for my app.
If I just type 80 on the left-hand side, Serveo will answer Forwarding HTTP traffic from https://xxxx.serveo.net where xxxx is an available subdomain, with https support.
However, if I type another port, like 59000, the app will be available through serveo.net:59000, but without SSL.
Problem
Now, I would like to do this with NodeJS, to automate things in a tool I'm building for my coworkers and my company's partners, so that they don't need to worry about it, nor to have an SSH client on their machine. I'm using the SSH2 Node module.
Here is an example of working code, using the custom port configuration (here, 59000), with an app listening on http://localhost:3000:
/**
* Want to try it out?
* Go to https://github.com/blex41/demo-ssh2-tunnel
*/
const Client = require("ssh2").Client; // To communicate with Serveo
const Socket = require("net").Socket; // To accept forwarded connections (native module)
// Create an SSH client
const conn = new Client();
// Config, just like the second example in my question
const config = {
remoteHost: "",
remotePort: 59000,
localHost: "localhost",
localPort: 3000
};
conn
.on("ready", () => {
// When the connection is ready
console.log("Connection ready");
// Start an interactive shell session
conn.shell((err, stream) => {
if (err) throw err;
// And display the shell output (so I can see how Serveo responds)
stream.on("data", data => {
console.log("SHELL OUTPUT: " + data);
});
});
// Request port forwarding from the remote server
conn.forwardIn(config.remoteHost, config.remotePort, (err, port) => {
if (err) throw err;
conn.emit("forward-in", port);
});
})
// ===== Note: this part is irrelevant to my problem, but here for the demo to work
.on("tcp connection", (info, accept, reject) => {
console.log("Incoming TCP connection", JSON.stringify(info));
let remote;
const srcSocket = new Socket();
srcSocket
.on("error", err => {
if (remote === undefined) reject();
else remote.end();
})
.connect(config.localPort, config.localPort, () => {
remote = accept()
.on("close", () => {
console.log("TCP :: CLOSED");
})
.on("data", data => {
console.log(
"TCP :: DATA: " +
data
.toString()
.split(/\n/g)
.slice(0, 2)
.join("\n")
);
});
console.log("Accept remote connection");
srcSocket.pipe(remote).pipe(srcSocket);
});
})
// ===== End Note
// Connect to Serveo
.connect({
host: "serveo.net",
username: "johndoe",
tryKeyboard: true
});
// Just for the demo, create a server listening on port 3000
// Accessible both on:
// http://localhost:3000
// https://serveo.net:59000
const http = require("http"); // native module
http
.createServer((req, res) => {
res.writeHead(200, {
"Content-Type": "text/plain"
});
res.write("Hello world!");
res.end();
})
.listen(config.localPort);
This works fine, I can access my app on http://serveo.net:59000. But it does not support HTTPS, which is one of my requirements. If I want HTTPS, I need to set the port to 80, and leave the remote host blank just like the plain SSH command given above, so that Servo assigns me an available subdomain:
// equivalent to `ssh -R 80:localhost:3000 serveo.net`
const config = {
remoteHost: "",
remotePort: 80,
localHost: "localhost",
localPort: 3000
};
However, this is throwing an error:
Error: Unable to bind to :80
at C:\workspace\demo-ssh2-tunnel\node_modules\ssh2\lib\client.js:939:21
at SSH2Stream.<anonymous> (C:\workspace\demo-ssh2-tunnel\node_modules\ssh2\lib\client.js:628:24)
at SSH2Stream.emit (events.js:182:13)
at parsePacket (C:\workspace\demo-ssh2-tunnel\node_modules\ssh2-streams\lib\ssh.js:3851:10)
at SSH2Stream._transform (C:\workspace\demo-ssh2-tunnel\node_modules\ssh2-streams\lib\ssh.js:693:13)
at SSH2Stream.Transform._read (_stream_transform.js:190:10)
at SSH2Stream._read (C:\workspace\demo-ssh2-tunnel\node_modules\ssh2-streams\lib\ssh.js:252:15)
at SSH2Stream.Transform._write (_stream_transform.js:178:12)
at doWrite (_stream_writable.js:410:12)
at writeOrBuffer (_stream_writable.js:394:5)
I've tried many things without any success. If anyone has an idea about what might be wrong in my example, I'll be really grateful. Thanks!
OpenSSH defaults to "localhost" for the remote host when it's not specified. You can also verify this by checking the debug output from the OpenSSH client by adding -vvv to the command line. You should see a line like:
debug1: Remote connections from LOCALHOST:80 forwarded to local address localhost:3000
If you mimic this by setting config.remoteHost = 'localhost' in your JS code you should get the same result as the OpenSSH client.
Related
this code is in my nodejs backend (https://backend.example.com) server.js file:
const WebSocket = require('ws');
const server = new WebSocket.Server({
port: 7500
},
() => {
console.log('Server started on port 7500');
}
);
This code is in my nextjs frontend chat (http://frontend.example.com/chat) page file:
React.useEffect(() => {
ws.current = new WebSocket("ws://localhost:7500");
ws.current.onopen = () => {
console.log("Connection opened");
setConnectionOpen(true);
};
ws.current.onmessage = (event) => {
const data = JSON.parse(event.data);
setMessages((_messages) => [..._messages, data]);
};
return () => {
console.log("Cleaning up...");
ws.current.close();
};
}, []);
it works fine in localhost but on deployed live server, the websocket is not communicating, what is wrong with my code?
EDIT: Have updated the useEffect() to:
React.useEffect(() => {
ws.current = new WebSocket("wss://backend.example.com:7500");
ws.current.onopen = () => {
console.log("Connection opened");
setConnectionOpen(true);
};
ws.current.onmessage = (event) => {
const data = JSON.parse(event.data);
setMessages((_messages) => [..._messages, data]);
};
return () => {
console.log("Cleaning up...");
ws.current.close();
};
}, []);
but still it does not work, If I visit the https://backend.example.com I get Upgrade Required
If this is the code you deploy on live server, then I think the following points have to be addressed.
On the client you point to localhost, you should have the server name, instead.
And more important, in local you're publishing the app in http, while in live server it is in https?
In this case the WebSocket url should change the protocol from ws to wss.
UPDATE
Another point of attention is your server.
I don't see code that is handling the connection, according to the documentation example.
import WebSocket from 'ws';
const ws = new WebSocket('ws://www.host.com/path');
// This handle the co
ws.on('open', function open() {
ws.send('something');
});
ws.on('message', function message(data) {
console.log('received: %s', data);
});
If this is not the library you're using, please update your question whit all the references to the library you're using or the documentation you're referring to to write your code.
About the port, there is nothing specific about the port, the only issue is that you can have the port blocked, on the firewall of your client or on the network gateway of your server.
But that depends on your environment, you should check if the port is usable.
A simple test is to try a small server with an html page published on the 7500 port on your server. If you can see the page the port is ok.
And more you should not use the same port of your server, pick another one, because the http server is reserving that port, and your WebSocket will fail attempting to bind.
But you should see an error on the server if that happened.
If you want to use your application server port, instead of starting a different server then follow this example.
// server.ts
import { Server } from 'net';
const port = 8081;
const hostname = 'localhost';
const server = new Server();
server.on('connection', (socket) => {
socket.setEncoding('utf8');
socket.on('data', (data) => {
console.log(data.toString());
});
});
server.listen({ port, host: hostname }, () => {
console.log('Server running at port', port);
});
When I try to console log the data of the buffer I am getting weird chars. I tried to assign it to a variable and split the data of the first request, but the same is happening. I tried to set 'utf-8' as well and all the other types and no one is giving me a the request properly.
This is the string I am getting when decoded to 'utf8':
��Lϻط�4�)]�3O�X\��M�ʏ����S�mN 9BRg����Vl�^A:�ʃ�"I���r�*��p� ���+�/�,�0̨̩����/5����
**
#
3+)** http/1.1
���]xO����
�[��ˈ�
I�B,}�J��
-+
ih2zz�
My purpose is to get the websocket key of the request. Then I will respond with the hash in base 64 to establish the connection, but that's another thing, only to put in context the situation.
PD: I am using Node 18.7.0, but I tried with 16 and the same happens.
PD2: Okay, I realized the problem. At the client I was setting a wss://localhost:8081. Switching this to ws://localhost:8081 solved the problem, however I am still interested in the solution for a secure connection.
Thank you.
I have a simple snippet on the front end as follows which I can verify is working. I can do this by changing the port to something other than 3000 and it will error.
It is definitely finding the server at that port:
// Create WebSocket connection .. will error if I change the port
const socket = new WebSocket('ws://localhost:3000');
console.log('DEBUG: Web socket is up: ');
// Connection opened
socket.addEventListener('open', function (event) {
socket.send('Hello Server!');
});
I am using ws-express on the server side as follows. This was the minimal example given in the NPM docs:
const expressWs = require('express-ws')(app);
app.ws('/echo', (ws, req) => {
ws.on('message', (msg) => {
ws.send(msg);
});
});
However, the open event on the client never fires. I would like to send messages from the client to the server, but I assume, that I need an open event to fire first.
I am trying to create a websocket server that listen to an external websocket clinet.
the point is I am laoding a web base application inside my browser window in electron.
for example : win.loadURL(www.something.com); so the websocket call coming from this url
meaning if I getinto this url in browser in my network tab I see websocket call is keep
calling but there is no server. so I want to implement the server inside my electron app main.js. and here is my code:
const WebSocket = require("ws");
const wss = new WebSocket.Server({port: 8102});
wss.on("connection", ws => {
ws.on("message", message => {
console.log("received: %s", message);
});
ws.send("something");
});
so far I did not get any success. any help would appriciate.
you need to start your http server
mine looks like this:
import http from "http";
import * as WebSocket from "ws";
const port = 4444;
const server = http.createServer();
const wss = new WebSocket.Server({ server });
wss.on("connection", (ws: WebSocket) => {
//connection is up, let's add a simple simple event
ws.on("message", (message: string) => {
//log the received message and send it back to the client
console.log("received: %s", message);
ws.send(`Hello, you sent -> ${message}`);
});
//send immediatly a feedback to the incoming connection
ws.send("Hi there, I am a WebSocket server");
});
//start our server
server.listen(port, () => {
console.log(`Data stream server started on port ${port}`);
});
This test program connects to an https server and gets some content. I've checked my server in browsers and with curl and the certificate is working correctly. If I run curl to grab data from the server it correctly complains about the certificate being unknown unless I pass it in with --cacert or turn security off with -k.
So the problem I am having is that although I think my client should be doing certificate authentication and I am telling it where the public certificate is, it just always works. If I remove the ca: option so it has no idea what the certificate is from the server then it silently works. I would like to catch the authentication error but I can't seem to do so.
var https = require('https');
var fs = require('fs');
function main() {
var data = '';
var get = https.get({
path: '/',
host: 'localhost',
port: 8000,
agent: false,
ca: [ fs.readFileSync('https_simple/cacert.pem') ]
}, function(x) {
x.setEncoding('utf8');
x.on('data', function(c) {data += c});
x.on('error', function(e) {
throw e;
});
x.on('end', function() {
console.log('Hai!. Here is the response:');
console.log(data);
});
});
get.on('error', function(e) {throw e});
get.end();
}
main();
In order to make this work I needed to upgrade to v0.7.8 (although any v0.7 should be fine) where the rejectUnauthorized functionality has been added to https.get
This combination of options is needed:
agent: false, // or you can supply your own agent, but if you don't you must set to false
rejectUnauthorized: true,
ca: [ fs.readFileSync('https_simple/cacert.pem') ]
Now if the authentication fails you will get an 'error' event and the request will not go ahead.
See the https.request documentation for details on making your own Agent
The bug fix was committed in this change: https://github.com/joyent/node/commit/f8c335d0
As per the documentation for https.request, the ca option of both https.get and https.request is an option from tls.connect. The documentation for the options to the tls.connect module function states:
ca: An array of strings or Buffers of trusted certificates. If this is
omitted several well known "root" CAs will be used, like VeriSign.
These are used to authorize connections.
Digging into the node.js source, the root certs used can be found here: https://github.com/joyent/node/blob/master/src/node_root_certs.h
So in short, with no authority cert provided as an option to https.get the tls module will attempt to authenticate the connection using the list of root certs anyway.
I do this in npm, using the request module. It goes like this:
var cacert = ... // in npm, this is a config setting
var request = require("request")
request.get({ url: "https://...",
ca: cacert,
strictSSL: true })
.on("response", function (resp) { ... })
.on("error", function (er) { ... })
The error event will be raised if the ssl isn't valid.
In V 0.6.15 you need to explicitly check whether or not the certificate validation passed or failed.
if (x.connection.authorized === false) {
console.log('SSL Authentication failed');
} else if (x.connection.authorized === true) {
console.log('SSL Authentication succeeded');
}