I am running a website server and a websocket server on two different ports. I'd like two-way SSL enabled on both. The website is using a gunicorn/flask app and is working fine for the two-way SSL on all browsers (desktop/mobile, except mobile Firefox). Inside that webpage is a script to connect to the websocket (python websockets server) which also has two-way SSL enabled.
This only works in desktop Firefox; anything else gives the error WebSocket connection to '<domain>:<port>' failed: WebSocket opening handshake was canceled.
On desktop Firefox, it requests user certificates for both the webpage and websocket, but like I said anything else returns the above error.
I've been scouring the internet for solutions to this and have seen some info on using proxies with certain hosting providers but I am hosting these myself and would rather not get too deep in setting up further stuff, if possible. For clarification, turning off two-way ssl and only doing server cert verification does work. It is only when the websocket asks for the clients cert that nothing happens and the error is in the console (client side, nothing shows on the server side).
Does anyone have ideas on how to keep two-way SSL for the site and websocket? Security is a major concern for this application.
This looks like the same problem as described in Cannot create WSS connection with Chrome, other browser work. The problem is that Chrome and Safari will not prompt for a client certificate for a Websockets connection. See this bug from 2013 which is still open.
If the user made already a choice which client certificate to use on the same origin it will work though. Only, an origin includes the port and your Websocket is on a different port than your website. This kind of setup can also cause other problems (like with restrictive firewalls) so it is a good idea to use a single origin and reverse proxy the Websocket from this origin. See this comment on the same bug on how to do this with nginx.
I want to test service workers but I have a virtual host setup and I can't seem to be able to enable https on localhost.
How can I whitelist my local virtual host url to test service workers
whenever I try and register for the service worker on the localhost? Chrome says https is required to enable service worker. How can I get past this restriction at least for local testing.
In general, you need to serve both your page and your service worker script via HTTPS in order to use service workers. The rationale is described at Prefer Secure Origins For Powerful New Features.
There's an exception to the HTTPS requirement in place to facilitate local development: if you access your page and service worker script via http://localhost[:port], or via http://127.x.y.z[:port], then service workers should be enabled without any further actions.
In recent versions of Chrome, you can work around this requirement duriing local development via chrome://flags/#unsafely-treat-insecure-origin-as-secure, as explained in this answer.
Firefox offers similar functionality, via the devtools.serviceWorkers.testing.enabled setting.
Please note that this functionality is only meant to facilitate testing that wouldn't otherwise be able to take place, and you should always plan on using HTTPS when serving the production version of your site. Don't ask real users to go through the steps of enabling those flags!
If you want to debug a plugged-in mobile device's service worker for a real behavior testing of a progressive web app, the ssl chrome start options do not help and you definitely do not need to buy certificates.
#chris-ruppel mentioned installing proxy software, but there is actually an easier way using port forwarding:
Assuming you connect and debug your device using Chrome:
In the Chrome Dev Tools "Remote devices" open "Settings" and add a "Port forwarding" rule.
If your localhost setup is running on localhost:80,
just add a rule "Device port 8080" (can be any unpriviliged port > 1024)
and local address "localhost:80" (or mytestserver.sometestdomainwithoutssl.company:8181 or whatever)
After you did that, you can call the URL "http://localhost:8080" on your mobile device and it will be answered by the "localhost:80" on your actual PC/test server. Works perfectly with service workers as if it were your local machine running on your mobile.
Works also for multiple port forwardings and different target domains as long as you remember to use unprivileged ports on your mobile device. See screenshot:
Source of this info is the google remote devices documentation: https://developers.google.com/web/tools/chrome-devtools/remote-debugging/local-server
(but as of Apr 2017 it is not very clear to read this simple answer out of it)
I often want to debug and test on a real device. One method I've come up with involves routing the phone's network traffic through Charles Proxy during local development. Unlike all the Chrome-specific solutions, this works with any browser on your phone.
Run Charles on my laptop (which also serves my website with the Service Worker). Once Charles is running, note the IP/port for Step 2.
Configure the mobile device to use my laptop as a proxy.
For Android just tap and hold on your WiFi in settings > Modify network > Advanced Settings > Proxy. Use Manual to set the IP/port.
For iOS click the (i) icon > HTTP Proxy section. Select Manual, then set the IP/port.
Visiting localhost on my mobile device now allows the Service Worker to be registered and tested.
The easiest way to test pwa, in my case, was using ngrok.
https://ngrok.com/download log in, get ur token and set it!
When you run ./ngrok http {your server port} make sure that you use https which will be shown in the terminal after you run this command above.
You could use https://surge.sh too, it is for host a static webpage, if you visit here: https://surge.sh/help/securing-your-custom-domain-with-ssl
will be able to see how to set up a ssl certificate
I used ngrok to tunnel the local IP (it is really not that because it is on Google Colab) to a public one.
Going to the ngrok console I can see all the tunnels created. I created just one tunnel for localhost:port but here there are 2, one for HTTP and other for HTTPS (isn't that nice?).
If I go to the https address of my Web App, on the console I see
But if I go to the http address, on the console I get
Q: Can you work with service workers that need HTTPs through tunnels to a remote machine?
A: Apparently yes!
The code behind that registration is (important to know where it fails):
// Here we register the SERVICE WORKER
IndexController.prototype._registerServiceWorker = function() {
console.log("1.Starting SW function.");
if (!navigator.serviceWorker) {
console.log("2.Browser is NOT compatible with SW or something is not working.");
return; }
console.log("2.Browser is compatible with SW.");
navigator.serviceWorker.register('/sw.js').then(function() {
console.log('3.Registration worked!');
}).catch(function() {
console.log('3.Registration failed!');
});
};
And to make it more complicated, my web App using Service Workers is running inside Colab (Google Colab). The web App is running on Node.js inside Colab.
If you are working from localhost it should be easier for you, because the https requirement is not enforced when connecting to localhost (according to theory). [A] and [B]
That is not the same that the browser will be nice with your App just because it runs on localhost.
Note: My experiment above..
Firefox: worked with and without the settings below.
Chrome: Without adding the urls to the whitelist and restartting I got
Going to https web app I got:
IndexController.js:49 Mixed Content: The page at 'https://0a4e1e0095b0.ngrok.io/' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://0a4e1e0095b0.ngrok.io/updates?since=1602934673264&'. This request has been blocked; this endpoint must be available over WSS.
IndexController._openSocket # IndexController.js:49
IndexController # IndexController.js:10
(anonymous) # index.js:16
loadScripts # loadScripts.js:5
46.../utils/loadScripts # index.js:15
s # _prelude.js:1
e # _prelude.js:1
(anonymous) # _prelude.js:1
IndexController.js:49 Uncaught DOMException: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.
at IndexController._openSocket (https://0a4e1e0095b0.ngrok.io/js/main.js:2251:12)
Going to http web app I got:
Navigated to http://0a4e1e0095b0.ngrok.io/
IndexController.js:17 1.Starting SW function.
IndexController.js:19 2.Browser is NOT compatible with SW or something is not working.
If you are not on localhost AND can't use https, then you may need to change these settings on your browser.
Some people already explained this but here it goes again.
Chrome:
Go to chrome://flags/#unsafely-treat-insecure-origin-as-secure
Add the urls you want to whitelist.
Restart chrome
Note that this will restart all Chrome windows. This is not a solution for me because my tunnels change name every time they are created and I can't be restarting a bunch of windows every time.
Firefox / Waterfox
Open Developer Tools
Open settings
Mark "Enable service workers over HTTP (when toolbox is open)"
Firefox/Waterfox
You probably don't need to do the changes below, but I did (my browser may be a bit old). More info here.
In about:config
I enabled
dom.serviceWorkers.testing.enabled
dom.serviceWorkers.enabled
I highly recommend looking at this https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers and related pages on that same site.
If someone is interested in the ngrok setup, it is very simple (python version).
# Install pyngrok python package on your Google Colab Session
!pip install pyngrok
# Set up your ngrok Authtoken (requires free registration)
!ngrok authtoken YOUR_TOKEN_HERE
# Invoke ngrok from Python and start tunneling/connecting
from pyngrok import ngrok
# Open a HTTP tunnel on the default port 80 if not specified
ngrok_tunnel = ngrok.connect('8888')
# You can print it, or go to the ngrok console on https://dashboard.ngrok.com/status/tunnels
print (ngrok_tunnel.public_url)
If you want to test service workers on a client device that cannot run a web server on localhost, the general technique is as follows:
Give your server a hostname.
Give this hostname a certificate.
Make IPs trust the CA that issued this certificate.
But this is easier said than done. In a November 2016 AMA on Reddit, a Let's Encrypt representative acknowledged that HTTPS on a private LAN "is a really tough question, and I think no one has come up with a satisfactory answer so far."
Common ways to give your computer a hostname involve giving it a stable internal IP address, not one that changes daily or every time you power-cycle your Internet gateway appliance. You'll need to configure the DHCP server on your network, usually that in your gateway, to set up a "reservation" that associates a particular private address (usualy within 10/8 or 192.168/16) with the MAC address of your development workstation's Ethernet card. For this, read the manual of your gateway.
Now that your development workstation has a stable IP address, there's a time/money tradeoff. If you're willing to learn advanced DNS and OpenSSL usage and install a root certificate on all devices with which you plan to test:
Run an internal DNS server on your network. This could be on your gateway or on your development workstation.
Configure your DNS server to be authoritative for some made-up TLD and recursive for other TLDs.
Give a stable name to your development workstation's private IP address. This gives it an internal name.
Configure your DHCP server to give the address of this DNS server to other devices obtaining leases.
On your development workstation, use OpenSSL to generate keypairs for a private certificate authority and the web server.
Using OpenSSL, issue a root certificate for the CA and a certificate for the web server's internal name.
Configure HTTPS in the web server on your development workstation using this certificate.
Install the CA's root certificate as a trusted root certificate on all devices.
On all devices, access this internal name.
If you cannot add a root certificate or control local DNS, such as if you plan to test with devices owned by others (BYOD) or with more locked-down browsers that do not allow users to add trusted root certificates, such as those in major video game consoles, you'll need a fully-qualified domain name (FQDN):
Buy a domain from a registrar that offers DNS with an API. This could be directly within a TLD or from one of the dynamic DNS providers that has made it onto the Public Suffix List. (Non-PSL dynamic DNS providers are unacceptable because of rate limits imposed by Let's Encrypt.)
In this domain's zone file, point an A record at your development workstation's private IP address. This gives your development workstation a FQDN.
Use Dehydrated, an ACME client that supports the dns-01 challenge, to obtain a certificate for this FQDN from the Let's Encrypt certificate authority.
Configure HTTPS in the web server on your development workstation using this certificate.
On all devices, access this name.
As Jeff mentioned in the first response, you do not need https at the localhost level to test Service Workers. Service workers will register and work just fine as long as you access the localhost domain -- without HTTPS.
Once you have your application tested on localhost and you want to see how it works with https for real, the simplest approach would be to upload your app to GitHub. You can create a public domain for free (and with HTTPS!).
Here are the instructions: https://pages.github.com/
I think the easiest way to test service worker is to find a free hosting provider.
nowadays, there many sites that provide free hosting. you can easily host your app on this free servers.
I mostly use heroku and netlify. this are free and easy to use.
For mobile testing with Android over the network with plain HTTP, a simple solution is to install Termux on your phone, then use socat to port-forward from a local port to your development host:
socat tcp-l:8000,fork,reuseaddr tcp:192.168.1.170:8000
This will run in the background while the terminal is running. Then in Chrome you can connect over plain HTTP to http://127.0.0.1:8000.
Works great over the local network and no certs, apps or services needed.
I'm trying to use web sockets to connect from a Google Chrome browser on my phone to a server running node.js and socket.io.
Using the remote debugging tool in Google Chrome I get this error in the console
Failed to load resource http://localhost/socket.io/1/?t=1368744562638
This happens despite me specifying my internal LAN IP in code for the client like so:
var socket = io.connect('http://192.168.1.3');
Furthermore it seems like the first heartbeat request makes it but starts to fail after that.
The code runs as expected when running the client on the server.
I am of course a idiot. I had another javascript file that had not been updated to connect to the specific IP I had set and was still set to "localhost".
After updating the host that socket.io should connect to in that javascript file everything is now running smoothly :)
I have a problem with websockets and socket.io. When I try to connect to my node server with socket.io it initially connects using websockets but when reverts to jsonp-polling shortly after.
This is the output from the node sever when I connect:
8 Jun 07:01:15 - Initializing client with transport "websocket"
8 Jun 07:01:19 - Initializing client with transport "jsonp-polling"
8 Jun 07:01:19 - Client 16630339180119336 connected
This happens in Chrome & Safari.
I have updated to the latest socket.io version 0.6.17 and am running node 0.4.7.
I have tried deleteing my cookies and cache as suggested on github and SO, however the problem remains. Also, when I try to force websockets it never fully connects with a session ID.
Does anyone have any ideas?
Websocket API is not supported by default in all the browsers at the moment (as per my knowledge) it should work on chromium though try testing it on chromium or firefox(after editing the default settings)and see if that still reverts to XHRPolling.
I am running it on a different IP as I need to run node on port 80 which causes conflict on my web server with Apache. Can websockets/flashsockets not be use cross-domain?
Now there might be 2 different reasons for the bug from here
Web/Flash Sockets will not let u connect to the node.js client unless either u specify a differnt port like 81 or u specially specify apache to proxy the incoming request to Node.
an easy solution could be writing the Node.js based HTTP server to just relay data from Apache (and setting Apache to run on a differnt port then 80)
This link tells how to do that... in this process you can make Node.js do something like check if the request is from a websocket/httpbrowser if thats an http browser forward the request to Apache if not ie if thats from web/flash sockets then handle the socket accordingly. or as commented on the question. Specify APACHE to proxy to Node.js.
Flashsockets require you to serve a crossdomain policy file on port 843 are you sure you are providing a cross domain file? (I think socket.io has inbuilt functionality to do that but still its always good to check.)
As told on the socket.io main website
In order to provide realtime connectivity on every browser, Socket.IO selects the most capable transport at runtime, without it affecting the API.
WebSocket
Adobe® Flash® Socket
AJAX long polling
AJAX multipart streaming
Forever Iframe
JSONP Polling
It's pretty clear that it will revert to AJAX Long Polling if websockets are disabled and Adobe Flash Socket fails to connect (this might be due to the unavailability of the policy file).
Here's a sample code for the cross domain file which you can include in your code and see if that makes your server run with websockets.
var net = require("net");
// Node.js
var Policy = net.createServer(function(socket)
{
socket.setEncoding('utf8');
socket.on('connect',function(){
console.log("Policy Request");
socket.end("<?xml version=\"1.0\"?><!DOCTYPE cross-domain-policy SYSTEM \"/xml/dtds/cross-domain-policy.dtd\"><cross-domain-policy><allow-access-from domain=\"*\" to-ports=\"*\" secure=\"false\"/></cross-domain-policy>");
});
});
Policy.listen(843);
I am just getting to grips with socket.IO and nodeJS. Got my web app working ok with them. I then got a friend to try it out at work on an office machine and found that it failed to connect.
I set up these two test cases:
http://thebeer.co/labs/rt/chat.php (server JS here) - This is an exact copy of the socket.IO chat example.
http://thebeer.co/labs/rt/test.php (server JS here)
Both of them fail for him. I also got a friend to try on a University computer and that too failed to connect! I've tried node servers listening on ports 8100, 8080, and 81-90 after being told that lower port numbers are less likely to be blocked by secure networks.
Really don't understand, it is very important that that real-time functionality is accessible to everyone, what am I doing wrong?
How can I get socket.IO to connect within secure Office and University networks?
A lot of big corporate networks will block everything other than port 80 (http) and port 443 (https) for most of their users. Try and put everything over one of those two for maximum compatibility.
Even when you get the everything running on port 80, also keep in mind that a lot enterprisey networks run HTTP traffic through content filters or other proxies that might not understand the websockets Upgrade header... you might want to try forcing socket.io to use one of the background compatibility transports and see if that helps.
FWIW, +1 to Colin Pickard because he is probably right... I just thought I would add this point just in case.
Take a look at both these links. I helped another so member who wanted the same.
I'm running nodejs on port 80 and I have no issues with corporate networks. (One thing I needed to have working as a priority)
How do I run Node.js on port 80?
https://serverfault.com/questions/273143/binding-apache-to-specific-ip-address/273181#273181