Connecting to a third-party service that has a self-signed certificate - javascript

I am building a website, which access a Philips HUE bridge over local network.
The site is https.
My problem is, that the certificate of the bridge is self-signed, which causes an error when I try to call it:
Failed to load resource: net::ERR_CERT_AUTHORITY_INVALID
From the documentation:
This means that in case you would access the bridge using a browser (e.g. going to https://) the browser will complain that the connection is not secure due to the fact that the certificate has not been certified by a trusted third party, and will ask you whether you want to continue and trust the certificate.
For your application it is best practice to pin (with the bridge-id) the certificate on first connection with the bridge (“trust on first use”) and check upon later contacts with the same bridge.
So the error is expected. But how can I get around it?
I have little experience with certificates, and https for that matter.
Any help would be appreciated.

Related

What security threats when using a crypto library without https?

I'm playing with JavaScript and I know that webcrypto API is not available without https but I want encipherment capability between a web server on LAN and a browser.
Using https with a self signed certificate will display a ugly warning message to the user that makes it unsuitable for my use case.
I've also tried to embedded an iframe in an https web page hosted online with a valid certificate using a service worker so that the encryption is done by the parent page of the iframe through postmessage api but when the https page go offline the subtlcrypto API become unavailable on some browser.
So can you propose some hacks please?
Please don't kill me, I'm a beginner.
Using https with a self signed certificate will display a ugly warning message to the user that makes it unsuitable for my use case.
You want Let’s Encrypt. You can use Certbot to get free TLS certificates. Certbot will even renew them for you.
If you want to try to continue without HTTPS at all, please don't. It's a foolhardy idea.
HTTPS is good and free; use it!
The only way to avoid the ugly warning displayed by the browser is to have a certificate signed by a CA trusted by the browser. So you can roll your own CA as suggested by B. Fleming, but this is a lot of work to maintain and make it trusted on all major browsers.
The main and decisive security thread I have found is the ability for any attacker to control the code of your embedded crypto library by using something like a MITM attack.
So as others suggested, using a Let's Encrypt certificate is a good idea, but you will have to define a domain name you control for each of your LAN servers (may be changing only subdomains) and bind it to a local IP address on your router.
Also, be aware that other attacks exists and a single point of failure is very dangerous in this kind of configuration.

How Do You Program a Website to Use a Certificate for Authentication?

My company is looking at using card based certificates (like CACs) for authentication on a web site; instead of the user entering a username and password this information would be extracted from the certificate. How does one go about programming the front end to pass the certificate on to the webserver? I've been looking for specific directions but wind up with everything but that. The over all idea is to hook into something like an LDAP for authentication eventually, but the browser part is completely eluding me.
You don't really have to do much in the way of the client side stuff. For a website, the browser handles it for you. You just have to setup your server to require a client certificate and the browser will handle it. Under Apache, after setting up the SSL certificate, this is as simple as setting SSLVerifyClient required (manual) in the apache configuration. Your site will then require a client certificate for access. You can also set SSLVerifyClient to optional which allows someone to click "Cancel" when prompted to select a certificate and the site will still load, but is missing the environment variables (see below).
If you are using smart cards, you might need a driver for the smart card reader to prompt the user for a pin to query the card, but it isn't something you need to handle. In my experience (with CAC cards), both Internet Explorer and Firefox use a third party software (we use ActiveIdentity) to ask for the user's pin (Firefox needs to be setup to use a "Security Device", but it is simple) and Chrome already has built in support for the smart cards without needing a separate program. It is also possible to install the certificate in the browser, but I haven't worked with that.
As for validating the user, in Apache, once the SSL handshake stuff has been completed by the browser and server, in PHP there are several environment variables that are available (you can see them on a phpinfo() page or if you print_r($_SERVER). They are all SSL_* and include stuff like the domain or common name). This can differ based on what information was supplied from the client. We just found one that held a unique id for the card $_SERVER['SSL_CLIENT_S_DN_CN'] and store that with each account. Then we can use that id to query for the account for creating a validated user session.
SSL client certificates are presented in the SSL handshake, which is done in the lower layers and is therefore transparent to the frontend (browser handles it internally). The web server needs to be configured to accept or require the client certificates so they are requested in the SSL handshake.
The client certificate is then validated by the web server and you can then access the client's identity typically via some API, depending on the server/platform.
EDIT:
Here's a guide for nginx/PHP: http://nategood.com/client-side-certificate-authentication-in-ngi

Options for testing service workers via HTTP

I want to test service workers but I have a virtual host setup and I can't seem to be able to enable https on localhost.
How can I whitelist my local virtual host url to test service workers
whenever I try and register for the service worker on the localhost? Chrome says https is required to enable service worker. How can I get past this restriction at least for local testing.
In general, you need to serve both your page and your service worker script via HTTPS in order to use service workers. The rationale is described at Prefer Secure Origins For Powerful New Features.
There's an exception to the HTTPS requirement in place to facilitate local development: if you access your page and service worker script via http://localhost[:port], or via http://127.x.y.z[:port], then service workers should be enabled without any further actions.
In recent versions of Chrome, you can work around this requirement duriing local development via chrome://flags/#unsafely-treat-insecure-origin-as-secure, as explained in this answer.
Firefox offers similar functionality, via the devtools.serviceWorkers.testing.enabled setting.
Please note that this functionality is only meant to facilitate testing that wouldn't otherwise be able to take place, and you should always plan on using HTTPS when serving the production version of your site. Don't ask real users to go through the steps of enabling those flags!
If you want to debug a plugged-in mobile device's service worker for a real behavior testing of a progressive web app, the ssl chrome start options do not help and you definitely do not need to buy certificates.
#chris-ruppel mentioned installing proxy software, but there is actually an easier way using port forwarding:
Assuming you connect and debug your device using Chrome:
In the Chrome Dev Tools "Remote devices" open "Settings" and add a "Port forwarding" rule.
If your localhost setup is running on localhost:80,
just add a rule "Device port 8080" (can be any unpriviliged port > 1024)
and local address "localhost:80" (or mytestserver.sometestdomainwithoutssl.company:8181 or whatever)
After you did that, you can call the URL "http://localhost:8080" on your mobile device and it will be answered by the "localhost:80" on your actual PC/test server. Works perfectly with service workers as if it were your local machine running on your mobile.
Works also for multiple port forwardings and different target domains as long as you remember to use unprivileged ports on your mobile device. See screenshot:
Source of this info is the google remote devices documentation: https://developers.google.com/web/tools/chrome-devtools/remote-debugging/local-server
(but as of Apr 2017 it is not very clear to read this simple answer out of it)
I often want to debug and test on a real device. One method I've come up with involves routing the phone's network traffic through Charles Proxy during local development. Unlike all the Chrome-specific solutions, this works with any browser on your phone.
Run Charles on my laptop (which also serves my website with the Service Worker). Once Charles is running, note the IP/port for Step 2.
Configure the mobile device to use my laptop as a proxy.
For Android just tap and hold on your WiFi in settings > Modify network > Advanced Settings > Proxy. Use Manual to set the IP/port.
For iOS click the (i) icon > HTTP Proxy section. Select Manual, then set the IP/port.
Visiting localhost on my mobile device now allows the Service Worker to be registered and tested.
The easiest way to test pwa, in my case, was using ngrok.
https://ngrok.com/download log in, get ur token and set it!
When you run ./ngrok http {your server port} make sure that you use https which will be shown in the terminal after you run this command above.
You could use https://surge.sh too, it is for host a static webpage, if you visit here: https://surge.sh/help/securing-your-custom-domain-with-ssl
will be able to see how to set up a ssl certificate
I used ngrok to tunnel the local IP (it is really not that because it is on Google Colab) to a public one.
Going to the ngrok console I can see all the tunnels created. I created just one tunnel for localhost:port but here there are 2, one for HTTP and other for HTTPS (isn't that nice?).
If I go to the https address of my Web App, on the console I see
But if I go to the http address, on the console I get
Q: Can you work with service workers that need HTTPs through tunnels to a remote machine?
A: Apparently yes!
The code behind that registration is (important to know where it fails):
// Here we register the SERVICE WORKER
IndexController.prototype._registerServiceWorker = function() {
console.log("1.Starting SW function.");
if (!navigator.serviceWorker) {
console.log("2.Browser is NOT compatible with SW or something is not working.");
return; }
console.log("2.Browser is compatible with SW.");
navigator.serviceWorker.register('/sw.js').then(function() {
console.log('3.Registration worked!');
}).catch(function() {
console.log('3.Registration failed!');
});
};
And to make it more complicated, my web App using Service Workers is running inside Colab (Google Colab). The web App is running on Node.js inside Colab.
If you are working from localhost it should be easier for you, because the https requirement is not enforced when connecting to localhost (according to theory). [A] and [B]
That is not the same that the browser will be nice with your App just because it runs on localhost.
Note: My experiment above..
Firefox: worked with and without the settings below.
Chrome: Without adding the urls to the whitelist and restartting I got
Going to https web app I got:
IndexController.js:49 Mixed Content: The page at 'https://0a4e1e0095b0.ngrok.io/' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://0a4e1e0095b0.ngrok.io/updates?since=1602934673264&'. This request has been blocked; this endpoint must be available over WSS.
IndexController._openSocket # IndexController.js:49
IndexController # IndexController.js:10
(anonymous) # index.js:16
loadScripts # loadScripts.js:5
46.../utils/loadScripts # index.js:15
s # _prelude.js:1
e # _prelude.js:1
(anonymous) # _prelude.js:1
IndexController.js:49 Uncaught DOMException: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.
    at IndexController._openSocket (https://0a4e1e0095b0.ngrok.io/js/main.js:2251:12)
Going to http web app I got:
Navigated to http://0a4e1e0095b0.ngrok.io/
IndexController.js:17 1.Starting SW function.
IndexController.js:19 2.Browser is NOT compatible with SW or something is not working.
If you are not on localhost AND can't use https, then you may need to change these settings on your browser.
Some people already explained this but here it goes again.
Chrome:
Go to chrome://flags/#unsafely-treat-insecure-origin-as-secure
Add the urls you want to whitelist.
Restart chrome
Note that this will restart all Chrome windows. This is not a solution for me because my tunnels change name every time they are created and I can't be restarting a bunch of windows every time.
Firefox / Waterfox
Open Developer Tools
Open settings
Mark "Enable service workers over HTTP (when toolbox is open)"
Firefox/Waterfox
You probably don't need to do the changes below, but I did (my browser may be a bit old). More info here.
In about:config
I enabled
dom.serviceWorkers.testing.enabled
dom.serviceWorkers.enabled
I highly recommend looking at this https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers and related pages on that same site.
If someone is interested in the ngrok setup, it is very simple (python version).
# Install pyngrok python package on your Google Colab Session
!pip install pyngrok
# Set up your ngrok Authtoken (requires free registration)
!ngrok authtoken YOUR_TOKEN_HERE
# Invoke ngrok from Python and start tunneling/connecting
from pyngrok import ngrok
# Open a HTTP tunnel on the default port 80 if not specified
ngrok_tunnel = ngrok.connect('8888')
# You can print it, or go to the ngrok console on https://dashboard.ngrok.com/status/tunnels
print (ngrok_tunnel.public_url)
If you want to test service workers on a client device that cannot run a web server on localhost, the general technique is as follows:
Give your server a hostname.
Give this hostname a certificate.
Make IPs trust the CA that issued this certificate.
But this is easier said than done. In a November 2016 AMA on Reddit, a Let's Encrypt representative acknowledged that HTTPS on a private LAN "is a really tough question, and I think no one has come up with a satisfactory answer so far."
Common ways to give your computer a hostname involve giving it a stable internal IP address, not one that changes daily or every time you power-cycle your Internet gateway appliance. You'll need to configure the DHCP server on your network, usually that in your gateway, to set up a "reservation" that associates a particular private address (usualy within 10/8 or 192.168/16) with the MAC address of your development workstation's Ethernet card. For this, read the manual of your gateway.
Now that your development workstation has a stable IP address, there's a time/money tradeoff. If you're willing to learn advanced DNS and OpenSSL usage and install a root certificate on all devices with which you plan to test:
Run an internal DNS server on your network. This could be on your gateway or on your development workstation.
Configure your DNS server to be authoritative for some made-up TLD and recursive for other TLDs.
Give a stable name to your development workstation's private IP address. This gives it an internal name.
Configure your DHCP server to give the address of this DNS server to other devices obtaining leases.
On your development workstation, use OpenSSL to generate keypairs for a private certificate authority and the web server.
Using OpenSSL, issue a root certificate for the CA and a certificate for the web server's internal name.
Configure HTTPS in the web server on your development workstation using this certificate.
Install the CA's root certificate as a trusted root certificate on all devices.
On all devices, access this internal name.
If you cannot add a root certificate or control local DNS, such as if you plan to test with devices owned by others (BYOD) or with more locked-down browsers that do not allow users to add trusted root certificates, such as those in major video game consoles, you'll need a fully-qualified domain name (FQDN):
Buy a domain from a registrar that offers DNS with an API. This could be directly within a TLD or from one of the dynamic DNS providers that has made it onto the Public Suffix List. (Non-PSL dynamic DNS providers are unacceptable because of rate limits imposed by Let's Encrypt.)
In this domain's zone file, point an A record at your development workstation's private IP address. This gives your development workstation a FQDN.
Use Dehydrated, an ACME client that supports the dns-01 challenge, to obtain a certificate for this FQDN from the Let's Encrypt certificate authority.
Configure HTTPS in the web server on your development workstation using this certificate.
On all devices, access this name.
As Jeff mentioned in the first response, you do not need https at the localhost level to test Service Workers. Service workers will register and work just fine as long as you access the localhost domain -- without HTTPS.
Once you have your application tested on localhost and you want to see how it works with https for real, the simplest approach would be to upload your app to GitHub. You can create a public domain for free (and with HTTPS!).
Here are the instructions: https://pages.github.com/
I think the easiest way to test service worker is to find a free hosting provider.
nowadays, there many sites that provide free hosting. you can easily host your app on this free servers.
I mostly use heroku and netlify. this are free and easy to use.
For mobile testing with Android over the network with plain HTTP, a simple solution is to install Termux on your phone, then use socat to port-forward from a local port to your development host:
socat tcp-l:8000,fork,reuseaddr tcp:192.168.1.170:8000
This will run in the background while the terminal is running. Then in Chrome you can connect over plain HTTP to http://127.0.0.1:8000.
Works great over the local network and no certs, apps or services needed.

websockets & VPN

I understand that websocket is still being worked on. Now, I don't know if what I'm considering is even technically possible but I'm just bouncing off ideas.
What I'm thinking of is a client less SSL VPN using websockets. Is it possible to create a websocket & redirect all the traffic from the browser (on that particular site/domain) through this socket. So lets us say you go to a site http://example.com & this site will set up a websocket back to it's server. Now can we in any way capture all the traffic going from that browser tab & push it through that websocket tunnel (wss://). This way you can have a client less SSL VPN solution.
Now, the biggest problem I can see is how do you actually grab all the traffic going from that browser tab or window. I don't think javascript has or will have enough privileges or even capabilities to do that. Any thoughts?
You could present your own browser UI (URL bar + rendering area), push out HTTP requests over your tunnel and parse & present the returned HTML in the rendering area. But you are correct, you aren't going to be able to capture all browser traffic in javascript without somehow escalating privileges (for example, as a Firefox extension).
A web proxy is really what you are describing: http://en.wikipedia.org/wiki/Proxy_server
All browsers have support for HTTP proxy server settings. If the proxy encapsulated the data with SSL and sent it on to another proxy within the firewall (I assume that's why you mention VPN) then I think you have what you are asking. I don't think WebSockets really has any relevance here. You could use it, but it would just be harder.

Self-signed certs -- helping users know they need to add root CA to trusted cert store

I have a desktop product which uses an embedded webserver which will use self-signed certs.
Is there something that I can put in a web page that would detect that they haven't added the root CA to their trusted list, and display a link or DIV or something directing them how to do it?
I'm thinking maybe a DIV that has instructions on install the CA, and a Javascript that runs some test (tries to access something without internal warnings??), and hides the DIV if the test succeeds. Or something like that...
Any ideas from the brilliant SO community ? :)
Why do you want to do this? It is a bad idea to train users to indiscriminately install root CA certificates just because a web site tells them to. You are undermining the entire chain of trust. A security conscious user would ignore your advice to install the certificate, and might conclude that you are not taking security seriously since you did not bother to acquire a certificate from an existing CA.
Do you really need HTTPS? If so, you should probably bite the bullet and make a deal with a CA to facilitate providing your customers with proper CA signed server certificates. If the web server is only used for local connections from the desktop app, you should either add the self-signed certificate to the trusted list as part of the installation process, or switch to HTTP instead.
Assuming you know C# and you want to install a pfx file.Create a exe that will be run from a url.Follow this URL
and this
The only idea I have is to use frames and some javascript.
The first element of the frame will act as a watchdog waiting x amount of time (javascript setTimeout) before showing your custom ssl failure message to the user with hyperlinks or instructions to download the self-signed cert.
The second frame element attempts the https connection and if successful resets the watchdog frame so that it never fires. If it fails (assume https cert validation failed) the watchdog message would then fire and be presented to the user.
Depending on your browser you will most likely still see some security warning with the approach but you would at least be able to push your own content without requiring users to run untrusted code with no proper trust chain (This would be much much worse from a security POV than accepting the cert validation errors and establishing an untrusted ssl session)
Improvements to the concept may be possible using other testing methods such as XMLHttpRequest et al.
You should not do this. Root certificates are not something you just install, since adding one could compromise any security given to you by https.
However if you are making a desktop app then just only listen to 127.0.0.1. That way the traffic never leaves the users computer and no attacker can listen in.
You might try to add some (hidden) Flex element or Java Applet once per user session.
It will just download any https page of your server and will get all information about connection:
com.sun.deploy.security.CertificateHostnameVerifier.verify()
or
javax.security.cert.X509Certificate.checkValidity()
I suppose Flex (which is more common to users) shoul have similar ways of validating https certificate from user's point of view. It should also share OS' trusted cert. store while Java might have its own.
Since the server is running on the client machine (desktop product) can it not check the supported browsers for installed certs using winapi/os functions? I know Firefox has a cert database in the user's profile directory and IE probably keeps information in the registry. It wouldn't be reliable for all browsers but if the server simply chooses between "Certificate Found" and "Please ensure you have installed the cert before continuing" then no harm is done as the user can choose to continue either way.
You could also simplify matters by providing an embedded browser as well (ie, gecko), this way you only have 1 browser to deal with which simplifies a lot of things (including pre-installing the root CA).
To recap: you are setting up webservers on desktop apps; each desktop will have its own webserver, but you want to use SSL to secure the connection to that webserver.
I guess there are several problems here with certificates, one being that the hostname used to access the desktop has to match the certificate. In this case you have little choice but to generate certificates on the client. You'll need to allow the user some way to specify the host name in case the name used by outsiders can't be detected from the host itself.
I'd also suggest allowing for an admin to install a trusted cert, for those who don't want to rely on self-signed certs. This way you can also offload the cost of trusted cert maintenance to the admins who really want it.
Finally, in my experience browsers either allow or refuse the self-signed cert and there is no way for the server to know if the cert is denied, or temporarily accepted, or permanently accepted. I assume there must be a mechanism somewhere to handle SSL failures but typical web programming doesn't operate at that layer. In any case, the only thing a webserver can do if SSL fails is to fallback to non-SSL, and you've indicated in a comment that you can't have anything non-SSL. I think you should try to have that restriction lifted; a non-SSL start page would be extremely helpful in this situation: it can test (using frames or images or JSON or AJAX) the https connection, and it can link to documentation about how to set up the certificate, or where to download an installer for the cert.
If the browser won't connect because of a self-signed cert, and you're not allowed to use plain HTTP at all, by what other means could you communicate with the user? There are no other channels and you can't establish one because you don't have any communication.
You mentioned in a comment writing a win32 app for installing the cert. You could install a cert at the time you install the application itself, but that doesn't help any remote browsers, and a local browser doesn't need SSL to access localhost.
We've been working on an opensource JavaScript project, called Forge, that's related to this problem. Do you have a website that your users could access? If so, then you could provide a secure connection to those desktop apps via your website using a combination of Flash for cross-domain + JavaScript for TLS. It will require you to implement some web services on your website to handle signing certificates the desktop app certificates (or having your desktop apps upload the self-signed certs so they can be accessed via JavaScript). We describe how it works here:
http://blog.digitalbazaar.com/2010/07/20/javascript-tls-1/
An alternative to setting up a website, but is less secure because it allows for a MiTM attack is to host the JavaScript+Flash directly on the desktop app server. You could have your users hit your desktop app over regular http to download the JS+Flash+SSL cert, but then start using TLS afterwards via the JS. If you're on a localhost connection the MiTM attack might be a little less worrisome -- perhaps enough for you to consider this option.
An ActiveX control could do the trick. But I really didn't chime in to help with the solution, more to disagree with the stance that what you are doing is a security risk.
To be clear, you are needing a secure cipher (hopefully AES and not DES), and are already in control of your endpoints, just not able to completely rule out promiscuous-mode network sniffers that could catch clear-text passwords or other sensitive data.
SSL is a "Secure Socket Layer", and by definition, is NOT dependent upon ANY certificates.
However, all effective modern ciphers require it to authenticate the tunnel endpoints, which is not always a necessity for every application; a frustration I have dealt with in numerous back-end datacenter automation routines using web service APIs to manage nodes, where the "users" were actually processes that needed encrypted key exchange prior to a RESTful command negotiation.
In my case, the VLANs were secured via ACLs, so I really "could" send clear-text authentication headers. But just typing that made me throw up in my mouth a little bit.
I'm sure I'll get flamed for typing this, but I'm extremely battle-hardened and would've made the same comments to you in years 10-15 of my IT career. So I empathize with their worries, and very much appreciate if they are passionate enough about security to flame me. They'll figure it out eventually.....
But I do agree with the fact that it is a BAD idea to "train" users to install root CA's on their own. On the other hand, if you use a self-signed cert, you have to train them to install that. And if a user doesn't know how to determine if a CA Cert is trustworthy, they definitely won't be able to determine a self-signed cert from a CA Cert, and thus either process would have the same effect.
If it were me, I would automate the process instead of having it assist the end-users, so that it becomes as hidden from them as possible, just like a proper PKI would do for an enterprise.
Speaking of which, I just thought of a potential solution. Use the Microsoft PKI Model. With Server 2012 R2, you can deliver trusted keys to endpoints that are not even domain members using "device control" via "workspaces", and the client machines can subscribe to multiple workspaces, so they are not committed solely to yours if they subscribe. Once they do, and authenticate, the AD Certificate Services Role will push all root CA Certs necessary, as are present in active directory, or specified LDAP server. (In case you are using offline CA servers)
Also, I realize this thread is like 7 years old, but am sure it still gets referenced by a good number of people needing similar solutions, and felt obligated to share a contrasting opinion. (Ok Microsoft, where's my kickback for the plug I gave you?)
-cashman

Categories