Ignore HSTS in Chrome app/extension - javascript

I'm writing a Chrome packaged app for diagnosing web services. I want to be able to send a GET request to a URL and look at the headers and data in the response.
My problem is if a users visits a site that has the HSTS header set before using my app, my app will then be unable send GET requests to the http:// URLs for that domain because Chrome will automatically convert the http:// URLs to https:// ones before the request is sent out.
Is there anything at all I can do to prevent this? I've looked into the webrequest API and webview tag but I'm finding nothing that lets me ignore HSTS.
Is it possible to use https://developer.chrome.com/apps/sockets_tcp for this (I would need to be able to support http, https and gzipped data)?

Is there anything at all I can do to prevent this?
Probably not. If you already tested <webview> and it shares the HSTS list with the browser, then the network layer will transparently rewrite this for you.
Is it possible to use chrome.sockets.tcp for this?
Technically, yes, HSTS shouldn't matter for that. Practically, you would need to implement something like wget+SSL+gzip from ground up (in JS, NaCl or a Native Host - but in the latter case you don't really need built-in sockets).

Related

Page indexing problem with Google search console

Recently, I created my first website and registered at Google search console.
The address is https://www.chemjix.com. However, as I always landed on the insecure http address when entering just www.chemjix.com in the address bar, I use the index.html to forward to https and directly to the correct language (English or German). After adding the website to Google search console, I got the message for the http that it is not indexed because of "Soft 404 Website" and "Page with redirect".
Is that a problem and does Google therefore rank my website lower or something like that? And what would be the alternative, do I need to get rid of that redirection?
Thanks a lot
celdri
You should use your hosting provider's tools to setup your domain correctly. You should use only https, as these days this is a must have for websites.
Most providers have some kind SSL or integrated the free Let's Encrypt solution for a real free for life SSL.
If you have a working site in https, you may use the hosting provider's own solution (like in Cloudways for example we have a swith) to redirect every URL to https, or you should use .htaccess redirect rules.
You also can additionally take advantage of Cloudflare not just to domain managament and caching, but to setup, and handle and check full SSL route from user to server.

Migrate localStorage data from HTTP to HTTPS

I've made a web app that stores persistent user data in localStorage. I've enabled HTTPS on my site, and I'd really like to flip the switch on HSTS. However, as far as I can tell, localStorage considers http://example.com and https://example.com to be different, so if existing users are redirected to the HTTPS version of my site, they can no longer access their data (though it still exists).
Long term, I want to build a new version of this app that offers more options for data storage. But in the short term, all I can think of is having a transition period where users at the http version are asked to migrate their data via some other (unknown) mechanism that both versions can access.
Is this a fair assessment of my options? Is there a way for https://example.com to access the localStorage of http://example.com? If not, is there anywhere I can put user data such that both versions can access it but other sites can't? Or should I ask them to download their data and re-upload it? That doesn't see ideal from a UX or (user) security standpoint.
Note that this web app doesn't interact with a server at all; everything happens with localStorage and the client.
Unfortunately there does not seem to be any way to directly retrieve localStorage contents for the less secure http copy of the site from the https site. reference
The workaround I have seen is to use an iframe loading a special page on the insecure site similar to this answer. The general theory is to have code in the iframe that sends messages using postMessage with localStorage data back to the secure page.
Unfortunately this approach does not allow you to disable http entirely, because otherwise your iframed http copy would not load.

Why script tags referencing "localhost:3000" fail

I am in the middle of a massive refactoring of my companies' web services and in doing so we are splitting out our static web files (HTML, CSS, JS) from our auth server and our API service. Our migration plan requires us to keep the old web server for backwards compatibility, therefore in order to do local dev I need to setup all my script tags to reference "localhost:3000", however, when I do that the requests fail in the browser
I can go directly to "localhost:3000/scripts/core.js" just fine in the browser, but if I put the exact same URL into a tag, all of a sudden the request fails in Chrome. Doesn't give me an error or anything, just says "failed"
Anyone know what I need to do to get this to work?
Because "localhost:3000/scripts/core.js" is a filepath name but "http://localhost:3000/scripts/core.js" is not. The first gives no transport mechanism but tries to attempt to open file:/// while the second specifies HTTP.
So test Or https:// as the case may be.
As far as SSL goes, this may be a separate issue. Is your SSL certificate signed for your local server or at least self-signed and accepted by your browser? Enquiring minds need to know but this may need to be a separate question.

Open URL with Custom Request headers

My team hosts a set of central web pages that is used by many different organizations. These pages change their look and feel (fonts, images, etc.) based on which organization calls the page. This is determined by a custom HTTP Request Header: "organization". I am building a test site to test the look and feel of ALL of the different organizations.
My plan was to have a web site with a drop down where our QA people can choose an Org then click something (button/link) to open the central web pages with the look and feel for that org. Note that when calling these central pages the URL in the browser MUST change to the URL of the page. So far all ideas/samples I can find involve getting the page content from the remote server and displaying it IN the calling page (URL does NOT change). Bottom line is I need to be able to set HTTP Request Headers then open a new URL with those headers.
I can use JavaScript, ASP Classic, Java and/or other similar technologies/languages. Any ideas to get me started?
I did find some similar questions to mine but none of them allow the URL to change in the browser so it doesn't work.
EDIT:
OK so it seems to not be possible via code. We cannot use a proxy since corporate has locked our browsers down and we cannot change proxy settings (even on dynamically created browser profiles). So is it possible to add custom HTTP Request Headers in IIS Express? If so I can write java test cases that modify the config files of IIS Express then start the server and load a central test page that redirects to the appropriate pages. Can this be done?
I am dying to know how the architect of this solution intended for end users to add headers indicating their organizational membership. Corporate proxy perhaps? Who knows. Seems cray cray. Whatever the intention was, you should be trying to adhere to it, if you want an accurate test.
But if can't do that, and your sole need is to perform testing under the assumption that the headers can be added somehow (just not by you), and you have control over your browser, you can use an add-on, such as Modify Headers for FireFox.
You could also build something separate to act as a proxy and add the headers, such as the solution to this question. Point your browser at the proxy and voila.

How can I override the Origin header in Chrome when connecting to a WebSocket?

I am trying to connect to an external web socket server, which is not run by myself. I would like to connect to it from a localhost javascript file, therefore the origin header has null value.
I understand that this is a measure against cross-site forgery. However, since I am on localhost, I should be able to fake this, by getting Chrome to send a custom Origin header.
Is it possible? (if I need an extension, that is fine)
If not, what is my best option to achieve the above? Thank you.
Web pages cannot change the Origin header, but extensions can modify the request headers via the chrome.webRequest API. But ws:// and wss:// are not supported by this API, so this doesn't help unless the server also supports other means of communication via http(s) (e.g. long-polling).
There is still a solution though: Simply load a (known) web page at the desired origin in an iframe (e.g. https://example.com/favicon.ico or https://example.com/robots.txt) and use a content script to open the WebSocket from there.
The Origin header is one of the headers that are set automatically by the user agent (as part of the browser implementation), and cannot be altered programatically or through extensions. This makes sense because web service providers cannot allow random connections from localhosts.
You can connect to an external WebSocket only if you do it from a host explicitly accepted by the web service provider. Many headers cannot be trusted (because they can be overridden), but this is not the case with Origin as it offers security not only for users, but also for service providers against unwanted connections.
As far as I know this will not be possible, it would break the security guards against CSRF in Chrome.
If you were able to do that the whole concept of XHR would fall apart.
Here is an Extension you can use to manipulate header on the fly, but so far I have not been able to get it to manipulate socket headers.
Look here if you want to read more about this.
But this doesn't stop you from implementing your own client (in place of chrome) where you can literally send whatever headers you want, not sure if this helps you, sorry.
It depends how you want to use your chrome browser. Since you mention localhost I assume you develop and will use this for some kind of scraping. I suggest that you explore Chrome DevTools Protocol which will render (almost) any kind of protection useless because you use a real browser. CORS, Origin, Cookie or any arbitrary header value will be under your control, and you can send a custom header for xhr/websocket request(s). If you want to manipulate in a more advanced way you can use Network.continueInterceptedRequest. You might only want to start chrome using parameters like "--disable-web-security, --disable-xss-auditor, --disable-client-side-phishing-detection, --allow-insecure-localhost" more about such options at peter.sh. However, the last option require a plugin in order to spoof origin header so I recommend the first option.

Categories