We Keep Coding

How to overcome Ajax Content Security Policy directive?

I am injecting some JS code on various websites (using Selenium and Python) to send POST requests to my local web server. On some websites it works fine. But mostly, I don't get the requests. Figured it's because of the Content Security Policy.
For example, when I try to run the code using Console in Chrome on github.com, I get a following error:
Refused to connect to 'http://10.50.50.127:7777/' because it violates
the following Content Security Policy directive: "connect-src 'self'
uploads.github.com status.github.com collector.githubapp.com
api.github.com www.google-analytics.com ...".
My code looks like this:
function sendData() {
var request = new XMLHttpRequest();
request.open('POST', 'http://10.50.50.127:7777', false);
request.send("test");
}
I did some research on my own, and found a possible solution - to use a local proxy server and send data to a relative path like "/test". But it's pretty complicated to write a proxy server from scratch.
So, what can I do to overcome this Content Security Policy?

If your using Chrome and you want to disable Content Security Policy you can also use a plugin called Disable Content-Security-Policy from Chrome Web Store. This is the plugin for Chrome to disable headers.
I inject JS via Tampermonkey (Chrome) and this works fine.

If you controlled over both of sides then you can use.
https://easyxdm.net/wp/
Regards

I figured it! Turns out you can just disable all of the security checks:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--disable-web-security')
chrome_options.add_argument('--allow-running-insecure-content')
browser = webdriver.Chrome(chrome_options=chrome_options)

Geoserver access REST in javascript - Why 403 Forbidden? [duplicate]

I am building a web API. I found whenever I use Chrome to POST, GET to my API, there is always an OPTIONS request sent before the real request, which is quite annoying. Currently, I get the server to ignore any OPTIONS requests. Now my question is what's good to send an OPTIONS request to double the server's load? Is there any way to completely stop the browser from sending OPTIONS requests?

edit 2018-09-13: added some precisions about this pre-flight request and how to avoid it at the end of this reponse.
OPTIONS requests are what we call pre-flight requests in Cross-origin resource sharing (CORS).
They are necessary when you're making requests across different origins in specific situations.
This pre-flight request is made by some browsers as a safety measure to ensure that the request being done is trusted by the server.
Meaning the server understands that the method, origin and headers being sent on the request are safe to act upon.
Your server should not ignore but handle these requests whenever you're attempting to do cross origin requests.
A good resource can be found here http://enable-cors.org/
A way to handle these to get comfortable is to ensure that for any path with OPTIONS method the server sends a response with this header
Access-Control-Allow-Origin: *
This will tell the browser that the server is willing to answer requests from any origin.
For more information on how to add CORS support to your server see the following flowchart
http://www.html5rocks.com/static/images/cors_server_flowchart.png
edit 2018-09-13
CORS OPTIONS request is triggered only in somes cases, as explained in MDN docs:
Some requests don’t trigger a CORS preflight. Those are called “simple requests” in this article, though the Fetch spec (which defines CORS) doesn’t use that term. A request that doesn’t trigger a CORS preflight—a so-called “simple request”—is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (for example, Connection, User-Agent, or any of the other headers with names defined in the Fetch spec as a “forbidden header name”), the only headers which are allowed to be manually set are those which the Fetch spec defines as being a “CORS-safelisted request-header”, which are:
Accept
Accept-Language
Content-Language
Content-Type (but note the additional requirements below)
DPR
Downlink
Save-Data
Viewport-Width
Width
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
No event listeners are registered on any XMLHttpRequestUpload object used in the request; these are accessed using the XMLHttpRequest.upload property.
No ReadableStream object is used in the request.

Have gone through this issue, below is my conclusion to this issue and my solution.
According to the CORS strategy (highly recommend you read about it) You can't just force the browser to stop sending OPTIONS request if it thinks it needs to.
There are two ways you can work around it:
Make sure your request is a "simple request"
Set Access-Control-Max-Age for the OPTIONS request
Simple request
A simple cross-site request is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (e.g. Connection, User-Agent, etc.), the only headers which are allowed to be manually set are:
Accept
Accept-Language
Content-Language
Content-Type
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
A simple request will not cause a pre-flight OPTIONS request.
Set a cache for the OPTIONS check
You can set a Access-Control-Max-Age for the OPTIONS request, so that it will not check the permission again until it is expired.
Access-Control-Max-Age gives the value in seconds for how long the response to the preflight request can be cached for without sending another preflight request.
Limitation Noted
For Chrome, the maximum seconds for Access-Control-Max-Age is 600 which is 10 minutes, according to chrome source code
Access-Control-Max-Age only works for one resource every time, for example, GET requests with same URL path but different queries will be treated as different resources. So the request to the second resource will still trigger a preflight request.

Please refer this answer on the actual need for pre-flighted OPTIONS request: CORS - What is the motivation behind introducing preflight requests?
To disable the OPTIONS request, below conditions must be satisfied for ajax request:
Request does not set custom HTTP headers like 'application/xml' or 'application/json' etc
The request method has to be one of GET, HEAD or POST. If POST, content type should be one of application/x-www-form-urlencoded, multipart/form-data, or text/plain
Reference:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS

When you have the debug console open and the Disable Cache option turned on, preflight requests will always be sent (i.e. before each and every request). if you don't disable the cache, a pre-flight request will be sent only once (per server)

Yes it's possible to avoid options request. Options request is a preflight request when you send (post) any data to another domain. It's a browser security issue. But we can use another technology: iframe transport layer. I strongly recommend you forget about any CORS configuration and use readymade solution and it will work anywhere.
Take a look here:
https://github.com/jpillora/xdomain
And working example:
http://jpillora.com/xdomain/

For a developer who understands the reason it exists but needs to access an API that doesn't handle OPTIONS calls without auth, I need a temporary answer so I can develop locally until the API owner adds proper SPA CORS support or I get a proxy API up and running.
I found you can disable CORS in Safari and Chrome on a Mac.
Disable same origin policy in Chrome
Chrome: Quit Chrome, open an terminal and paste this command: open /Applications/Google\ Chrome.app --args --disable-web-security --user-data-dir
Safari: Disabling same-origin policy in Safari
If you want to disable the same-origin policy on Safari (I have 9.1.1), then you only need to enable the developer menu, and select "Disable Cross-Origin Restrictions" from the develop menu.

As mentioned in previous posts already, OPTIONS requests are there for a reason. If you have an issue with large response times from your server (e.g. overseas connection) you can also have your browser cache the preflight requests.
Have your server reply with the Access-Control-Max-Age header and for requests that go to the same endpoint the preflight request will have been cached and not occur anymore.

I have solved this problem like.
if($_SERVER['REQUEST_METHOD'] == 'OPTIONS' && ENV == 'devel') {
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Headers: X-Requested-With');
header("HTTP/1.1 200 OK");
die();
}
It is only for development. With this I am waiting 9ms and 500ms and not 8s and 500ms. I can do that because production JS app will be on the same machine as production so there will be no OPTIONS but development is my local.

You can't but you could avoid CORS using JSONP.

you can also use a API Manager (like Open Sources Gravitee.io) to prevent CORS issues between frontend app and backend services by manipulating headers in preflight.
Header used in response to a preflight request to indicate which HTTP headers can be used when making the actual request :
content-type
access-control-allow-header
authorization
x-requested-with
and specify the "allow-origin" = localhost:4200 for example

After spending a whole day and a half trying to work through a similar problem I found it had to do with IIS.
My Web API project was set up as follows:
// WebApiConfig.cs
public static void Register(HttpConfiguration config)
{
var cors = new EnableCorsAttribute("*", "*", "*");
config.EnableCors(cors);
//...
}
I did not have CORS specific config options in the web.config > system.webServer node like I have seen in so many posts
No CORS specific code in the global.asax or in the controller as a decorator
The problem was the app pool settings.
The managed pipeline mode was set to classic (changed it to integrated) and the Identity was set to Network Service (changed it to ApplicationPoolIdentity)
Changing those settings (and refreshing the app pool) fixed it for me.

OPTIONS request is a feature of web browsers, so it's not easy to disable it. But I found a way to redirect it away with proxy. It's useful in case that the service endpoint just cannot handle CORS/OPTIONS yet, maybe still under development, or mal-configured.
Steps:
Setup a reverse proxy for such requests with tools of choice (nginx, YARP, ...)
Create an endpoint just to handle the OPTIONS request. It might be easier to create a normal empty endpoint, and make sure it handles CORS well.
Configure two sets of rules for the proxy. One is to route all OPTIONS requests to the dummy endpoint above. Another to route all other requests to actual endpoint in question.
Update the web site to use proxy instead.
Basically this approach is to cheat browser that OPTIONS request works. Considering CORS is not to enhance security, but to relax the same-origin policy, I hope this trick could work for a while. :)

One solution I have used in the past - lets say your site is on mydomain.com, and you need to make an ajax request to foreigndomain.com
Configure an IIS rewrite from your domain to the foreign domain - e.g.
<rewrite>
<rules>
<rule name="ForeignRewrite" stopProcessing="true">
<match url="^api/v1/(.*)$" />
<action type="Rewrite" url="https://foreigndomain.com/{R:1}" />
</rule>
</rules>
</rewrite>
on your mydomain.com site - you can then make a same origin request, and there's no need for any options request :)

It can be solved in case of use of a proxy that intercept the request and write the appropriate headers.
In the particular case of Varnish these would be the rules:
if (req.http.host == "CUSTOM_URL" ) {
set resp.http.Access-Control-Allow-Origin = "*";
if (req.method == "OPTIONS") {
set resp.http.Access-Control-Max-Age = "1728000";
set resp.http.Access-Control-Allow-Methods = "GET, POST, PUT, DELETE, PATCH, OPTIONS";
set resp.http.Access-Control-Allow-Headers = "Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since";
set resp.http.Content-Length = "0";
set resp.http.Content-Type = "text/plain charset=UTF-8";
set resp.status = 204;
}
}

What worked for me was to import "github.com/gorilla/handlers" and then use it this way:
router := mux.NewRouter()
router.HandleFunc("/config", getConfig).Methods("GET")
router.HandleFunc("/config/emcServer", createEmcServers).Methods("POST")
headersOk := handlers.AllowedHeaders([]string{"X-Requested-With", "Content-Type"})
originsOk := handlers.AllowedOrigins([]string{"*"})
methodsOk := handlers.AllowedMethods([]string{"GET", "HEAD", "POST", "PUT", "OPTIONS"})
log.Fatal(http.ListenAndServe(":" + webServicePort, handlers.CORS(originsOk, headersOk, methodsOk)(router)))
As soon as I executed an Ajax POST request and attaching JSON data to it, Chrome would always add the Content-Type header which was not in my previous AllowedHeaders config.

Issue with reading Response headers

I am trying to build a chrome extension for which i need to ping to different machines.The code which i tried with is able to read the response headers for a https site but not for http.I am new to Javascripting. Any help would be great.I understand it is a CORS issue and tried setting the headers in the client code.many forums mention it setting al the server side but where can I do in this case? Please find the code below and the plugin UI and response returned from https site in the snapshot.
Code--
url="https://www.icicibank.com/";
//url = "www.rediff.com/";
ping = new XMLHttpRequest();
ping.open("get", url,true);
//ping.setRequestHeader("Access-Control-Allow-Origin","*");
// ping.setRequestHeader("Access-Control-Allow-Credentials", "true");
ping.send(null)
ping.onreadystatechange=function() {
if (ping.readyState==4) {
alert(ping.getAllResponseHeaders());
//alertify.alert(ping.getAllResponseHeaders());
}
}
Thanks

CORS is indeed part of the server response, not the request. So you cannot "set" it on your side.
However, extensions are allowed to bypass CORS restrictions and make cross-origin requests. But for that you need to list domains you're going to connect to in manifest permissions. The user will be warned, at install time, that you'll interact with those domains.
For example, to allow requests to http://example.com and https://example.com domains regardless of CORS, you need to include in the manifest:
"permissions" : [
"*://example.com/"
],
If you can't say which sites you'll need to connect to in advance, you'll either need permissions for all urls (special permission, literally, "<all_urls>") or use Optional Permissions to request that at runtime.

Cross domain Ajax JSON POST support against RESTful WCF service using transportCredentialOnly security

I've posted before on this subject, but after a year of getting on with other things, I've managed to get into a pickle once again. I'll try and give a brief overview of the scenario and the current attempts to make things work:
IIS web server hosting HTML, JS etc. on host: iis.mycompany.com (referred to as foo)
WCF RESTful web services hosted via a Windows Service on host: wcf.mycompany.com (referred to as bar)
The Javascript served from foo works by making RESTful ajax calls (GET or POST depending on the action) to the WCF services on bar, obviously these are cross domain calls as they aren't on the same host.
The Javascript uses the jQuery (1.7.2) framework to manipulate the DOM and perform ajax calls to bar, the expected content type for POSTS is JSON, and the response from GETS is expected to be JSON too (application/json).
Bar has it's WCF services configured using TransportCredentialOnly as the security mode and the transport client credentail type is NTLM, so only authed users to contact the services.
CORS Support has been added to bar's WCF services using an extension to WCF:
http://blogs.msdn.com/b/carlosfigueira/archive/2012/05/15/implementing-cors-support-in-wcf.aspx
We have added additional headers and modfied some that the post already contained based on numerous internet articles:
property.Headers.Add("Access-Control-Allow-Headers", "Accept, Content-Type");
property.Headers.Add("Access-Control-Allow-Methods", "POST, GET, OPTIONS");
property.Headers.Add("Access-Control-Max-Age", "172800");
property.Headers.Add("Access-Control-Allow-Origin", "http://iis.mycompany.com");
property.Headers.Add("Access-Control-Allow-Credentials", "true");
property.Headers.Add("Content-type", "application/json");
Sites giving information on enabling CORS suggest that the Access-Control-Allow-Origin response header should be set to "*" however, this is not possible in our case as we make jQuery ajax calls using the following setup:
$.ajaxSetup({
cache: "false",
crossDomain: true,
xhrFields: {
withCredentials: true
}
});
As it turns out you cannot use "*" for the accepted origin when you are using "withCredentials" in the ajax call:
https://developer.mozilla.org/en/http_access_control
"Important note: when responding to a credentialed request, server
must specify a domain, and cannot use wild carding."
Currently in our development lab, this doesn't matter as we can hard code the requests to the IIS (foo) server URL.
The main problem now appears to be attempting POST requests (GET is working using the above configuration). When the browser attempts the POST process, it first sends an OPTIONS header to the server requesting allowed OPTIONS for the subsequent post. This is where we would like to see the headers we've configured in the CORS Support WCF extension being passed back, however we aren't getting that far; before the response comes back as "401 Unauthorized", I believe this is to do with the transport security binding configuration requesting NTLM, but I'm not sure.
Also, I'm not very experienced with this, but I haven't seen much information about POST using application/json content type as opposed to text/plain when performing cross domain requests.
I know that people will probably suggest JSONP as the one true solution, I'm not against different approaches, indeed I encourage anyone to suggest best practices as it would help others reading this question later. However, please attempt to answer the question before suggestion alternatives to it.
Many thanks in advance for anyone who contributes.
peteski
:)
UPDATE:
It appears that Chrome (20.x.x) doesn't suffer the problem of not negotiating NTLM to retrieve the OPTIONS header response from the server, but Firefox (13.0.1) does.
We've also noticed that someone has already posted a bug up on the Firefox forum, which we've added information to:
http://bugzilla.mozilla.org/show_bug.cgi?id=751552
Please vote for this bug to be fixed on the bugzilla site!
Using the following code, we can watch the network trace to see Firefox failing and Chrome working fine:
var url = "http://myWebServiceServer/InstantMessagingService/chat/message/send";
var data = '{ "remoteUserUri" : "sip:foo.bar#mydomain.com", "message" : "This is my message" }';
var request = new XMLHttpRequest();
request.open("POST", url, true);
request.withCredentials = true;
request.setRequestHeader("Content-Type", "application/json");
request.send(data);
console.log(request);
On a separate note, IE8 doesn't support the XMLHttpRequest for cross domain calls, favouring it's own magical XDomainRequest object, so we've got some work to do in changing the client side code to handle IE8 vs the world cases. (Thanks IE8).
/me crosses fingers that Mozilla fix the Firefox bug.
UPDATE 2:
After some digging it appears that IE8's XDomainRequest cannot be used to make cross domain requests where NTLM must be negotiated, this basically means that the security on our WCF binding can't be used thanks to limitations in a web browser.
http://blogs.msdn.com/b/ieinternals/archive/2010/05/13/xdomainrequest-restrictions-limitations-and-workarounds.aspx
"No authentication or cookies will be sent with the request"
So, I guess we've taken this as far as it is going to go for now.. It looks like we're going to have to create our own custom token authentication and pass it across to the WCF service in a cookie, or in IE8's case, POST it with the JSON. The WCF service will then have to handle decrypting the data and using that instead of the ServiceSecurityContext.Current.WindowsIdentity we previously had access to with NTLM auth.

I know you said you would rather have the problem itself addressed, but you may consider using a "reverse proxy."
I don't know what technologies you are using, but we use Apache web server and have a Java RESTful API running on a different server that required authentication. For a while, we messed with JSONP and CORS, but were not satisfied.
In the end, we setup an Apache Reverse Proxy and it worked miracles. The web browser believes it is communicating with its own domain and acts appropriately. The RESTful API doesn't know it is being used via a proxy. Therefore, everything just works. And Apache does all the magic.
Hopefully, all web servers have a feature like Apache's reverse proxy.
Here is some documentation on the feature: http://httpd.apache.org/docs/2.2/mod/mod_proxy.html
All we had to do is ensure the mod_proxy module was installed, then add the following lines to our Apache config file:
ProxyPass /restapi http://restfulserver.com/restapi
ProxyPassReverse /restapi http://restfulserver.com/restapi
Then restart the web server and voila!

Cross-site AJAX requests

I need to make an AJAX request from a website to a REST web service hosted in another domain.
Although this is works just fine in Internet Explorer, other browsers such as Mozilla and Google Chrome impose far stricter security restrictions, which prohibit cross-site AJAX requests.
The problem is that I have no control over the domain nor the web server where the site is hosted. This means that my REST web service must run somewhere else, and I can't put in place any redirection mechanism.
Here is the JavaScript code that makes the asynchronous call:
var serviceUrl = "http://myservicedomain";
var payload = "<myRequest><content>Some content</content></myRequest>";
var request = new XMLHttpRequest();
request.open("POST", serviceUrl, true); // <-- This fails in Mozilla Firefox amongst other browsers
request.setRequestHeader("Content-type", "text/xml");
request.send(payload);
How can I have this work in other browsers beside Internet Explorer?

maybe JSONP can help.
NB youll have to change your messages to use json instead of xml
Edit
Major sites such as flickr and twitter support jsonp with callbacks etc

The post marked as the answer is erroneous: the iframes document is NOT able to access the parent. The same origin policy works both ways.
The fact is that it is not possible in any way to consume a rest based webservice using xmlhttprequest. The only way to load data from a different domain (without any framework) is to use JSONP. Any other solutions demand a serverside proxy located on your own domain, or a client side proxy located on the remote domain and som sort of cross-site communication (like easyXDM) to communicate between the documents.

The fact that this works in IE is a security issue with IE, not a feature.
Unfortunately cross-site scripting is prohibited, and the accepted work around is to proxy the requests through your own domain: do you really have no ability to add or modify server side code?
Furthermore, the secondary workaround - involving the aquisition of data through script tags - is only going to support GET requests, which you might be able to hack with a SOAP service, but not so much with the POST request to a RESTful service you describe.
I'm really not sure an AJAX solution exists, you might be back to a <form> solution.

The not very clear workaround (but works) is using iframe as container for requests to another sites. The problem is, the parent can not access iframe's content, can only navigate iframe's "src" attribut. But the iframe content can access parent's content.
So, if the iframe's content know, they can call some javascript content in parent page or directly access parent's DOM.
EDIT:
Sample:
function ajaxWorkaroung() {
var frm = gewtElementById("myIFrame")
frm.src = "http://some_other_domain"
}
function ajaxCallback(parameter){
// this function will be called from myIFrame's content
}

Make your service domain accept cross origin resource sharing (CORS).
Typical scenario: Most CORS compliant browsers will first send an OPTIONS header, to which, the server should return information about which headers are accepted. If the headers satisfy the service's requirements for the request provided (Allowed Methods being GET and POST, Allowed-Origin *, etc), the browser will then resend the request with the appropriate method (GET, POST, etc.).
Everything this point forward is the same as when you are using IE, or more simply, if you were posting to the same domain.
Caviots: Some service development SDK's (WCF in particular) will attempt to process the request, in which case you need to preprocess the OPTIONS Method to respond to the request and avoid the method being called twice on the server.
In short, the problem lies server-side.
Edit There is one issue with IE 9 and below with CORS, in that it is not fully implemented. Luckily, you can solve this problem by making your calls from server-side code to the service and have it come back through your server (e.g. mypage.aspx?service=blah&method=blahblah&p0=firstParam=something). From here, your server side code should implement a request/response stream model.

Just use a server side proxy on your origin domain. Here is an example: http://jquery-howto.blogspot.com/2009/04/cross-domain-ajax-querying-with-jquery.html

This can also be done using a webserver setup localy that calls curl with the correct arguments and returns the curl output.
app.rb
require 'sinatra'
require 'curb'
set :views,lambda {"views/"+self.name.to_s.downcase.sub("controller","")}
set :haml, :layout => :'../layout', :format => :html5, :escape_html=>true
disable :raise_errors
get '/data/:brand' do
data_link = "https://externalsite.com/#{params[:brand]}"
c = Curl::Easy.perform(data_link)
c.body_str
end
Sending an ajax request to localhost:4567/data/something will return the result from externalsite.com/something.

Another option would be to setup a CNAME record on your own domain to "Mask" the remote domain hostname.