Cross-domain workaround to load an external page - javascript

Here's the jQuery documentation:
http://api.jquery.com/load/
As mentioned there as an additional note:
Due to browser security restrictions, most "Ajax" requests are subject
to the same origin policy; the request can not successfully retrieve
data from a different domain, subdomain, or protocol.
Is there any way to bypass this limitation?

One way is to create a proxy page that requests the external page on the server. The implementation is dependent on the technology being used, but the idea is that you can then make an ajax call to your proxy page instead which will be on the same domain as the calling page.

Yes, there are several ways to get around this. But I would strongly suggest CORS (Cross-Domain Resource Sharing). CORS is the emerging standard for addressing cross-domain Ajax. This is replacing approaches such as JSONP (which has known security issues).
Unfortunately, CORS is not supported by older IE versions (6-9), but there are established polyfills for browsers which do not natively implement CORS. easyXDM is one such polyfill.
At a basic level, below is a sample request (from the MDN link above), where the browser generates a request with the Origin header, indicating the domain of the request, which is validated by the endpoint:
GET /resources/access-control-with-credentials/ HTTP/1.1
Host: bar.other
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b3pre) Gecko/20081130 Minefield/3.1b3pre
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://foo.example/examples/credential.html
Origin: http://foo.example
Cookie: pageAccess=2
An Ajax endpoint supporting CORS may respond with the appropriate headers, which the browser validates. See Access-Control-Allow-Origin and Access-Control-Allow-Credentials in this example:
HTTP/1.1 200 OK
Date: Mon, 01 Dec 2008 01:34:52 GMT
Server: Apache/2.0.61 (Unix) PHP/4.4.7 mod_ssl/2.0.61 OpenSSL/0.9.7e mod_fastcgi/2.4.2 DAV/2 SVN/1.4.2
X-Powered-By: PHP/5.2.6
Access-Control-Allow-Origin: http://foo.example
Access-Control-Allow-Credentials: true
Cache-Control: no-cache
Pragma: no-cache
Set-Cookie: pageAccess=3; expires=Wed, 31-Dec-2008 01:34:53 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 106
Keep-Alive: timeout=2, max=100
Connection: Keep-Alive
Content-Type: text/plain
The browser may also send a pre-flight (using the OPTIONS verb) which can indicate to the server what headers or verbs the Ajax call will use. The pre-flight is actually a separate call from the primary Ajax call, but is only invoked under certain conditions.
Many web servers (IIS, Apache) support CORS through modules. The servers will require configuration to white-list origins, allowed verbs, headers, etc. Wildcarding can also be used but is not recommended.

Related

Http cookie not sent even though CORS is configured to allow access

I have a web app that needs to access siteA and siteB (both are under my administration). After the web app talks with siteA (and gets a cookie from siteA) and it jumps to siteB, it needs to make a call with siteA, but the http request doesn't contain any cookie even though I configured (via cors) siteA to trust siteB and allow credential.
Have been googling and reading documents both in SO and other sites on this, however, I just can't get the cookie to be sent in the HTTP request. Here is the javascript that makes the API call with siteA (running in a page on siteB, browser is chrome)
await (await fetch("http://siteA/api/1/greeting", {credentials: 'same-origin'})).text()
"Greetings from Spring Boot!"
Here is the HTTP request sent by the above javascript.
GET /api/1/greeting HTTP/1.1
Host: siteA
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4117.2 Safari/537.36
Accept: */*
Origin: https://siteB
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Here is the response:
HTTP/1.1 200
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Access-Control-Allow-Origin: http://siteB
Access-Control-Allow-Credentials: true
Set-Cookie: jname=joe; HttpOnly
Content-Type: text/plain;charset=UTF-8
Content-Length: 27
Date: Sun, 26 Apr 2020 17:58:48 GMT
Keep-Alive: timeout=60
Connection: keep-alive
Any help or ideas are appreciated!
PS1: When I refresh a web page of siteA, I can see (from dev console) that the request has the cookie.
PS2: tried to use credentials: 'include' in the javascript snippet, it didn't work either.
Can you try changing your code above to:
await (await fetch("http://siteA/api/1/greeting", {credentials: 'include'})).text()
See here: https://javascript.info/fetch-crossorigin

Access Control Allow Origin header Error Only in Chrome,Safari but not Opera

I have enable Access-Control-Allow-Origin on my apache webserver. I am able to load all requests on the first load. However, subsequent loading/refreshing of the pages would leave half of the requests failed, leaving an error of,
XMLHttpRequest cannot load http://***. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access.
In Chrome Developer Tool, under the Network tab, enabling "Disable Cache" removes this error.
I am using AngularJS $http to call my request, I have changed the cache option to true/false but this abnormality persist. For your information, I am also using an angular-cache module, https://github.com/jmdobry/angular-cache, I have tried enabling/disabling this too, but to no avail.
Here is a copy of the request header:
GET ***Valid_Url_With_Parameters** HTTP/1.1
Host: ********
Connection: keep-alive
Accept: application/json, text/plain, */*
Origin: http://localhost:3000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36
Referer: http://localhost:3000/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
I have a 2 different types response header with different HTTP status codes for the failed requests.
HTTP/1.1 200 OK
Content-Length: 564121
Content-Type: application/json
Server: Apache/2.2.15 (CentOS)
Vary: Accept
Last-Modified: Fri, 16 Sep 2016 12:41:41 GMT
Connection: keep-alive
Date: Fri, 16 Sep 2016 12:43:44 GMT
HTTP/1.1 206 Partial Content
Content-Type: application/json
Server: Apache/2.2.15 (CentOS)
Vary: Accept
Last-Modified: Fri, 16 Sep 2016 13:39:05 GMT
Content-Range: bytes 29200-29200/785592
Content-Length: 1
Connection: keep-alive
Date: Fri, 16 Sep 2016 14:20:48 GMT
It seems something to do with caching. The first time the page loads, all of the request have the correct access control header. Subsequent refresh/reload removes this header. How can I continue using caching and keep the access control header?
Edit:
Just to add on, I tried using Safari, the same abnormality appeared. It load on the first try but subsequent loading will throw a no access control allow origin error. However, this time, the response header in Safari shows blank.
Edit 2:
Abnormality appears in Firefox as well but not in Opera Beta version 40.0.2308.52. I am beginning to wonder if this is the cause of the browser. Access-Control-Allow-Origin header is present too.
Response Header in Opera
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Allow: GET, POST, HEAD, OPTIONS
Content-Type: application/json
Date: Sat, 17 Sep 2016 06:56:53 GMT
Server: Apache/2.2.15 (CentOS)
Vary: Accept
Transfer-Encoding: chunked
Edit 3:
I would like to add that the requests that I'm pulling is a large amount of data. I am pulling the last 7 days history of data, there are about 20 JSON files, each ranging from 500KB to 1000KB. I noticed that when I change the query parameter to pull in a small history of data, this error does not happen.
Any help is much appreciated.
One work around I've found is to dynamically generate a javascript file on the server as resource included into your page. This doesn't require AJAX the which gets blocked by CORS and can be cached.
We had this same problem in Chrome when returning a specific value in the 'Access-Control-Allow-Origin' header. We were able to solve it by also including Vary: Origin in the response headers.
It appears Chrome considers response headers, including 'Access-Control-Allow-Origin', as part of the cached response. So if it requests the same file from a different origin, that header from the cache will not match. But if you include Vary: Origin it lets Chrome know the response (just this one header in our case) will be different based on the Origin of the request.
This page on MDN explains it:
If the server sends a response with an Access-Control-Allow-Origin
value that is an explicit origin (rather than the "*" wildcard), then
the response should also include a Vary response header with the value
Origin — to indicate to browsers that server responses can differ
based on the value of the Origin request header.
I would recommend you to add chrome extension in your browser
Allow-Control-Allow-Origin: *

Cross domain jQuery ajax call with credentials

I've followed the following steps:
Get the server to allow cross domain calls (with all the headers and stuff) This works
Test the server with some cross domain calls This works
Get the server to force a certificate This works
Go to a file on the server with a browser, choose the right certificate and see the file Still works
Now we get to the nice part
Combine the cross domain calls with the certificate <-- this does not work
Problem
I am getting the certificate request from the browser, but when I select the same certificate as I do when using the browser, the call is made but I get a 403 Forbidden.
Code
$.ajax({
type: "POST",
xhrFields: {withCredentials: true},
dataType: "xml",
contentType: "text/xml; charset=\"utf-8\"",
url: "https://www.myOtherServer.com/testfile.asp",
});
Any ideas?
Edit
The Access-Control-Allow-Credentials: true and the Access-Control-Allow-Origin are properly configured.
Additional information
I'm starting to think that it has something to do with the content type. When I change it to "text/html" I get a 415 error, but I do really need to send xml because it is a SOAP server.
Response headers
Access-Control-Allow-Cred... true
Access-Control-Allow-Head... Content-Type, Origin, Man, Messagetype, Soapaction, X-Test-Header
Access-Control-Allow-Meth... GET,POST,HEAD,DELETE,PUT,OPTIONS
Access-Control-Allow-Orig... https://www.mywebsite.com
Access-Control-Max-Age 1800
Cache-Control private
Content-Length 5561
Content-Type text/html; charset=utf-8
Date Wed, 19 Dec 2012 15:06:46 GMT
Server Microsoft-IIS/7.5
X-Powered-By ASP.NET
Request headers
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language nl,en-us;q=0.7,en;q=0.3
Access-Control-Request-He... content-type
Access-Control-Request-Me... POST
Cache-Control no-cache
Connection keep-alive
Host myhoast.com
Origin https://www.mywebsite.com
Pragma no-cache
User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0
My best guess is that this is a problem not with your Javascript but with your CORS configuration. Did you set up your server with the Access-Control-Allow-Credentials: true header? http://www.w3.org/TR/cors/#access-control-allow-credentials-response-header
Also note that, even when the allow-credentials header is set, the browser will not allow responses to credentialed requests if Access-Control-Allow-Origin is *, according to these docs: https://developer.mozilla.org/en-US/docs/HTTP/Access_control_CORS?redirectlocale=en-US&redirectslug=HTTP_access_control#Requests_with_credentials.
Edit: Since the OP has the CORS headers set up properly, the problem seems to be that the server is rejecting OPTIONS requests with a 403 status code. OPTIONS requests (known as the "preflight request") are sent before certain cross-domain requests (such as POSTs with application/xml content types), to allow the server to notify the browser of what types of requests are allowed. Since the browser doesn't see the 200 response that it expects from the OPTIONS request, it doesn't fire the actual POST request.
basicly we just have to write on htaccess
Header set Access-Control-Allow-Origin “*”
but when we need cookie etc, we had to add script on your ajax code and htaccess
i write about cross domain XHR on my blog, blog.imammubin.com/cross-domain-xhr/2014/05/28/ (Edit: site no longer exists)
hope this help..

jQuery getJSON doesnt send cookies

i am including JS on domain1 form domain2
<script type="text/javascript" src="http://www.domain2.com/script.js"></script>
that script doesn onload and on button click a JSONP request to domain2
$.getJSON( 'http://www.domain2.com/process?callback=?',
function(data){
if ( data ) processData( data );
}
);
and then displaying the data on domain1.
So here is my problem:
The getJSON request doesnt send cookies to the domain2.
The weirdest thing is that it does send the cookies half a day and the other half not. :-)
This is how the request looks like when it doesnt work:
Request details
GET /ajax/embed-user-library?detail=98&callback=jsonp1312398534998 HTTP/1.1
User-Agent: Opera/9.80 (Windows NT 6.1; U; en) Presto/2.9.168 Version/11.50
Host: www.floowie.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en,sk-SK;q=0.9,sk;q=0.8
Accept-Encoding: gzip, deflate
Referer: http://www.sokker.cz/en/test2
Connection: Keep-Alive
Response details
HTTP/1.1 200 OK
Date: Wed, 03 Aug 2011 19:06:51 GMT
Server: Apache/2.2.16 (Debian)
X-Powered-By: PHP/5.3.5-0.dotdeb.1
Set-Cookie: SESSID=64292b70dc28d7c6c9f13f70070353d8; path=/; domain=.floowie.com
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-cache, must-revalidate
Pragma: no-cache
Content-Length: 34
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: application/json
And this when it works(nothing changed in the scripts):
Request details
GET /ajax/embed-user-library?detail=99&test=1&callback=jsonp1312398534999 HTTP/1.1
User-Agent: Opera/9.80 (Windows NT 6.1; U; en) Presto/2.9.168 Version/11.50
Host: test1.floowie.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en,sk-SK;q=0.9,sk;q=0.8
Accept-Encoding: gzip, deflate
Referer: http://www.sokker.cz/en/test2
Cookie: __utma=254918925.1489796832.1301725317.1312260335.1312298033.44; __utmz=254918925.1312298033.44.11.utmcsr=sokker.cz|utmccn=(referral)|utmcmd=referral|utmcct=/en/test2; lang=en; FLWSESSID=ddd1bc696f83f5a70b5f0f3ae30b4691; __utma=121955676.1030804516.1282595153.1312390656.1312397285.194; __utmb=121955676.8.10.1312397285; __utmc=121955676; __utmz=121955676.1312397285.194.21.utmcsr=floowie.crmserver.cz|utmccn=(referral)|utmcmd=referral|utmcct=/index.php
Connection: Keep-Alive
Response details
HTTP/1.1 200 OK
Date: Wed, 03 Aug 2011 19:07:45 GMT
Server: Apache/2.2.16 (Debian)
X-Powered-By: PHP/5.3.5-0.dotdeb.1
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-cache, must-revalidate
Pragma: no-cache
Content-Length: 20
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: application/json
Did someone see such a behaviour?
Is it solvable?
Thank you
If you want to use AJAX petitions over different domains/subdomains you have to implement Cross Origin Requests.
References:
http://hacks.mozilla.org/2009/07/cross-site-xmlhttprequest-with-cors/
https://developer.mozilla.org/en/http_access_control
Examples:
http://arunranga.com/examples/access-control/
Your server needs to send this headers:
Access-Control-Allow-Origin: test1.floowie.com
Access-Control-Allow-Credentials: true // allow cookie/session credentials
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
You can return the Access-Control-Allow-Origin globally or set specifically dependent of your input Origin ($_SERVER['HTTP_ORIGIN']) request header. Also apply for Access-Control-Allow-Methods.
You must implement the OPTIONS petition. Before the first AJAX call, modern browsers call that URL with an OPTIONS method to retrieve the above headers.
Ok this is the first part, the second is with jQuery. Read very carefully this page: http://api.jquery.com/jQuery.ajax/
You will need to add some options to every AJAX call, you can do it globally:
$(document).ajaxSend(function (event, xhr, settings) {
settings.xhrFields = {
withCredentials: true
};
});
Or specific:
$.ajax({
url: a_cross_domain_url,
xhrFields: {
withCredentials: true
}
});
This issue made ​​me lose many hours... hope it helps.
Note that you won't need to set your cookie domain as ".floowie.com" if you want.
You must properly implement CORS requests with credentials to send and receive cookies via Ajax. See developer.mozilla.org, specifically under the section titled "Requests with credentials."
First off, here is a simple CORS Ajax request with credentials, using jQuery 1.5.1+:
$.ajax({
url: "http://www.domain2.com/process",
xhrFields: {
withCredentials: true
}
}).done(function (data) { console.log(data); });
Note the withCredentials flag in the xhrFields. This flag tells the browser to send cookies with the request for the external domain, not the origin domain. In your case, cookies for www.domain2.com will be sent, and you will have access to them server-side.
On the server-side, you need to add certain headers to the response:
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: www.domain1.com
Important: requests with credentials cannot set the Access-Control-Allow-Origin header to global (Access-Control-Allow-Origin: *). It must specify domains (Access-Control-Allow-Origin: www.domain1.com).
It's obviously better if you specify a domain for the Access-Control-Allow-Origin header. But if you don't know or care where the CORS request is coming from, you could use the Origin header from the request and simply set the Access-Control-Allow-Origin header of your response to that. In C#, this is how we did this:
this.Response.AddHeader("Access-Control-Allow-Origin", this.Request.Headers["Origin"]);
After doing all of this, cookies that you set server-side will be sent back with the response, and the browser will be able to properly handle them and insert them into the browser's cookie store for www.domain2.com. And any subsequent CORS requests you send will send these cookies in the request as well.
If you are sending a request other than with the GET, POST, or HEAD methods, you will need to implement Preflighted requests (see under section titled "Preflighted requests"):
Unlike simple requests (discussed above), "preflighted" requests first send an HTTP request by the OPTIONS method to the resource on the other domain, in order to determine whether the actual request is safe to send. Cross-site requests are preflighted like this since they may have implications to user data. In particular, a request is preflighted if:
It uses methods other than GET, HEAD or POST. Also, if POST is used to send request data with a Content-Type other than application/x-www-form-urlencoded, multipart/form-data, or text/plain, e.g. if the POST request sends an XML payload to the server using application/xml or text/xml, then the request is preflighted.
It sets custom headers in the request (e.g. the request uses a header such as X-PINGOTHER)
Side-note about IE8 and IE9:
The Ajax call above will fail in IE8 and 9. I included the JS file from MoonScript/jQuery-ajaxTransport-XDomainRequest on my page, and this automagically allowed CORS requests to work in those old IE versions. But sadly, the XDomainRequest object that MS created for IE8 and 9 does not allow cookies to be sent or received. (see this MSDN blog post for more information)
You have different hosts. In the first example the host is "Host: www.floowie.com". In the second it is "Host: test1.floowie.com".
I'm guessing that the cookies are originally set by 'test1.floowie.com' and you haven't specified that they should be available to '.floowie.com' (i.e. the whole domain and all subdomains).
Can you post the code that sets the cookies in the first place?
If you get this fixed, it should at least show consistent behaviour. However, IE will probably still not pass cookies across subdomains. That's what I'm wrestling with at the moment, which is how I can across your question.

empty response body in ajax (or 206 Partial Content)

I'm feeling completely stupid because I've spent two hours solving task which should be very simple and which I solved many times before. But now I'm not even sure in which direction to dig.
I fail to fetch static content using ajax from local servers (Apache and Mongrel). I get responses 200 and 206 (depending on the server), empty response text (although Content-Length header is always correct), firebug shows request in red.
Javascript is very generic, I'm getting same results even here: http://www.w3schools.com/ajax/tryit.asp?filename=tryajax_first
(just change document location to 'http://localhost:3000/whatever')
So, it's probably not the cause.
Well, now I'm out of ideas. I can also post http headers, if it'll help.
Thanks!
Response Headers
Connection close
Date Sat, 01 May 2010 21:05:23 GMT
Last-Modified Sun, 18 Apr 2010 19:33:26 GMT
Content-Type text/html
Content-Length 7466
Request Headers
Host localhost:3000
User-Agent Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip,deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive 115
Connection keep-alive
Referer http://www.w3schools.com/ajax/tryit_view.asp
Origin http://www.w3schools.com
Response Headers
Date Sat, 01 May 2010 21:54:59 GMT
Server Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8l DAV/2 mod_jk/1.2.28
Etag "3d5cbdb-fb4-4819c460d4a40"
Accept-Ranges bytes
Content-Length 4020
Cache-Control max-age=7200, public, proxy-revalidate
Expires Sat, 01 May 2010 23:54:59 GMT
Content-Range bytes 0-4019/4020
Keep-Alive timeout=5, max=100
Connection Keep-Alive
Content-Type application/javascript
Request Headers
Host localhost
User-Agent Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip,deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive 115
Connection keep-alive
Origin null
UPDATED:
I've found a problem, it was about cross-domain requests. I knew that there are restrictions, but thought they're relaxed for local filesystem and local servers. (and expected more descriptive error message, anyway)
Thanks everybody!
It seems lake a caching problem only. Just delete cache inside of Internet Explorer and repeat your experiment. All HTTP GET requests will be cached. IE cache also ajax responses. If you don't like it you can append the URL with the text like '?p=blala'. Where 'p' is a name which will be interpret as a name of a parameter and text 'blala' must be unique in every request. Typically one use (new Date).getTime() construct to generate such 'blala'. IE will "think", that URL is new and will always send request to server.
UPDATED: Caching of static data will be made in all browsers, especially if Web server explicitly allow this: see Cache-Control: max-age=7200, public, proxy-revalidate is the response from server. Just try to go to http://www.w3schools.com/ajax/tryit.asp?filename=tryajax_first and modify the line
xmlhttp.open("GET","ajax_info.txt",true);
to
xmlhttp.open("GET","ajax_info.txt?p=" + (new Date).getTime(),true);
then click on "Edit and Click Me >>". Then if you click on "Change Content" button you will see in the HTTP traffic full data contain. I don't see any time 206 code. What it means if Response has "Accept-Ranges: bytes" and "Content-Range" like "bytes 0-4019/4020" inside of HTTP header you can read on http://benramsey.com/archives/206-partial-content-and-range-requests/

Categories