Using Javascript or jQuery, how can I check if an external link is available?
An $.ajax() call is not available as it violates SOP (same problem as here). I have read about JSONP, but I would like to know if there is a more straight solution, as I don't want to load any data from the external server; what I need is only to check if it is reachable.
EDIT (answer)
I solved it with the following code:
$.ajax({url: 'http://www.example.com',
type: 'GET',
dataType: 'jsonp',
complete: function(jqXHR, textStatus) {
console.log(jqXHR.status); // '200' = url reachable
},
timeout: 2000
});
The only problem now is that I get a Parse error, but in any case it can be checked if the external link is working.
You could use YQL to check if a URL is available via its JSONP interface.
YQL Console.
Do you have access to server side languages? You could use AJAX to call a local file which uses something like PHP to check the external address.
If you need help with the PHP part, have a look at http://www.catswhocode.com/blog/amazing-things-to-do-with-php-and-curl
Yes, in browsers that support flash you can use this plugin. It has the same api as native xhr. Further if there is no flash plugin in browsers you still can access the other domain. In some browsers if your target domain response has the following headers there will not be an error.
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Content-Type, *
Replace * with appropriate domains.
All above techniques are applicable only if the second domain is under your control. If not then you have to use a proxy as Jleagle said.
Related
I attempt to send a GET request in a jQuery AJAX request.
$.ajax({
type: 'GET',
url: /* <the link as string> */,
dataType: 'text/html',
success: function() { alert("Success"); },
error: function() { alert("Error"); },
});
However, whatever I've tried, I got XMLHttpRequest cannot load <page>. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:7776' is therefore not allowed access.
I tried everything, from adding header : {} definitions to the AJAX request to setting dataType to JSONP, or even text/plain, using simple AJAX instead of jQuery, even downloading a plugin that enables CORS - but nothing could help.
And the same happens if I attempt to reach any other sites.
Any ideas for a proper and simple solution? Is there any at all?
This is by design. You can't make an arbitrary HTTP request to another server using XMLHttpRequest unless that server allows it by putting out an Access-Control-Allow-Origin header for the requesting host.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
You could retrieve it in a script tag (there isn't the same restriction on scripts and images and stylesheets), but unless the content returned is a script, it won't do you much good.
Here's a tutorial on CORS:
http://www.bennadel.com/blog/2327-cross-origin-resource-sharing-cors-ajax-requests-between-jquery-and-node-js.htm
This is all done to protect the end user. Assuming that an image is actually an image, a stylesheet is just a stylesheet and a script is just a script, requesting those resources from another server can't really do any harm.
But in general, cross-origin requests can do really bad things. Say that you, Zoltan, are using coolsharks.com. Say also that you are logged into mybank.com and there is a cookie for mybank.com in your browser. Now, suppose that coolsharks.com sends an AJAX request to mybank.com, asking to transfer all your money into another account. Because you have a mybank.com cookie stored, they successfully complete the request. And all of this happens without your knowledge, because no page reload occurred. This is the danger of allowing general cross-site AJAX requests.
If you want to perform cross-site requests, you have two options:
Get the server you are making the request to to either
a. Admit you by putting out a Access-Control-Allow-Origin header that includes you (or *)
b. Provide you with a JSONP API.
or
Write your own browser that doesn't follow the standards and has no restrictions.
In (1), you must have the cooperation of the server you are making requests to, and in (2), you must have control over the end user's browser. If you can't fulfill (1) or (2), you're pretty much out of luck.
However, there is a third option (pointed out by charlietfl). You can make the request from a server that you do control and then pass the result back to your page. E.g.
<script>
$.ajax({
type: 'GET',
url: '/proxyAjax.php?url=http%3A%2F%2Fstackoverflow.com%2F10m',
dataType: 'text/html',
success: function() { alert("Success"); },
error: function() { alert("Error"); }
});
</script>
And then on your server, at its most simple:
<?php
// proxyAjax.php
// ... validation of params
// and checking of url against whitelist would happen here ...
// assume that $url now contains "http://stackoverflow.com/10m"
echo file_get_contents($url);
Of course, this method may run into other issues:
Does the site you are a proxy for require the correct referrer or a certain IP address?
Do cookies need to be passed through to the target server?
Does your whitelist sufficiently protect you from making arbitrary requests?
Which headers (e.g. modify time, etc) will you be passing back to the browser as your server received them and which ones will you omit or change?
Will your server be implicated as having made a request that was unlawful (since you are acting as a proxy)?
I'm sure there are others. But if none of those issues prevent it, this third method could work quite well.
you can ask the developers of that domain if they would set the appropriate header for you, this restriction is only for javascript, basically you can request the ressource from your server with php or whatever and the javascript requests the data from your domain then
Old question, but I'm not seeing this solution, which worked for me, anywhere. So hoping this can be helpful for someone.
First, remember that it makes no sense to try modifying the headers of the request to get around a cross-origin resource request. If that were all it took, it would be easy for malicious users to then circumvent this security measure.
Cross-origin requests in this context are only possible if the partner site's server allows it through their response headers.
I got this to work in Django without any CORS middleware by setting the following headers on the response:
response["Access-Control-Allow-Origin"] = "requesting_site.com"
response["Access-Control-Allow-Methods"] = "GET"
response["Access-Control-Allow-Headers"] = "requesting_site.com"
Most answers on here seem to mention the first one, but not the second two. I've just confirmed they are all required. You'll want to modify as needed for your framework or request method (GET, POST, OPTION).
p.s. You can try "*" instead of "requesting_site.com" for initial development just to get it working, but it would be a security hole to allow every site access. Once working, you can restrict it for your requesting site only to make sure you don't have any formatting typos.
I'm trying to implement some code that will allow me to have one sub-domain communicate with another sub-domain, e.g. one.example.com and two.example.com. The two sites have the ability to share cookies and session data as I have set the cookie to use .example.com.
When I visit either site via standard HTTP I can dump the session and see the expected data. However if I do this via JavaScript with jQuery $.ajax() the dumped session data is empty. In both cases I use PHP to dump the session data.
I have tried the following solution but to no luck yet (http://forum.kohanaframework.org/discussion/9895/problem-session-expired-with-ajax/p1). I'm also using a slightly newer version of Kohana (3.3).
I've also tried setting the headers as soon as they reach the controller:
$this->response->headers('Access-Control-Allow-Origin', 'http://one.example.com');
$this->response->headers('Access-Control-Allow-Credentials', 'true');
$this->response->headers('Access-Control-Allow-Methods', 'POST, GET, OPTIONS');
However the Chrome inspector still shows the Access-Control-Allow-Origin as *.
The Problem
The problem I was having was due to a setting in my Apache config file which looked like this:
<IfModule mod_headers.c>
Header set Access-Control-Allow-Origin "*"
</IfModule>
In order to solve my particular problem I simple removed / commented out the above code as it was overriding the headers I sent from PHP.
My implemented solution was then quite simple. In the following example we'll assume that I am making a call from one.example.com (the main website) to two.example.com (a sub-site).
Kohana / PHP
In my PHP I set the following headers, I've chosen to do this in my parent Controller. You could create your own Cors class or helper if you prefer. Basically you don't want to have this code duplicated hundreds of times throughout your project.
$this->response->headers('Access-Control-Allow-Origin', 'http://one.example.com');
$this->response->headers('Access-Control-Allow-Credentials', 'true');
$this->response->headers('Access-Control-Allow-Methods', 'POST, GET, OPTIONS');
JavaScript / jQuery
In my $.ajax() requests I then have to make sure to set the xhrFields.withCredentials property to true.
$.ajax({
url: 'two.example.com',
xhrFields: {
withCredentials: true
}
});
Or I could set it globally for all ajax requests like so:
$(document).ajaxSend(function (event, xhr, settings) {
settings.xhrFields = {
withCredentials: true
};
});
For more information check the $.ajax documentation: http://api.jquery.com/jQuery.ajax/
Further Reading
For further information checkout the following resources:
https://code.google.com/p/html5security/wiki/CrossOriginRequestSecurity
http://enable-cors.org/
Dictionary.com provides absolutely no documentation on how to grab data from them. I'm trying to use jQuery's ajax requests, but those are not working.
They provide a URL through which I'm supposed to be able to use. I'll provide this below.
http://api-pub.dictionary.com/v001?vid=<VID>&type=random&site=dictionary
They also provide a key, which I assume that I place in the <VID> spot.
Could someone please tell me what I'm doing wrong with this? I'll provide the code I'm using below:
$("#btnGetData").click(function() {
$.ajax({
url: "http://api-pub.dictionary.com/",
type: "GET",
data: "v001?vid=<VID>&type=random&site=dictionary",
success: function() { alert("success") },
});
});
Could someone please tell me what I'm doing wrong?
You are not passing the data correctly. Also the request url is http://api-pub.dictionary.com/v001. Try this:
$("#btnGetData").click(function() {
$.ajax({
url: "http://api-pub.dictionary.com/v001",
type: "GET",
dataType: "jsonp", //For external apis
data: {"vid": <VID>,
"type":"random"
"site": "dictionary"},
success: function() {
alert("success") },
});
});
UPDATE: Maybe you're being blocked by the same origin policy as #thordarson note, in that case add:
dataType: "jsonp" to your ajax call.
Possibility 1
You don't have access to their API. According to their API page they're very selective about who they give access to their API.
We are selective about our API partners and approve use as well as
develop terms on a case-by-case basis. If you are interested in using
our API, please contact us directly [...].
Possibility 2
You're being blocked by the same origin policy. Browsers are generally very strict about requests made by JavaScript across different domains. This can be bypassed using Cross-origin resource sharing. This requires configuring the server you're requesting, so in this case it's not viable.
Your best bet would be to create a server-side script that requests the URL and then using AJAX to request the file.
Example in PHP, let's call this request_dictionary.php:
<?php
$vid = "Your API key";
$type = $_GET['type'];
$site = $_GET['dictionary'];
$request_url = "http://api-pub.dictionary.com/v001?vid=$vid&type=$type&site=$site";
echo file_get_contents($request_url);
?>
Note: You probably need to change this to fit your needs (error handling, 404s, caching etc.). This code is untested.
Then change your jQuery so that it requests your file.
$("#btnGetData").click(function() {
$.ajax({
url: "/request_dictionary.php",
type: "GET",
data: "type=random&site=dictionary",
success: function() { alert("success") },
});
});
Warning
Using an AJAX call without a proxy (as explained in possibility 2) will expose your API key. This is against Dictionary.com's API Terms of Service.
2.1 Dictionary.com will assign and deliver to your Customer Application an Application Key to access the API. All Calls must
contain such Application Key. The Application Key is uniquely
associated with each Customer Application and all versions, upgrades
and updates thereof. The Application Key is Confidential Information
as defined in these Terms and Conditions. You must maintain and keep
the Application Key in a secure, embedded manner not accessible by any
third party. The Application Key is fully revocable by Dictionary.com
at any time. Dictionary.com may block attempts to access the API with
an invalid or revoked Application Key.
I've updated the code in possibility 2 to conceal the API key by keeping it in the PHP file.
Update/Note: Simply passing dataType: 'jsonp' to enable cross-origin calls is not enough. The responding server needs to respond with an Access-Control-Allow-Origin header containing your domain (or a rule that includes your domain). Any configuration of this kind is out of your hand as the requester.
Update/Note 2: Upon investigating the returned headers from your request URL, I can see no evidence of Access-Control-Allow-Origin. This means that even if you had access to their API, you wouldn't be able to access it using AJAX. Full headers below:
GET /v001?vid=%3CVID%3E&type=random&site=dictionary HTTP/1.1[CRLF]
Host: api-pub.dictionary.com[CRLF]
Connection: close[CRLF]
User-Agent: Web-sniffer/1.0.44 (+http://web-sniffer.net/)[CRLF]
Accept-Encoding: gzip[CRLF]
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8[CRLF]
Accept-Language: en-US,en;q=0.8[CRLF]
Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7[CRLF]
Cache-Control: no-cache[CRLF]
Referer: http://web-sniffer.net/[CRLF]
I'm trying to validate a feed, using the web service that is in this question.
But browser does not allow me to send a ajax GET request to another server. And there is a one second restriction per each request in that web service, so I can't mirror requests from my server.
This is my current jQuery code:
var reqUrl = "http://validator.w3.org/feed/check.cgi?url=" + encodeURIComponent(theUrl);
$.get(reqUrl, function(data) {
// do something
});
Isn't there any other way?
Ajax calls are not valid across different domains unless you use JSONP. JQuery-ajax-cross-domain is a similar question that may give you some insight. Also as noted by Luis in the comments, JSONP has to also be implemented on the domain that you are getting the data from.
Here is an example for jquery ajax(), but you may want to look into $.getJSON():
$.ajax({
url: 'http://yourUrl?callback=?',
dataType: 'jsonp',
success: processJSON
});
Another option is CORS (Cross Origin Resource Sharing), however, this requires that the other server to enable CORS which most likely will not happen in this case.
I searched google and found this. the third answer says that:
In computing, the same origin policy is an important security concept for a number of browser-side programming languages, such as JavaScript. The policy permits scripts running on pages originating from the same site to access each other's methods and properties with no specific restrictions, but prevents access to most methods and properties across pages on different sites.(source)
you'd better see the answers of this question.
I think you can't use JSONP because you haven't any access to W3C script.
Update (explanations)
I the question I linked to there is another way that I can explain it to you. if you set Access-Control-Allow-Origin header to * as the answer said you can send requests to another domains. and to use it easily in an MVC application you can see this solution. Good luck
Update2
to allow just http://validator.w3.org/ you just should set Access-Control-Allow-Origin to http://validator.w3.org/
for more details as answer said go here.
As said you can use JSONP but the endpoint must also implement it, And its only used if you are requesting json data from the call. It looks like you are retrieving html.
You can also implement a simple proxy in your domain that pulls the data from the external location and serves it to the ajax call. You can develop a simple proxy in php using for instance CURL.
Make sure you understand the implications of this security wise making sure for instance that you protect your proxy to only make calls to that external url (whitelisting).
Update: I just noticed you cannot use the proxy solution. And after following the link you have suggested I have came across CORS, which I didnt event know about. So apparentry you can set some headers when you are serving the pages in your domain that will instruct the browser that requests to some domains can be done.
Check this page for how to implement it:
http://enable-cors.org/
I have read that you might have to tweak it a bit to work with IE but it seems that all browsers are now implementing it.
I know this is an old question, but I myself have been trying to create AJAX requests to validator.w3.org as well, hit the exact same issues and stumbled on this SO question.
However, I did find a solution;
As people have already stated, the main problem here is that the server must issue valid CORS headers, i.e.
Access-Control-Allow-Origin: *
I used Fiddler to check the response headers from validator.w3.org and sure enough, the headers were not set. However, they also have another tool that does at validator.w3.org/nu/.
Here is an example: http://codepen.io/DDN-Shep/pen/ogdGgO/
$('form').on('submit', function(e) {
e.preventDefault();
var data = new FormData(this);
$.ajax({
url: 'http://validator.w3.org/nu/', // Trailing '/' is important or you get a 301 HTTP response status code
type: 'POST',
data: data,
processData: false, // Required for jQuery & FormData
contentType: false, // Set by FormData, required by the server
success: function(data, status, xhr) { /* TODO */ },
error: function(xhr, status, error) { /* TODO */ },
complete: function(xhr, status) { /* TODO */ }
});
});
If you are ever unsure whether or not a server allows CORS, you can use this very helpful online tool;
test-cors.org = http://client.cors-api.appspot.com/client
Here are two pages, test.php and testserver.php.
test.php
<script src="scripts/jq.js" type="text/javascript"></script>
<script>
$(function() {
$.ajax({url:"testserver.php",
success:function() {
alert("Success");
},
error:function() {
alert("Error");
},
dataType:"json",
type:"get"
}
)})
</script>
testserver.php
<?php
$arr = array("element1",
"element2",
array("element31","element32"));
$arr['name'] = "response";
echo json_encode($arr);
?>
Now my problem: when both of these files are on the same server (either localhost or web server), it works and alert("Success") is called; If it is on different servers, meaning testserver.php on web server and test.php on localhost, its not working, and alert("Error") is executing. Even if the URL inside AJAX is changed to http://domain.example/path/to/file/testserver.php
Use JSONP.
jQuery:
$.ajax({
url:"testserver.php",
dataType: 'jsonp', // Notice! JSONP <-- P (lowercase)
success:function(json){
// do stuff with json (in this case an array)
alert("Success");
},
error:function(){
alert("Error");
}
});
PHP:
<?php
$arr = array("element1","element2",array("element31","element32"));
$arr['name'] = "response";
echo $_GET['callback']."(".json_encode($arr).");";
?>
The echo might be wrong, it's been a while since I've used php. In any case you need to output callbackName('jsonString') notice the quotes. jQuery will pass its own callback name, so you need to get that from the GET params.
And as Stefan Kendall posted, $.getJSON() is a shorthand method, but then you need to append 'callback=?' to the url as GET parameter (yes, value is ?, jQuery replaces this with its own generated callback method).
JSONP is a good option, but there is an easier way. You can simply set the Access-Control-Allow-Origin header on your server. Setting it to * will accept cross-domain AJAX requests from any domain. (https://developer.mozilla.org/en/http_access_control)
The method to do this will vary from language to language, of course. Here it is in Rails:
class HelloController < ApplicationController
def say_hello
headers['Access-Control-Allow-Origin'] = "*"
render text: "hello!"
end
end
In this example, the say_hello action will accept AJAX requests from any domain and return a response of "hello!".
Here is an example of the headers it might return:
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: text/html; charset=utf-8
X-Ua-Compatible: IE=Edge
Etag: "c4ca4238a0b923820dcc509a6f75849b"
X-Runtime: 0.913606
Content-Length: 6
Server: WEBrick/1.3.1 (Ruby/1.9.2/2011-07-09)
Date: Thu, 01 Mar 2012 20:44:28 GMT
Connection: Keep-Alive
Easy as it is, it does have some browser limitations. See http://caniuse.com/#feat=cors.
You can control this via HTTP header by adding Access-Control-Allow-Origin. Setting it to * will accept cross-domain AJAX requests from any domain.
Using PHP it's really simple, just add the following line into the script that you want to have access outside from your domain:
header("Access-Control-Allow-Origin: *");
Don't forget to enable mod_headers module in httpd.conf.
You need to have a look at Same Origin Policy:
In computing, the same origin policy
is an important security concept for a
number of browser-side programming
languages, such as JavaScript. The
policy permits scripts running on
pages originating from the same site
to access each other's methods and
properties with no specific
restrictions, but prevents access to
most methods and properties across
pages on different sites.
For you to be able to get data, it has to be:
Same protocol and host
You need to implement JSONP to workaround it.
I had to load webpage from local disk "file:///C:/test/htmlpage.html", call "http://localhost/getxml.php" url, and do this in IE8+ and Firefox12+ browsers, use jQuery v1.7.2 lib to minimize boilerplate code. After reading dozens of articles finally figured it out. Here is my summary.
server script (.php, .jsp, ...) must return http response header Access-Control-Allow-Origin: *
before using jQuery ajax set this flag in javascript: jQuery.support.cors = true;
you may set flag once or everytime before using jQuery ajax function
now I can read .xml document in IE and Firefox. Other browsers I did not test.
response document can be plain/text, xml, json or anything else
Here is an example jQuery ajax call with some debug sysouts.
jQuery.support.cors = true;
$.ajax({
url: "http://localhost/getxml.php",
data: { "id":"doc1", "rows":"100" },
type: "GET",
timeout: 30000,
dataType: "text", // "xml", "json"
success: function(data) {
// show text reply as-is (debug)
alert(data);
// show xml field values (debug)
//alert( $(data).find("title").text() );
// loop JSON array (debug)
//var str="";
//$.each(data.items, function(i,item) {
// str += item.title + "\n";
//});
//alert(str);
},
error: function(jqXHR, textStatus, ex) {
alert(textStatus + "," + ex + "," + jqXHR.responseText);
}
});
It is true that the same-origin policy prevents JavaScript from making requests across domains, but the CORS specification allows just the sort of API access you are looking for, and is supported by the current batch of major browsers.
See how to enable cross-origin resource sharing for client and server:
http://enable-cors.org/
"Cross-Origin Resource Sharing (CORS) is a specification that enables truly open access across domain-boundaries. If you serve public content, please consider using CORS to open it up for universal JavaScript/browser access."
This is possible, but you need to use JSONP, not JSON. Stefan's link pointed you in the right direction. The jQuery AJAX page has more information on JSONP.
Remy Sharp has a detailed example using PHP.
I use Apache server, so I've used mod_proxy module. Enable modules:
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
Then add:
ProxyPass /your-proxy-url/ http://service-url:serviceport/
Finally, pass proxy-url to your script.
Browser security prevents making an ajax call from a page hosted on one domain to a page hosted on a different domain; this is called the "same-origin policy".
From the Jquery docs (link):
Due to browser security restrictions, most "Ajax" requests are subject to the same origin policy; the request can not successfully retrieve data from a different domain, subdomain, or protocol.
Script and JSONP requests are not subject to the same origin policy restrictions.
So I would take it that you need to use jsonp for the request. But haven't tried this myself.
There are few examples for using JSONP which include error handling.
However, please note that the error-event is not triggered when using JSONP! See: http://api.jquery.com/jQuery.ajax/ or jQuery ajax request using jsonp error
I know 3 way to resolve your problem:
First if you have access to both domains you can allow access for all other domain using :
header("Access-Control-Allow-Origin: *");
or just a domain by adding code bellow to .htaccess file:
<FilesMatch "\.(ttf|otf|eot|woff)$">
<IfModule mod_headers.c>
SetEnvIf Origin "http(s)?://(www\.)?(google.com|staging.google.com|development.google.com|otherdomain.net|dev02.otherdomain.net)$" AccessControlAllowOrigin=$0
Header add Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin
</IfModule>
</FilesMatch>
you can have ajax request to a php file in your server and handle request to another domain using this php file.
you can use jsonp , because it doesn't need permission. for this you can read our friend #BGerrissen answer.
For Microsoft Azure, it's slightly different.
Azure has a special CORS setting that needs to be set. It's essentially the same thing behind the scenes, but simply setting the header joshuarh mentions will not work. The Azure documentation for enabling cross domain can be found here:
https://learn.microsoft.com/en-us/azure/app-service-api/app-service-api-cors-consume-javascript
I fiddled around with this for a few hours before realizing my hosting platform had this special setting.
it works, all you need:
PHP:
header('Access-Control-Allow-Origin: http://www.example.com');
header("Access-Control-Allow-Credentials: true");
header('Access-Control-Allow-Methods: GET, PUT, POST, DELETE, OPTIONS');
JS (jQuery ajax):
var getWBody = $.ajax({ cache: false,
url: URL,
dataType : 'json',
type: 'GET',
xhrFields: { withCredentials: true }
});
It got to work in PHP just adding this to the page served:
header("Access-Control-Allow-Origin: *");
header('Access-Control-Allow-Methods: GET, POST, OPTIONS');
PS: I have made my own xhr suite called sa_ajax with handy and useful methods. (https://github.com/osergioabreu/sa_ajax/)