cross-site XmlHttpRequest in Firefox? - javascript

I understand (I think) that XmlHttpRequest objects adhere to the "same-domain" policy. However, I want to create a simple (POC) local html file that downloads XML from a web server and does something with it (let's start with a simple "alert()").
Is it possible at all? Do I need a special Firefox config option?
The server from which I'm trying to download is not under my control (it's actually a Google API).
My simple attempt is the code from Mozilla's "Using XMLHttpRequest" page. It returns an error I don't really understand from the "send" method.
Disclaimer: I'm mainly a C/C++ developer - never done any serious JS programming, never tried using these APIs.

XMLHttpRequest actually adheres to a much stricter implementation of the same domain policy: while you can set the document.domain property to allow JavaScript served from two sub-domains to talk with each other, you can't do that with XMLHttpRequestObject. In your case, going to a completely different domain, you couldn't do that with JavaScript either.
There are a couple of options. First, you can use a reverse proxy to make it appear that the external site is a sub-domain of your site. Take a look at Apache's mod_proxy, in particular ProxyPassReverse
Another alternative is to return data as a JSON object: <script src="foo"> can retrieve whatever data it wants from wherever it wants. Downside of this is that it's not (easily) repeatable (as in multiple requests from same page).
I also recommend that you Google for "google mashups". Most of these live on the "googlemashops.com" domain, which makes implementation a lot easier. A few live outside that domain, and may give you some ideas.
Edit: rather than use the XMLHttpRequest object directly, I recommend going through a third-party library such as prototype.js

If the XML that you're trying to retrieve is returned by one of Google's JS APIs, then there's no need for XmlHttpRequest(since that can only be used on the same domain as your page anway).
So in the case of using a Google API, such as the Maps one, usually begin by adding a reference to their common API somewhere on your page:
<script type="text/javascript" src="http://www.google.com/jsapi?key=your_google_api_key"></script>
Then add a reference to the specific API(s) you plan to use to your page:
<script type="text/javascript">google.load("maps", "2");</script>
Now you can call the various functions provided by that API:
<script type="text/javascript">
function initialize() {
var map = new google.maps.Map2(document.getElementById("map"));
map.setCenter(new google.maps.LatLng(41.86, 87.68), 13);
}
google.setOnLoadCallback(initialize);
</script>
No XmlHttpRequest required :)

You can use JSONP to do this. I do it here using jQuery and PHP. Basically I use the PHP proxy to wrap the JSON reply so jQuery can handle it. It's under BSD.

Alternatively try using IE8. If you are running from disk and not a website IE8 will ignore all normal domain restrictions and get the data you want.

With jQuery (and presumably XHR, but I try not to use it directly) you can happily do cross-domain requests, provided that you do not specify unusual headers or non-GET methods. If you do want to use these, you must have control of the server to enable OPTIONS requests.
See https://developer.mozilla.org/En/HTTP_access_control for details.

Related

cross-site requests & browsers addons

I'm trying to develop a basic Firefox Addon with the "new" WebExtensions system.
I'd like to
Extract some text from a web page (not owned by me)
evaluate it using a remote website
post in the same page the result
The problem is how to make the web request with the addon (point 2). I found that I could use XMLHttpRequest, but as I imagined for security reasons I can't get access to remote paths.
That is because (I guess) the javascript code is run inside the page, even though I had thought that an addon would be... external.
Of course the result would be inside the page, but I assume the addon could work as a proxy to make this request. That said I have no idea how and what should I do.
I don't want to use some strange trick (like removing some security control), I'd like to do it the "right" way.
What I also don't understand is if the addons are bounded to run within the page they are made for.
EDIT: OK, turns out the chrome docs is actually better than the mozilla one. To actually use the XHR to cross-site req you have to put an additional line of code in your manifest.
{...
"permissions": [
"http://random.com/"
],
}..
I'm still not sure if this is the proper way to do what I aim to though.
To actually use the XHR to cross-site req you have to put an additional line of code in your manifest.
{...
"permissions": [
"http://random.com/"
],
}

GET arbitrary URLs with JavaScript

I want to be able to access /robots.txt from a variety of sites using JavaScript. This is for a side project that tests the availability of sites, not all of which are under my control. I've tried this:
$.get(robotsUrl, function() {
console.log('success!');
}, "text")
.fail(function() {
console.log('failed :(');
});
However, this fails with
XMLHttpRequest cannot load https://my.test.url/robots.txt. Origin http://localhost:8000 is not allowed by Access-Control-Allow-Origin
MDN's page on Same-Origin-Policy says that it's possible to embed content with some elements, such as <script>, <iframe> <embed>. Could I load /robots.txt from an arbitrary site with any of these? Is there any other way I can access this file on other domains?
You could load it with any of them, you just won't be able to make the data available to JavaScript. That's rather the point of the Same Origin Policy.
If you want to get arbitrary data from arbitrary sites, you need to do it server side.
To get around a same origin policy, you need to either have control over the host site and set the allow-origin (not an option here), or load it by a method other than JavaScript (which JSONP does; it is loaded as a standard script).
That means you could display the robots.txt in an iframe, for example, by just setting its src attribute.
If you want to manipulate the contents in JavaScript, that won't work (even after you load the content in an iframe, you're still not allowed to interact with it). Your final option is to set up a proxy. Have a script on your server which when called will load the relevant file and redirect the content. It's not hard to do, but means your server will have higher traffic (and you'll need to lock it down so that it isn't used maliciously).
iframes won't let you peek at the content. You could show it to your user, but I'm guessing you want to analyze it with code.
You could do it on your server. Even if you just have a /cors/robots/domain.tld handler (and others for other files you need to access). This is probably the best way, if it's feasible for your situation.
AnyOrigin, is a free service allows you to make cross-origin requests.
$.getJSON('http://anyorigin.com/get?url=google.com/robots.txt&callback=?', function(data){
console.log(data.contents); // contents of Robots.txt
});
Pretty sure this is possible with Chrome by runnning the browser with the Same Origin Policy disabled: Disable same origin policy in Chrome.
It may be preferable to do something like this outside the context of a browser however, on the command line perhaps using something like CURL?

Bypassing a browser security limitation on my own computer

I'm trying to create a list of languages' directions (Left-To-Right or Right-To-Left. Other directions don't exist in Wikipedia) by their prefixes (en, fr, es, …) in Wikipedia. To do that I wrote a JS code that can be executed on this page and that stores the data in a variable. The code opens the main page of each Wikipedia and tries to check the direction of the language according to the 'dir' attribute of its <html> element. However, since each Wikipedia has a different domain, the browser won't let me access this data, for security reasons. Is there a browser that would? Is it possible to change my preferences to enable it?
P.S. The code:
var as = document.querySelectorAll('a.extiw'), pre, win, dirByPre = {};
for each (var a in as)
if (pre = /^http:\/\/(\w+)\.wikipedia\.org\/wiki\/$/.exec(a.href)) {
win = open(pre[0]);
win.onload = function () {
opener.dirByPre[pre[1]] = document.documentElement.dir;
close();
};
}
Thanks a lot!
What you're running into is an Access Control Allow Origin error. Read up on it, Access Control Allow Origin not allowed by.
In general, if you want to access another page's data for use in JS, you need to do so on your own server and pass that to Javascript.
However, the best solution in this case is to instead access Wikipedia via its API and request a JSONP response. Remember, Wikipedia is built on MediaWiki, so see its API Docs http://www.mediawiki.org/wiki/API:Main_page.
title = "List_of_Wikipedias";
$.getJSON("http://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvprop=content&titles="+title+"&format=json&callback=?", function(data) {
console.log(data);
})
Note the "format=json" and the "callback=?"
This JSONP approach will work in all modern browsers.
There is a reason that this is generally disallowed in modern browsers. There are many security issues that exist when arbitrary JavaScript is allowed to access other domains. A single example is XSS (cross-site scripting).
What you're trying to do would be much easier if you weren't trying to run your script inside the browser. Python, Ruby, or any other scripting language, really, could do this with ease using individual HTTP requests. And if you know JavaScript, how about Node.JS?
This might be an odd solution, but you could write a browser extension which would allow you to make cross domain requests. There are easy ways to develop extensions such as Crossrider which has an X-Domain addon that allows you to make cross domain requests via Javascript.

JQuery Cross-Domain .load() (self-constructing widget)

I'm creating a widget for people to use on their websites, however to keep the widget future-proof, i want it to be self constructing using an external javascript.
So the widget code i would ask people to put on their websites would be something like:
<div id="my_widget"></div>
<script type="text/javascript" src="http://www.external-domain.com/mywidget.js"></script>
mywidget.js would then use jquery's .load() to populate the #my_widget div with an iframe.
Problem is this doesn't work....
what do i need to do?
(note i dont want to have the iframe in the code i give to my customers)
It depends on what url you are specifying in the load function. If this url is not hosted on the same domain that executes the page containing this script won't work due to same origin policy restriction. One possible workaround to make cross domain ajax calls is to use JSON-P if you have control over the server side script which is used in the load function.
Here's the idea behind JSON-P:
You provide a server side script hosted on Domain A which will return JSONP formatted response. Domain A is your domain for which you have full control.
This server side script could be called from Domain B using AJAX.
Let's suppose that http://domainA.com/myscript?jsoncallback=foo returns the following response:
foo({ result: '<strong>Hello World</strong>' });
Now inside mywidget.js you could call this script:
$.getJSON('http://domainA.com/myscript?jsoncallback=?', function(data) {
$('#my_widget').html(data.result);
});
All that is left is to tell the users include mywidget.js script and provide a placeholder with id="my_widget" to host the results (you could even generate this placeholder in the success callback).
Remark: When using JSONP you are limited to GET requests only. This means that there's a limit in the size of the request you can send.
You have total control of their page since you're executing your code on their site.
You can create iframes document.createElement("iframe"), inject it anywhere on the page document.getElementById("my_widget").appendChild(iframe) and do whatever else you feel like.
One thing to be careful with this is to not clutter their namespaces... avoid any usual namespace and make up your own (__my_widget or whatever else is weird). And try to keep the namespaces counts as low as 1, or even none if possible.
Don't use load, use an iframe if you're just trying to load html from your site.

Dashboard Cross-domain AJAX with jquery

Hey everyone, I'm working on a widget for Apple's Dashboard and I've run into a problem while trying to get data from my server using jquery's ajax function. Here's my javascript code:
$.getJSON("http://example.com/getData.php?act=data",function(json) {
$("#devMessage").html(json.message)
if(json.version != version) {
$("#latestVersion").css("color","red")
}
$("#latestVersion").html(json.version)
})
And the server responds with this json:
{"message":"Hello World","version":"1.0"}
For some reason though, when I run this the fields on the widget don't change. From debugging, I've learned that the widget doesn't even make the request to the server, so it makes me think that Apple has some kind of external URL block in place. I know this can't be true though, because many widgets phone home to check for updates.
Does anyone have any ideas as to what could be wrong?
EDIT: Also, this code works perfectly fine in Safari.
As requested by Luca, here's the PHP and Javascript code that's running right now:
PHP:
echo $_GET["callback"].'({"message":"Hello World","version":"1.0"});';
Javascript:
function showBack(event)
{
var front = document.getElementById("front");
var back = document.getElementById("back");
if (window.widget) {
widget.prepareForTransition("ToBack");
}
front.style.display = "none";
back.style.display = "block";
stopTime();
if (window.widget) {
setTimeout('widget.performTransition();', 0);
}
$.getJSON('http://nakedsteve.com/data/the-button.php?callback=?',function(json) {
$("#devMessage").html(json.message)
if(json.version != version) {
$("#latestVersion").css("color","red")
}
$("#latestVersion").html(json.version)
})
}
In Dashcode click Widget Attributes then Allow Network Access make sure that option is checked. I've built something that simply refused to work, and this was the solution.
Cross-domain Ajax requests ( Using the XMLHttpRequest / ActiveX object ) are not allowed in the current standard, as per the W3C spec:
This specification does not include
the following features which are being
considered for a future version of
this specification:
Cross-site XMLHttpRequest;
However there's 1 technique of doing ajax requests cross-domain, JSONP, by including a script tag on the page, and with a little server configuration.
jQuery supports this, but instead of responding on your server with this
{"message":"Hello World","version":"1.0"}
you'll want to respond with this:
myCallback({"message":"Hello World","version":"1.0"});
myCallback must be the value in the "callback" parameter you passed in the $.getJSON() function. So if I was using PHP, this would work:
echo $_GET["callback"].'({"message":"Hello World","version":"1.0"});';
Apple has some kind of external URL block in place.
In your Info.plist you need to have the key AllowNetworkAccess set to true.
<key>allowNetworkAccess</key>
<true/>
Your code works in Safari because it is not constrained in the dashboard sever and it is not standards complient in that it DOES allow cross site AJAX. FF IS standards complient in that it DOES NOT allow cross site ajax.
If you are creating a dashboard widget, why don't you use the XMLHttpRequest Setup function in the code library of DashCode. Apple built these in so you don't need to install 3rd party JS libraries. I'm not sure about JSON support but perhaps starting here will lead you in a better direction.
So another solution is to create your own server side web service where you can control the CORS of, the users web browser can't access another site, but if you wrap that other site in your own web service (on the same domain) then it does not cause an issue.
Interesting that it works in Safari. As far as I know to do x-domain ajax requests you need to use the jsonp dataType.
http://docs.jquery.com/Ajax/jQuery.getJSON
http://bob.pythonmac.org/archives/2005/12/05/remote-json-jsonp/
Basically you need to add callback=? to your query string and jquery will automatically replace it with the correct method eg:
$.getJSON("http://example.com/getData.php?act=data&callback=?",function(){ ... });
EDIT: put the callback=? bit at the end of the query string just to be safe.

Categories