Cross Domain HTML Get Request - javascript

Im using Coda, and Im trying to write a program to use Javascript/Jquery to load nfl's html on their stats page (http://www.nfl.com/stats/player), and then remove all of the excess html. Resulting several lists of players and their stat's.
I've tried using `$('#container').load('http://www.nfl.com/stats/player').
This works fine in Coda, but I can't parse the html the way I want to.
In google chrome I get the error
XMLHttpRequest cannot load http://www.nfl.com/stats/player. Origin null is not
allowed by Access-Control-Allow-Origin.
From what I understand this is a security feature built into all browsers. Is there a workaround to this issue? Can I use a different type of request?
I understand that i should be using JSONP for this type of request, but I dont believe the nfl has an API that would make this possible.
I've seen questions like this get thrown around, but I don't think anyone's given a really good solid answer yet.
I think theres still a lot of people wondering if theres an easy way to $.get cross-domain HTML and parse it.

You're not allowed to do this because it can be used for XSS (cross site scripting) where scripts are accessed by scripts outside of the domain of the site. i.e. you can get cookie information or such this way.
You will have to do this server side. If you're using php you can use $content = file_get_contents('http://nfl.com/stats/player'); or you can do it using curl if you wish.
Otherwise the legit way to do it is through an API, but as you've pointed out that isn't an option in your case.

Related

Relax Same Origin Rule for Javascript Widget (deprecated google finance backfill data)

Background:
I have a small javascript-powered widget that uses data from another origin. I tested it successfully in my local IDE using external (non-origin) data from:
https://www.google.com/finance/getprices?q=.NSEI&x=NSE&i=600&p=1d&f=d,o,h,l,c,v
When I uploaded this widget to my wordpress site I got the error:
SEC7120: Origin http://www.my-wordpress-site.com not found in
Access-Control-Allow-Origin header.
Which in turn made my calls to the data return something like this:
SCRIPT5007: SCRIPT5007: Unable to get property 'split' of undefined or
null reference
Disclaimer:
I know that google finance has long since been deprecated, but my understanding is that the servers were left running. This means that the data is still there. Google clearly states that using the data for consumption violates the terms of usage for this data. What I am building is not an API, or anything consumption related. It's merely a front-end widget serving strictly as aesthetic ornamentation. I just want it to make the site look cooler by plotting some finance data.
Nonetheless, I'm not sure if google is not allowing me to fetch the data because it doesn't recognize my website. Maybe I have misunderstood the terms of use, but I don't think I have, but who knows. I'm hoping its a small protocol type of fix where I add a few lines and everything is happily ever after. Alternatively, if it is indeed the other way around, I don't have any control over what domains google trusts and I don't imagine contacting google and saying "hey google, don't leave my website out in the cold" would work.
Question: How do I relax the same origin rule? A few similar questions have been asked on stack overflow for c#, but I still wonder how to do this for my wordpress javascript widget. Can javascript alone do it?
I see a lot of examples having:
header("Access-Control-Allow-Origin: http://mozilla.com");
I don't know what language that is. I'm guessing PHP or c#. I don't know anything about those languages. I just want to design my widgets without having to learn a billion things about back-end protocols. I'm prepared to put in some effort though.
Clarification
I have made widgets this way (pointing an iframe to my widget's html located on my cpanel server where wordpress is in wp-content/uploads/2018/03) before and had no trouble at all. This time around I'm getting the origin errors as noted above in my post. The only difference is now I'm using some external data from a google server. Maybe it's google that doesn't recognize my site? The widget worked perfectly offline in my IDE. I'm not sure how to proceed and whether or not word-press has anything to bring to bear for these things to help widget designers like myself out, or if it's another matter altogether.
Error result confirmed in the following browsers:
edge
chrome
The issue is fairly straightforward. Browsers enforce the Same Origin Policy, which prevents sites from accessing data fetched from a foreign domain. CORS is a protocol that servers can use to instruct browsers to allow foreign domains to read their data.
Public APIs that are intended to be used in a browser context will generally use CORS to make this possible. For whatever reason, Google does not seem to be using CORS to allow public access to this API. In particular, their response does not include the appropriate Access-Control-Allow-Origin header (part of the CORS protocol) to allow Javascript from your site to read the data. There is really nothing you can do about that.
The standard workaround is to proxy the data. That is, instead of having your code use the API directly, it would instead hit a URL on your server, which would then fetch the requested data from Google and return it. The server will be able to read the data because the Same Origin Policy (being a browser concern) won't apply. And your Javascript will be able to read the result since the request was made to the same domain.
If there are ways to get around the Same Origin Policy in the browser, I'm not familiar with them.

How to ajax to different domain

This question have been bothering me for a while. So here is the story. There is another domain name which consist of a php file that produce json result :
lets say http://www.otherdomain.com/token.php and it will produce:
{"token":"123455"}
I want to ajax from mywebsite.com to get and act on the response given. The problem is otherdomain.com is not own my me and I can't modify the content in it or whatsoever.
So I have searched for several answer on stackoverflow and will explain why this doesn't work.
set the CORS header.
It didnt work because the domain is not own by me and i can't do anything about it.
Use my server as a proxy to request the data. (This is workable for a few times, but when the visitor on my site get more and more there will be more data sent from single ip to the site which is from my server which then might triggered 403 error)
Use plugin on chrome or firefox. (This option is okay if i am the only one who using this website)
Use JsonP. It shown in jQuery getJSON works locally, but not cross domain but the answer shows you need to modify the response data on the otherdomain.com file and since i dont own the domain, I cant do anything.
You may not use your option number 4 correctly. jsonp is especially made for cross domain requests. So, jsonp is your solution. If you stuck, then look at google maps or other services, they have working jsonp examples.

Using a client's IP to get content

I'm a bit embarrassed here because I am trying to get content remotely, by using the client's browser and not the server. But I have specifications which make it look impossible to me, I literally spent all day on it with no success.
The data I need to fetch is on a distant server.
I don't own this server (I can't do any modification to it).
It's a string, and I need to get it and pass it to PHP.
It must be the client's (user browsing the website) browser that actually gets the data (it needs to be it's IP, and not the servers).
And, with the cross-domain policy I don't seem to be able to get around it. I already knew about it, still tried a simple Ajax query, which failed. Then I though 'why not use iFrames', but the same limitation seems to apply to them too. I then read about using YQL (http://developer.yahoo.com/yql/) but I noticed the server I was trying to reach blocked YQL's user-agent making it impossible to use this technique.
So, that's all I could think about or find. But I can't believe it's not possible to achieve such a thing, that doesn't even look hard...
Oh, and my Javascript knowledge is very basic, this mustn't help either.
This is one reason that the same-origin policy exists. You're trying to have your webpage access data on a different server, without the user knowing, and without having "permission" from the other server to do so.
Without establishing a two-way trust system (ie modifying the 'other' server), I believe this is not possible.
Even with new xhr and crossdomain support, two-way trust is still required for the communication to work.
You could consider a fat-client approach, or try #selbie suggestion and require manual user interaction.
The same origin policy prevents document or script loaded from one
origin from getting or setting properties of a document from a different
origin.
-- From http://www.mozilla.org/projects/security/components/same-origin.html
Now if you wish to do some hackery to get it... visit this site
Note: I have never tried any of the methods on the aforementioned site and cannot guarantee their success
I can only see a really ugly solution: iFrames. This article contains some good informations to start with.
You could do it with flash application:
flash with a crossdomain.xml file (won't help though since you don't control the other server)
On new browsers there is CORS - requires Access-Control-Allow-Origin header set on the server side.
You can also try to use JSONP (but I think that won't work since you don't own the other server).
I think you need to bite the bullet and find some other way to get the content (on the server side for example).

Have someone modify javascript code without write access

I would like to test out some interviewees by having them write some javascript that will make requests against my server. I don't want to give them write access to my code base. Javascript runs on the client side, so this should technically be possible. However I know there are browser restrictions that say that the javascript has to come from the server?
I apologize that this is a really dumb question, but how should I proceed?
Edit:::
Ok, so I failed to mention that the entire application is based off sending JSON objects to and from the server.
I think you're thinking of the javascript XMLHttpRequest object itself, in which case, yes the current standard is for browsers to block cross-domain calls. So, you can tell them to pretend they either have a separate proxy script on their own personal domain which will allow them to leap to yours, or to pretend they're building a page directly served from your domain. There's talk of browsers supporting trust certificates or honoring special setups between source and target servers, but nothing universally accepted AFAIK.
Javascript source itself does not need to come from the server, and in fact this is how you can get around this little XMLHttpRequest block by using json. Here's a simple json tutorial I just dug up from Yahoo. But, this calls for your server to provide a json format feed if your server is the intended target.
"there are browser restrictions that say that the javascript has to come from the server"
I'm not aware of any situation where that's the case, only a lack of a console. But...even in those cases there's the address bar and javascript:.
If I misunderstood and you're talking about cross-domain restrictions, then your server needs to support JSONP for the requests so they can make cross-domain calls for data.
You could POST the javascript to your server and then your server can send back some HTML that has the javascript inlined. Sounds like you're trying to set up something is, more or less, a homebrew version of http://jsfiddle.net/; AFAIK, jsfiddle just does a "POST javascript/CSS/HTML and return HTML".

Avoid x-domain solutions

I'm currently working on a web application that customers can add to their webpages by adding a javascript link to a js file on my server. The application read all the javascriptfiles from my sever, but I still get an error when trying to use ajax to get data from my database. I didn't think that would be a problem because the files is on my server.
Can I fix this or do I have to make a cross-browser solution? I don't have any control over the costumers server.
Thanks in advance
Mikael
This is not possible: When you execute a remote script, it runs in the context of the containing document.
There are some popular workarounds for this:
Using an iframe, which fixes the cross-domain problem but doesn't integrate well with the remote site (e.g. no custom styling)
Using JSONP to make cross-domain Ajax requests (detailed explanation here)
Using a server-side proxy script (not an option in this scenario)
Using YQL (I'm not familiar with this but it's said to work)
The same origin policy is based on the host document not the script itself.
You need to use a cross domain ajax technique.

Categories