We Keep Coding

Soundcloud embedded via https and some resources responses with 403

This is a two-part question.
FIRST:
We're having issues when embedding soundcloud resource, our site uses https.
When an embedded resource is loaded, it makes a request (example url: http://soundcloud.com/oembed?format=js&url=https://soundcloud.com/user908553597%2Fann-marie-in-my-feelings-treat-me-like-somebody-remake&iframe=true), it is properly responding with this:
({
"version":1.0,
"type":"rich",
"provider_name":"SoundCloud",
"provider_url":"http://soundcloud.com",
"height":400,"width":"100%",
"title":"Ann Marie - In My Feelings (Treat Me Like Somebody Remake) by user908553597",
"description":"",
"thumbnail_url":"http://i1.sndcdn.com/artworks-000146535766-ycam16-t500x500.jpg",
"html":"\u003Ciframe width=\"100%\" height=\"400\" scrolling=\"no\" frameborder=\"no\" src=\"https://w.soundcloud.com/player/?visual=true\u0026url=http%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F246270367\u0026show_artwork=true\"\u003E\u003C/iframe\u003E",
"author_name":"user908553597",
"author_url":"http://soundcloud.com/user908553597"
});
However, "provider_url" and "thumbnail_url" is returned with http protocol, as our site is running via https this causes issues.
Is it possible to use a flag or parameter to assure that all resources in the response uses https, or does this require changes in your API ?
SECOND:
Some resources seems to return a http 403 response code, example: http://soundcloud.com/oembed?format=js&url=https://soundcloud.com/richgurl_1%2Ftink-treat-me-like-somebody&iframe=true is this how the API is supposed to handle certain resource requests ? I.e. Is this a predictable behaviour, so that one can act on returned 403`s.
Thanks in advance

So i figured it out,
or at least a solution that is managable. Changing the protocol did it
example url: https://soundcloud.com/oembed?format=js&url=https://soundcloud.com/user908553597%2Fann-marie-in-my-feelings-treat-me-like-somebody-remake&iframe=true
The thumbnail_url is retutrned via http, my solution was to change protocol to https via the .ts file in Angular that takes the response for the above url, using replace. The iframe using the thumbnail_url then has a url with the required protocol.

AJAX request gets "No 'Access-Control-Allow-Origin' header is present on the requested resource" error

I attempt to send a GET request in a jQuery AJAX request.
$.ajax({
type: 'GET',
url: /* <the link as string> */,
dataType: 'text/html',
success: function() { alert("Success"); },
error: function() { alert("Error"); },
});
However, whatever I've tried, I got XMLHttpRequest cannot load <page>. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:7776' is therefore not allowed access.
I tried everything, from adding header : {} definitions to the AJAX request to setting dataType to JSONP, or even text/plain, using simple AJAX instead of jQuery, even downloading a plugin that enables CORS - but nothing could help.
And the same happens if I attempt to reach any other sites.
Any ideas for a proper and simple solution? Is there any at all?

This is by design. You can't make an arbitrary HTTP request to another server using XMLHttpRequest unless that server allows it by putting out an Access-Control-Allow-Origin header for the requesting host.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
You could retrieve it in a script tag (there isn't the same restriction on scripts and images and stylesheets), but unless the content returned is a script, it won't do you much good.
Here's a tutorial on CORS:
http://www.bennadel.com/blog/2327-cross-origin-resource-sharing-cors-ajax-requests-between-jquery-and-node-js.htm
This is all done to protect the end user. Assuming that an image is actually an image, a stylesheet is just a stylesheet and a script is just a script, requesting those resources from another server can't really do any harm.
But in general, cross-origin requests can do really bad things. Say that you, Zoltan, are using coolsharks.com. Say also that you are logged into mybank.com and there is a cookie for mybank.com in your browser. Now, suppose that coolsharks.com sends an AJAX request to mybank.com, asking to transfer all your money into another account. Because you have a mybank.com cookie stored, they successfully complete the request. And all of this happens without your knowledge, because no page reload occurred. This is the danger of allowing general cross-site AJAX requests.
If you want to perform cross-site requests, you have two options:
Get the server you are making the request to to either
a. Admit you by putting out a Access-Control-Allow-Origin header that includes you (or *)
b. Provide you with a JSONP API.
or
Write your own browser that doesn't follow the standards and has no restrictions.
In (1), you must have the cooperation of the server you are making requests to, and in (2), you must have control over the end user's browser. If you can't fulfill (1) or (2), you're pretty much out of luck.
However, there is a third option (pointed out by charlietfl). You can make the request from a server that you do control and then pass the result back to your page. E.g.
<script>
$.ajax({
type: 'GET',
url: '/proxyAjax.php?url=http%3A%2F%2Fstackoverflow.com%2F10m',
dataType: 'text/html',
success: function() { alert("Success"); },
error: function() { alert("Error"); }
});
</script>
And then on your server, at its most simple:
<?php
// proxyAjax.php
// ... validation of params
// and checking of url against whitelist would happen here ...
// assume that $url now contains "http://stackoverflow.com/10m"
echo file_get_contents($url);
Of course, this method may run into other issues:
Does the site you are a proxy for require the correct referrer or a certain IP address?
Do cookies need to be passed through to the target server?
Does your whitelist sufficiently protect you from making arbitrary requests?
Which headers (e.g. modify time, etc) will you be passing back to the browser as your server received them and which ones will you omit or change?
Will your server be implicated as having made a request that was unlawful (since you are acting as a proxy)?
I'm sure there are others. But if none of those issues prevent it, this third method could work quite well.

you can ask the developers of that domain if they would set the appropriate header for you, this restriction is only for javascript, basically you can request the ressource from your server with php or whatever and the javascript requests the data from your domain then

Old question, but I'm not seeing this solution, which worked for me, anywhere. So hoping this can be helpful for someone.
First, remember that it makes no sense to try modifying the headers of the request to get around a cross-origin resource request. If that were all it took, it would be easy for malicious users to then circumvent this security measure.
Cross-origin requests in this context are only possible if the partner site's server allows it through their response headers.
I got this to work in Django without any CORS middleware by setting the following headers on the response:
response["Access-Control-Allow-Origin"] = "requesting_site.com"
response["Access-Control-Allow-Methods"] = "GET"
response["Access-Control-Allow-Headers"] = "requesting_site.com"
Most answers on here seem to mention the first one, but not the second two. I've just confirmed they are all required. You'll want to modify as needed for your framework or request method (GET, POST, OPTION).
p.s. You can try "*" instead of "requesting_site.com" for initial development just to get it working, but it would be a security hole to allow every site access. Once working, you can restrict it for your requesting site only to make sure you don't have any formatting typos.

Cross domain Ajax JSON POST support against RESTful WCF service using transportCredentialOnly security

I've posted before on this subject, but after a year of getting on with other things, I've managed to get into a pickle once again. I'll try and give a brief overview of the scenario and the current attempts to make things work:
IIS web server hosting HTML, JS etc. on host: iis.mycompany.com (referred to as foo)
WCF RESTful web services hosted via a Windows Service on host: wcf.mycompany.com (referred to as bar)
The Javascript served from foo works by making RESTful ajax calls (GET or POST depending on the action) to the WCF services on bar, obviously these are cross domain calls as they aren't on the same host.
The Javascript uses the jQuery (1.7.2) framework to manipulate the DOM and perform ajax calls to bar, the expected content type for POSTS is JSON, and the response from GETS is expected to be JSON too (application/json).
Bar has it's WCF services configured using TransportCredentialOnly as the security mode and the transport client credentail type is NTLM, so only authed users to contact the services.
CORS Support has been added to bar's WCF services using an extension to WCF:
http://blogs.msdn.com/b/carlosfigueira/archive/2012/05/15/implementing-cors-support-in-wcf.aspx
We have added additional headers and modfied some that the post already contained based on numerous internet articles:
property.Headers.Add("Access-Control-Allow-Headers", "Accept, Content-Type");
property.Headers.Add("Access-Control-Allow-Methods", "POST, GET, OPTIONS");
property.Headers.Add("Access-Control-Max-Age", "172800");
property.Headers.Add("Access-Control-Allow-Origin", "http://iis.mycompany.com");
property.Headers.Add("Access-Control-Allow-Credentials", "true");
property.Headers.Add("Content-type", "application/json");
Sites giving information on enabling CORS suggest that the Access-Control-Allow-Origin response header should be set to "*" however, this is not possible in our case as we make jQuery ajax calls using the following setup:
$.ajaxSetup({
cache: "false",
crossDomain: true,
xhrFields: {
withCredentials: true
}
});
As it turns out you cannot use "*" for the accepted origin when you are using "withCredentials" in the ajax call:
https://developer.mozilla.org/en/http_access_control
"Important note: when responding to a credentialed request, server
must specify a domain, and cannot use wild carding."
Currently in our development lab, this doesn't matter as we can hard code the requests to the IIS (foo) server URL.
The main problem now appears to be attempting POST requests (GET is working using the above configuration). When the browser attempts the POST process, it first sends an OPTIONS header to the server requesting allowed OPTIONS for the subsequent post. This is where we would like to see the headers we've configured in the CORS Support WCF extension being passed back, however we aren't getting that far; before the response comes back as "401 Unauthorized", I believe this is to do with the transport security binding configuration requesting NTLM, but I'm not sure.
Also, I'm not very experienced with this, but I haven't seen much information about POST using application/json content type as opposed to text/plain when performing cross domain requests.
I know that people will probably suggest JSONP as the one true solution, I'm not against different approaches, indeed I encourage anyone to suggest best practices as it would help others reading this question later. However, please attempt to answer the question before suggestion alternatives to it.
Many thanks in advance for anyone who contributes.
peteski
:)
UPDATE:
It appears that Chrome (20.x.x) doesn't suffer the problem of not negotiating NTLM to retrieve the OPTIONS header response from the server, but Firefox (13.0.1) does.
We've also noticed that someone has already posted a bug up on the Firefox forum, which we've added information to:
http://bugzilla.mozilla.org/show_bug.cgi?id=751552
Please vote for this bug to be fixed on the bugzilla site!
Using the following code, we can watch the network trace to see Firefox failing and Chrome working fine:
var url = "http://myWebServiceServer/InstantMessagingService/chat/message/send";
var data = '{ "remoteUserUri" : "sip:foo.bar#mydomain.com", "message" : "This is my message" }';
var request = new XMLHttpRequest();
request.open("POST", url, true);
request.withCredentials = true;
request.setRequestHeader("Content-Type", "application/json");
request.send(data);
console.log(request);
On a separate note, IE8 doesn't support the XMLHttpRequest for cross domain calls, favouring it's own magical XDomainRequest object, so we've got some work to do in changing the client side code to handle IE8 vs the world cases. (Thanks IE8).
/me crosses fingers that Mozilla fix the Firefox bug.
UPDATE 2:
After some digging it appears that IE8's XDomainRequest cannot be used to make cross domain requests where NTLM must be negotiated, this basically means that the security on our WCF binding can't be used thanks to limitations in a web browser.
http://blogs.msdn.com/b/ieinternals/archive/2010/05/13/xdomainrequest-restrictions-limitations-and-workarounds.aspx
"No authentication or cookies will be sent with the request"
So, I guess we've taken this as far as it is going to go for now.. It looks like we're going to have to create our own custom token authentication and pass it across to the WCF service in a cookie, or in IE8's case, POST it with the JSON. The WCF service will then have to handle decrypting the data and using that instead of the ServiceSecurityContext.Current.WindowsIdentity we previously had access to with NTLM auth.

I know you said you would rather have the problem itself addressed, but you may consider using a "reverse proxy."
I don't know what technologies you are using, but we use Apache web server and have a Java RESTful API running on a different server that required authentication. For a while, we messed with JSONP and CORS, but were not satisfied.
In the end, we setup an Apache Reverse Proxy and it worked miracles. The web browser believes it is communicating with its own domain and acts appropriately. The RESTful API doesn't know it is being used via a proxy. Therefore, everything just works. And Apache does all the magic.
Hopefully, all web servers have a feature like Apache's reverse proxy.
Here is some documentation on the feature: http://httpd.apache.org/docs/2.2/mod/mod_proxy.html
All we had to do is ensure the mod_proxy module was installed, then add the following lines to our Apache config file:
ProxyPass /restapi http://restfulserver.com/restapi
ProxyPassReverse /restapi http://restfulserver.com/restapi
Then restart the web server and voila!

Submit cross domain ajax POST request

I swear I saw an article about this at one point but can not find it...
How can I perform a jQuery ajax request of type POST on another domain? Must be accomplished without a proxy. Is this possible?

Yes you can POST all you want, even $.post() works...but you won't get a response back.
This works, the other domain will get the POST:
$.post("http://othersite.com/somePage.php", { thing: "value" }, function(data) {
//data will always be null
});
But the response, data in the above example, will be null due to the same-origin policy.

All the options I've experimented with:
1) PORK: http://www.schizofreend.nl/Pork.Iframe/Examples/ Creates an iframe and submits the post there, then reads the response. Still requires same base domain per
request (i.e. www.foo.com can request
data from www2.foo.com, but not from
www.google.com) . Also requires you to
fiddle with the document.domain
property, which causes adverse side
effects. And there's a pervasive problem in all the major browsers where reloading the page basically shuffles the cached contents of all iframes on the page if any of them are dynamically written. Your response data will show up in the box where an ad is supposed to be.
2) flxhr: http://flxhr.flensed.com/ Can even be used to mask jQuery's built-in ajax so you don't even notice it. Requires flash though, so iPhone is out
3) jsonp: Doesn't work if you're posting a lot of data. boo.
4) chunked jsonp: When your jsonp request is too big, break the query string up into manageable chunks and send multiple get requests. Reconstruct them on the server. This is helpful but breaks down if you're load balancing users between servers.
5) CORS: http://www.w3.org/TR/cors/ doesn't work in older browsers (IE7, IE6, Firefox 2, etc)
So we currently do the following algorithm:
If request is small enough, use JSONP
If not small enough, but user has flash, use FlXHR
Else use chunked JSONP
Spend one afternoon writing that up and you'll be able to use it for good. Adding CORS to our algorithm might be good for faster iPhone support.

If you have control over the code running at the other domain, just let it return an appropriate Access-Control-Allow-Origin header in the response. See also HTTP Access-Control at MDC.

If you want a fire and forget POST where you don't care about the response then just submit a form to a hidden iframe. This requires a Transitional Doctype.
<form method="POST" action="http://example.com/" target="name_of_iframe">
If you want to parse the response, then using a proxy if the only real option.
If you are desperate, and control the remote site, then you can:
Submit a form as above
Set a cookie in the response (which might be blocked before the iframe could cause the cookie to be considered '3rd party' (i.e. likely to be advertising tracking).
Wait long enough for the response to come back
Dynamically generate a script element with the src pointing to the remote site
Use JSON-P in the response and take advantage of the data previously stored in the cookie
This approach is subject to race conditions and generally ugly. Proxing the data through the current domain is a much better approach.

If you need to know that the POST was successful, and don't have control over the remote server:
$.ajax({
type:"POST",
url:"http://www.somesite.com/submit",
data:'firstname=test&lastname=person&email=test#test.com',
complete: function(response){
if(response.status == 0 && response.statusText == "success")
{
/* CORS POST was successful */
}
else
{
/* Show error message */
}
}
});
If there was a problem with the submission then response.statusText should equal "error".
Note: some remote servers will send the HTTP header Access-Control-Allow-Origin: *, which will result in a 200 OK HTTP status code response. In that case, ajax will execute the success handler, and this method is not needed. To look at the response just do console.log(JSON.stringify(response)); or use FireBug's 'Net' panel.

Cross-site AJAX requests

I need to make an AJAX request from a website to a REST web service hosted in another domain.
Although this is works just fine in Internet Explorer, other browsers such as Mozilla and Google Chrome impose far stricter security restrictions, which prohibit cross-site AJAX requests.
The problem is that I have no control over the domain nor the web server where the site is hosted. This means that my REST web service must run somewhere else, and I can't put in place any redirection mechanism.
Here is the JavaScript code that makes the asynchronous call:
var serviceUrl = "http://myservicedomain";
var payload = "<myRequest><content>Some content</content></myRequest>";
var request = new XMLHttpRequest();
request.open("POST", serviceUrl, true); // <-- This fails in Mozilla Firefox amongst other browsers
request.setRequestHeader("Content-type", "text/xml");
request.send(payload);
How can I have this work in other browsers beside Internet Explorer?

maybe JSONP can help.
NB youll have to change your messages to use json instead of xml
Edit
Major sites such as flickr and twitter support jsonp with callbacks etc

The post marked as the answer is erroneous: the iframes document is NOT able to access the parent. The same origin policy works both ways.
The fact is that it is not possible in any way to consume a rest based webservice using xmlhttprequest. The only way to load data from a different domain (without any framework) is to use JSONP. Any other solutions demand a serverside proxy located on your own domain, or a client side proxy located on the remote domain and som sort of cross-site communication (like easyXDM) to communicate between the documents.

The fact that this works in IE is a security issue with IE, not a feature.
Unfortunately cross-site scripting is prohibited, and the accepted work around is to proxy the requests through your own domain: do you really have no ability to add or modify server side code?
Furthermore, the secondary workaround - involving the aquisition of data through script tags - is only going to support GET requests, which you might be able to hack with a SOAP service, but not so much with the POST request to a RESTful service you describe.
I'm really not sure an AJAX solution exists, you might be back to a <form> solution.

The not very clear workaround (but works) is using iframe as container for requests to another sites. The problem is, the parent can not access iframe's content, can only navigate iframe's "src" attribut. But the iframe content can access parent's content.
So, if the iframe's content know, they can call some javascript content in parent page or directly access parent's DOM.
EDIT:
Sample:
function ajaxWorkaroung() {
var frm = gewtElementById("myIFrame")
frm.src = "http://some_other_domain"
}
function ajaxCallback(parameter){
// this function will be called from myIFrame's content
}

Make your service domain accept cross origin resource sharing (CORS).
Typical scenario: Most CORS compliant browsers will first send an OPTIONS header, to which, the server should return information about which headers are accepted. If the headers satisfy the service's requirements for the request provided (Allowed Methods being GET and POST, Allowed-Origin *, etc), the browser will then resend the request with the appropriate method (GET, POST, etc.).
Everything this point forward is the same as when you are using IE, or more simply, if you were posting to the same domain.
Caviots: Some service development SDK's (WCF in particular) will attempt to process the request, in which case you need to preprocess the OPTIONS Method to respond to the request and avoid the method being called twice on the server.
In short, the problem lies server-side.
Edit There is one issue with IE 9 and below with CORS, in that it is not fully implemented. Luckily, you can solve this problem by making your calls from server-side code to the service and have it come back through your server (e.g. mypage.aspx?service=blah&method=blahblah&p0=firstParam=something). From here, your server side code should implement a request/response stream model.

Just use a server side proxy on your origin domain. Here is an example: http://jquery-howto.blogspot.com/2009/04/cross-domain-ajax-querying-with-jquery.html

This can also be done using a webserver setup localy that calls curl with the correct arguments and returns the curl output.
app.rb
require 'sinatra'
require 'curb'
set :views,lambda {"views/"+self.name.to_s.downcase.sub("controller","")}
set :haml, :layout => :'../layout', :format => :html5, :escape_html=>true
disable :raise_errors
get '/data/:brand' do
data_link = "https://externalsite.com/#{params[:brand]}"
c = Curl::Easy.perform(data_link)
c.body_str
end
Sending an ajax request to localhost:4567/data/something will return the result from externalsite.com/something.

Another option would be to setup a CNAME record on your own domain to "Mask" the remote domain hostname.