Load large json file from external website with D3 - javascript

On my site "mywebsite.com" I have a D3 javascript code running on some data set located at "otherwebsite.com/data.json", so I naively tried
d3.json("otherwebsite.com/data.json", function(error, json) {
if (!error) {
console.log('done loading',json)
} else {
console.log(error)
}
})
but of course it does not work :)
Access to XMLHttpRequest at 'otherwebsite/data.json' from origin 'https://mywebsite.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Anyone has a better idea? I should emphasize that it is a large file (200MB).
Thanks

Taken from the MDN page on CORS:
For security reasons, browsers restrict cross-origin HTTP requests initiated from scripts. For example, XMLHttpRequest and the Fetch API follow the same-origin policy. This means that a web application using those APIs can only request resources from the same origin the application was loaded from unless the response from other origins includes the right CORS headers.
The CORS header in question is Access-Control-Allow-Origin which needs to either be explicitly set to https://mywebsite.com or be set to something that includes that, such as *.
If you have access to the server for "otherwebsite.com" you can change the header being sent with the request.
Otherwise you'll probably have to download the data server side on "mywebsite.com" and then have the front end request it from your back end, rather than making a cross origin request, so you'll change the JS to look like:
d3.json("/data.json", function(error, json) {...

Related

Can fetch API from node.js but not from client [duplicate]

I tried to fetch the URL of an old website, and an error happened:
Fetch API cannot load http://xyz.
No 'Access-Control-Allow-Origin' header is present on the requested resource.
Origin 'http://abc' is therefore not allowed access.
If an opaque response serves your needs, set the request's mode to 'no-cors'
to fetch the resource with CORS disabled.
I understood the message, and tried to do a request that returns an opaque response:
fetch("http://xyz", {'mode': 'no-cors'})
Ok, it now works... but I can't read it. =\
What's the purpose then, of an opaque response?
Consider the case in which a service worker acts as an agnostic cache. Your only goal is serve the same resources that you would get from the network, but faster. Of course you can't ensure all the resources will be part of your origin (consider libraries served from CDNs, for instance). As the service worker has the potential of altering network responses, you need to guarantee you are not interested in the contents of the response, nor on its headers, nor even on the result. You're only interested on the response as a black box to possibly cache it and serve it faster.
This is what { mode: 'no-cors' } was made for.
Opaque responses can't be accessed by JavaScript, but you can still cache them with the Cache API and respond with them in the fetch event handler in a service worker. So they're useful for making your app offline, also for resources that you can't control (e.g. resources on a CDN that doesn't set the CORS headers).
There's also solution for Node JS app. CORS Anywhere is a NodeJS proxy which adds CORS headers to the proxied request.
The url to proxy is literally taken from the path, validated and proxied. The protocol part of the proxied URI is optional, and defaults to "http". If port 443 is specified, the protocol defaults to "https".
This package does not put any restrictions on the http methods or headers, except for cookies. Requesting user credentials is disallowed. The app can be configured to require a header for proxying a request, for example to avoid a direct visit from the browser. https://robwu.nl/cors-anywhere.html
javascript is a bit tricky getting the answer, I fixed it by getting the api from the backend and then calling it to the frontend.
public function get_typechange () {
$ url = "https://........";
$ json = file_get_contents ($url);
$ data = json_decode ($ json, true);
$ resp = json_encode ($data);
$ error = json_last_error_msg ();
return $ resp;
}

What is the difference between getJSON() function and post() function when make a request across domain?

I am trying to access a Google Apps Script WebAPI from my website using javascript to pass some value and create an excel file and download it through this API.
I tried 2 following way:
Using POST request with $.post.
My values are many. So, at first, I use a POST request with a body is JSON of list values. Browser rejects API response, because of CORS error.
I researched about CORS to understand it. At some topics, I found a solution is the following second way.
Access to XMLHttpRequest at 'https://script.google.com/macros/s/xxxxxxx' from origin 'https://example.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Using GET request with $.getJSON.
I pass JSON of list values to URL parameter and make GET request. It worked fine.
var url = 'https://script.google.com/macros/s/' + api_id + '/exec?' + request_parameter_string;
$.post(url, payload, function(data, textStatus) {
// Do something
}, 'json');
$.getJSON(url, function(json_result) {
// Do something
})
.fail(function() {
// Do something
});
What I do not understand is why? Why it works with getJSON but not work with post?
I think CORS work with both of GET and POST requests. And I checked the response header with Postman. The headers are the same Access-Control-Allow-Origin →*.
I think have something is different inside getJSON and post functions.
*UPDATE: Update POST CORS error message.
GET requests are not bound by CORS we can host images and static files in CDN which is different from the origin and would help in improving the performance by caching and making parallel requests.
Similarly GET is used for serving ads, trackers and analytics from third party domains as well.
More information about Same Origin Policy and GET is at https://security.stackexchange.com/a/16221/9517
How the browsers identify Other HTTP Verbs are allowed for the cross origin request is elaborated # https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request

Cannot load Deezer API resources from localhost with the Fetch API

I'm trying to access the Deezer API from localhost, but I'm keep getting the following error:
Fetch API cannot load http://api.deezer.com/search/track/autocomplete?limit=1&q=eminem.
Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Origin 'http://localhost' is therefore not allowed access.
If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
localhost's response's headers does have Access-Control-Allow-Origin header
(Access-Control-Allow-Origin:*).
I'm using fetch like:
fetch('http://api.deezer.com/search/track/autocomplete?limit=1&q=eminem').
What am I doing wrong?
You can make the request through a public CORS proxy; to do that try changing your code to:
fetch('https://cors-anywhere.herokuapp.com/http://api.deezer.com/search/track/autocomplete?limit=1&q=eminem')
That sends the request through https://cors-anywhere.herokuapp.com, which forwards the request to http://api.deezer.com/search/track/autocomplete?limit=1&q=eminem and then receives the response. The https://cors-anywhere.herokuapp.com backend adds the Access-Control-Allow-Origin header to the response and passes that back to your requesting frontend code.
The browser will then allow your frontend code to access the response, because that response with the Access-Control-Allow-Origin response header is what the browser sees.
You can also set up your own CORS proxy using https://github.com/Rob--W/cors-anywhere/
For details about what browsers do when you send cross-origin requests from frontend JS code using XHR or the Fetch API or AJAX methods from JavaScript libraries—and details about what response headers must be received in order for browsers to allow frontend code to access the responses—see https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS.
For now, it is impossible to make this request. You can proxy request to API from your own backend or use jsonp. Here is a library with fetch-like syntax https://github.com/camsong/fetch-jsonp. Usage example https://jsfiddle.net/4dmfo0dd/1/
fetchJsonp('https://api.deezer.com/search/track/autocomplete?limit=1&q=eminem&output=jsonp')
.then(function(response) {
return response.json();
})
.then(json => console.log(json))
.catch(function(error) { console.log(error); });
CORS or Cross Origin requests made to servers
http://api.deezer.com/search/track/autocomplete?limit=1&q=eminem
in this case, have a preflight check enabled by all modern browsers.
and usually fail, if the server does not respond with Access-control headers.
In case of a fetch too, since you are basically fiddling with Javascript ,
You Need the Server to respond with Access-control-Allow-Origin Headers, which are flexible.
You Can not do Much about it Unless, the API itself becomes flexible and more open.
You however can Use fetch with mode set to no-cors
IFFF you only wish to cache the result of the request you make, to serve as a response, and not consume it yourself
Read Opaque Responses
No-CORS Definition
no-cors — Prevents the method from being anything other than HEAD, GET or POST. If any ServiceWorkers intercept these requests, they may not add or override any headers except for these. In addition, JavaScript may not access any properties of the resulting Response. This ensures that ServiceWorkers do not affect the semantics of the Web and prevents security and privacy issues arising from leaking data across domains

Ajax call to API not working

I am trying to retrieve data from an API using Jquery's ajax(), but it doesn't work with this implemenation:
$.ajax('http://api.forismatic.com/api/1.0/?method=getQuote&format=json').done(function(data) {
alert(1);
alert(JSON.stringify(data));
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
After running the code the alert function doesn't work, so I concluded that the success callback function isn't working, but I have no idea why.
After inspecting the server response headers, there is no Access-Control-Allow-Origin, this means that the server doesn't allow cross-origin access. Since you make a cross-origin HTTP request, your request will be rejected by the browser following the Same-origin policy:
The same-origin policy restricts how a document or script loaded from
one origin can interact with a resource from another origin. It is a
critical security mechanism for isolating potentially malicious
documents.
Look at your console you will see the following error (Chrome):
XMLHttpRequest cannot load
http://api.forismatic.com/api/1.0/?method=getQuote&format=json. No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://s.codepen.io' is therefore not allowed
access.
For more details please refer to: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS

No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://fiddle.jshell.net'

Getting the below when doing a JQUERY ajax call in jsfiddle
No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://fiddle.jshell.net' is therefore not allowed access. The response had HTTP status code 404
URL works fine when I put it in a browser?
Does not work in JsFiddle?
var URL="http://query.yahooapis.com/v1/public/yql&env=http%3A%2F%2Fdatatables.org%2Falltables.env&format=json&q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22ACM%22)%0A%09%09";
$.get(URL).done(function (data) {
alert("HELLO");
}).fail(function (jqXHR, textStatus) {
alert("NOPE");
}).always(function () {
alert("OH WELLL");
});
I was struggling with the same problem on both JSFiddle and Codepen. I needed to load an external js-data file and it was blocked due to browser-security. My workaround was to include the whole js-data as script with HTML in the header:
<script src="http://yoururlto/yourdata.js"></script>
The data js itself looks like this:
var data = {
"type": "FeatureCollection",
"features": [{
}]
};
Then access the data in any way you want:
geojsonLayer.addData(data);
map.fitBounds(geojsonLayer.getBounds());
This is a standard CORS response when an XHR call is made from the browser. The server needs to allow the domain/location of the browser to make requests.
res.header('Access-Control-Allow-Origin', '*');
Otherwise, the server will block incoming requests.
Another way to get around this is to use a known server as a proxy to make requests to the external API.
It's because your API does not return a CORS header. This header can restrict what hosts to allow access to that specific page and this behaviour is embedded in the XMLHttpRequest functions that jQuery uses. It's used as security measure so there isn't really a workaround for it.

Categories