How to get html/text from webpage [duplicate] - javascript

This question already has answers here:
Ways to circumvent the same-origin policy
(8 answers)
Closed 8 years ago.
I'd like to get the text on this page:
https://cvo-v025.cvo-zwfryslan.nl/display/ToonBerichten.aspx?uid=ctl14&pid=723df4e4-248f-4df6-b3ad-751b410daab7&id=1c76d69d-d858-44d9-8a47-e65e9f294898
Php cUrl isn't working, YQL isn't working (but didn't give an error), javascript didn't work, the error message was:
XMLHttpRequest cannot load https://cvo-v025.cvo-zwfryslan.nl/display/ToonBerichten.aspx?uid=ctl14&pid=…3df4e4-248f-4df6-b3ad-751b410daab7&id=1c76d69d-d858-44d9-8a47-e65e9f294898. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://--------.nl' is therefore not allowed access.
Is there a way for me to get the text from that page?

The page you're trying to fetch text from actually makes a POST request to another resource to fetch the text via XHR. When you open Firebug or similar, you should see the POST request, its URL and response. It would appear that you need to have a session on the site to actually fetch anything, as making a POST request to that URL fails to retrieve anything useful.
Copying the request as CURL does yield a working terminal command, along with all sent headers, but I doubt it will be helpful if you wish to do this programmatically.

Related

API working with POSTMAN but not with fetch [duplicate]

This question already has an answer here:
Fetching API json data alert: Content Security Policy: The page’s settings blocked the loading of a resource at
(1 answer)
Closed 4 months ago.
I know there have been lots of questions on this topic and I read them all but not able to resolve my issue so asking here.
I am getting the following error while trying to access an api using fetch-
Refused to connect to 'https://xxxxx/api' because it violates the
following Content Security Policy directive: "connect-src 'self'
updates.developer.mozilla.org
www.google-analytics.com stats.g.doubleclick.net"
I don't have any control over this api so I can't add anything to connect-src. When making a fetch(api_url) request I am getting the above error but while making the same GET request from POSTMAN(or directly from url bar of browser) then I am able to receive the response body(which is an xml).
Also while making a fetch request with mode set to 'no-cors' I am able to get a response but not the response body. I want the response body.
What is POSTMAN doing differently? How can I achieve the same using fetch()?
Try installing the 'CORS Unblock' chrome extension and then run your application. Also, this error generally occurs when we use an API link as http://'apilink' when the link is https://'apilink' or vice-versa. Please check that as well.

How to make a CORS request from the client [duplicate]

This question already has answers here:
Why doesn't adding CORS headers to an OPTIONS route allow browsers to access my API?
(36 answers)
How does the 'Access-Control-Allow-Origin' header work?
(19 answers)
No 'Access-Control-Allow-Origin' header is present on the requested resource—when trying to get data from a REST API
(26 answers)
Closed 4 years ago.
im trying to make a simple request to an API
fetch('someurl').then((data) => console.log(data))
but im getting the classic No 'Access-Control-Allow-Origin' header is present on the requested resource.
how can I fix this on the client side? or is the only way to fix it for the API author to change it and add the correct response headers?
To deepen you understand on CORS have a look at MDN's article on Cross-Origin Resource Sharing (CORS). It's pretty extensive.
Using jsonP you would be able to work around this when making simple GET requests. See also this older, short and sweet article that explains it in more detail. How JSONP Works.
The Wikipedia Definition of JSONP is as follows:
a communication technique used in JavaScript programs which run in Web
browsers. It provides a method to request data from a server in a
different domain, something prohibited by typical web browsers because
of the same origin policy.
With that in mind, let look at the following example and make the request.
$(document).ready(function() {
$.getJSON("https://jsonplaceholder.typicode.com/users?callback=?", function(json){
console.log('getJSON call: ', json);
});
})
FETCH does not support jsonp
After a bit of research, it does turn out that the Fetch API does not support jsonP requests. If you have a look at this jsFiddle example you'll see that the $.getJSON call returns data when used with the suffix ?callback=? while the 'fetch()' call fails and returns a CORS message. Open the console to see the result of both calls.
Your question in the comments
Also, do you know why fetch({ url : 'https://randomurl" }) would not
get a CORS blockage but fetch('https://randomurl') would?
The first argument you provide to fetch is a string/URL, the second (optional) argument can be an options object {}. Because you provide an object as the first argument, that URL cannot be found. The reason why it doesn't give you a CORS blockage is because you provided an invalid URL, which returns a 404 status. Fetch deals with page cannot be found errors by returning a 200 OK status and in the JSON returned it will provide you with more info.
The Promise returned from fetch() won’t reject on HTTP error status
even if the response is an HTTP 404 or 500. Instead, it will resolve
normally (with ok status set to false), and it will only reject on
network failure or if anything prevented the request from completing.
Source: MDN docs
I hope this helped a bit in broadening your understanding of CORS and the how Fetch works.
You can find some workaounds in Why does my JavaScript get a "No 'Access-Control-Allow-Origin' header is present on the requested resource" error when Postman does not? but you can't solve the problem from the client. It must be solved on server by setting correct headers that allow it...

Cross Domain JSON - does it matter what server provides source? [duplicate]

This question already has answers here:
XMLHttpRequest cannot load XXX No 'Access-Control-Allow-Origin' header
(11 answers)
Closed 3 years ago.
I've found simple tutorial how to make cross domain json call here
And it works perfectly fine, so i decided to use this example, just change url from:
var url = "http://api.myjson.com/bins/23xvb";
to
var url = "http://dl.sniper.pl/test.json"
Unfortunately changing it returns such an error (in chrome):
XMLHttpRequest cannot load http://dl.sniper.pl/test.json. Response to
preflight request doesn't pass access control check: No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'null' is therefore not allowed access.
Googling that error didnt provide any answers to find a solution so here's the question:
Why i get such an error and how to solve it?
The http://dl.sniper.pl/ server must be configured to send the Access-Control-Allow-Origin response header in responses to requests for http://dl.sniper.pl/test.json.
But because that server isn’t sending the Access-Control-Allow-Origin response header, your browser is refusing to allow your frontend JavaScript code to access that response.
So you either nust configure the http://dl.sniper.pl/ server to send Access-Control-Allow-Origin or else you can make the request through a CORS proxy.
There’s an open CORS proxy you can make you request through by changing your code to this:
var url = "https://cors-anywhere.herokuapp.com/http://dl.sniper.pl/test.json"
That sends the request through the open CORS proxy https://cors-anywhere.herokuapp.com which adds the Access-Control-Allow-Origin response header to it and then passes that back to your requesting frontend code as the response.
That response with the Access-Control-Allow-Origin response header is what the browser sees, so the browser allows your frontend JavaScript code to actually access the response.
You can also easily set up your own CORS proxy using https://github.com/Rob--W/cors-anywhere/
See https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS for an explanation of how browsers behave when you send cross-origin requests frontend JavaScript code using XHR or the Fetch API or AJAX methods from JavaScript libraries—and for details about what response headers must be received in order for browsers to allow frontend code to access the responses.
you should configure you server todo this in your htaccess
u need something like this
<RequireAll>
Require all granted
</RequireAll>

No 'Access-Control-Allow-Origin' header is present on the requested resource error. Request succeeds but failure is triggered [duplicate]

This question already has answers here:
XMLHttpRequest cannot load XXX No 'Access-Control-Allow-Origin' header
(11 answers)
Closed 5 years ago.
I'm encountering a bit of a strange issue when making an Ajax cross domain request. I get the following error in the console of chrome dev tools:
No 'Access-Control-Allow-Origin' header is present on the requested resource error
However, when I look at the network requests, it passes the browsers CORS preflight request because request changes from OPTIONS which it was when it was failing preflight request to GET, and the response is as I would get via postman. However, the Ajax failure message is triggered so even though in dev tools the request appears to succeed, I can't access the successful response via the JavaScript.
Additional info is that the file that is making the ajax request is just an HTML file with inline JavaScript that I open directly from the file directory. I'm thinking this might be my problem but couldn't find anything that explicitly says this so I am wanting confirmation.
Note with respect to the API: the appropriate access control headers are set
You have to pass some (if not all, I haven't checked) with every response, not only the response to the pre-flight OPTIONS request.

How do I load a static, plaintext json file into Javascript with a GET request? [duplicate]

This question already has answers here:
Get JSON data from external URL and display it in a div as plain text
(7 answers)
Closed 6 years ago.
This is the one I am testing with, specifically:
static json data
I have run into many cross-site request errors, though I don't see how this should be different than requesting an image that is statically hosted on the same site, which I can easily load into my test html file.
I have tried the following:
$.getJSON('http://anyorigin.com/get?url=maxcandocia.com/static/blog/test_output3.json&callback=?',
function(data){
$('#output').html(data.contents);
});
var network_data
$.getJSON("http://maxcandocia.com/static/blog/test_output3.json",
function(data){
network_data = data.contents;
})
In general, you can't access a resource on another origin (in essence domain-port combination) through JavaScript unless that origin explicitly allows you to through the use of specific headers, specifically the Access-Control-Allow-Origin.
It seems that anyorigin.com, the service that you're using, for some reason fails to redirect correctly; this does not appear to be your fault, but something wrong with the service. I would recommend that you try some other, equivalent service, such as https://crossorigin.me/ (just add https://crossorigin.me/ in front of the URL):
var network_data;
$.getJSON("https://crossorigin.me/https://maxcandocia.com/static/blog/test_output3.json",
function(data){
network_data = data.contents;
})
If you control the server yourself, it would be better to just set up the server to send the Access-Control-Allow-Origin header for your JSON file.
Access-Control-Allow-Origin is not an issue with your code. It means you have an issue with security policies.
If you own maxcandocia.com you need to adjust the headers to allow your origin (where you're running the script). If not, you have no chance here unless they change their policies for you (unlikely.
You would instead be looking for a server-side solution, such as what Frxstrem has suggested. Or if you own the script send back this header with the origins you require:
Access-Control-Allow-Origin: http://yourorigin.com http://maxcandocia.com https://maxcandocia.com
I have an in depth answer here on using json with jquery: https://stackoverflow.com/a/17768384/2376468

Categories