ServiceWorker fails to fetch external resource without reason

ServiceWorker fails to fetch external resource without reason - javascript

I'm trying to fetch and cache some external resources/websites using a service worker.
My code in service-worker.js is the following:
'use strict';
var static_urls = [
'https://quiqqer.local/test?app=1',
'https://quiqqer.local/calendar?app=1'
];
self.addEventListener('install', function (event)
{
event.waitUntil(
caches.open('ionic-cache').then(function(cache) {
cache.addAll(static_urls.map(function (urlToPrefetch)
{
console.log(urlToPrefetch);
return new Request(urlToPrefetch, {mode: 'no-cors'});
})).catch(function(error) {
console.error(error);
}).then(function() {
console.log('All fetched and cached');
});
})
);
});
Which creates this output:
service-worker.js: https://quiqqer.local/test?app=1
service-worker.js: https://quiqqer.local/calendar?app=1
service-worker.js: TypeError: failed to fetch
service-worker.js: All fetched and cached
(index): service worker installed
What is the reason for the failing fetch?
My site https://quiqqer.local has set the header Access-Control-Allow-Origin to '*'
Maybe it's my self-signed certificate for my site?
I added an exception for this certificate, so if I open the site Chrome shows that the site isn't secure next to the URL bar but the content is still displayed.

Since you're explicitly setting {mode: 'no-cors'}, you're going to get back an opaque Response—one that always has a response code of 0.
cache.add() will throw a TypeError if the Response that you're attempting to add to the cache has a code outside of the 2xx range.
Because you have CORS enabled on your server, via the Access-Control-Allow-Origin header, you can just make a request with its mode set to 'cors'. 'cors' mode is the default for cross-origin URLs, so
cache.addAll(static_urls)
.then(...)
.catch(...);
should give you the behavior that you want.
If you were making a request against a server that didn't support CORS, and you wanted to cache it, then you'd have to explicitly set {mode: 'no-cors'} and use a combination of fetch() + cache.put() to add the opaque response the cache. (And you'd have to assume the risk that the opaque response isn't a 4xx or 5xx error, since there's no way of knowing.) But, you don't have to worry about that because your server does support CORS.

Related

how to convert this curl command to javascript fetch [duplicate]

I can hit this endpoint, http://catfacts-api.appspot.com/api/facts?number=99 via Postman and it returns JSON
Additionally I am using create-react-app and would like to avoid setting up any server config.
In my client code I am trying to use fetch to do the same thing, but I get the error:
No 'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://localhost:3000' is therefore not allowed
access. If an opaque response serves your needs, set the request's
mode to 'no-cors' to fetch the resource with CORS disabled.
So I am trying to pass in an object, to my Fetch which will disable CORS, like so:
fetch('http://catfacts-api.appspot.com/api/facts?number=99', { mode: 'no-cors'})
.then(blob => blob.json())
.then(data => {
console.table(data);
return data;
})
.catch(e => {
console.log(e);
return e;
});
Interestingly enough the error I get is actually a syntax error with this function. I am not sure my actual fetch is broken, because when I remove the { mode: 'no-cors' } object, and supply it with a different URL it works just fine.
I have also tried to pass in the object { mode: 'opaque'} , but this returns the original error from above.
I belive all I need to do is disable CORS.. What am I missing?

mode: 'no-cors' won’t magically make things work. In fact it makes things worse, because one effect it has is to tell browsers, “Block my frontend JavaScript code from seeing contents of the response body and headers under all circumstances.” Of course you never want that.
What happens with cross-origin requests from frontend JavaScript is that browsers by default block frontend code from accessing resources cross-origin. If Access-Control-Allow-Origin is in a response, then browsers relax that blocking and allow your code to access the response.
But if a site sends no Access-Control-Allow-Origin in its responses, your frontend code can’t directly access responses from that site. In particular, you can’t fix it by specifying mode: 'no-cors' (in fact that’ll ensure your frontend code can’t access the response contents).
However, one thing that will work: if you send your request through a CORS proxy.
You can also easily deploy your own proxy to Heroku in just 2-3 minutes, with 5 commands:
git clone https://github.com/Rob--W/cors-anywhere.git
cd cors-anywhere/
npm install
heroku create
git push heroku master
After running those commands, you’ll end up with your own CORS Anywhere server running at, for example, https://cryptic-headland-94862.herokuapp.com/.
Prefix your request URL with your proxy URL; for example:
https://cryptic-headland-94862.herokuapp.com/https://example.com
Adding the proxy URL as a prefix causes the request to get made through your proxy, which:
Forwards the request to https://example.com.
Receives the response from https://example.com.
Adds the Access-Control-Allow-Origin header to the response.
Passes that response, with that added header, back to the requesting frontend code.
The browser then allows the frontend code to access the response, because that response with the Access-Control-Allow-Origin response header is what the browser sees.
This works even if the request is one that triggers browsers to do a CORS preflight OPTIONS request, because in that case, the proxy also sends back the Access-Control-Allow-Headers and Access-Control-Allow-Methods headers needed to make the preflight successful.
I can hit this endpoint, http://catfacts-api.appspot.com/api/facts?number=99 via Postman
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS explains why it is that even though you can access the response with Postman, browsers won’t let you access the response cross-origin from frontend JavaScript code running in a web app unless the response includes an Access-Control-Allow-Origin response header.
http://catfacts-api.appspot.com/api/facts?number=99 has no Access-Control-Allow-Origin response header, so there’s no way your frontend code can access the response cross-origin.
Your browser can get the response fine and you can see it in Postman and even in browser devtools—but that doesn’t mean browsers expose it to your code. They won’t, because it has no Access-Control-Allow-Origin response header. So you must instead use a proxy to get it.
The proxy makes the request to that site, gets the response, adds the Access-Control-Allow-Origin response header and any other CORS headers needed, then passes that back to your requesting code. And that response with the Access-Control-Allow-Origin header added is what the browser sees, so the browser lets your frontend code actually access the response.
So I am trying to pass in an object, to my Fetch which will disable CORS
You don’t want to do that. To be clear, when you say you want to “disable CORS” it seems you actually mean you want to disable the same-origin policy. CORS itself is actually a way to do that — CORS is a way to loosen the same-origin policy, not a way to restrict it.
But anyway, it’s true you can—in your local environment—do suff like give a browser runtime flags to disable security and run insecurely, or you can install a browser extension locally to get around the same-origin policy, but all that does is change the situation just for you locally.
No matter what you change locally, anybody else trying to use your app is still going to run into the same-origin policy, and there’s no way you can disable that for other users of your app.
You most likely never want to use mode: 'no-cors' in practice except in a few limited cases, and even then only if you know exactly what you’re doing and what the effects are. That’s because what setting mode: 'no-cors' actually says to the browser is, “Block my frontend JavaScript code from looking into the contents of the response body and headers under all circumstances.” In most cases that’s obviously really not what you want.
As far as the cases when you would want to consider using mode: 'no-cors', see the answer at What limitations apply to opaque responses? for the details. The gist of it is:
In the limited case when you’re using JavaScript to put content from another origin into a <script>, <link rel=stylesheet>, <img>, <video>, <audio>, <object>, <embed>, or <iframe> element (which works because embedding of resources cross-origin is allowed for those)—but for some reason you don’t want to/can’t do that just by having the markup of the document use the resource URL as the href or src attribute for the element.
When the only thing you want to do with a resource is to cache it. As alluded to in What limitations apply to opaque responses?, in practice the scenario that’s for is when you’re using Service Workers, in which case the API that’s relevant is the Cache Storage API.
But even in those limited cases, there are some important gotchas to be aware of; see the answer at What limitations apply to opaque responses? for the details.
I have also tried to pass in the object { mode: 'opaque'}
There is no 'opaque' request mode — opaque is instead just a property of the response, and browsers set that opaque property on responses from requests sent with no-cors mode.
But incidentally the word opaque is a pretty explicit signal about the nature of the response you end up with: “opaque” means you can’t see into any of its details; it blocks you from seeing.

If you are trying to address this issue temporarily on your localhost, you can use this chrome extension : Allow CORS Access-Control-Allow-Origin
https://chrome.google.com/webstore/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlf

If you are using Express as back-end you just have to install cors and import and use it in app.use(cors());.
If it is not resolved then try switching ports.
It will surely resolve after switching ports

So if you're like me and developing a website on localhost where you're trying to fetch data from Laravel API and use it in your Vue front-end, and you see this problem, here is how I solved it:
In your Laravel project, run command php artisan make:middleware Cors. This will create app/Http/Middleware/Cors.php for you.
Add the following code inside the handles function in Cors.php:
return $next($request)
->header('Access-Control-Allow-Origin', '*')
->header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS');
In app/Http/kernel.php, add the following entry in $routeMiddleware array:
‘cors’ => \App\Http\Middleware\Cors::class
(There would be other entries in the array like auth, guest etc. Also make sure you're doing this in app/Http/kernel.php because there is another kernel.php too in Laravel)
Add this middleware on route registration for all the routes where you want to allow access, like this:
Route::group(['middleware' => 'cors'], function () {
Route::get('getData', 'v1\MyController#getData');
Route::get('getData2', 'v1\MyController#getData2');
});
In Vue front-end, make sure you call this API in mounted() function and not in data(). Also make sure you use http:// or https:// with the URL in your fetch() call.
Full credits to Pete Houston's blog article.

You can also set up a reverse proxy which adds the CORS headers using a self-hosted CORS Anywhere or Just CORS if you want a managed solution.
https://justcors.com/<id>/<your-requested-resource>
http://cors-anywhere.com/<your-requested-resource>

Very easy solution (2 min to config) is to use local-ssl-proxy package from npm
The usage is straight pretty forward:
1. Install the package:
npm install -g local-ssl-proxy
2. While running your local-server mask it with the local-ssl-proxy --source 9001 --target 9000
P.S: Replace --target 9000 with the -- "number of your port" and --source 9001 with --source "number of your port +1"

Solution for me was to just do it server side
I used the C# WebClient library to get the data (in my case it was image data) and send it back to the client. There's probably something very similar in your chosen server-side language.
//Server side, api controller
[Route("api/ItemImage/GetItemImageFromURL")]
public IActionResult GetItemImageFromURL([FromQuery] string url)
{
ItemImage image = new ItemImage();
using(WebClient client = new WebClient()){
image.Bytes = client.DownloadData(url);
return Ok(image);
}
}
You can tweak it to whatever your own use case is. The main point is client.DownloadData() worked without any CORS errors. Typically CORS issues are only between websites, hence it being okay to make 'cross-site' requests from your server.
Then the React fetch call is as simple as:
//React component
fetch(`api/ItemImage/GetItemImageFromURL?url=${imageURL}`, {
method: 'GET',
})
.then(resp => resp.json() as Promise<ItemImage>)
.then(imgResponse => {
// Do more stuff....
)}

I had a similar problem with my browser debugger saying my response.body was null but fiddler and the developer tools show it as populated that turned out to be basically the same scenario as this. I was using a local Angular application hitting a Web Api service running on IISExpress. I fixed it by following the steps outlined here to find the correct applicationhost.config file to add a Access-Control-Allow-Origin header like so:
<customHeaders>
<clear />
<add name="X-Powered-By" value="ASP.NET" />
<add name="Access-Control-Allow-Origin" value="*" />
<add name="Access-Control-Allow-Headers" value="Content-Type" />
</customHeaders>

If all the above solutions don't work, probably it's because of the file permissions as sometimes even if you have fixed the non-cors problem using Heroku or another way, it throws 403 forbidden error. Set the directory/file permissions like this:
Permissions and ownership errors
A 403 Forbidden error can also be caused by incorrect ownership or permissions on your web content files and folders.
Permissions
Rule of thumb for correct permissions:
Folders: 755
Static Content: 644
Dynamic Content: 700

Parse external json using Fetch API in jsfiddle [duplicate]

I can hit this endpoint, http://catfacts-api.appspot.com/api/facts?number=99 via Postman and it returns JSON
Additionally I am using create-react-app and would like to avoid setting up any server config.
In my client code I am trying to use fetch to do the same thing, but I get the error:
No 'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://localhost:3000' is therefore not allowed
access. If an opaque response serves your needs, set the request's
mode to 'no-cors' to fetch the resource with CORS disabled.
So I am trying to pass in an object, to my Fetch which will disable CORS, like so:
fetch('http://catfacts-api.appspot.com/api/facts?number=99', { mode: 'no-cors'})
.then(blob => blob.json())
.then(data => {
console.table(data);
return data;
})
.catch(e => {
console.log(e);
return e;
});
Interestingly enough the error I get is actually a syntax error with this function. I am not sure my actual fetch is broken, because when I remove the { mode: 'no-cors' } object, and supply it with a different URL it works just fine.
I have also tried to pass in the object { mode: 'opaque'} , but this returns the original error from above.
I belive all I need to do is disable CORS.. What am I missing?

If you are trying to address this issue temporarily on your localhost, you can use this chrome extension : Allow CORS Access-Control-Allow-Origin
https://chrome.google.com/webstore/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlf

If you are using Express as back-end you just have to install cors and import and use it in app.use(cors());.
If it is not resolved then try switching ports.
It will surely resolve after switching ports

So if you're like me and developing a website on localhost where you're trying to fetch data from Laravel API and use it in your Vue front-end, and you see this problem, here is how I solved it:
In your Laravel project, run command php artisan make:middleware Cors. This will create app/Http/Middleware/Cors.php for you.
Add the following code inside the handles function in Cors.php:
return $next($request)
->header('Access-Control-Allow-Origin', '*')
->header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS');
In app/Http/kernel.php, add the following entry in $routeMiddleware array:
‘cors’ => \App\Http\Middleware\Cors::class
(There would be other entries in the array like auth, guest etc. Also make sure you're doing this in app/Http/kernel.php because there is another kernel.php too in Laravel)
Add this middleware on route registration for all the routes where you want to allow access, like this:
Route::group(['middleware' => 'cors'], function () {
Route::get('getData', 'v1\MyController#getData');
Route::get('getData2', 'v1\MyController#getData2');
});
In Vue front-end, make sure you call this API in mounted() function and not in data(). Also make sure you use http:// or https:// with the URL in your fetch() call.
Full credits to Pete Houston's blog article.

You can also set up a reverse proxy which adds the CORS headers using a self-hosted CORS Anywhere or Just CORS if you want a managed solution.
https://justcors.com/<id>/<your-requested-resource>
http://cors-anywhere.com/<your-requested-resource>

Very easy solution (2 min to config) is to use local-ssl-proxy package from npm
The usage is straight pretty forward:
1. Install the package:
npm install -g local-ssl-proxy
2. While running your local-server mask it with the local-ssl-proxy --source 9001 --target 9000
P.S: Replace --target 9000 with the -- "number of your port" and --source 9001 with --source "number of your port +1"

Solution for me was to just do it server side
I used the C# WebClient library to get the data (in my case it was image data) and send it back to the client. There's probably something very similar in your chosen server-side language.
//Server side, api controller
[Route("api/ItemImage/GetItemImageFromURL")]
public IActionResult GetItemImageFromURL([FromQuery] string url)
{
ItemImage image = new ItemImage();
using(WebClient client = new WebClient()){
image.Bytes = client.DownloadData(url);
return Ok(image);
}
}
You can tweak it to whatever your own use case is. The main point is client.DownloadData() worked without any CORS errors. Typically CORS issues are only between websites, hence it being okay to make 'cross-site' requests from your server.
Then the React fetch call is as simple as:
//React component
fetch(`api/ItemImage/GetItemImageFromURL?url=${imageURL}`, {
method: 'GET',
})
.then(resp => resp.json() as Promise<ItemImage>)
.then(imgResponse => {
// Do more stuff....
)}

I had a similar problem with my browser debugger saying my response.body was null but fiddler and the developer tools show it as populated that turned out to be basically the same scenario as this. I was using a local Angular application hitting a Web Api service running on IISExpress. I fixed it by following the steps outlined here to find the correct applicationhost.config file to add a Access-Control-Allow-Origin header like so:
<customHeaders>
<clear />
<add name="X-Powered-By" value="ASP.NET" />
<add name="Access-Control-Allow-Origin" value="*" />
<add name="Access-Control-Allow-Headers" value="Content-Type" />
</customHeaders>

If all the above solutions don't work, probably it's because of the file permissions as sometimes even if you have fixed the non-cors problem using Heroku or another way, it throws 403 forbidden error. Set the directory/file permissions like this:
Permissions and ownership errors
A 403 Forbidden error can also be caused by incorrect ownership or permissions on your web content files and folders.
Permissions
Rule of thumb for correct permissions:
Folders: 755
Static Content: 644
Dynamic Content: 700

Get Header Request Javascript: cURL working, fetch forbidden

i want to GET Request the Header of this link https://soundcloud.app.goo.gl/u6XXuqp3Q7Q7WXgH8 in a React App. When doing a simple request
request('https://soundcloud.app.goo.gl/u6XXuqp3Q7Q7WXgH8', function (error, response, body) {
console.error('error:', error);
console.log('body:', body);
});
I get
Access to fetch at 'https://soundcloud.app.goo.gl/u6XXuqp3Q7Q7WXgH8' from
origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-
Control-Allow-Origin' header is present on the requested resource. If an
opaque response serves your needs, set the request's mode to 'no-cors' to
fetch the resource with CORS disabled.
When using the cors-anywhere proxy
request('https://cors-anywhere.herokuapp.com/https://soundcloud.app.goo.gl/u6XXuqp3Q7Q7WXgH8', function (error, response, body) {
console.error('error:', error);
console.log('body:', body);
});
I get
request.js:150 GET https://cors-anywhere.herokuapp.com/https://soundcloud.app.goo.gl/u6XXuqp3Q7Q7WXgH8 403 (Forbidden)
But when I just cURL I get the right response
curl -s -o /dev/null -D - https://soundcloud.app.goo.gl/u6XXuqp3Q7Q7WXgH8
Can anyone explain this behaviour to me and maybe tell me how to overcome this?
Thanks!

CORS error usually comes up when you are trying to call an API from your browser directly and that API doesn't allow any other website to get the data.
There can be 2 workarounds to this problem I can think as of now -
Either you call that API from your backend code and you gather the data received from the website in backend and transfer the data on the client-side. Basically, you will be Wrapping the original API.
You can use Web Servers like NGINX and configure a specific path on the domain pointing to External API and other paths to your project. Which will help in making both the External API and Your React code on the same domain name.
Mainly your objective is to have that external API to work through your domain working along with your react code.

In browser context JS, such as a in a fetch call to an API, CORS applies. Rather with a CURL request, you are essentially running server side context, therefore CORS doesn't apply. This is why you can hit instagrams unofficial api from a server context or node.js context, but not from the same fetch call in a browser. Possible solutions:
Allow-Content-Orgin header set by the server must include the domain your page is hosted on.
Use Eletron or NodeWebkit.js for visual apps running client side with a browser (chromium) to launch your app in (mix of both browser and server context, CORS does not apply)
Run a node.js app (no browser, server context) and run your app from the terminal. (CORS does not apply)

Can fetch API from node.js but not from client [duplicate]

I tried to fetch the URL of an old website, and an error happened:
Fetch API cannot load http://xyz.
No 'Access-Control-Allow-Origin' header is present on the requested resource.
Origin 'http://abc' is therefore not allowed access.
If an opaque response serves your needs, set the request's mode to 'no-cors'
to fetch the resource with CORS disabled.
I understood the message, and tried to do a request that returns an opaque response:
fetch("http://xyz", {'mode': 'no-cors'})
Ok, it now works... but I can't read it. =\
What's the purpose then, of an opaque response?

Consider the case in which a service worker acts as an agnostic cache. Your only goal is serve the same resources that you would get from the network, but faster. Of course you can't ensure all the resources will be part of your origin (consider libraries served from CDNs, for instance). As the service worker has the potential of altering network responses, you need to guarantee you are not interested in the contents of the response, nor on its headers, nor even on the result. You're only interested on the response as a black box to possibly cache it and serve it faster.
This is what { mode: 'no-cors' } was made for.

Opaque responses can't be accessed by JavaScript, but you can still cache them with the Cache API and respond with them in the fetch event handler in a service worker. So they're useful for making your app offline, also for resources that you can't control (e.g. resources on a CDN that doesn't set the CORS headers).

There's also solution for Node JS app. CORS Anywhere is a NodeJS proxy which adds CORS headers to the proxied request.
The url to proxy is literally taken from the path, validated and proxied. The protocol part of the proxied URI is optional, and defaults to "http". If port 443 is specified, the protocol defaults to "https".
This package does not put any restrictions on the http methods or headers, except for cookies. Requesting user credentials is disallowed. The app can be configured to require a header for proxying a request, for example to avoid a direct visit from the browser. https://robwu.nl/cors-anywhere.html

javascript is a bit tricky getting the answer, I fixed it by getting the api from the backend and then calling it to the frontend.
public function get_typechange () {
$ url = "https://........";
$ json = file_get_contents ($url);
$ data = json_decode ($ json, true);
$ resp = json_encode ($data);
$ error = json_last_error_msg ();
return $ resp;
}

Fetch throws "TypeError: Failed to fetch" for successful, same-origin request

We have been encountering inconsistent client errors with a single-page JavaScript application making fetch requests. Of note, they are all same-origin requests.
let request = new Request(url, options);
...
window.fetch(request)
.then(response => response.json())
.then(data => ...)
.catch(error => ...)
Around 5% of the promises are rejecting with the following error despite the server and the browser receiving a 200 OK response:
TypeError: Failed to fetch
I'm stumped... All of my searches lead to discussions about CORS errors. That doesn't seem to apply given these are all same-origin requests. What is causing the fetch to throw the TypeError?
I can confirm using the Network tab in Chrome DevTools that the fetch request completes with a 200 OK response and valid JSON. I can also confirm that the URLs are same-origin. I can also confirm that there are no CORS pre-flight requests. I have reproduced this issue on Chrome 66 and Safari 11.1. However, we've received a stream of error reports from a mix of Chrome and Safari versions, both desktop and mobile.
EDIT:
This does not appear to be a duplicate of the linked question as we are not sending CORS requests, not setting mode: "no-cors", and not setting the Access-Control-Allow-Origin header.
Additionally, I re-ran tests with the mode: 'same-origin' option set explicitly. The requests are (still) successful; however, we (still) receive the intermittent TypeError.

I know that this is an old issue, but after searching the entire evening I want to share my findings so you can spend your time better.
My web app also worked well for most users but from time to time visitors received the error mentioned in the question. I'm not using any complicated infrastructure (reverse proxy etc.) setup nor do I communicate with services on a different domain/protocol/port. I'm just sending a POST request to a PHP-File on the same server where the React app is served from.
The short answer: My problem was that I've sent the request to the backend by using an absolute URL, like https://my-fancy-domain.com/funky_service.php. After changing this to a relative path like /funky-service.php the issue was gone.
My explanation: Most users come to the site without www in the URL, but some users actually do type this part in their address bars (www.my-fancy...). It turned out that the www is part of the origin, so when these users submit the form and send post requests to https://my-fancy... it's technically another origin. This is why the browser expects CORS headers and sometimes even sends an OPTIONS preflight request. When you use a relative path in your JavaScript-Code the post request will also include the www-part (uses the origin from the address bar) -> same-origin -> no CORS hassle. As it only affects visitors that come with the www to your site it also explains the fact that it worked for most users even with the absolute URL.
Also important to know: The request fails in the browser/ JavaScript-Code but is actually sent to the backend (very ugly!).
Let me know if you need more information. Actually, it is very simple but hard to explain (and to find)

The issue could be with the response you are receiving from back-end. If it was working fine on the server then the problem could be with the response headers. Check the Access-Control-Allow-Origin (ACAO) in the response headers. Usually react's fetch API will throw fail to fetch even after receiving response when the response headers' ACAO and the origin of request won't match.
Ref: Getting "TypeError: failed to fetch" when the request hasn't actually failed

We Keep Coding

JavaScript is the programming language of the Web.