I'd like to intercept fetch from all parts and libraries in my application, and at the same time I'd like not to break possibility of working with the application via file URL - it is useful for Electron and mobile devices (via WebView). For now, I've found two possible ways of doing this:
something like here
const realFetch = window.fetch;
window.fetch = function() {
// do something
return realFetch.apply(this, arguments)
}
something like here, with service worker registration:
main.js:
if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('sw.js').then(function(registration) {
console.log('Service worker registered with scope: ', registration.scope);
}, function(err) {
console.log('ServiceWorker registration failed: ', err);
});
});
}
sw.js:
self.addEventListener('fetch', function(event) {
event.respondWith(
// intercept requests by handling event.request here
);
});
With the first approach I cannot intercept fetch requests from web workers. The second approach doesn't work with file URLs, and I want my application to work via file URL due to it allows me to use the app via Electron for desktops or WebView for Android. Is there any other way for intercepting fetch requests?
P.S. I cannot modify the worker I'm trying to intercept requests from.
Update:
On basis of #Ciro Corvino's answer, I've tried the third approach: to start my own worker before anythnig else and try to redefine fetch from there. Didn't work for me, unfortunately, here is the code:
function redefineFetch() {
console.log('inside worker');
if (self.fetch == null) {
console.log('null!');
} else {
console.log(self.fetch.toString());
}
const originalFetch: WindowOrWorkerGlobalScope['fetch'] = self.fetch;
self.fetch = (input: RequestInfo, init: RequestInit) => {
console.log('overridden');
return originalFetch(input, init);
}
}
const blob = new Blob(['(' +
redefineFetch.toString() + ')()'], {type: 'text/javascript'});
const blobUrl = window.URL.createObjectURL(blob);
const w = new Worker(blobUrl);
I'm sure that this code starts before the other workers (I've added a timeout), but this doesn't redefine fetch for the other workers. Can someone explain why or fix the solution?
Update 2:
Apparently each worker has it's own private WorkerGlobalScope, otherwise there would be no sense to use messages for inter-worker communications. Probably, another fix for my problem could be in overriding Worker constructor, if this is possible. Will check it.
Just try to override the fetch method of the current WorkerGlobalScope into main javascript context (window) and into each js file run in a dedicated worker context calling this function:
note that the self property returns the specialized scope for each context
//works in each worker context you call it and enable fetch interception
function EnableFetchWithArguments() {
const originalCtxFetch = self.fetch;
self.fetch = function() {
// Get the parameter in arguments
// Intercept the parameter here
return originalCtxFetch.apply(this, arguments)
}
}
see for reference and browser compatiblity: WorkerGlobalScope
Related
I have a set of integration tests which run in Karma. Unfortunately, they call out to an external, production API endpoint. I do not want integration tests to call out and am exploring my options.
I am wondering if service workers are a viable solution. My assumption is that they do not work because https://github.com/w3c/ServiceWorker/issues/1188 makes it clear that cross-origin fetch is not supported and localhost is not the same origin as a production API endpoint.
For clarity, here is some code I am running:
try {
const { scope, installing, waiting, active } = await navigator.serviceWorker.register('./base/htdocs/test/imageMock.sw.js');
console.log('ServiceWorker registration successful with scope: ', scope, installing, waiting, active);
(installing || waiting || active).addEventListener('statechange', (e) => {
console.log('state', e.target.state);
});
} catch (error) {
console.error('ServiceWorker registration failed: ', error);
}
and the service worker
// imageMock.sw.js
if (typeof self.skipWaiting === 'function') {
console.log('self.skipWaiting() is supported.');
self.addEventListener('install', (e) => {
// See https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html#service-worker-global-scope-skipwaiting
e.waitUntil(self.skipWaiting());
});
} else {
console.log('self.skipWaiting() is not supported.');
}
if (self.clients && (typeof self.clients.claim === 'function')) {
console.log('self.clients.claim() is supported.');
self.addEventListener('activate', (e) => {
// See https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html#clients-claim-method
e.waitUntil(self.clients.claim());
});
} else {
console.log('self.clients.claim() is not supported.');
}
self.addEventListener('fetch', (event) => {
console.log('fetching resource', event);
if (/\.jpg$/.test(event.request.url)) {
const response = new Response('<p>This is a response that comes from your service worker!</p>', {
headers: { 'Content-Type': 'text/html' },
});
event.respondWith(response);
}
});
and when this code is ran I see in the console
ServiceWorker registration successful with scope: http://localhost:9876/base/htdocs/test/ null null ServiceWorker
and then requests to https://<productionServer>.com/image.php are not intercepted by the fetch handler.
Is it correct that there is no way to intercept in this scenario?
You can use a service worker to intercept requests made by a browser as part of a test suite. As long a service worker is in control of a web page, it can intercept cross-origin requests and generate any response you'd like.
(The issue you link to about "foreign fetch" is something different; think of it as the remote production server deploying its own service worker. This was abandoned.)
"Stop mocking fetch" is a comprehensive article covering how to use the msw service worker library within the context of a test suite.
I can't say off the top of my head exactly why your current setup isn't working, but from past experience, I can say that the most important thing to remember when doing this is that you need to delay making any requests from a client test page until the page itself is being controlled by an active service worker. Otherwise, there's a race condition in which you might end up firing off a request that needs to trigger a fetch handler, but won't if the service worker isn't yet in control.
You can wait for this to happen with logic like:
const controlledPromise = new Promise((resolve) => {
// Resolve right away if this page is already controlled.
if (navigator.serviceWorker.controller) {
resolve();
} else {
navigator.serviceWorker.addEventListener('controllerchange', () => {
resolve();
});
}
});
await controlledPromise;
// At this point, the page will be controlled by a service worker.
// You can start making requests at this point.
Note that for this use case, await navigator.serviceWorker.ready will not give you the behavior you need, as there can be a gap in time between when the navigator.serviceWorker.ready promise resolves and when the newly activated service worker takes control of the current page. You don't want that gap of time to lead to flaky tests.
I am using service worker to provide a fallback page that shows the user is offline. The service worker during interception of request, fetches the same request and on error on fetching, provides response for 'offline.html' request from the cache. A small snippet of doing this is.
self.addEventListener("fetch", (event) => {
event.respondWith(
caches.match(event.request).then(() => {
return fetch(event.request).catch((err) => {
return caches.match("offline.html");
});
})
);
});
now if the offline html has other request, probably to its css files, or images, how do I load them from cache. I've tried doing the following:
self.addEventListener("fetch", (event) => {
event.respondWith(
caches.match(event.request).then(() => {
return fetch(event.request).catch((err) => {
let url = event.request.url;
if(url.endsWith('.css')) return caches.match('offline.css');
if(url.endsWith('.jpg') || url.endsWith('.png')) return caches.match('images/banner.jpg');
return caches.match("offline.html");
});
})
);
});
But is there a better way of doing this? Is there a standard way of doing this?
First off, I would recommend checking to see whether event.request.destination === 'document' before you decide whether or not to use offline.html as the fallback content. That ensure that you're not accidentally returning an HTML document to satisfy, say, a random API request that happens to fail.
Additionally, your current code includes caches.match(event.request) but then doesn't actually used the cached response, which is likely not what you intend.
That said, let's walk through what I think is your desired logic:
Your service worker attempts to make a request against the network.
If that request returns a valid response, use it, and you'd done.
If that request fails, then:
If it was a navigation request, regardless of the destination URL, use the cached offline.html for the response.
Otherwise, for non-navigation requests (like CSS or JS requests), use the cached entry matching the desired URL for the response.
Here's a service worker that implements that. You'll need to ensure that the CSS, JS, and offline.html assets are cached during service worker installation; this just includes the fetch handler logic.
self.addEventListener('install', (event) => {
event.waitUntil(
/* Cache your offline.html and the CSS and JS it uses here. */
);
});
async function fetchLogic(request) {
try {
// If the network request succeeds, just use
// that as the response.
return await fetch(request);
} catch(error) {
// Otherwise, implement fallback logic.
if (request.mode === 'navigate') {
// Use the cached fallback.html for failed navigations.
return await caches.match('offline.html');
}
// Otherwise, return a cached copy of the actual
// subresource that was requested.
// If there's a cache miss for that given URL, you'll
// end up with a NetworkError, just like you would if
// there were no service worker involvement.
return await caches.match(request.url);
}
}
self.addEventListener('fetch', (event) => {
event.respondWith(fetchLogic(event.request));
});
There's also some formal guidance in this article.
I have this service worker:
//IMPORT POLYFILL
importScripts('cache-polyfill.js');
//INSTALL
self.addEventListener('install', function(e) {
e.waitUntil(
caches.open('stock_item_balance_v1').then(function(cache) {
return cache.addAll([
'/app/offline_content/purchase/stock_items/stock_items_balance.php',
'/app/offline_content/purchase/stock_items/js/stock_items_balance.js'
]);
})
);
});
//FETCH (FETCH IS WHEN YOU CHECK FOR INTERNET)
self.addEventListener('fetch', function(event) {
//console.log(event.request.url);
event.respondWith(
caches.match(event.request).then(function(response) {
return response || fetch(event.request);
})
);
});
In "stock_items_balance.php" i fetch data from my DB. So in every 30 minutes i would like to update my cached pages and reload the window.
So first i have a script that checks for internet connection.
If true, i want to clean/update the cache and reload the page.
How can i do that?
//INTERVAL
setInterval(function(){
//CLEAN/UPDATE CACHED FILES
serviceworker.update(); // ???
//RELOAD PAGE
window.location.reload();
}, 180000);
(I think you have a larger question as to whether the approach you describe is actually going to give a good, predictable, offline-capable experience for your users, but I'm just going to focus on the actual technical question you asked.)
Messaging the service worker
First off, you should keep in mind that it's possible to have multiple tabs open for the same URL, and if you, you're going to end up with your update code potentially running multiple times. The code in this answer handles the "reload" step for you from inside of the service worker, after the asynchronous cache update has completed, by getting a list of all the active clients of the service worker and telling each to navigate to the current URL (which is effectively a reload).
// Additions to your service worker code:
self.addEventListener('message', (event) => {
// Optional: if you need to potentially send different
// messages, use a different identifier for each.
if (event.data === 'update') {
event.waitUntil((async () => {
// TODO: Move these URLs and cache names into constants.
const cache = await caches.open('stock_item_balance_v1');
await cache.addAll([
'/app/offline_content/purchase/stock_items/stock_items_balance.php',
'/app/offline_content/purchase/stock_items/js/stock_items_balance.js'
]);
const windowClients = await clients.matchAll();
for (const windowClient of windowClients) {
// Optional: check windowClient.url first and
// only call navigate() if it's the URL for one
// specific page.
windowClient.navigate(windowClient.url);
}
})());
}
});
// Additions to your window/page code:
setInterval(() => {
if (navigator.serviceWorker.controller) {
navigator.serviceWorker.controller.postMessage('update');
}
}, 180000);
What won't work
The Cache Storage API is available from both inside a service worker and inside of your page's window scope. Normally what I'd recommend that folks do is to open up the same cache from the window context, and call cache.add() to update the cached entry with the latest from the network. However, calling cache.add() from the window context will cause the network request to be intercepted by your fetch handler, and at that point, your response won't actually come from the network. By calling cache.add() from inside your service worker, you can guarantee that the resulting network request won't trigger your fetch handler.
I am trying to do some performance improvements for our web app by using web workers. I need to include some scripts that I was using importScripts(). But the conundrum is that importScripts fails when trying to access offline. How do I access these files offline using Cache API? Do I need to implement custom reader for reading ReadableStream? Is there a better standard to implement offline cache access inside web workers?
Details
These files are javascript scripts which have some custom js and external libraries like CryptoJS and LocalForage. I would like to implement - Network falling back to Cache paradigm using CacheAPI/Service Workers.
I initially implemented a standard Service Worker with an install and fetch event listeners but I believe the scope between the service worker and the web worker was not the same. After some research on MDN and exploration, I see that Cache API is available within the WebWorkerScope so I moved the cache call within the web worker scope.
I have tried various ways of accessing these files by using fetch events and just getting the files from cache. I get a response back after my promises resolve but the body of the response is a readable stream and I am not sure how to resolve that.
Any help or pointers would be really appreciated.
My web worker invocation
var worker = new Worker('Path');
I have attempted to follow the write up as a guide -
https://developers.google.com/web/ilt/pwa/caching-files-with-service-worker
// Web Worker
self.addEventListener('fetch', function(event){
event.respondWith(
fetch(event.request).catch(function(){
return caches.match(event.request);
})
)
});
caches.open('mcaseworker').then(function(cache){
var urlList = ['/resources/scripts/custom/globalConfig.js',
'/resources/scripts/localforage/localforage.min.js'
'/resources/scripts/utility/pako.js',
'/resources/scripts/cryptojs/aes.js',
'/resources/scripts/cryptojs/sha1.js'
];
// Initialize a promise all with the list of urls
Promise.all(urlList.map(function(url){
return fetch(url, {
headers : {
'Content-Type' : 'application/x-javascript'
}
})
.then(function(response){
if (response){
return response;
}
});
}))
.then(function(values){
Promise.all(values.map(function(value){
return value;
}))
.then(function(res){
// Custom Code
// Would like to access localforage and other javascript libraries.
})
})
})
Response after promises resolve.
Web workers don't have a fetch event, so your code listening on the fetch event will never trigger. You should put your cache and fetch event listener in a service worker.
Main code:
if ('serviceWorker' in navigator) {
// Register a service worker hosted at the root of the
// site using the default scope.
navigator.serviceWorker.register('/sw.js').then(function(registration) {
console.log('Service worker registration succeeded:', registration);
}, /*catch*/ function(error) {
console.log('Service worker registration failed:', error);
});
} else {
console.log('Service workers are not supported.');
}
const worker = new Worker("/worker.js");
sw.js
self.addEventListener('fetch', function(event){
event.respondWith(
fetch(event.request).catch(function(){
return caches.match(event.request);
})
)
});
//Add cache opening code here
worker.js
// Import scripts here
importScripts('/resources/scripts/localforage/localforage.min.js');
You can see this answer for more information about the difference between web workers and service workers.
I'm introducing service worker on my site.And i'm using app-shell approach for responding to requests.Below is my code structure.
serviceWorker.js
self.addEventListener("fetch", function(event) {
if (requestUri.indexOf('-spid-') !== -1) {
reponsePdPage(event,requestUri);
}else{
event.respondWith(fetch(requestUri,{mode: 'no-cors'}).catch(function (error){
console.log("error in fetching => "+error);
return new Response("not found");
})
);
}
});
function reponsePdPage(event,requestUri){
var appShellResponse=appShellPro();
event.respondWith(appShellResponse); //responds with app-shell
event.waitUntil(
apiResponse(requestUri) //responds with dynamic content
);
}
function appShellPro(){
return fetch(app-shell.html);
}
function apiResponse(requestUri){
var message=['price':'12.45cr'];
self.clients.matchAll().then(function(clients){
clients.forEach(function (client) {
if(client.url == requestUri)
client.postMessage(JSON.stringify(message));
});
});
}
App-shell.html
<html>
<head>
<script>
if ('serviceWorker' in navigator) {
navigator.serviceWorker.onmessage = function (evt) {
var message = JSON.parse(evt.data);
document.getElementById('price').innerHTML=message['price'];
}
}
</script>
</head>
<body>
<div id="price"></div>
</body>
</html>
serviceWorker.js is my only service worker file. whenever i'm getting request of -spid- in url i calls reponsePdPage function.In reponsePdPage function i'm first responding with app-shell.html. after that i'm calling apiResponse function which calls postmessage and send the dynamic data.The listener of post message is written in app-shell.html.
The issue i'm facing is, sometimes post message gets called before the listener registration.It means the apiResponse calls post message but their is not register listener to that event. So i cant capture the data.?Is their something wrong in my implementation.
I'm going to focus on just the last bit, about the communication between the service worker and the controlled page. That question is separate from many of the other details you provide, such as using PHP and adopting the App Shell model.
As you've observed, there's a race condition there, due to the fact that the code in the service worker and the parsing and execution of the HTML are performed in separate processes. I'm not surprised that the onmessage handler isn't established in the page yet at the time the service worker calls client.postMessage().
You've got a few options if you want to pass information from the service worker to controlled pages, while avoiding race conditions.
The first, and probably simplest, option is to change the direction of communication, and have the controlled page use postMessage() to send a request to the service worker, which then responds with the same information. If you take that approach, you'll be sure that the controlled page is ready for the service worker's response. There's a full example here, but here's a simplified version of the relevant bit, which uses a Promise-based wrapper to handle the asynchronous response received from the service worker:
Inside the controlled page:
function sendMessage(message) {
// Return a promise that will eventually resolve with the response.
return new Promise(function(resolve) {
var messageChannel = new MessageChannel();
messageChannel.port1.onmessage = function(event) {
resolve(event.data);
};
navigator.serviceWorker.controller.postMessage(message,
[messageChannel.port2]);
});
}
Inside the service worker:
self.addEventListener('message', function(event) {
// Check event.data to see what the message was.
// Put your response in responseMessage, then send it back:
event.ports[0].postMessage(responseMessage);
});
Other approaches include setting a value in IndexedDB inside the service worker, which is then read from the controlled page once it loads.
And finally, you could actually take the HTML you retrieve from the Cache Storage API, convert it into a string, modify that string to include the relevant information inline, and then respond with a new Response that includes the modified HTML. That's probably the most heavyweight and fragile approach, though.