I am trying to do some performance improvements for our web app by using web workers. I need to include some scripts that I was using importScripts(). But the conundrum is that importScripts fails when trying to access offline. How do I access these files offline using Cache API? Do I need to implement custom reader for reading ReadableStream? Is there a better standard to implement offline cache access inside web workers?
Details
These files are javascript scripts which have some custom js and external libraries like CryptoJS and LocalForage. I would like to implement - Network falling back to Cache paradigm using CacheAPI/Service Workers.
I initially implemented a standard Service Worker with an install and fetch event listeners but I believe the scope between the service worker and the web worker was not the same. After some research on MDN and exploration, I see that Cache API is available within the WebWorkerScope so I moved the cache call within the web worker scope.
I have tried various ways of accessing these files by using fetch events and just getting the files from cache. I get a response back after my promises resolve but the body of the response is a readable stream and I am not sure how to resolve that.
Any help or pointers would be really appreciated.
My web worker invocation
var worker = new Worker('Path');
I have attempted to follow the write up as a guide -
https://developers.google.com/web/ilt/pwa/caching-files-with-service-worker
// Web Worker
self.addEventListener('fetch', function(event){
event.respondWith(
fetch(event.request).catch(function(){
return caches.match(event.request);
})
)
});
caches.open('mcaseworker').then(function(cache){
var urlList = ['/resources/scripts/custom/globalConfig.js',
'/resources/scripts/localforage/localforage.min.js'
'/resources/scripts/utility/pako.js',
'/resources/scripts/cryptojs/aes.js',
'/resources/scripts/cryptojs/sha1.js'
];
// Initialize a promise all with the list of urls
Promise.all(urlList.map(function(url){
return fetch(url, {
headers : {
'Content-Type' : 'application/x-javascript'
}
})
.then(function(response){
if (response){
return response;
}
});
}))
.then(function(values){
Promise.all(values.map(function(value){
return value;
}))
.then(function(res){
// Custom Code
// Would like to access localforage and other javascript libraries.
})
})
})
Response after promises resolve.
Web workers don't have a fetch event, so your code listening on the fetch event will never trigger. You should put your cache and fetch event listener in a service worker.
Main code:
if ('serviceWorker' in navigator) {
// Register a service worker hosted at the root of the
// site using the default scope.
navigator.serviceWorker.register('/sw.js').then(function(registration) {
console.log('Service worker registration succeeded:', registration);
}, /*catch*/ function(error) {
console.log('Service worker registration failed:', error);
});
} else {
console.log('Service workers are not supported.');
}
const worker = new Worker("/worker.js");
sw.js
self.addEventListener('fetch', function(event){
event.respondWith(
fetch(event.request).catch(function(){
return caches.match(event.request);
})
)
});
//Add cache opening code here
worker.js
// Import scripts here
importScripts('/resources/scripts/localforage/localforage.min.js');
You can see this answer for more information about the difference between web workers and service workers.
Related
I have a set of integration tests which run in Karma. Unfortunately, they call out to an external, production API endpoint. I do not want integration tests to call out and am exploring my options.
I am wondering if service workers are a viable solution. My assumption is that they do not work because https://github.com/w3c/ServiceWorker/issues/1188 makes it clear that cross-origin fetch is not supported and localhost is not the same origin as a production API endpoint.
For clarity, here is some code I am running:
try {
const { scope, installing, waiting, active } = await navigator.serviceWorker.register('./base/htdocs/test/imageMock.sw.js');
console.log('ServiceWorker registration successful with scope: ', scope, installing, waiting, active);
(installing || waiting || active).addEventListener('statechange', (e) => {
console.log('state', e.target.state);
});
} catch (error) {
console.error('ServiceWorker registration failed: ', error);
}
and the service worker
// imageMock.sw.js
if (typeof self.skipWaiting === 'function') {
console.log('self.skipWaiting() is supported.');
self.addEventListener('install', (e) => {
// See https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html#service-worker-global-scope-skipwaiting
e.waitUntil(self.skipWaiting());
});
} else {
console.log('self.skipWaiting() is not supported.');
}
if (self.clients && (typeof self.clients.claim === 'function')) {
console.log('self.clients.claim() is supported.');
self.addEventListener('activate', (e) => {
// See https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html#clients-claim-method
e.waitUntil(self.clients.claim());
});
} else {
console.log('self.clients.claim() is not supported.');
}
self.addEventListener('fetch', (event) => {
console.log('fetching resource', event);
if (/\.jpg$/.test(event.request.url)) {
const response = new Response('<p>This is a response that comes from your service worker!</p>', {
headers: { 'Content-Type': 'text/html' },
});
event.respondWith(response);
}
});
and when this code is ran I see in the console
ServiceWorker registration successful with scope: http://localhost:9876/base/htdocs/test/ null null ServiceWorker
and then requests to https://<productionServer>.com/image.php are not intercepted by the fetch handler.
Is it correct that there is no way to intercept in this scenario?
You can use a service worker to intercept requests made by a browser as part of a test suite. As long a service worker is in control of a web page, it can intercept cross-origin requests and generate any response you'd like.
(The issue you link to about "foreign fetch" is something different; think of it as the remote production server deploying its own service worker. This was abandoned.)
"Stop mocking fetch" is a comprehensive article covering how to use the msw service worker library within the context of a test suite.
I can't say off the top of my head exactly why your current setup isn't working, but from past experience, I can say that the most important thing to remember when doing this is that you need to delay making any requests from a client test page until the page itself is being controlled by an active service worker. Otherwise, there's a race condition in which you might end up firing off a request that needs to trigger a fetch handler, but won't if the service worker isn't yet in control.
You can wait for this to happen with logic like:
const controlledPromise = new Promise((resolve) => {
// Resolve right away if this page is already controlled.
if (navigator.serviceWorker.controller) {
resolve();
} else {
navigator.serviceWorker.addEventListener('controllerchange', () => {
resolve();
});
}
});
await controlledPromise;
// At this point, the page will be controlled by a service worker.
// You can start making requests at this point.
Note that for this use case, await navigator.serviceWorker.ready will not give you the behavior you need, as there can be a gap in time between when the navigator.serviceWorker.ready promise resolves and when the newly activated service worker takes control of the current page. You don't want that gap of time to lead to flaky tests.
I'm trying to load a service worker before all subresource requests on the page so I can apply some optimizations to the way subresources are loaded (e.g. lazy-loading, loading minified versions of assets instead of full assets). However, I cannot find a way load my SW before other subresource requests begin.
I created a simple proof-of-concept of what I'm trying to do to 401 any requests handled by my Service Worker (just to make it easier to find when my SW begins handling requests).
Here's my HTML:
<!doctype html>
<head>
<script>
if ('serviceWorker' in navigator) {
navigator.serviceWorker.register('/dl-wps-sw.js', { scope: '/' }).then(function (registration) {
console.log('Service Worker registration successful with scope: ', registration.scope);
}, function (err) {
console.error(err);
});
}
</script>
...
and here's my Service Worker:
self.addEventListener('install', function (event) {
self.skipWaiting();
});
self.addEventListener('activate', () => {
self.clients.claim();
});
self.addEventListener('fetch', (event) => {
const init = {status: 401, statusText: 'Blocked!'};
event.respondWith(new Response(null, init));
});
This is what happens in my browser:
As you can see in the screenshot, even though my code to register the Service Worker is at the very top of the page, it doesn't activate and begin handling requests until a bit later, and by then a large number of critical requests I need to catch have already fired.
I found someone else who seemed to be trying to do what I'm trying to accomplish (2 years earlier) in a Github issue for the Service Worker spec: https://github.com/w3c/ServiceWorker/issues/1282
It seems they suggested using a <link rel="serviceworker"... tag to do this, but it appears this link type has since been removed from Chrome for some reason: https://www.chromestatus.com/feature/5682681044008960
I've tried several other ideas to attempt to load my SW first:
Preloading my SW with a <link rel="preload"... tag
Preloading my SW with a fetch/XMLHttpRequest
Inlining the Service Worker in the HTML (not possible apparently)
Delaying the execution of the page by running a while loop for a few seconds (this kinda worked, but its a terrible unpredictable hack)
Any ideas or strategies I'm missing? My Google-fu has failed me on coming up with a solution.
You'll need to install the SW alone first, then refresh the page. The SW can then serve content for a SW-enabled page and intercept all other requests. Example: fetch-progress.anthum.com
index.html
<p>Installing Service Worker, please wait...</p>
<script>
navigator.serviceWorker.register('sw.js')
.then(reg => {
if (reg.installing) {
const sw = reg.installing || reg.waiting;
sw.onstatechange = function() {
if (sw.state === 'installed') {
// SW installed. Refresh page so SW can respond with SW-enabled page.
window.location.reload();
}
};
} else if (reg.active) {
// something's not right or SW is bypassed. previously-installed SW should have redirected this request to different page
handleError(new Error('Service Worker is installed and not redirecting.'))
}
})
.catch(handleError)
function handleError(error) {}
</script>
sw.js
self.addEventListener('fetch', event => {
const url = event.request.url;
const scope = self.registration.scope;
// serve index.html with service-worker-enabled page
if (url === scope || url === scope+'index.html') {
const newUrl = scope+'index-sw-enabled.html';
event.respondWith(fetch(newUrl))
} else {
// process other files here
}
});
index-sw-enabled.html
<!--
This page shows after SW installs.
Put your main app content on this page.
-->
I'd like to intercept fetch from all parts and libraries in my application, and at the same time I'd like not to break possibility of working with the application via file URL - it is useful for Electron and mobile devices (via WebView). For now, I've found two possible ways of doing this:
something like here
const realFetch = window.fetch;
window.fetch = function() {
// do something
return realFetch.apply(this, arguments)
}
something like here, with service worker registration:
main.js:
if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('sw.js').then(function(registration) {
console.log('Service worker registered with scope: ', registration.scope);
}, function(err) {
console.log('ServiceWorker registration failed: ', err);
});
});
}
sw.js:
self.addEventListener('fetch', function(event) {
event.respondWith(
// intercept requests by handling event.request here
);
});
With the first approach I cannot intercept fetch requests from web workers. The second approach doesn't work with file URLs, and I want my application to work via file URL due to it allows me to use the app via Electron for desktops or WebView for Android. Is there any other way for intercepting fetch requests?
P.S. I cannot modify the worker I'm trying to intercept requests from.
Update:
On basis of #Ciro Corvino's answer, I've tried the third approach: to start my own worker before anythnig else and try to redefine fetch from there. Didn't work for me, unfortunately, here is the code:
function redefineFetch() {
console.log('inside worker');
if (self.fetch == null) {
console.log('null!');
} else {
console.log(self.fetch.toString());
}
const originalFetch: WindowOrWorkerGlobalScope['fetch'] = self.fetch;
self.fetch = (input: RequestInfo, init: RequestInit) => {
console.log('overridden');
return originalFetch(input, init);
}
}
const blob = new Blob(['(' +
redefineFetch.toString() + ')()'], {type: 'text/javascript'});
const blobUrl = window.URL.createObjectURL(blob);
const w = new Worker(blobUrl);
I'm sure that this code starts before the other workers (I've added a timeout), but this doesn't redefine fetch for the other workers. Can someone explain why or fix the solution?
Update 2:
Apparently each worker has it's own private WorkerGlobalScope, otherwise there would be no sense to use messages for inter-worker communications. Probably, another fix for my problem could be in overriding Worker constructor, if this is possible. Will check it.
Just try to override the fetch method of the current WorkerGlobalScope into main javascript context (window) and into each js file run in a dedicated worker context calling this function:
note that the self property returns the specialized scope for each context
//works in each worker context you call it and enable fetch interception
function EnableFetchWithArguments() {
const originalCtxFetch = self.fetch;
self.fetch = function() {
// Get the parameter in arguments
// Intercept the parameter here
return originalCtxFetch.apply(this, arguments)
}
}
see for reference and browser compatiblity: WorkerGlobalScope
I'm using a service worker to introduce offline functionality for my single page web app. It's pretty straightforward - use the network when available, or try and fetch from the cache if not:
service-worker.js:
self.addEventListener("fetch", event => {
if(event.request.method !== "GET") {
return;
}
event.respondWith(
fetch(event.request)
.then(networkResponse => {
var responseClone = networkResponse.clone();
if (networkResponse.status == 200) {
caches.open("mycache").then(cache => cache.put(event.request, responseClone));
}
return networkResponse;
})
.catch(_ => {
return caches.match(event.request);
})
)
})
So it intercepts all GET requests and caches them for future use, including the initial page load.
Switching to "offline" in DevTools and refreshing at the root of the application works as expected.
However, my app uses HTML5 pushstate and a client side router. The user could navigate to a new route, then go offline, then hit refresh, and will get a "no internet" message, because the service worker was never told about this new URL.
I can't think of a way around it. As with most SPAs, my server is configured to serve the index.html for a number of catch-all URLs. I need some sort of similar behaviour for the service worker.
Inside your fetch handler, you need to check whether event.request.mode is set to 'navigate'. If so, it's a navigation, and instead of responding with a cached response that matches the specific URL, you can respond with a cached response for your index.html. (Or app-shell.html, or whatever URL you use for the generic HTML for your SPA.)
Your updated fetch handler would look roughly like:
self.addEventListener('fetch', event => {
if (event.request.method !== 'GET') {
return;
}
if (event.request.mode === 'navigate') {
event.respondWith(caches.match('index.html'));
return;
}
// The rest of your fetch handler logic goes here.
});
This is a common use case for service workers, and if you'd prefer to use a pre-packaged solution, the NavigationRoute class in the workbox-routing module can automate it for you.
I'm introducing service worker on my site.And i'm using app-shell approach for responding to requests.Below is my code structure.
serviceWorker.js
self.addEventListener("fetch", function(event) {
if (requestUri.indexOf('-spid-') !== -1) {
reponsePdPage(event,requestUri);
}else{
event.respondWith(fetch(requestUri,{mode: 'no-cors'}).catch(function (error){
console.log("error in fetching => "+error);
return new Response("not found");
})
);
}
});
function reponsePdPage(event,requestUri){
var appShellResponse=appShellPro();
event.respondWith(appShellResponse); //responds with app-shell
event.waitUntil(
apiResponse(requestUri) //responds with dynamic content
);
}
function appShellPro(){
return fetch(app-shell.html);
}
function apiResponse(requestUri){
var message=['price':'12.45cr'];
self.clients.matchAll().then(function(clients){
clients.forEach(function (client) {
if(client.url == requestUri)
client.postMessage(JSON.stringify(message));
});
});
}
App-shell.html
<html>
<head>
<script>
if ('serviceWorker' in navigator) {
navigator.serviceWorker.onmessage = function (evt) {
var message = JSON.parse(evt.data);
document.getElementById('price').innerHTML=message['price'];
}
}
</script>
</head>
<body>
<div id="price"></div>
</body>
</html>
serviceWorker.js is my only service worker file. whenever i'm getting request of -spid- in url i calls reponsePdPage function.In reponsePdPage function i'm first responding with app-shell.html. after that i'm calling apiResponse function which calls postmessage and send the dynamic data.The listener of post message is written in app-shell.html.
The issue i'm facing is, sometimes post message gets called before the listener registration.It means the apiResponse calls post message but their is not register listener to that event. So i cant capture the data.?Is their something wrong in my implementation.
I'm going to focus on just the last bit, about the communication between the service worker and the controlled page. That question is separate from many of the other details you provide, such as using PHP and adopting the App Shell model.
As you've observed, there's a race condition there, due to the fact that the code in the service worker and the parsing and execution of the HTML are performed in separate processes. I'm not surprised that the onmessage handler isn't established in the page yet at the time the service worker calls client.postMessage().
You've got a few options if you want to pass information from the service worker to controlled pages, while avoiding race conditions.
The first, and probably simplest, option is to change the direction of communication, and have the controlled page use postMessage() to send a request to the service worker, which then responds with the same information. If you take that approach, you'll be sure that the controlled page is ready for the service worker's response. There's a full example here, but here's a simplified version of the relevant bit, which uses a Promise-based wrapper to handle the asynchronous response received from the service worker:
Inside the controlled page:
function sendMessage(message) {
// Return a promise that will eventually resolve with the response.
return new Promise(function(resolve) {
var messageChannel = new MessageChannel();
messageChannel.port1.onmessage = function(event) {
resolve(event.data);
};
navigator.serviceWorker.controller.postMessage(message,
[messageChannel.port2]);
});
}
Inside the service worker:
self.addEventListener('message', function(event) {
// Check event.data to see what the message was.
// Put your response in responseMessage, then send it back:
event.ports[0].postMessage(responseMessage);
});
Other approaches include setting a value in IndexedDB inside the service worker, which is then read from the controlled page once it loads.
And finally, you could actually take the HTML you retrieve from the Cache Storage API, convert it into a string, modify that string to include the relevant information inline, and then respond with a new Response that includes the modified HTML. That's probably the most heavyweight and fragile approach, though.