We Keep Coding

How to save cache storage date with JavaScript Cache API?

I'm using JavaScript's Cache Web API to store responses from my server to requests made by my client application. Moreover, I need a way to programmatically remove them based on when the request was originally made. Here is the code I use to store the responses:
/** Searches for the corresponding cache for the given request. If found, returns
* the cached response. Otherwise, performs the fetch request and adds the response
* to cache. Returns the HTTP response.
*/
export async function fetchCachedData(request: Request) {
const cache = await caches.open(CACHE_NAME);
// Check if response is already cached
const cachedResponse = await cache.match(request);
if (cachedResponse) {
console.debug("Using cached response for", request.url);
return cachedResponse.clone();
}
// Fetch new response
console.debug("Fetching", request.url, "...");
const response = await fetchFromAPI(request);
const responseDate = new Date().getTime().toString();
response.headers.set("Date", responseDate);
// Cache the new response
if (
response.ok /*&& response.clone().headers.get("Cache-Control") !== "no-store"*/
) {
await cache.put(request, response.clone());
console.info("Cached response as", response.url);
}
return response.clone();
}
This approach seems to work on browsers like Firefox, however on Chrome I get an error telling me that headers is read-only:
TypeError: Failed to execute 'set' on 'Headers': Headers are immutable
I have also tried setting the Date header on the server side, however it appears that not all headers set in the express-based app are honoured when cloning and retrieving them from the cache. This is why I wish to manually set the request date when the response is retrieved on the client side.
I don't necessarily need the date to be stored in the cached response's headers, that's just the way I have my cache filtering code set up. Ideally, the request date should be stored somewhere in the response object so that it is preserved when using clone() and is present in the cache.

I've solved this by using the pragma HTTP header -- it appears to be unused as of the HTTP/1.1 spec, however when set on the server side it is preserved in the headers of the response object obtained after making a request from the fetch() API.
Server code (Express.js):
export function headerMiddleware(
_req: Request,
res: Response,
next: NextFunction
) {
const now = new Date().getTime();
res.setHeader("pragma", now);
next();
}
This implementation is probably discouraged since pragma is a deprecated header and its original intention was to signal if no-cache is how the response is to be handled, however when I set its value to a numeric string there appear to be no errors and the solution runs smoothly. Ideally, I'd use the Date header, however if I set that on the server side then the header is removed upon inspection on the client.

When to set the headers explicitly when making HTTP request in NodeJS application

So I was having a look at a codebase of a NodeJS application and there were some specific functions making HTTP requests to the backend. To be exact, those functions were making a GET request to the backend and one thing that I found confusing was that in some of the functions, the headers were mentioned explicitly whereas, in some other functions who were making the GET request, there was no mention of headers (i.e. headers were not being set explicitly). Below is an example:
In the code below, the function is making a GET request and there's no mention of headers (i.e. the headers are not being set explicitly):
// Method for fetching a single post from the backend on the basis of the post ID
export const singlePost = (postID) => {
return fetch(http://localhost:8080/post/${postID}, {
method: "GET",
})
.then((response) => {
return response.json();
})
.catch((error) => {
console.log(error);
});
};
In the code below, the function is making a GET request and the headers are being set explicitly:
// Helper Method for making the call to the backend and fetching all their details of all the posts
export const list = (page) => {
return fetch(http://localhost:8080/posts/?page=${page}, {
method: "GET",
headers: {
Accept: "application/json",
"Content-Type": "application/json",
},
})
.then((response) => {
return response.json();
})
.catch((error) => console.log(error));
};
Now coming to the main question, could someone please explain to me when are we supposed to set the headers explicitly not only in just GET request but in other general HTTP requests as well (i.e. POST, PUT, OPTION etc).
It would be really great if some could refer a source or explain this concept here. Thanks!

HTTP request header is the information, in the form of a text record, that a user's browser sends to a Web server containing the details of what the browser wants and will accept back from the server. The request header also contains the type, version and capabilities of the browser that is making the request so that server returns compatible data.
Check this https://code.tutsplus.com/tutorials/http-headers-for-dummies--net-8039

Cannot create httponly cookie containing jwt in ASP.NET Core and React

I'm trying to implement an authentication scheme in my app. The controller, more specifically the method, responsible for checking user's credentials and generating jwt which it has to put into the httponly cookie afterward looks as follows
[HttpPost]
[Route("authenticate")]
public async Task<IActionResult> Authenticate([FromBody] User user)
{
var response = await _repository.User.Authenticate(user.Login, user.Password);
if (!response) return Forbid();
var claims = new List<Claim>
{
new Claim("value1", user.Login)
};
string token = _jwtService.GenerateJwt(claims);
HttpContext.Response.Cookies.Append(
"SESSION_TOKEN",
"Bearer " + token,
new CookieOptions
{
Expires = DateTime.Now.AddDays(7),
HttpOnly = true,
Secure = false
});
return Ok();
}
I tested this method in Postman - everything works gently and correctly in there. The cookie is being created as well. Moreover, recently I created an app using Angular where I was using the same authentication method, but with Angular's HTTP module the cookie was being created all the time. Here is what that method looks like in my React app with the usage of Axios
export const authenticate = async (login, password) => {
return await axiosLocal.post('/api/auth/authenticate',
{login, password}).then(response => {
return response.status === 200;
}, () => {
return false;
});
Everything I'm getting in response trying to log in is response code 200. I'm pretty sure it's something about Axios's settings.
Also if someone's curios the variable "axiosLocal" contains the baseURL to the API.
- Update 1
Ok. If I'm not mistaken in order to set a cookie from the response I have to send all the requests with { withCredentials: true } option. But when I'm trying to do that the request is being blocked by CORS, although I had already set a cors policy which has to allow processing requests from any origin like that
app.UseCors(builder => builder.AllowAnyHeader()
.AllowAnyMethod()
.AllowAnyOrigin()
.AllowCredentials());

I just had the same issue. I fixed it.
Problem:
In browsers, the httpOnly cookie was received and not returned to the server
In Postman working
// Problemable server code for settings httpOnly cookie
Response.Cookies.Append("refreshToken", refreshToken.Token, new CookieOptions
{
HttpOnly = true,
Expires = DateTime.UtcNow.AddDays(7),
});
Solution:
On the server .AllowCredentials() and
.SetOriginAllowed(host => true) or
.WithOrigins("https://localhost:3000")
On the client (react, axios) withCredentials:true in the headers
If still not working open the Network tab in DevTools in Chrome(current v.91.0.4472.124), select the failed request and when you put the mouse over the yellow triangle you can see very detailed information why the cookie is blocked.
// End server code for setting httpOnly cookie after following the DevTools warnings
Response.Cookies.Append("refreshToken", refreshToken.Token, new CookieOptions
{
HttpOnly = true,
Expires = DateTime.UtcNow.AddDays(7),
IsEssential=true,
SameSite=SameSiteMode.None,
Secure=true,
});

Finally solved. Passing .SetIsOriginAllowed(host => true) instead of .AllowAnyOrigin() to CORS settings with { withCredentials: true } as an option in Axios request helped me.

How can I make a Cloudflare worker which overwrites a response status code but preserves the rest of the response?

Specifically I am interested in changing all responses with code 403 to code 404, and changing all responses with code 301 to 302. I do not want any other part of the response to change, except the status text (which I want to be empty). Below is my own attempt at this:
addEventListener("fetch", event => {
event.respondWith(fetchAndModify(event.request));
});
async function fetchAndModify(request) {
// Send the request on to the origin server.
const response = await fetch(request);
const body = await response.body
newStatus = response.status
if (response.status == 403) {
newStatus = 404
} else if (response.status == 301) {
newStatus = 302
}
// Return modified response.
return new Response(body, {
status: newStatus,
statusText: "",
headers: response.headers
});
}
I have confirmed that this code works. I would like to know if there is any possibility at all that this overwrites part of the response other than the status code or text, and if so, how can I avoid that? If this goes against certain best practices of Cloudflare workers or javascript, please describe which ones and why.

You've stumbled on a real problem with the Fetch API spec as it is written today.
As of now, status, statusText, and headers are the only standard properties of Response's init structure. However, there's no guarantee that they will remain the only properties forever, and no guarantee that an implementation doesn't provide additional non-standard or not-yet-standard properties.
In fact, Cloudflare Workers today implements a non-standard property: webSocket, which is used to implement WebSocket proxying. This property is present if the request passed to fetch() was a WebSocket initiation request and the origin server completed a WebSocket handshake. In this case, if you drop the webSocket field from the Response, WebSocket proxying will break -- which may or may not matter to you.
Unfortunately, the standard does not specify any good way to rewrite a single property of a Response without potentially dropping unanticipated properties. This differs from Request objects, which do offer a (somewhat awkward) way to do such rewrites: Request's constructor can take another Request object as the first parameter, in which case the second parameter specifies only the properties to modify. Alternately, to modify only the URL, you can pass the URL as the first parameter and a Request object as the second parameter. This works because a Request object happens to be the same "shape" as the constructor's initializer structure (it's unclear if the spec authors intended this or if it was a happy accident). Exmaples:
// change URL
request = new Request(newUrl, request);
// change method (or any other property)
request = new Request(request, {method: "GET"});
But for Response, you cannot pass an existing Response object as the first parameter to Response's constructor. There are straightforward ways to modify the body and headers:
// change response body
response = new Response(newBody, response);
// change response headers
// Making a copy of a Response object makes headers mutable.
response = new Response(response.body, response);
response.headers.set("Foo", "bar");
But if you want to modify status... well, there's a trick you can do, but it's not pretty:
// Create an initializer by copying the Response's enumerable fields
// into a new object.
let init = {...response};
// Modify it.
init.status = 404;
init.statusText = "Not Found";
// Work around a bug where `webSocket` is `null` but needs to be `undefined`.
// (Sorry, I only just noticed this when testing this answer! We'll fix this
// in the future.)
init.webSocket = init.webSocket || undefined;
// Create a new Response.
response = new Response(response.body, init);
But, ugh, that sure was ugly.
I have proposed improvements to the Fetch API to solve this, but I haven't yet had time to follow through on them. :(

Cannot construct a Request with a Request whose mode is 'navigate' and a non-empty RequestInit

Consider this sample index.html file.
<!DOCTYPE html>
<html><head><title>test page</title>
<script>navigator.serviceWorker.register('sw.js');</script>
</head>
<body>
<p>test page</p>
</body>
</html>
Using this Service Worker, designed to load from the cache, then fallback to the network if necessary.
cacheFirst = (request) => {
var mycache;
return caches.open('mycache')
.then(cache => {
mycache = cache;
cache.match(request);
})
.then(match => match || fetch(request, {credentials: 'include'}))
.then(response => {
mycache.put(request, response.clone());
return response;
})
}
addEventListener('fetch', event => event.respondWith(cacheFirst(event.request)));
This fails badly on Chrome 62. Refreshing the HTML fails to load in the browser at all, with a "This site can't be reached" error; I have to shift refresh to get out of this broken state. In the console, it says:
Uncaught (in promise) TypeError: Failed to execute 'fetch' on 'ServiceWorkerGlobalScope': Cannot construct a Request with a Request whose mode is 'navigate' and a non-empty RequestInit.
"construct a Request"?! I'm not constructing a request. I'm using the event's request, unmodified. What am I doing wrong here?

Based on further research, it turns out that I am constructing a Request when I fetch(request, {credentials: 'include'})!
Whenever you pass an options object to fetch, that object is the RequestInit, and it creates a new Request object when you do that. And, uh, apparently you can't ask fetch() to create a new Request in navigate mode and a non-empty RequestInit for some reason.
In my case, the event's navigation Request already allowed credentials, so the fix is to convert fetch(request, {credentials: 'include'}) into fetch(request).
I was fooled into thinking I needed {credentials: 'include'} due to this Google documentation article.
When you use fetch, by default, requests won't contain credentials such as cookies. If you want credentials, instead call:
fetch(url, {
credentials: 'include'
})
That's only true if you pass fetch a URL, as they do in the code sample. If you have a Request object on hand, as we normally do in a Service Worker, the Request knows whether it wants to use credentials or not, so fetch(request) will use credentials normally.

https://developers.google.com/web/ilt/pwa/caching-files-with-service-worker
var networkDataReceived = false;
// fetch fresh data
var networkUpdate = fetch('/data.json').then(function(response) {
return response.json();
}).then(function(data) {
networkDataReceived = true;
updatePage(data);
});
// fetch cached data
caches.match('mycache').then(function(response) {
if (!response) throw Error("No data");
return response.json();
}).then(function(data) {
// don't overwrite newer network data
if (!networkDataReceived) {
updatePage(data);
}
}).catch(function() {
// we didn't get cached data, the network is our last hope:
return networkUpdate;
}).catch(showErrorMessage).then(console.log('error');
Best example of what you are trying to do, though you have to update your code accordingly. The web example is taken from under Cache then network.
for the service worker:
self.addEventListener('fetch', function(event) {
event.respondWith(
caches.open('mycache').then(function(cache) {
return fetch(event.request).then(function(response) {
cache.put(event.request, response.clone());
return response;
});
})
);
});

Problem
I came across this problem when trying to override fetch for all kinds of different assets. navigate mode was set for the initial Request that gets the index.html (or other html) file; and I wanted the same caching rules applied to it as I wanted to several other static assets.
Here are the two things I wanted to be able to accomplish:
When fetching static assets, I want to sometimes be able to override the url, meaning I want something like: fetch(new Request(newUrl))
At the same time, I want them to be fetched just as the sender intended; meaning I want to set second argument of fetch (i.e. the RequestInit object mentioned in the error message) to the originalRequest itself, like so: fetch(new Request(newUrl), originalRequest)
However the second part is not possible for requests in navigate mode (i.e. the initial html file); at the same time it is not needed, as explained by others, since it will already keep it's cookies, credentials etc.
Solution
Here is my work-around: a versatile fetch that...
can override the URL
can override RequestInit config object
works with both, navigate as well as any other requests
function fetchOverride(originalRequest, newUrl) {
const fetchArgs = [new Request(newUrl)];
if (request.mode !== 'navigate') {
// customize the request only if NOT in navigate mode
// (since in "navigate" that is not allowed)
fetchArgs.push(request);
}
return fetch(...fetchArgs);
}

In my case I was contructing a request from a serialized form in a service worker (to handle failed POSTs). In the original request it had the mode attribute set, which is readonly, so before one reconstructs the request, delete the mode attribute:
delete serializedRequest["mode"];
request = new Request(serializedRequest.url, serializedRequest);