I have a javascript project that I need help, which is to build a cache layer using cloudflare workers and cloudflare KV.
These were my given tasks:
Send API requests to a new url on cloudflare like: https://maps.shiply.com rather than https://maps.googleapis.com/maps/api/place/js/AutocompletionService.GetPredictionsJson?1ssw&4sen-GB&5sGB&6m6&1m2&1d34.5626609&2d-8.649357199999999&2m2&1d60.8608663&2d33.9165549&9sgeocode&15e3&20sDDC1BA1D-F381-44DA-9F99-B6F465F95056ap5a8xfuid3p&21m1&2e1&callback=_xdc_.24ids0&key=APIKEY&token=TOKEN
The cloudflare worker script first checks the Cloudflare KV to see if the response has been saved in the past.
If it has been saved in the past, return the KV.
If it has not been saved in the past, forward the request onto Google Maps API, save the response in KV and return the response
export default {
async fetch(request, env) {
try {
const { pathname } = new URL(request.url);
const [, key, value] = pathname.split("/");
if (!key) {
return new Response("A key is required in the URL path", {
status: 400,
});
}
if (request.method === "GET") {
// Check if the response has been saved in the past
const savedValue = await env.MAPS.get(key, {
namespace: env.NAMESPACE_ID,
});
if (savedValue) {
// If it has been saved, return the saved value
return new Response(savedValue, { status: 200 });
} else {
// If it has not been saved, forward the request to Google Maps API
const response = await fetch(
request.url.replace(
"maps.mapior.workers.dev",
`maps.googleapis.com/maps/api/place/js/AutocompletionService.GetPredictionsJson?1s${key}&4sen-GB&5sGB&6m6&1m2&1d34.5626609&2d-8.649357199999999&2m2&1d60.8608663&2d33.9165549&9sgeocode&15e3&20sDDC1BA1D-F381-44DA-9F99-B6F465F95056ap5a8xfuid3p&21m1&2e1&callback=_xdc_._24ids0&key=API_KEY&token=MY_TOKEN`
)
);
if (!response.ok) {
return new Response(
`Request to Google Maps API failed with status code ${response.status}`,
{ status: response.status }
);
}
const responseText = await response.text();
// Save the response in the Key-Value store
await env.MAPS.put(key, responseText, {
namespace: env.NAMESPACE_ID,
});
// Return the response from Google Maps API
return new Response(responseText, { status: 200 });
}
} else if (request.method === "PUT") {
// Store the value in the Key-Value store
if (!value) {
return new Response("A value is required in the URL path", {
status: 400,
});
}
await env.MAPS.put(key, value, { namespace: env.NAMESPACE_ID });
return new Response(`Saved in KV: ${key} = ${value}`);
} else {
return new Response("Unsupported method", { status: 405 });
}
} catch (e) {
return new Response(e.stack, { status: 500 });
}
},
};
As you can see from my above script, i was able to complete most of the tasks, but i have no idea about the google map AutocompletionService, the documentation were very confusing for a beginner like me.
When i run the above code in cloudfare workers enviornment, i am getting this error due to the request failure to google maps api:
502 Bad Gateway, content-length:54 content-type:text/plain;charset=UTF-8 Request to Google Maps API failed with status code 502
A Screenshot from my cloudflare workers dashboard where i am currently working on: My Screenshot
Something is wrong with the URL or my implementation.
Please help, i am a beginner and haven't worked on something like this before, can you please help me out.
As an output i need to see a google maps api response and store it as a value in cloudflarekv like so.
From the perspective of how the Google Maps JavaScript API and its Autocomplete service work, there are at least two reasons why what you are attempting seems problematic (not recommended):
The JavaScript API is designed to run in the users' browsers and send requests directly to Google servers. Anything that interferes with that direct connection is bound to cause problems. Unpredictable problems. The sensible recommendation to avoid problems is to use the API only as described in the documentation. If it's not documented, it's not an API.
Caching is disallowed by the Google Maps Platform Terms of Service. There are a few exceptions in the Google Maps Platform Service Specific Terms but none that allow caching entire API responses.
If you are concerned about having sensible caching, note that the Place Autocomplete service already sets cache-control: private, max-age=300 so the users' browsers will cache API responses for a short time, so duplicate requests are not sent in the short term.
Related
I created a key in the google cloud console. I even tried remaking it and using a the new one.
Im trying to use it like so:
export const getSheet= async () => {
try {
const sheetId ='xxxxxxx'
const tabName = 'myTab'
const accountKey = 'xxxxxxx'
const url =' https://sheets.googleapis.com/v4/spreadsheets/'+ sheetId +'/values/'+tabName+'?key='+ accountKey
console.log(url)
const response = await fetch(url);
console.log(response);
return '';
} catch (error) {
console.log(error);
} finally {
console.log('finally');
}
};
The request being sent is:
https://sheets.googleapis.com/v4/spreadsheets/xxxxxxx/values/myTab?key=xxxxxxx
No matter what I do I get
error: {code: 403, message: "The caller does not have permission", status: "PERMISSION_DENIED"}
Ive refered to these stack overflow posts regarding the same issue with no luck
Error 403 on Google Sheets API
Google Sheets API V4 403 Error
Getting 403 from Google Sheets API using apikey
Has anyone come across this and was able to fix it?
Thanks
-Coffee
You can't use an API key to access (Google Workplace) user data such as sheets; you may (!?) be able to get away with only use an API Key if the sheet were public (anyone with the link).
The options are admittedly confusing but:
API keys authenticate apps
OAuth is used to authenticate users
Have a look at authentication & authorization and OAuth for client-side web apps.
You can look through the Javascript Quickstart guide for Sheets API for the OAuth Setup. And you can access the sheet using Spreadsheet.values.get method similar to this sample script on the provided URL reference.
async function listMajors() {
let response;
try {
// Fetch first 10 files
response = await gapi.client.sheets.spreadsheets.values.get({
spreadsheetId: 'SHEETID',
range: 'SHEETRANGE',
});
} catch (err) {
document.getElementById('content').innerText = err.message;
return;
}
const range = response.result;
if (!range || !range.values || range.values.length == 0) {
document.getElementById('content').innerText = 'No values found.';
return;
}
And just like what DazWilkin provided. Make sure that your app complies with Google's OAuth2 policies.
You can also test your request url first on a browser and see if it returns a JSON response of the sheet data. Tested this https://sheets.googleapis.com/v4/spreadsheets/xxxxxxx/values/myTab?key=xxxxxxx format on a public sheet, and it returned the sheet data as long as it is set to 'Anyone with the link'
I am working on a React app using Leaflet 1.6.0 with Leaflet.nontiledlayer to display images from a WMS service. This WMS service requires authentication, and after some period of time, my session will expire, and all calls to the WMS service will fail with a HTTP 401 unauthorized status code.
In my error handling code, I would like to detect this HTTP 401 code, so that I can redirect the user back to the login page. But when I debug my error handling code, I do not see a field that contains the HTTP status code from the response. How can I check the HTTP status code when a WMS request fails?
I have thoroughly inspected both the error and target objects, and I do not see any field that corresponds to the HTTP status code.
My map layer is initialized as follows:
let mapDefault = L.nonTiledLayer.wms(appconfig.baseurl + "/OGC/WMS", {
...wmsOptions,
...{ layers: "DEFAULT", name: "Default" }
});
My error handling function:
const errorhandler = function (error, tile) {
const { target, sourceTarget } = error;
if (
target &&
target.wmsParams &&
target.wmsParams.name &&
(target.wmsParams.name === "Default")
) {
this.props.dispatch(
crewAlertActions.error(`Error in loading base map layer`)
);
}
};
mapDefault.on("error", errorhandler.bind(this));
Consider this sample index.html file.
<!DOCTYPE html>
<html><head><title>test page</title>
<script>navigator.serviceWorker.register('sw.js');</script>
</head>
<body>
<p>test page</p>
</body>
</html>
Using this Service Worker, designed to load from the cache, then fallback to the network if necessary.
cacheFirst = (request) => {
var mycache;
return caches.open('mycache')
.then(cache => {
mycache = cache;
cache.match(request);
})
.then(match => match || fetch(request, {credentials: 'include'}))
.then(response => {
mycache.put(request, response.clone());
return response;
})
}
addEventListener('fetch', event => event.respondWith(cacheFirst(event.request)));
This fails badly on Chrome 62. Refreshing the HTML fails to load in the browser at all, with a "This site can't be reached" error; I have to shift refresh to get out of this broken state. In the console, it says:
Uncaught (in promise) TypeError: Failed to execute 'fetch' on 'ServiceWorkerGlobalScope': Cannot construct a Request with a Request whose mode is 'navigate' and a non-empty RequestInit.
"construct a Request"?! I'm not constructing a request. I'm using the event's request, unmodified. What am I doing wrong here?
Based on further research, it turns out that I am constructing a Request when I fetch(request, {credentials: 'include'})!
Whenever you pass an options object to fetch, that object is the RequestInit, and it creates a new Request object when you do that. And, uh, apparently you can't ask fetch() to create a new Request in navigate mode and a non-empty RequestInit for some reason.
In my case, the event's navigation Request already allowed credentials, so the fix is to convert fetch(request, {credentials: 'include'}) into fetch(request).
I was fooled into thinking I needed {credentials: 'include'} due to this Google documentation article.
When you use fetch, by default, requests won't contain credentials such as cookies. If you want credentials, instead call:
fetch(url, {
credentials: 'include'
})
That's only true if you pass fetch a URL, as they do in the code sample. If you have a Request object on hand, as we normally do in a Service Worker, the Request knows whether it wants to use credentials or not, so fetch(request) will use credentials normally.
https://developers.google.com/web/ilt/pwa/caching-files-with-service-worker
var networkDataReceived = false;
// fetch fresh data
var networkUpdate = fetch('/data.json').then(function(response) {
return response.json();
}).then(function(data) {
networkDataReceived = true;
updatePage(data);
});
// fetch cached data
caches.match('mycache').then(function(response) {
if (!response) throw Error("No data");
return response.json();
}).then(function(data) {
// don't overwrite newer network data
if (!networkDataReceived) {
updatePage(data);
}
}).catch(function() {
// we didn't get cached data, the network is our last hope:
return networkUpdate;
}).catch(showErrorMessage).then(console.log('error');
Best example of what you are trying to do, though you have to update your code accordingly. The web example is taken from under Cache then network.
for the service worker:
self.addEventListener('fetch', function(event) {
event.respondWith(
caches.open('mycache').then(function(cache) {
return fetch(event.request).then(function(response) {
cache.put(event.request, response.clone());
return response;
});
})
);
});
Problem
I came across this problem when trying to override fetch for all kinds of different assets. navigate mode was set for the initial Request that gets the index.html (or other html) file; and I wanted the same caching rules applied to it as I wanted to several other static assets.
Here are the two things I wanted to be able to accomplish:
When fetching static assets, I want to sometimes be able to override the url, meaning I want something like: fetch(new Request(newUrl))
At the same time, I want them to be fetched just as the sender intended; meaning I want to set second argument of fetch (i.e. the RequestInit object mentioned in the error message) to the originalRequest itself, like so: fetch(new Request(newUrl), originalRequest)
However the second part is not possible for requests in navigate mode (i.e. the initial html file); at the same time it is not needed, as explained by others, since it will already keep it's cookies, credentials etc.
Solution
Here is my work-around: a versatile fetch that...
can override the URL
can override RequestInit config object
works with both, navigate as well as any other requests
function fetchOverride(originalRequest, newUrl) {
const fetchArgs = [new Request(newUrl)];
if (request.mode !== 'navigate') {
// customize the request only if NOT in navigate mode
// (since in "navigate" that is not allowed)
fetchArgs.push(request);
}
return fetch(...fetchArgs);
}
In my case I was contructing a request from a serialized form in a service worker (to handle failed POSTs). In the original request it had the mode attribute set, which is readonly, so before one reconstructs the request, delete the mode attribute:
delete serializedRequest["mode"];
request = new Request(serializedRequest.url, serializedRequest);
I am using AngularJS and trying to work with Google's reCAPTCHA,
I am using the "Explicitly render the reCAPTCHA widget" method for displaying the reCAPTCHA on my web page,
HTML code -
<script type="text/javascript">
var onloadCallback = function()
{
grecaptcha.render('loginCapcha', {
'sitekey' : 'someSiteKey',
'callback' : verifyCallback,
'theme':'dark'
});
};
var auth='';
var verifyCallback = function(response)
{
//storing the Google response in a Global js variable auth, to be used in the controller
auth = response;
var scope = angular.element(document.getElementById('loginCapcha')).scope();
scope.auth();
};
</script>
<div id="loginCapcha"></div>
<script src="https://www.google.com/recaptcha/api.js?onload=onloadCallback&render=explicit" async defer></script>
So far, I am able to achieve the needed functionality of whether the user is a Human or a Bot,
As per my code above, I have a Callback function called 'verifyCallback' in my code,
which is storing the response created by Google, in a global variable called 'auth'.
Now, the final part of reCAPCHA is calling the Google API, with "https://www.google.com/recaptcha/api/siteverify" as the URL and using a POST method,And passing it the Secret Key and the Response created by Google, which I've done in the code below.
My Controller -
_myApp.controller('loginController',['$rootScope','$scope','$http',
function($rootScope,$scope,$http){
var verified = '';
$scope.auth = function()
{
//Secret key provided by Google
secret = "someSecretKey";
/*calling the Google API, passing it the Secretkey and Response,
to the specified URL, using POST method*/
var verificationReq = {
method: 'POST',
url: 'https://www.google.com/recaptcha/api/siteverify',
headers: {
'Access-Control-Allow-Origin':'*'
},
params:{
secret: secret,
response: auth
}
}
$http(verificationReq).then(function(response)
{
if(response.data.success==true)
{
console.log("Not a Bot");
verified = true;
}
else
{
console.log("Bot or some problem");
}
}, function() {
// do on response failure
});
}
So, the Problem I am actually facing is that I am unable to hit the Google's URL, Following is the screenshot of the request I am sending and the error.
Request made -
Error Response -
As far as I understand it is related to CORS and Preflight request.So what am I doing wrong? How do I fix this problem?
As stated in google's docs https://developers.google.com/recaptcha/docs/verify
This page explains how to verify a user's response to a reCAPTCHA challenge from your application's backend.
Verification is initiated from the server, not the client.
This is an extra security step for the server to ensure requests coming from clients are legitimate. Otherwise a client could fake a response and the server would be blindly trusting that the client is a verified human.
If you get a cors error when trying to sign in with recaptcha, it could be that your backend server deployment is down.
I tried to cache a POST request in a service worker on fetch event.
I used cache.put(event.request, response), but the returned promise was rejected with TypeError: Invalid request method POST..
When I tried to hit the same POST API, caches.match(event.request) was giving me undefined.
But when I did the same for GET methods, it worked: caches.match(event.request) for a GET request was giving me a response.
Can service workers cache POST requests?
In case they can't, what approach can we use to make apps truly offline?
You can't cache POST requests using the Cache API. See https://w3c.github.io/ServiceWorker/#cache-put (point 4).
There's a related discussion in the spec repository: https://github.com/slightlyoff/ServiceWorker/issues/693
An interesting solution is the one presented in the ServiceWorker Cookbook: https://serviceworke.rs/request-deferrer.html
Basically, the solution serializes requests to IndexedDB.
I've used the following solution in a recent project with a GraphQL API: I cached all responses from API routes in an IndexedDB object store using a serialized representation of the Request as cache key. Then I used the cache as a fallback if the network was unavailable:
// ServiceWorker.js
self.addEventListener('fetch', function(event) {
// We will cache all POST requests to matching URLs
if(event.request.method === "POST" || event.request.url.href.match(/*...*/)){
event.respondWith(
// First try to fetch the request from the server
fetch(event.request.clone())
// If it works, put the response into IndexedDB
.then(function(response) {
// Compute a unique key for the POST request
var key = getPostId(request);
// Create a cache entry
var entry = {
key: key,
response: serializeResponse(response),
timestamp: Date.now()
};
/* ... save entry to IndexedDB ... */
// Return the (fresh) response
return response;
})
.catch(function() {
// If it does not work, return the cached response. If the cache does not
// contain a response for our request, it will give us a 503-response
var key = getPostId(request);
var cachedResponse = /* query IndexedDB using the key */;
return response;
})
);
}
})
function getPostId(request) {
/* ... compute a unique key for the request incl. it's body: e.g. serialize it to a string */
}
Here is the full code for my specific solution using Dexie.js as IndexedDB-wrapper. Feel free to use it!
If you are talking about form data, then you could intercept the fetch event and read the form data in a similar way as below and then save the data in indexedDB.
//service-worker.js
self.addEventListener('fetch', function(event) {
if(event.request.method === "POST"){
var newObj = {};
event.request.formData().then(formData => {
for(var pair of formData.entries()) {
var key = pair[0];
var value = pair[1];
newObj[key] = value;
}
}).then( ...save object in indexedDB... )
}
})
Another approach to provide a full offline experience can be obtained by using Cloud Firestore offline persistence.
POST / PUT requests are executed on the local cached database and then automatically synchronised to the server as soon as the user restores its internet connectivity (note though that there is a limit of 500 offline requests).
Another aspect to be taken into account by following this solution is that if multiple users have offline changes that get concurrently synchronised, there is no warranty that the changes will be executed in the right chronological order on the server as Firestore uses a first come first served logic.
According to https://w3c.github.io/ServiceWorker/#cache-put (point 4).
if(request.method !== "GET") {
return Promise.reject('no-match')
}