I'm looking for an example of what these two lines of code look like in a functioning A/B testing worker. From https://developers.cloudflare.com/workers/examples/ab-testing
const TEST_RESPONSE = new Response("Test group") // e.g. await fetch("/test/sompath", request)
const CONTROL_RESPONSE = new Response("Control group") // e.g. await fetch("/control/sompath", request)
I used the examples, subsituting the paths I’m using, and got a syntax error saying await can only be used in async. So I changed the function to async function handleRequest(request) and got a 500 error.
What should these two lines look like for the code to work?
Working example
document.getElementById("myBtn").addEventListener("click", myFun);
async function myFun() {
const isTest = Math.random() > 0.5 //here you place your condition
const res = isTest ? await fetch1() : await fetch1() //here you should have different fetches for A and B
console.log(res)
document.getElementById("demo").innerHTML = res.message;
}
async function fetch1() {
//I took the link from some other SO answer as I was running into CORS problems with e.g. google.com
return fetch("https://currency-converter5.p.rapidapi.com/currency/list?format=json", {
"method": "GET",
"headers": {
"x-rapidapi-host": "currency-converter5.p.rapidapi.com",
"x-rapidapi-key": "**redacted**"
}
})
.then(response => response.json())
}
<button id="myBtn">button</button>
<div id="demo">demo</div>
<script src="https://unpkg.com/#babel/standalone#7/babel.min.js"></script>
<script type="text/babel">
</script>
Okay, I've worked on this a bit and this worker below seems to fulfill your requirements:
async function handleRequest(request) {
const NAME = "experiment-0";
try {
// The Responses below are placeholders. You can set up a custom path for each test (e.g. /control/somepath ).
const TEST_RESPONSE = await fetch(
"https://httpbin.org/image/jpeg",
request
);
const CONTROL_RESPONSE = await fetch(
"https://httpbin.org/image/png",
request
);
// Determine which group this requester is in.
const cookie = request.headers.get("cookie");
if (cookie && cookie.includes(`${NAME}=control`)) {
return CONTROL_RESPONSE;
} else if (cookie && cookie.includes(`${NAME}=test`)) {
return TEST_RESPONSE;
} else {
// If there is no cookie, this is a new client. Choose a group and set the cookie.
const group = Math.random() < 0.5 ? "test" : "control"; // 50/50 split
const response = group === "control" ? CONTROL_RESPONSE : TEST_RESPONSE;
return new Response(response.body, {
status: response.status,
headers: {
...response.headers,
"Set-Cookie": `${NAME}=${group}; path=/`,
},
});
}
} catch (e) {
var response = [`internal server error: ${e.message}`];
if (e.stack) {
response.push(e.stack);
}
return new Response(response.join("\n"), {
status: 500,
headers: {
"Content-type": "text/plain",
},
});
}
}
addEventListener("fetch", (event) => {
event.respondWith(handleRequest(event.request));
});
There are some issues with the snippet once you uncomment the fetch() parts:
Yes, the function needs to be async in order to use await keyword, so I added that.
Additionally, I think the 500 error you were seeing was also due to a bug in the snippet: You'll notice I form a new Response instead of modifying the one that was chosen from the ternary expression. This is because responses are immutable in the workers runtime, so adding headers to an already instantiated response will result in an error. Therefore, you can just create an entirely new Response with all the bits from the original one and it seems to work.
Also, in order to gain insight into errors in a worker, always a good idea to add the try/catch and render those errors in the worker response.
To get yours working on this, just replace the httpbin.org urls with whatever you need to A/B test.
Related
I am trying to get the joke from https://icanhazdadjoke.com/. This is the code I used
const getDadJoke = async () => {
const res = await axios.get('https://icanhazdadjoke.com/', {headers: {Accept: 'application/json'}})
console.log(res.data.joke)
}
getDadJoke()
I expected to get the joke but instead I got the full html page, as if I didn't specify the headers at all. What am I doing wrong?
If you look at the API documentation for icanhazdadjoke.com, there is a section titled "Custom user agent." In that section, they explain how they want any requests to have a User Agent header. If you use Axios in a browser context, the User Agent is set for you by your browser. But I'm going to go out on a limb and say that you are running this code via Node, in which case, you may manually need to set the User Agent header, like so:
const getDadJoke = async () => {
const res = await axios.get(
'https://icanhazdadjoke.com/',
{
headers:
{
'Accept': 'application/json',
'User-Agent': 'my URL, email or whatever'
}
}
)
console.log(res.data.joke)
}
getDadJoke()
The docs say what they want you to put for the User Agent, but I think it would honestly work if there were any User Agent field at all.
The HTML page you're getting is a 503 response from Cloudflare.
As per the API documentation
Custom user agent
If you intend on using the icanhazdadjoke.com API we kindly ask that you set a custom User-Agent header for all requests.
My guess is they have a Cloudflare Browser Integrity Check configured that's triggering for the default Node / Axios user-agent.
Setting a custom user-agent appears to get around this...
const getDadJoke = async () => {
try {
const res = await axios.get("https://icanhazdadjoke.com/", {
headers: {
accept: "application/json",
"user-agent": "My Node and Axios app", // use something better than this
},
});
console.log(res.data.joke);
} catch (err) {
console.error(err.response?.data, err.toJSON());
}
};
Given how unreliable Axios releases have been since v1.0.0, I highly recommend you switch to something else. The Fetch API is available natively in Node since v18
const getDadJoke = async () => {
try {
const res = await fetch("https://icanhazdadjoke.com/", {
headers: {
accept: "application/json",
"user-agent": "My Node and Fetch app", // use something better than this
},
});
if (!res.ok) {
const err = new Error(`${res.status} ${res.statusText}`);
err.text = await res.text();
throw err;
}
console.log((await res.json()).joke);
} catch (err) {
console.error(err, err.text);
}
};
Using Axios REST API call which response JSON format.
If you using API from https://icanhazdadjoke.com/api#authentication
, you can use Axios.
Here is example.
Alternative method.
You needs to use web scrapping method for this case. Because HTML response from https://icanhazdadjoke.com/.
This is example how to scrap using puppeteer library in node.js
Demo code
Save as get-joke.js file.
const puppeteer = require("puppeteer");
async function getJoke() {
try {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://icanhazdadjoke.com/');
const joke = await page.evaluate(() => {
const jokes = Array.from(document.querySelectorAll('p[class="subtitle"]'))
return jokes[0].innerText;
});
await browser.close();
return Promise.resolve(joke);
} catch (error) {
return Promise.reject(error);
}
}
getJoke()
.then((joke) => {
console.log(joke);
})
Selector
Main Idea to use DOM tree selector
In the Chrome's DevTool (by pressing F12), shows HTML DOM tree structures.
<p> tag has class name is subtitle
document.querySelectorAll('p[class="subtitle"]')
Install dependency and run it
npm install puppeteer
node get-joke.js
Result
You can get the joke from that web site.
I am looking for a generic way to replay a fetch if I get a 401 response.
My application is a SPA using OIDC. Our frontend developers utilise fetch, and a ServieWorker injects the access_token into the AJAX request before sending the request to the API(s). There are times when a fetch occurs but access_token is expired. When that happens, I want to use the refresh_token to get a new access_token and then replay the fetch, returning the replayed fetch in the Promise. Ideally, this would be something the frontend developers would not even know is happening.
Meaning that a UI developer will code something like what's below (remember, the access_token is injected via ServiceWorker):
fetch("https://backend.api/user/get/1")
.then(resp =>
{
console.log("user information is XYZ. Raw response:", resp);
})
When really what's happening in the background is:
[Initial request] > [Expired token response] > [Request new token] > [Initial request replayed]
I've experimented with overriding the fetch method with what's below, but I can't figure out a generic way to recreate/clone the original fetch:
window.fetch = new Proxy(window.fetch, {
apply(fetch, that, args) {
// Forward function call to the original fetch
const result = fetch.apply(that, args);
// Do whatever you want with the resulting Promise
result.then(resp =>
{
if(resp.status == 400 || resp.status == 401)
{
let rt = getRefreshToken();
return fetch("https://idaas.provider/get/new/token", {
"method": "POST",
"body": new URLSearchParams({
grant_type: "refresh_token",
refresh_token: rt,
client_id: client_id_str
})
});
}
}).then(resp =>
{
let new_token = resp.new_token;
send_new_token_to_service_worker(new_token);
return new_token
}).then(tok =>
{
// How do I replay the original request?
})
return result;
}
});
The goal is to simplify what the UI developers need to handle. I want them focused on UX and have this sort of error handling done in the background.
Note: If necessary, I would be open to not using fetch and instead utilising a wrapper method. Obviously, because of the code already written surrounding fetch, the new method would need to accept the same arguments and produce the same return.
window.fetch = new Proxy(window.fetch, {
apply(fetch, that, args) {
let fetchApi = [
fetch(args.shift())
];
let fetchToken = [
fetch("https://idaas.provider/get/new/token", {
method: "POST",
body: new URLSearchParams({
grant_type: "refresh_token",
refresh_token: getRefreshToken(),
client_id: client_id_str,
})
}).then(token => send_new_token_to_service_worker(token)),
];
return Promise.allSettled(fetchApi).then(results => {
let rejected = results
.map(result => result.status)
.includes("rejected");
if (rejected) {
return Promise.all([...fetchToken, ...fetchApi]).then(results => results.at(1));
} else {
return results.at(0).value;
}
});
},
});
Usage fetch("https://your-api").then(resp => resp);
I am trying to call the API of a Synology NAS to retrieve information on free space and total space available.
My Login works, and I am getting my Token, requests with that token are shown as "successful".
However, when I try to call for the list_share method, with the additional value set to volume_status which according the documentation of the API should give me the desired data, all I get is:
{
"data": {
"offset": 0,
"shares": [],
"total": 0
},
"success": true
}
The request I am making is:
https://NAME:5001/webapi/entry.cgi?api=SYNO.FileStation.List&version=2&method=list_share&additional=volume_status&_sid=TOKEN
Am I making the wrong request? Do I need to change the Method from list_share to list?
Am I passing the additional value wrong?
The Documentation I am using can be found when googling for 'Synology NAS rest API'. I inserted the direct download link for the PDF here.
According to the linked documentation, the additional parameter needs additional brackets and quotes like this:
https://NAME:5001/webapi/entry.cgi?api=SYNO.FileStation.List&version=2&method=list_share&additional=["volume_status"]&_sid=TOKEN
Depending on the library you use you might need to encode the brackets and the quotes, but most libraries should do that for you :)
To come to that conclusion, this is what i did:
In the linked documentation i found the section about list_share and the provided example:
GET /webapi/entry.cgi?api=SYNO.FileStation.List&version=2&method=list_share& additional=%5B%22real_path%22%2C%22owner%2Ctime%22%5D
This is completely unreadable, since the url is already encoded, but opening the browser console and running
decodeURIComponent("/webapi/entry.cgi?api=SYNO.FileStation.List&version=2&method=list_share&additional=%5B%22real_path%22%2C%22owner%2Ctime%22%5D")
leads me to a readable url:
/webapi/entry.cgi?api=SYNO.FileStation.List&version=2&method=list_share&additional=["real_path","owner,time"]
This is an example, how to login, print out the data and logout again:
const http = require("http");
// Helper function to get JSON from an endpoint
function getJson(url) {
return new Promise((resolve, reject) => {
http.get(url, (res) => {
res.setEncoding("utf8");
let rawData = "";
res.on("data", (chunk) => {
rawData += chunk;
});
res.on("end", () => {
try {
const parsedData = JSON.parse(rawData);
resolve(parsedData);
} catch (e) {
reject(e);
}
});
});
});
}
(async () => {
const host = "synology:5000";
const user = "userHere";
const password = "passwordHere";
// login
const {
data: { sid },
} = await getJson(
`http://${host}/webapi/query.cgi?api=SYNO.API.Auth&method=login&version=3&account=${user}&passwd=${password}&session=test&format=sid`
);
const url = `http://${host}/webapi/query.cgi?api=SYNO.FileStation.List&method=list_share&version=2&_sid=${sid}&additional=["volume_status"]`;
const { data } = await getJson(url);
data.shares.forEach((share) => {
console.log(share.name, "-", share.path);
console.log(share.additional.volume_status);
});
//logout
await getJson(
`http://${host}/webapi/query.cgi?api=SYNO.API.Auth&method=logout&version=3&session=test`
);
})();
Good day I have a custom adonisjs command that pulls from an API.
async handle (args, options) {
// Status
// Open = 1979
// Get all jobs with open status.
const pullJobController = new PullJobsFromJobAdderController;
let token = await pullJobController.get_token();
if(token){
const jobs = await this._getOpenJobs('https://jobs/open-jobs', token , 1979);
}
}
async _getOpenJobs(url, accessToken, status) {
url = url + '?statusId=' + status
const headers = {
'Authorization': 'Bearer ' + accessToken
}
const options = {
method: 'GET',
url: url,
headers: headers
}
return (await rp(options).then(function (result) {
return {
status: true,
info: JSON.parse(result)
}
}).catch(function (error) {
return {
status: false
}
}));
} // _getOpenJobs()
PullJobsFromJobAdderController
async get_token()
{
// This works if directly returning the token.
// return "9ade34acxxa4265fxx4b5x6ss7fs61ez";
const settings = await this.settings();
const jobAdderObject = new this.JobAdder(settings.jobadder['client.id'], settings.jobadder['client.secret'])
const jobadderOauthObject = this.model('JobadderOauth');
const accessInfo = await jobadderOauthObject.jobdderLatestAccess();
let isAccessExpired = await this.checkAccessValidity(accessInfo.created_at);
let accessToken = accessInfo.access_token;
let apiEndpoint = accessInfo.api_endpoint;
if(isAccessExpired === true){
let refreshTokenInfo = await jobAdderObject.refrehToken(accessInfo.refresh_token)
if (refreshTokenInfo.status === true) {
let refreshTokenDetails = JSON.parse(refreshTokenInfo.info)
accessToken = refreshTokenDetails.access_token
apiEndpoint = refreshTokenDetails.api
await jobadderOauthObject.create({
code: accessInfo.code,
access_token: refreshTokenDetails.access_token,
refresh_token: refreshTokenDetails.refresh_token,
scope: 'read write offline_access',
api_endpoint: refreshTokenDetails.api
})
}
}
return accessToken;
} // get_token()
The function async get_token works as expected, it supplies me with a fresh token to be used by the adonisjs command. However it freezes after running the command.
But if I return the string token directly. The custom command handle() works as expected and terminates after running.
Scenario 1: (Directly returning the token string from PullJobsFromJobAdderController)
I run my custom command "adonis pull:jobs" and it runs as expected displaying in the terminal the result of the pulled data from the api.
Terminal is ready to accept another command.
Scenario 2: (Comment out the directly returned string token from PullJobsFromJobAdderController)
I run my custom command "adonis pull:jobs" and it runs as expected
displaying in the terminal the result of the pulled data from the
api.
Terminal is not accepting commands until I press ctrl+c and terminate the current job/command.
Perhaps I am missing something regarding async await calls.
Can someone point / help me to the right direction?
TIA
I got it, for anyone else having this kind of problem with adonis commands:
wrap the task inside your handle in a try... catch block then always have Database.close() and process.exit() in finally.
I'm building a node app that uses Gfycat's API to upload a local MP4 to it and give me the URL of the converted gif. Here's what I have so far:
const fs = require("fs");
const axios = require("axios");
async function main() {
const res = await axios({
method: "POST",
url: "https://api.gfycat.com/v1/gfycats",
data: { title: "test" }
});
const name = res.data.gfyname;
console.log(`The key name is: ${name}`)
const stream = fs.createReadStream("./Video.mp4");
const { size } = fs.statSync("./Video.mp4");
stream.on("error", console.warn);
const sendResult = await axios({
method: "PUT",
url: `https://filedrop.gfycat.com/${name}`,
data: stream,
headers: {
"Content-Type": "video/mp4",
"Content-Length": size
}
});
// Check the status of the encoding every n seconds until it says complete. When complete, return the gfyname.
await waitTillPosted(name);
return gfyname = checkResult.data.gfyname;
}
// Basic GET API request to check the encoding status of the gif
async function isPosted(name) {
var checkResult = await axios({
method: "GET",
url: `https://api.gfycat.com/v1/gfycats/fetch/status/${name}`,
});
console.log(`Current status: ${checkResult.data.task}`)
if (checkResult.data.task == "complete") {
return true;
}
throw new Error('The gif is still encoding.');
}
// Generic sleep function
function sleep(ms) {
console.log(`Sleeping for ${ms} ms.`)
return new Promise(resolve => setTimeout(resolve, ms));
}
// If the gif is not done encoding, sleep 30s
async function waitTillPosted(name) {
return isPosted(name).catch(() => sleep(30e3).then(waitTillPosted));
}
// ======================================================
main()
.then(name => {
console.log(`https://gfycat.com/${gfyname}`);
})
.catch(err => {
console.error(err);
});
According to the API, you can check the status of the conversion task with GET https://api.gfycat.com/v1/gfycats/fetch/status/gfyname.
When I run this app, the first GET request it makes reports encoding, as expected. I'm making a GET request every 30s until eventually, I expect it to say complete, where I'll return the Gfycat URL. However I'm get a NotFoundo instead.
I took one of the key names I got in my testing and checked the status with a GET request separately, and I got a proper complete response with the details of the gif. It works and the gif exists at the URL.
Why am I getting a NotFoundo when I do the same check as part of my main function?
It may be a bug with gfycat. I often get it during rush-hours when I think their servers are overloaded.
My solution: Start all over a couple of times after waiting a short time-out. That's what "works" for me.
Also you may want to try a different video. Maybe they can't decode your file.