Fetch api return same data always - javascript

Every time one of my button is clicked, I trigger a call to the adviceslip API to get an advice data. I expect the payload to vary. But the exact same data is returned after each call.
I've tried to call getAdvice function as a callback but it didn't work.
Am I missing something?
'use strict'
const title = document.querySelector('.advice-title')
const desc = document.querySelector('.advice-text')
const btn = document.querySelector('.btn')
const getAdvice = async () =>{
try{
const response = await fetch('https://api.adviceslip.com/advice');
if (!response.ok){
throw new Error('Api connection problem')
}
const responseJson = await response.json();
const data = responseJson.slip;
const id = `ADVICE #${data.id}`;
const advice = `"${data.advice}"`;
title.textContent = id;
desc.textContent = advice;
}
catch (e) {
console.error(e)
}
}
btn.addEventListener('click', () => {
getAdvice()
})

There appears to be an issue with caching where multiple requests closely spaced in time from a web page will cause the same result to come back. If I add a uniquely changing value to the URL to defeat caching like this:
const response = await fetch('https://api.adviceslip.com/advice?t=' + Math.random());
Then, I get a unique response every time.
I experimented with sending various headers to disable caching instead of the t=xxx hack, but those headers trigger a CORs error and the fetch() does not succeed because custom headers like this don't pass the test for a "simple" CORS request and thus require pre-flight which the target site does not support. So, the ?t=xxx where xxx is different on each request seems to do the trick.

I found this and it worked for me. Add
const response = await fetch("https://api.adviceslip.com/advice", { cache: "no-cache" });

Related

await fetch(file_url) returns sometimes doesn't return the full file contents

I have the following javascript code to fetch and process the contents of the .csv file
async function fetchCsv() {
const response = await fetch("levels.csv");
const reader = response.body.getReader();
const result = await reader.read();
const decoder = new TextDecoder("utf-8");
const csv = await decoder.decode(result.value);
return csv;
}
useEffect(() => {
fetchCsv().then((csv) => {
// process csv
(...)
When running this code 99% of the time the csv variable contains the correct contents of the file, but in rare cases the csv variable is only truncated part of the actual file.
What could be the reason and how to improve the code to handle that?
It's in a React App if that's relevant.
Extra info:
I have verified that when the problem occurs the network response for the levels.csv file is a proper response (200 and full 38kb are returned)
What you get when calling response.body.getReader() is a ReadableStreamDefaultReader object.
Calling its .read() method will return a Promise that will resolve with either the full content of the response body, in case the request was honored fast enough and the body size isn't too big (apparently 256MB in Firefox), or with just one chunk of the response body.
This allows you to handle the response as a stream, before it's entirely fetched.
If you wish to process this stream as text, you could either use a TextDecoderStream, which finally got support in all major browsers:
const response = await fetch("levels.csv");
const textStream = response.body.pipeThrough(new TextDecoderStream());
// now you can handle each chunk as text from textStream.getReader();
// or pipe it in yet another TransformStream
or in more old-school style, you could use the { stream: true } option of the TextDecoder#decode() method and handle each chunk one by one in there:
const response = await fetch("levels.csv");
const decoder = new TextDecoder();
const reader = response.body.getReader();
while (true) {
const {value, done} = await reader.read();
if (value) {
csv_chunks.push(decoder.decode(value, {stream: true}));
// do something with all the chunks we have so far
}
if (done) {
break;
}
}
But maybe you don't want to handle this response as a stream at all, in which case it might very well be enough for you to ask the browser to first fetch the whole response body before it itself decodes it as text. For this, if you need to decode the text as UTF-8, you'd use the Response#text() method:
const response = await fetch("levels.csv");
if (!response.ok) { // don't forget to handle possible network errors
throw new Error("NetworkError");
}
return response.text();
And if you need to handle an other encoding, then first consume the response as an ArrayBuffer then decode it to text:
const response = await fetch("levels.csv");
if (!response.ok) { // don't forget to handle possible network errors
throw new Error("NetworkError");
}
const buf = await response.arrayBuffer();
const decoder = new TextDecoder(encoding);
return decoder.decode(buf);

How to find how much bytes were sent on fetch request?

I am fetching data from different API with javascript's fetch API. But how can I find out how many bytes are sent on each request for analytics?
The request could be in any method.
I know that I can get the amount of bytes received with
response.headers["content-length"].
I need to find out a way to get the amount of bytes sent on the frontend (browser or mobile using React Native). Ideally, it would be the total size of the request, but just the size of the request body would be good enough.
You can get the value that will be set in the Content-Length header by reading the Request's body as text and checking the length of the returned string:
(async () => {
const formdata = new FormData();
const file = new Blob(["data".repeat(1024)])
formdata.append("key", file)
const req = new Request("/", { method: "POST", body: formdata });
// note that we .clone() the Request
// so that we can still use the original one with fetch()
console.log((await req.clone().text()).length);
fetch(req);
console.log("check the Network panel of your dev tools to see the sent header");
})();
However this only applies for requests where this header is sent, i.e not for GET and HEAD requests.
A quick solution that I used - a tiny middleware (I use Express):
const socketBytes = new Map();
app.use((req, res, next) => {
req.socketProgress = getSocketProgress(req.socket);
next();
});
/**
* return kb read delta for given socket
*/
function getSocketProgress(socket) {
const currBytesRead = socket.bytesRead;
let prevBytesRead;
if (!socketBytes.has(socket)) {
prevBytesRead = 0;
} else {
prevBytesRead = socketBytes.get(socket).prevBytesRead;
}
socketBytes.set(socket, {prevBytesRead: currBytesRead})
return (currBytesRead-prevBytesRead)/1024;
}
And then you can use req.socketProgress in your middlewares.

AWS Lambda - Only getting answer after the second test trigger

I'm developing an AWS Lambda in TypeScript that uses Axios to get data from an API and that data will be filtered and be put into a dynamoDb.
The code looks as follows:
export {};
const axios = require("axios");
const AWS = require('aws-sdk');
exports.handler = async (event: any) => {
const shuttleDB = new AWS.DynamoDB.DocumentClient();
const startDate = "2021-08-16";
const endDate = "2021-08-16";
const startTime = "16:00:00";
const endTime = "17:00:00";
const response = await axios.post('URL', {
data:{
"von": startDate+"T"+startTime,
"bis": endDate+"T"+endTime
}}, {
headers: {
'x-rs-api-key': KEY
}
}
);
const params = response.data.data;
const putPromise = params.map(async(elem: object) => {
delete elem.feat1;
delete elem.feat2;
delete elem.feat3;
delete elem.feat4;
delete elem.feat5;
const paramsDynamoDB = {
TableName: String(process.env.TABLE_NAME),
Item: elem
}
shuttleDB.put(paramsDynamoDB).promise();
});
await Promise.all(putPromise);
};
This all works kind of fine. If the test button gets pushed the first time, everything seems fine and is working. E.g. I received all the console.logs during developing but the data is not put into the db.
With the second try it is the same output but the data is successfully put into the Db.
Any ideas regarding this issue? How can I solve this problem and have the data put into the Db after the first try?
Thanks in advance!
you need to return the promise from the db call -
return shuttleDB.put(paramsDynamoDB).promise();
also, Promise.all will complete early if any call fails (compared to Promise.allSettled), so it may be worth logging out any errors that may be happening too.
Better still, take a look at transactWrite - https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html#transactWrite-property to ensure all or nothing gets written

Can not get AEC model Data in Autodesk Forge

i try to activate Revit Levels and 2D Minimap extension in autodesk forge viewer, but can not get AEC Model Data. I got this worning`
i tried to get AEC data with this code
const url = window.location.search;
console.log(url);
const svf_path = `${url.replace("?", "/storage/").replace(/%20/g, " ")}`;
Autodesk.Viewing.endpoint.getItemApi = (endpoint, derivativeUrn, api) => {
return svf_path;
};
Autodesk.Viewing.Initializer(options, async () => {
const paths = svf_path.split("/");
const [dest, svf_dir] = [paths[2], paths[3]];
const url = `/api/viewer/dest/${dest}/svf/${svf_dir}/manifest`;
const response = await fetch(url);
const manifest = await response.json();
const init_div = document.getElementById("init_div");
viewer = new Autodesk.Viewing.GuiViewer3D(init_div, config3d);
const viewerDocument = new Autodesk.Viewing.Document(manifest);
const viewable = viewerDocument.getRoot().getDefaultGeometry();
viewer.start();
await viewerDocument.downloadAecModelData();
viewer.loadDocumentNode(viewerDocument, viewable)
.then(function (result) {
Autodesk.Viewing.Document.getAecModelData(viewable);
})
});
wats wrong in my code?
The warning comes from the BubbleNode.prototype.getAecModelData method. You are not calling it in your code but it's possible that it's being called by the LevelsExtension itself. Try configuring the extension so that it doesn't detect the AEC data automatically by passing in { autoDetectAecModelData: false } as the extension options.
Btw. to debug the issue on your side, you can also try getting the non-minified version of viewer3D.js, put a breakpoint to where the warning is being logged, and see the call stack when the breakpoint is hit.

How to return API response to conversation in Dialogflow Webhook

I have a Dialogflow Webhook fulfillment integration, which does a GET request to an API i have set up. But i can't seem to get the API's response as text in my conversation.
The API does receive a http GET request and returns a response with statuscode 200.
If i do the same request in my browser this is the result:
{
"avmid": "1011GZ 18",
"straat": "Snoekjesgracht",
"postcode": "1011GZ",
"stad": "AMSTERDAM",
"provincienaam": "Noord-Holland",
"date": "2013-12-30",
"koopsom": 199800,
"koopsom2018q2": 333849
}
I have tried several things but i don't seem to be able to get it to work.
This is my JavaScript:
'use strict';
const http = require('http');
const functions = require('firebase-functions');
const {WebhookClient} = require('dialogflow-fulfillment');
process.env.DEBUG = 'dialogflow:debug'; // enables lib debugging statements
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const agent = new WebhookClient({ request, response });
console.log('Dialogflow Request headers: ' + JSON.stringify(request.headers));
console.log('Dialogflow Request body: ' + JSON.stringify(request.body));
function address_api_request (agent) {
let postcode = agent.parameters.zipcode;
let housenumber = agent.parameters.housenumber;
let avmid = postcode.toString()+'+'+housenumber.toString();
let url = 'http://XXX.XXX.XXX.XXX:XX/api/1011GZ+18';
var http = require('http');
http.get(url, function(response) {
var body = '';
response.on('data', function(d) {
body += d;
});
response.on('end', function() {
var parsed = JSON.parse(body);
agent.add(parsed.toString());
});
});
}
function welcome (agent) {
agent.add(`Welcome to my agent!`);
agent.add(agent.request_.body.queryResult.fulfillmentText);
}
function fallback (agent) {
agent.add(`I didn't understand`);
agent.add(`I'm sorry, can you try again?`);
}
// Run the proper function handler based on the matched Dialogflow intent name
let intentMap = new Map();
intentMap.set('company.woningwaarde-get-address-api', address_api_request);
intentMap.set('Default Welcome Intent', welcome);
intentMap.set('Default Fallback Intent', fallback);
agent.handleRequest(intentMap);
});
The intents work, and if i replace the address_api_request function with the underneath code it returns "test:
function address_api_request (agent) {
agent.add('test');
}
There are two issues here.
The first is that if you are using Dialogflow's built-in editor, it is using Firebase Cloud Functions under the covers. The default level of Firebase Cloud Functions does not allow network access outside of Google's cloud.
You can resolve this by upgrading the project to Firebase's Blaze Plan, which does require a credit card on file and charges per-use, however there is a free tier which is more than sufficient for reasonable testing, and even some light use under production. Once your action has been approved, you'll be eligible to receive Google Cloud credits which may be used for this purpose.
The other problem is that you have an asynchronous function (the http.get()), but you aren't using a Promise to handle the function and let the handleRequest() method know that it needs to wait for the function to resolve before returning a result. If you are using async functions, the Dialogflow library requires that you return a Promise from the function.
You have a few choices for how to handle this. First, you can wrap your call to http.get() as part of creating a new Promise object and in your end handler, send the message as you've indicated and then call the resolve parameter that you need to accept in the Promise handler. Easier, however, would be to use a library such as request-promise-native which wraps much of this for you and lets you get a result as part of a then() clause, where you would then handle it.
I haven't tested it, but your code might then look something like this:
function address_api_request (agent) {
let postcode = agent.parameters.zipcode;
let housenumber = agent.parameters.housenumber;
let avmid = postcode.toString()+'+'+housenumber.toString();
let url = 'http://XXX.XXX.XXX.XXX:XX/api/1011GZ+18';
const rp = require('request-promise-native');
var options = {
uri: url,
json: true
};
return rp( options )
.then( body => {
// Since json was set true above, it parses it for you
// You wouldn't really want to send back the whole body
agent.add( body );
})
.catch( err => {
console.log('Problem with request', err );
agent.add( "Uh oh, something went wrong" );
});
}

Categories