I’m using the Got library for HTTP requests within my application. I’m making HTTP requests to an API where I can expect an HTTP code 404 under certain conditions. I’d like to use Got's internal retry functionality for rerunning the request until the 404 error is gone (which will happen; I just don't know if it takes 1 minute or 30 minutes).
From the documentation I know that HTTP code 404 is not a supported statusCode for the built-in retry functionality and therefore I cannot perform any action within the beforeRetry hook of Got; see here.
I’m extending a Got instance to allow some presets for the API I’m calling. For now I have not found a clean way to extend the existing retry statusCodes with 404.
const gitlabOnPrem = got.extend({
prefixUrl: ".." + "..",
mutableDefaults: true,
responseType: 'json',
//retry: { statusCode: got.defaults.options.retry.statusCode.push(404) }, //does not work | not clean
https: { rejectUnauthorized: false },
headers: {
'PRIVATE-TOKEN': "..",
Accept: 'application/json',
},
hooks: {
beforeError: [
(error) => {
const { response } = error;
console.log(response);
/*
Another idea: if I cannot extend retry statusCodes then I´d like to somehow force a retry from here
if (response.statusCode === 404 && response.path.endsWith('/export/download')) {
console.log('FORCE A RETRY AS THE DOWNLOAD MIGHT NOT BE READY YET');
}
*/
if (response && response.body) {
error.name = 'GitLabOnPremError';
error.message = `${response.body.message !== undefined
? response.body.message
: response.body.error
} (${response.statusCode})`;
}
return error;
},
],
},
});
How can I extend the HTTP statusCodes that allow running a retry?
If this is not possible, see my comment in the code. Is it somehow possible to force a retry manually by just using Got?
Sometimes it's best to just write it yourself. Trying to get a library to work in a way that it isn't totally made to do can be more pain than it's worth. It's also usually very brittle in the long run.
Why not just wrap it yourself? Something like this:
async function MyGotFn() {
let retries = 100;
let statusCode = 404;
let response;
while (statusCode === 404 && --retries > 0) {
response = await got('...');
statusCode = response.statusCode;
}
if (response.statusCode === 404) throw new Error('max retries reached');
return response;
}
I am building a react-native app, and I am starting to implement a more robust and sophisticated error-handling system, specifically for handling server errors when making http requests. Here is a basic example of how I am currently making http requests in my app.
I have a 'client.js' file which is essentially just a wrapper around axios. I have a 'get' method that looks like this:
const get = async (endpoint, config = {}) => {
try {
const result = await axios.get(domain + endpoint, config);
return result;
} catch (error) {
throw new Error(error.message);
}
};
Then, I have a file for each api endpoint that I need to access. For example, I have a 'posts.js' file, and in that file I have a 'getPosts' method:
const getPosts = async (userID, page, pageSize) => {
try {
const response = await client.get(
`${endpoint}?userID=${userID}&page=${page}&pageSize=${pageSize}`
);
return response.data;
} catch (error) {
throw new Error(error.message);
}
};
And then finally, in the component that is calling getPosts, I have a function that looks something like this:
const loadPosts = async () => {
try {
const response = await getPosts();
// do something with the response from the server
} catch (error) {
// display an error message to the client
}
}
Obviously this is a very simple example of what a request might look like, but this is the basic structure that I use throughout my app. The problem I am having is that it seems very repetitive and messy to have to wrap almost all of my functions in a try/catch block, and then basically raise an error object until I get to the function that is actually going to handle the error. Is there some sort of 'design method' for error handling that simplifies and centralizes this process? Perhaps something similar to an express-middleware when creating a node server? Or is this a standard way to handle errors in javascript?
Thank you to anyone who can help!
As you are using axios as the http library here, so you can take a look at axios interceptor in order to hook the response and do something with that before passing it to the consumer. This will help you to respond to errors raised from once cental place.
axios.interceptors.response.use((response) => {
return response;
}, function(error) {
// do what you want to do with the error.
return Promise.reject(error)
});
Or with ES5 syntax
axios.interceptors.response.use(function (response) {
// Do something with response data
return response;
}, function (error) {
// Not 200 Ok
// Do something with response error
return Promise.reject(error);
});
I have been trying to make a http PUT request for sending data to the server. Each time, when I try to send the request the data in the server becomes null. I have tested the server side code by sending correct format of data seems the server side is working fine. Going back to the client side code, I have noticed that during fetch operation the data in the response becomes in 'ReadableStream'. Don't know actually what does that mean. Can anybody help me finding out the mistakes if it is possible from the below information.
I am trying to send the data from the react component:
updateData(data)
the format of the data I am trying to send is an Object:
{first:"first", second:"second"}
And then in my api handler:
const updateData = async (data) => {
const resource = await getResource('PUT', {
data,
});
const url = `${process.env.updateAPI}/Update`;
return handleFetch(url, resource);
};
And then in my handleFetch function:
const handleFetch = async (url, resource) => {
const res = await fetch(url, resource);
if (!res.ok) {
throw new Error(`Request failed, status ${res.status}`);
}
const resData = await res.json();
return resData;
};
getResource function is accepting the parameters and returns the required methods, data and the header where 'Content-Type': 'application/json', is provided and handling the data like: body = JSON.stringify(data);
During debugging I have noticed that the res from fetchHandlers becomes like this:
I am actually confused, why in the body the data becomes 'ReadableStream' instead of the actual data I am trying to send or is this the problem I am having null values. Can anybody help me with that?, it will be highly appreciated. Thanks in Advance
If your data return type is not JSON or you don't want JSON then use text()
As an example:
fetch('https://jsonplaceholder.typicode.com/posts/1')
.then(function(response) {
return response.text();
}).then(function(data) {
console.log(data); // this will be a string
});
I've read many topics here, but nothing same was helpful. My problem is that I often get ECONNREFUSED error while using axios.all .. get on nodejs (in 50% of get requests). In the same time, curl works great.
This in my js code:
const axios = require('axios');
async function makeGetRequest () {
let config = {
headers: {"User-Agent": 'curl/7.64.1'}
};
try {
const [BYN, RUR] = await axios.all([
axios.get('https://www.nbrb.by/api/exrates/rates/145', config),
axios.get('https://www.nbrb.by/api/exrates/rates/298', config),
]);
return [BYN.data, RUR.data];
} catch(error) {
return error
}
}
makeGetRequest().then((value) => {
console.log("VAL: ", value)
})
As you can see, I tried to manipulate headers in order to imitate curl's but this doesn't work.
The command:
curl https://www.nbrb.by/api/exrates/rates/145
works fine. But I need SSR response for my gatsby site.
The error:
ECONNREFUSED
is the underlying HTTP protocol connection error which means the the HTTP server didn't accept your connection.
Probably it's not a problem with your code at all, you just need to ensure rates and agents allowed by the web server your trying to reach to call that API.
As selfagency commented, you may be reaching the max rate of requests per seconds, and IMO it's likely to be your case because you are dispatching 2 concurrents requests.
So, try the following:
const axios = require("axios");
async function makeGetRequest() {
let config = {
headers: { "User-Agent": "curl/7.64.1" }
};
try {
const BYN = await axios.get(
"https://www.nbrb.by/api/exrates/rates/145",
config
);
const RUR = await axios.get("https://www.nbrb.by/api/exrates/rates/298", config);
return [BYN.data, RUR.data];
} catch (error) {
return error;
}
}
makeGetRequest().then(value => {
console.log("VAL: ", value);
});
This way you will be doing only 1 request at a time, so maybe you bypass the rate limit. BTW this is very common for free mode APIs that want you to pay to leverage your usage.
I want to fetch my Json file in react js, for this I am using fetch. But it shows an error
Uncaught (in promise) SyntaxError: Unexpected token < in JSON at position 0
What could be the error, i am getting no clue. I even validated my JSON.
handleGetJson(){
console.log("inside handleGetJson");
fetch(`./fr.json`)
.then((response) => response.json())
.then((messages) => {console.log("messages");});
}
My Json (fr.json)
{
"greeting1": "(fr)choose an emoticon",
"addPhoto1": "(fr)add photo",
"close1": "(fr)close"
}
Add two headers Content-Type and Accept to be equal to application/json.
handleGetJson(){
console.log("inside handleGetJson");
fetch(`./fr.json`, {
headers : {
'Content-Type': 'application/json',
'Accept': 'application/json'
}
})
.then((response) => response.json())
.then((messages) => {console.log("messages");});
}
The solution that worked for me is that:-
I moved my data.json file from src to public directory.
Then used fetch API to fetch the file.
fetch('./data.json').then(response => {
console.log(response);
return response.json();
}).then(data => {
// Work with JSON data here
console.log(data);
}).catch(err => {
// Do something for an error here
console.log("Error Reading data " + err);
});
The problem was that after compiling react app the fetch request looks for the file at URL "http://localhost:3000/data.json" which is actually the public directory of my react app. But unfortunately while compiling react app data.json file is not moved from src to public directory. So we have to explicitly move data.json file from src to public directory.
This error can be received but be aware it can be a red herring to the real issue. In my case, there wasn't an issue with the JSON as the error states, but rather a 404 was occurring that it could not pull the JSON data to process in the 1st place thus resulting in this error.
The fix for this was that in order to use fetch on a .json file in a local project, the .json file must be accessible. This can be done by placing it in a folder such as the public folder in the root of the project. Once I moved the json file into that folder, the 404 turned into a 200, and the Unexpected token < in JSON at position 0 error was resolved.
I was getting the same error, for me, it was because API was just returning a string however in fetch call I was expecting json :
response => response.json()
Returning json from API resolved the issue for me, if your API is not supposed to return json then simply don't do response.json()
I also had the same issue when trying to fetch the data from "/src" folder. Moving the file into the "/public" solved the problem for me.
I had the same issue with fetch and decided to switch to axios. Fixed the issue right away, here's the code snippet.
var axios = require('axios');
var config = {
method: 'get',
url: 'http://localhost:4000/'
};
axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});
I had the same issue although I was requesting data from another web server and not locally. The response status code was 200 but I still didnt get the data, even though it was sent in JSON format by default. The (simple) problem was that I had forgot to include 'https://' in the url, so instead it used the local host in the beginning.
on your Promise response
you requested
response.json()
but this works well if your server sends json response in return
especially if you're using Node Js on the server side
So check again and make sure your server sends json as response
as said if its NodeJS the response could be
res.json(YOUR-DATA-RESPONSE)
I confirm some methods proposed here that also worked for me : you have to put your local .json file in your public directory where fetch() is looking for (looking in http://localhost:3000/)
for example : I use this fetch() in my src/App.js file:
componentDidMount(){
fetch('./data/json-data.json')
.then ( resp1 => resp1.json() )
.then ( users1 => this.setState( {cards : users1} ) )
}
so I created public/data/json-data.json
and everything was fine then :)
Sometime you API backend could not respect the contract, and send plain text (ie. Proxy error: Could not proxy request ..., or <html><body>NOT FOUND</body></html>).
In this case, you will need to handle both cases: 1) a valid json response error, or 2) text payload as fallback (when response payload is not a valid json).
I would suggest this to handle both cases:
// parse response as json, or else as txt
static consumeResponseBodyAs(response, jsonConsumer, txtConsumer) {
(async () => {
var responseString = await response.text();
try{
if (responseString && typeof responseString === "string"){
var responseParsed = JSON.parse(responseString);
if (Api.debug) {
console.log("RESPONSE(Json)", responseParsed);
}
return jsonConsumer(responseParsed);
}
} catch(error) {
// text is not a valid json so we will consume as text
}
if (Api.debug) {
console.log("RESPONSE(Txt)", responseString);
}
return txtConsumer(responseString);
})();
}
then it become more easy to tune the rest handler:
class Api {
static debug = true;
static contribute(entryToAdd) {
return new Promise((resolve, reject) => {
fetch('/api/contributions',
{ method: 'POST',
headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' },
body: JSON.stringify(entryToAdd) })
.catch(reject);
.then(response => Api.consumeResponseBodyAs(response,
(json) => {
if (!response.ok) {
// valid json: reject will use error.details or error.message or http status
reject((json && json.details) || (json && json.message) || response.status);
} else {
resolve(json);
}
},
(txt) => reject(txt)// not json: reject with text payload
)
);
});
}
Try converting the response to string and then parse it to JSON. This solves the error for me. Below is the code for the same.
let resp = JSON.stringify(response);
res = JSON.parse(resp);
It may come when the API(you are consuming) is not sending the corresponding JSON.
You may experience the response as 404 page or something like HTML/XML response.
I had the .json file in src folder. Simply moved it in the public folder and it worked
I struggled with the same issue but then found a solution after doing some research.
The issue sometimes arises from a typing error. Your console lets you know the type of error.
Here's is how I found out: In settings.py I wrote a double underscore: CORS__ORIGIN_ALLOW_ALL = True instead of
CORS_ORIGIN_ALLOW_ALL = True.
The issues persisted and I changed this 'the API Fetch method' and it worked just fine:
refreshList() {
fetch(process.env.REACT_APP_API+ "department")
.then((response) => response.json())
.then((data) => {
this.setState({ deps: data });
});
}
to:
refreshList() {
fetch("http://127.0.0.1:8000/" + "department")
.then((response) => response.json())
.then((data) => {
this.setState({ deps: data });
});
}
For me, I was making a call with fetch from a new computer and I forgot to add the env file to populate the process.env.REACT_APP_URL and process.env.REACT_APP_API_KEY fields. Adding back the .env file resolved the issue.
Add "method: 'GET'"
let query = {
method: 'GET',
headers: {
authToken: token,
"Content-Type": "application/json",
},
};
With datas in public/datas/datas.json :
export default class App extends React.Component {
constructor(props) {
super(props);
}
async componentDidMount() {
const response = await fetch("datas/datas.json");
const datas = await response.json();
console.log(datas);
}
...
At first, add the line of code that is given below at the top :
const port = process.env.PORT || 3000;
make sure your fr.json inside public folder.
then write the code as given below:
fetch(`http://localhost:${port}/fr.json`)
.then((response) => response.json())
.then((messages) => {console.log("messages");});
I was getting the error.
I simply added "proxy" in my package.json and the error went away.
The error was simply there because the API request was getting made at the same port as the react app was running. You need to provide the proxy so that the API call is made to the port where your backend server is running.
Mostly this is caused with an issue in your React/Client app. Adding this line to your client package.json solves it
"proxy": "http://localhost:5000/"
Note: Replace 5000, with the port number where your server is running
Reference: How to get create-react-app to work with a Node.js back-end API