Why does these two async fetches blocks each other? - javascript

Currently I'm trying to make two fetches at the beginning of the website loading process. I'm getting my data via sftp and if I'm fetching both endpoints alone its working. But if I'm trying to get both at the same time, my image is "forever" fetching in the network tab and does not show up. So it seems that they block each other, but I don't understand this, because they are working asynchronously?
This is my main Page, where the image should load:
useEffect(() => {
fetch(`${URLS.apiUrl}/image/label`)
.then(async (x) => {
let resp = await x.json()
if (resp !== undefined && resp.id === undefined && resp.data === undefined) {
setNoImageAvailable(true)
return
}
setImage(resp.data)
})
}, [reRenderValue])
Now I made another component for a better overview, which is linked into my main page and uses a own useEffect (I already tried it with using both fetches in one useEffect, but that also does not work)
useEffect(() => {
fetch(`${URLS.apiUrl}/files/gaugeNames`)
.then(async (v) => {
try {
let resp = (await v.json())
setFilenames(resp)
} catch {
console.log('Could not parse response')
}
})
}, [reload])
As I already said, I'm using sftp to get my data from a ftp, that's the way i do it:
async getGaugeFiles(req, res, next){
// res.json([]);
//Connect FTP Client
await SFTPClient.connect()
//Get all Gauge files from host
let fileList = (await SFTPClient.listFiles(environment.FTP_PATH_GAUGE))?.map(f => `gauge_${f.name}`)
//Disconnect FTP Clients
await SFTPClient.disconnect()
return res.json(fileList)
}
I already checked the return and it return me the correct fileList, so its working well.
Now comes my problem, I'm getting also an error in my backend if my trying to fetch both routes. It says "Error downloading file {FILENAME}: Error: _get: No response from server" and it also tells me "ERR_GENERIC_CLIENT"
This comes through my last function, where I'm loading my files
async download(file, path) {
try {
return await this.client.get(`${path}/${file}`)
} catch (err) {
console.log(`Error downloading file ${file}:`, err)
return null
}
}

Fixed it with a timer. With a timer of 1 Second im letting the first fetch through and after that i can fetch the other things. Not the best solution but currently the only one which is working.

Related

How to avoid nesting promises when handling errors from multiple APIs

I’m working on an application where I need to make requests to two apis. I’m using cognito to handle authentication, and then a lambda that communicates with a database. But, I don’t think that my problem is specific to either of those implementations. It could arise with any two apis.
I’m trying to write the process of signing up a new user. I need to create a new user in cognito so that the new user is able to login, and then I need to create a corresponding user in the database that will store the non-authentication related data for the user. If one of the api requests encounters an error, then I need to delete the item that I created in the other api.
My current implementation is basically this:
const signUpNewUser = (userInfo) => {
API.post("user", "/user", userInfo)
.then((response) => {
return COGNITO.post("user", "/user", response.newUserID);
})
.then((res) => {
//BOTH REQUESTS OCCURED WITH NO ERRORS
})
.catch((error) => {
if (error.origin === "COGNITO_ERROR") {
//IF DB CHANGES CONFIRMED BUT COGNITO FAILED, DELETE CREATED GUEST IN DB
return API.delete("guest", "/guest", userInfo);
} else if (error.origin === "DATABASE_ERROR") {
//IF DB CHANGES FAILED THEN COGNITO HAS NOT RUN YET, SO DON'T NEED TO DELETE IN THIS CASE
}
});
};
This follows the pattern I see demonstrated on the internet. However, I’m having trouble distinguishing cognito errors from database errors. In the above code I sort them by error.origin but they don’t actually have a property that reliably indicates their origin. This problem must be common when working with multiple api’s that you don’t control, but I’m having trouble finding a good solution.
It feels I need to nest promises in this situation. I could nest a catch after API.Post and COGNITO.post, and use that catch to throw a new error that has an origin property. Then that would bubble up and get caught by the final catch that handles all errors. Like this:
const signUpNewUser2 = (userInfo) => {
API.post("user", "/user", userInfo)
.catch((err) => {
let parsedError = err;
parsedError.origin = "DATABASE_ERROR";
throw parsedError;
})
.then((response) => {
let newGuestID = response.id;
return COGNITO.post("user", "/user", newGuestID)
.then((res) => {
return res;
})
.catch((err) => {
let parsedError = err;
parsedError.origin = "COGNITO_ERROR";
throw parsedError;
});
})
.then((res) => {
//BOTH REQUESTS OCCURED WITH NO ERRORS
})
.catch((error) => {
if (error.origin === "COGNITO_ERROR") {
//IF DB CHANGES CONFIRMED BUT COGNITO FAILED, DELETE CREATED GUEST IN DB
return API.delete("guest", "/guest", guestInfo);
} else if (error.origin === "DATABASE_ERROR") {
//IF DB CHANGES FAILED THEN COGNITO HAS NOT RUN YET, SO DON'T NEED TO DELETE IN THIS CASE
}
});
};
But everything I've read says you should avoid nesting promises.
Alternatively, I could put API.post and COGNITO.post in separate functions with internal .then .catch statements, and then have those functions return a promise or throw an error with an added property to indicate origin. But I've seen people say that just hides the problem and makes code harder to follow.
The standard patter I see is that you have one catch, towards the end of a .then chain, that knows how to handle multiple kinds of errors. But if you don’t control the APIs you are working with, how can you confidently sort those errors? Is there something basic about the nature of errors in js that I'm missing?
Because you want to make the API calls in serial, this should be pretty easy to manage. All you need to do is do COGNITO.post in a .then after the first API call - no need to insert another .catch in between.
const signUpNewUser2 = (userInfo) => {
API.post("user", "/user", userInfo)
.then((response) => {
let newGuestID = response.id;
return COGNITO.post("user", "/user", newGuestID)
.then(handleBothSuccess)
.catch((err) => {
// COGNITO failed
return API.delete("guest", "/guest", guestInfo);
});
})
.then((res) => {
//BOTH REQUESTS OCCURED WITH NO ERRORS
})
.catch((error) => {
// Some error other than COGNITO failing occurred
});
};
There's nothing wrong with nesting Promises when the control flow you need to implement requires it - or with declaring the .then or .catch functions in separate standalone variables first, which avoids visual nesting.
Alternatively, consider async/await, which may be clearer to follow.
const signUpNewUser2 = async (userInfo) => {
let newGuestId;
try {
newGuestId = await API.post("user", "/user", userInfo);
} catch (e) {
// API failed, do something here if you want...
return;
}
let cognitoResponse;
try {
cognitoResponse = await COGNITO.post("user", "/user", newGuestID);
} catch (e) {
// COGNITO failed
// If deleting throws, the error will percolate to the caller
return API.delete("guest", "/guest", guestInfo);
}
//BOTH REQUESTS OCCURED WITH NO ERRORS
};

Why can't I memoize in NextJS' getServerSideProps?

I'm using React + NextJS to display a list of products on a category page.
I can get the products just fine using getServerSideProps, but I don't like that it re-requests the product list on each visit to the same page. I'm trying to memoize a function that gets the list, and while that seems to work (meaning there are no errors thrown), the supposedly memoized function is still called on subsequent visits to the same page.
See the code below, and note that the "get category" console log is shown in the terminal window when I revisit a page, and in Chrome's network tools I see a fetch request made by NextJS.
How can I make it cache the result of my getCategory function so it doesn't keep fetching it?
export async function getServerSideProps(context) {
let config = await import("../../config/config");
let getCategory = memoize(async (url) => {
console.log("getting category");
let response = await axios.get(url);
if ( response.status ) {
return response.data;
} else {
return false;
}
});
let response = await getCategory(`${config.default.apiEndpoint}&cAction=getCTGY&ctgyCode=${context.params.code}`);
if ( response ) {
return {
props: {
category: response
}
};
} else {
return {
props: {
category: null
}
};
}
}
This doesn't work becuase nextjs api routes are "serverless", which means the state that memoize is supposed to remember is destroyed after HTTP call.
The serverless solution is to use a separate service for caching, which is accessible from your api route.
Otherwise, you may need to look at using a custom server.

Express resolving route before API call completes

Ok, full disclosure I am a hobby coder so I understand there to be gaps in my knowledge. However I've tried all sorts of solutions for this and have been unable to get a working answer.
DESIRED RESULT
I make a call to my Express server, it fetches data from an external API and renders once the data has been retrieved.
PROBLEM
I cannot seem to make Express wait no matter how I lay out the async/await pattern. Currently my code is as follows:
Express.js
app.get('/getInventory', async (req, res) => {
try {
let inventory = await api.getInventory(req.query.id)
console.log(inventory)
res.json(inventory)
}
catch(err) {
console.log(err)
}
})
My api.js is currently as such:
exports.getInventory = async function(id){
let data = await manager.getInventoryContents(id, 570, 2, true, (err, inventory) => {
if (err) {
console.log(err)
Promise.reject(err)
}
else {
console.log('Success')
Promise.resolve(inventory)
}
})
return data
}
In case you're wondering I have the console.log() actions just to try and see when something is happening.
WHAT I AM GETTING SO FAR
With just about every variation I am getting the Express.js inventory as undefined (similarly no data being sent to the client), however I AM receiving a Success message (or even the inventory itself) from api.js.
I am guessing the right syntax is obvious once I complete it but I cannot seem to get it to function properly. What am I missing?
Try this:
exports.getInventory = async function(id){
return new Promise((resolve, reject) => {
manager.getInventoryContents(id, 570, 2, true, (err, inventory) => {
if (err) {
console.log(err);
reject(err);
}
else {
console.log('Success');
resolve(inventory);
}
}
}
}

How to check for specific backend error keys when using async/await?

I want to display a custom message in Vue (with Django as the backend) when creating an account if the entered email already exists in the database.
Currently the server returns a 400 with the following response:
{"email":["There is another user with this email"]}
How can I access this error object to check if it contains an "email" key?
I've found this approach creating a wrapper: https://dev.to/sadarshannaiynar/capture-error-and-data-in-async-await-without-try-catch-1no2
but I feel like there must be a better/simpler way of handling this
Things I tried:
1) const { error, data } = await usersAPI.create(userData)
where "create" is:
create (data) {
return Vue.axios.post(url, data, config)
}
then console.error(error) (but it doesn't print anything)
2) Using try-catch(error) - prints nothing
3) Appending catch(e => console.error(e) to the await call - still nothing
Axios return an object error, you can get the content with error.message.
axios.post('/badurl')
.then(response => {
console.log(response)
})
.catch(error => {
console.log(error.message)
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.19.0/axios.min.js"></script>

How Do I Resolve this 'Too Many Requests' Error in Node.js?

I'm trying to build a JSON file by making successive HTTP requests with Axios:
Get an array of objects (projects)
Create an array property in each project named attachments
Get each project's tasks
Get each task's attachments
Push each project's task's attachments in to the project's attachments array
Create a JSON file out of the modified projects array
Code:
let getProjects = function() {
try {
return axios.get('https://app.asana.com/api/1.0/projects/')
} catch (error) {
console.error(error)
}
}
let getTasks = function(project) {
try {
return axios.get('https://app.asana.com/api/1.0/projects/'+project+'/tasks')
} catch (error) {
console.error(error)
}
}
let getAttachments = function(task) {
try {
return axios.get('https://app.asana.com/api/1.0/tasks/'+task+'/attachments')
} catch (error) {
console.error(error)
}
}
async function getAsanaData() {
let projects = await getProjects()
return Promise.all(projects.data.data.map(async (project) => {
project.attachments = []
let tasks = await getTasks(project.gid)
return Promise.all(tasks.data.data.map(async (task) => {
let attachments = await getAttachments(task.gid)
project.attachments = !!attachments ? project.attachments.concat(attachments.data.data) : project.attachments
return project
}))
}))
}
getAsanaData()
.then((projects) => {
var asanaData = safeJsonStringify(projects);
fs.writeFile("thing.json", asanaData);
})
.catch(err=>console.log(err))
But I'm running into this error:
status: 429,
statusText: 'Too Many Requests
I haven't found anything helpful yet for figuring out how to resolve it. What can I do?
HTTP response status code 429 indicates sending too many requests than what server could handle. It has been documented at https://asana.com/developers/documentation/getting-started/errors too. The maximum allowed is 150 per minute as documented at https://asana.com/developers/documentation/getting-started/rate-limits.
So, yes, as #Randy Casburn commented, you will have to throttle your requests.
You're getting throttled by Asana for sending too many requests and reaching the maximum rate.
When it happens, you need to check for the Retry-After response header and wait for the specified amount of time before sending another request.
https://asana.com/developers/documentation/getting-started/rate-limits
You can also learn more in the RFC 6585 about HTTP 429

Categories