We Keep Coding

FetchError: request to url failed, reason ECONNREFUSED crashes server despite try catch being in place

The problem
FetchError: request to https://direct.faforever.com/wp-json/wp/v2/posts/?per_page=10&_embed&_fields=content.rendered,categories&categories=638 failed, reason: connect ECONNREFUSED
I'm doing some API calls for a website using fetch. Usually there are no issues, when a request "fails" usually the catch error gets it and my website continues to run. However, when the server that hosts the API calls is down/off, my fetch API calls crash the website entirely (despite being on a try catch loop).
As far as I'm concerned, shouldnt the catch block "catch" the error and continue to the next call? Why does it crash everything?
My wanted solution
For the website to just move on to the next fetch call / just catch the error and try again when the function is called again (rather than crashing the entire website).
The code
Here is an example of my fetch API call (process.env.WP_URL is = https:direct.faforever.com )
async function getTournamentNews() {
try {
let response = await fetch(`${process.env.WP_URL}/wp-json/wp/v2/posts/?per_page=10&_embed&_fields=content.rendered,categories&categories=638`);
let data = await response.json();
//Now we get a js array rather than a js object. Otherwise we can't sort it out.
let dataObjectToArray = Object.values(data);
let sortedData = dataObjectToArray.map(item => ({
content: item.content.rendered,
category: item.categories
}));
let clientNewsData = sortedData.filter(article => article.category[1] !== 284);
return await clientNewsData;
} catch (e) {
console.log(e);
return null;
}
}
Here's the whole code (this whole thing is being called by express.js in line 246 (the extractor file).
Extractor / Fetch API Calls file
https://github.com/FAForever/website/blob/New-Frontend/scripts/extractor.js
Express.js file in line 246
https://github.com/FAForever/website/blob/New-Frontend/express.js#:~:text=//%20Run%20scripts%20initially%20on%20startup

Catching Error With Custom Rest API For FireStore Javascript

I have been watching a tutorial on making a Rest API for Firestore which appears to work but I cannot figure out how to catch an error.
The code below basically uses an end point to retrieve a document id from the firestore database.
The client uses javascript fetch to call the API.
I am trying to workout how to return something back to the client from the API if the document id is not there. I thought I might get a 404 status returned but I always get status 200.
This is the API code I used
app.get("/api/read/:id", (req, res) => {
(async () => {
try {
const document = db.collection("users").doc(req.params.id);
let product = await document.get();
let response = product.data();
return res.status(200).send(response);
} catch (error) {
console.log(error);
return res.status(500).send(error);
}
})();
})

I'm fairly certain that the 404 message is for the server itself not being found (though I do need to brush up on my error codes).
However, if you're looking to check to see if a document exists there's a command specifically for that demonstrated in the examples in the firebase docs

How do you return a 'success' message to the frontend using onFinalize?

I am currently using a ReactJs frontend with the Firebase Javascript SDK. It is hooked up to a firebase-functions storage emulator using NodeJS in the backend.
Usually, I am able to call a functions.https.onCall(...) cloud function (that has been set up with nodejs) from my frontend and receive a promise, and then I can use the data in the front end normally.
However, using functions.storage.object().onFinalize(...) in the backend, I am trying to force some sort of way to return a success message to my front end after a file has successfully been processed.
Here is what I've tried:
My backend code (Nodejs):
exports.processMedia = functions.storage.object().onFinalize(async (object) =>{
// Here I successfully process any uploaded images
// Issue - After the media has been processed, I'd like to return a message that can be used to update the UI. I've tried to return a new promise like so:
return new Promise((resolve, reject) =>{
return resolve("Success");
});
})
And then in my React UI, I am trying to capture the return message from the backend using a httpsCallable('processMedia') function because I don't know how else to try to get the response.
async processMedia(){
const processMediaCallable = this.functions.httpsCallable('processMedia');
return processMediaCallable;
}
Then I try to log out the values from the promise:
async function getProcessedMessage(){
await firebase.processMedia().then((val) => {
console.log(val);
});
}
I get this output in the browser console for val after I've uploaded a file in the UI and processMedia() in the backend has completed:
ƒ (data) {
return _this.call(name, data, options || {});
}
I am trying to output a success message here. Any thoughts would be greatly appreciated. Thank you.

You cannot return a response to client from a background function. When a user uploads any file to your Firebase storage they get a success response and the function triggered from that event has nothing to do with client. There's a workaround for using Firebase Realtime Database which you can use to emit the response to user.
To break it down in simple steps:
Upload file to Firebase Storage
Create a realtime database listener on client side
After all processing in your Cloud Function, write something in the database and that'll be delivered to your user.
Client:
async function uploadFile() {
// upload to firebase storage
const alertRef = firebase.database().ref(`alerts/${userID}`);
alertRef.on('child_added', (snapshot) => {
const data = snapshot.val();
// Alert received from cloud function
// Delete this alert from database
snapshot.ref.set(null)
});
}
Cloud function:
exports.processMedia = functions.storage.object().onFinalize(async (object) => {
// Process the file
// Add alert in realtime database
const alertRef = firebase.database().ref(`alerts/${userID}`).push()
return alertRef.set({...}) // add any data you would like to pass to client side
})
child_added event is triggered once for each existing child and then again every time a new child is added to the specified path. The listener is passed a snapshot containing the new child's data.
That being said just make sure you are not alerting users multiple times with same notification.

How to compress json response size in Firebase cloud functions?

i have a firebase callable function that takes data from a sql cloud database (mysql).
this is the code:
// Get records
exports.getRecords = functions.region("europe-west1").https.onCall((data, context) => {
// get data
const table_name = data.table_name
return mysqlPromise.createPool(db.connectionOptions)
.then(pool => {
return pool.query(`
SELECT * FROM ${table_name};`
)
})
.then(res => {
return { result: res };
})
.catch(error => {
throw new functions.https.HttpsError('failed-precondition', error);
});
});
If I run the function on a table with few rows (up to 10000) it works perfectly, but if the table is bigger (100K records) the execution fails showing a "CORS error".
Analyzing the response, I noticed that the response size on a query that requires 100K rows is around 25MB.
I know that the limit quota of the firebase functions is 10MB for each response, and I am pretty sure that the error is due to that (even if CORS sounds strange).
The question is:
"Is there any way to run that query that returns a 100k rows json to me?"
maybe I have to compress the response or else ... what is the best solution?
Thanks in advance! :)

Processing fetch response - JS

I have the response after GET-request with fetch. I know that this response will return an Array. When I process it with .json() - everything is fine. But when I try to process same response with .formData()- it fails.
Here's the code:
fetch(fullPath)
.then(response => response.json())
.then((allData) => {
console.log(allData);
})
.catch((e) => console.log(e));
same one with response.formData() doesn't work.
So the question is - why we are not able to process the promise in first "then" with .formData() if the off. doc says that formData() is also acceptable method for extracting a body from the response ?

The formData() method will return an error if the response is not having the multipart/form-data type. It should have key=value entries. See this question where someone asks about designing a response with that format.
Something similar happens if you call the json() method on a response that is not valid JSON: that also triggers an exception.
Since multipart/form-data is rarely used as response content type, the formData() method is not often used either. The documentation on MDN mentions it has some use for service workers:
Note: This [method] is mainly relevant to service workers. If a user submits a form and a service worker intercepts the request, you could for example call formData() on it to obtain a key-value map, modify some fields, then send the form onwards to the server (or use it locally).