We Keep Coding

Fetch API stream doesn't work on server but does locally

As I explained in a previous question I have a FastAPI endpoint that returns a StreamingResponse response that is then consumed by a React application using fetch().body.getReader() API.
The problem I'm facing appears when I open my React application, select the image (s) using Uppy and send it to my FastAPI endpoint, locally it works just fine and the images are returned as a stream response:
But when I deploy my application on Heroku or Render the rendered response is all broken:
Adding more context to my previous question I'm rendering the stream using an async generator:
async function* submit({data}) {
const formData = new FormData()
data?.current?.files?.successful.map(image =>
formData.append('images', image.data)
)
formData.append('language', data.current.language.code)
try {
const response = await fetch(
'endpoint',
{
method: 'POST',
body: formData
}
)
const reader = response.body.getReader()
while (true) {
const { value, done } = await reader.read()
if (done) break
const base64 = `data:${response.headers.get(
'content-type'
)};base64,${btoa(String.fromCharCode(...new Uint8Array(value)))}`
yield base64
}
} catch (error) {
// ...
}
}
That is called when the "Gallery" component in the screenshot is rendered:
const [images, setImages] = useState([])
useEffect(() => {
;(async () => {
for await (const image of await translateSubmit({
data
})) {
setImages(previous => [...previous, image])
}
})()
// eslint-disable-next-line
}, [])
I was expecting to get the same result in the server when the application is deployed, but it just doesn't work as it should and I'm not sure how to approach the problem. Any tips?

Unable to display API data on React frontend

I'm trying to return data fetched from a private API and display it on a page. My frontend use React JS and my backend use node with Express and Axion. My code work up to the point of returning the data. I get my APi Key and fetch my data but the data is not transferred to my page (Quotes.js).
Backend
app.js
import express from "express";
import { getCase } from "./getCase.js";
const app = express();
app.use(function (req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header(
"Access-Control-Allow-Headers",
"Origin, X-Requested-With, Content-Type, Accept"
);
next();
});
app.get("/", function (req, res) {
console.log("app.js call getCase");
res.send(getCase());
//console.log(req);
});
//console.log(Quote.getQuote());
let port = process.env.PORT;
if (port == null || port == "") {
port = 5000;
}
app.listen(port, function () {
console.log(`Server started on port ${port}...`);
});
Backend getCase
import { getToken } from "./nsApiToken.js";
import axios from "axios";
let getData = "";
console.log("begin of getCase");
const getCase = async () => {
let tokenRes = await getToken();
const url =
"https://5156735-sb1.app.netsuite.com/app/site/hosting/restlet.nl?script=860&deploy=1&recordtype=supportcase&id=717986";
try {
const res = await axios.get(url, {
headers: {
Authorization: `Bearer ${tokenRes.data.access_token}`,
},
});
return res;
} catch (error) {
return error;
}
};
export { getCase };
Frontend App.js
import logo from "./logo.svg";
import "./App.css";
import Quotes from "./Quotes.js";
function App() {
return (
<div className="App">
<header className="App-header">
<Quotes />
</header>
</div>
);
}
export default App;
Frontend Quotes.js
import React, { useState, useEffect } from "react";
import axios from "axios";
const Quotes = async () => {
const [text, setText] = useState([]);
const [author, setAuthor] = useState("");
const getQuote = await axios
.get("http://localhost:5000", {
crossdomain: true,
})
.then((res) => res.data)
.then((data) => {
setText({
data: data,
});
console.log("res: ", text);
});
return (
<div>
<button onClick={getQuote}>Generate Quote</button>
<h1>{text}</h1>
<h3>{author}</h3>
</div>
);
};
export default Quotes;
Process:
When I run my process the front execute and call Quotes.js in the axios get process.
app.js then route to home ('/') and call getCase via the app.get.
The getCase process execute get the API token and add it in the headers Authorization. The process initiate the call and fetch the data (if I console.log(res.data.fields.phone) or console.log(res.data.id) I see the correct data.
In my Quotes.js I want to display the data but res.data is empty, yet I get back status 200.
I've been trying to understand why it is not passing the data from the backend to the frontend.

There are several problems and some improvements to be made.
Backend
Problem - You are sending the entire AxiosResponse in the response from your Express app
Just send the data
const getCase = async () =>
(
await axios.get(
"https://5156735-sb1.app.netsuite.com/app/site/hosting/restlet.nl",
{
params: {
script: 860,
deploy: 1,
recordtype: "supportcase",
id: 717986,
},
headers: {
Authorization: `Bearer ${(await getToken()).data.access_token}`,
},
}
)
).data; // Return the data, not the whole response
Problem - getCase() is async
You need to await the result
app.get("/", async (req, res, next) => {
try {
res.json(await getCase());
} catch (err) {
next(err); // send the error to the Express error handler
}
});
Improvement - Creating your own CORS middleware is a waste of time
By the time you create a comprehensive CORS middleware, it will look exactly the same as the standard one so just use that
import express from "express";
import cors from "cors";
const app = express();
express.use(cors());
Frontend
Problem - React function components cannot be async
Function components must return a valid JSX node. Remove async from Quotes
Problem - getQuote should be a function
In order to trigger getQuote by button click, it needs to be a function
// if text is an object, initialise it as one
const [text, setText] = useState({});
const getQuotes = async () => {
try {
// there is no "crossdomain" Axios option
const { data } = await axios.get("http://localhost:5000");
setText({ data });
} catch (err) {
console.error(err.toJSON());
}
};
Problem - the text state is an object
JSX cannot render plain objects, you instead need to reference properties that can be rendered.
<h1>{text.data?.some?.property}</h1>
No idea what your response object looks like so this is just generic advice

The reason why this is not working is for two reasons. Firstly, res.data is not an asynchronous function. And since you are doing await, you can just get data. Secondly, you need to make your API calls and setState in the useEffect hook or else it would just end up in an infinite rerender situation. You just have to do the following and it should work:
useEffect(() => {
const fetchData = async () => {
const {data} = await axios
.get('http://localhost:5000', {
crossdomain: true
})
setText(data)
}
fetchData()
}, [])

Unable to receive POST data unless I refresh client. why? How can I wait for post data to be received before page loads?

My app starts with a simple html form. the inputs are PIN# and Date Of Birth.
My express server runs on the same port 3000, when the user submits their data, puppeteer starts and logs into a specific webpage. Then I scrape the image on that webpage. Google Api takes the text from that image and saves it in an array. I then post that array string to src/results.html. But as soon as the user hits submit, they are redirected to /resuts route right immediately and the page says cannot post the data. but when I see in the console (roughly a minute later) that the post was successful, I refresh the page and I get the array of text I wanted to see.
How can I await for the data to finish being posted to the route before the page loads the data? Im using react for client side. below is my server side code. client side is just a basic react page login and a static /results page meant for the data.
const puppeteer = require("puppeteer");
const express = require("express");
const app = express();
const morgan = require("morgan");
const fs = require("fs");
const cors = require("cors");
const request = require("request-promise-native").defaults({ Jar: true });
const poll = require("promise-poller").default;
app.use(morgan("combined"));
const port = 3000;
// Imports the Google Cloud client library
const vision = require("#google-cloud/vision");
require("dotenv").config();
app.use(cors());
const textArray = [];
const App = (pinNum, dateOfB) => {
const config = {
sitekey: process.env.SITEKEY,
pageurl: process.env.PAGEURL,
apiKey: process.env.APIKEY,
apiSubmitUrl: "http://2captcha.com/in.php",
apiRetrieveUrl: "http://2captcha.com/res.php",
};
const chromeOptions = {
executablePath: "/Program Files/Google/Chrome/Application/chrome.exe",
headless: true,
slowMo: 60,
defaultViewport: null,
};
async function main() {
const browser = await puppeteer.launch(chromeOptions);
const page = await browser.newPage();
console.log(`Navigating to ${config.pageurl}`);
await page.goto(config.pageurl);
try {
const requestId = await initiateCaptchaRequest(config.apiKey);
// const pin = getPIN();
console.log(`Typing PIN ${pinNum}`);
await page.type("#PIN", pinNum);
// const dob = getDOB();
console.log(`Typing DOB ${dateOfB}`);
const input = await page.$("#DOB");
await input.click({ clickCount: 3 });
await input.type(dateOfB);
const response = await pollForRequestResults(config.apiKey, requestId);
console.log(`Entering recaptcha response ${response}`);
await page.evaluate(
`document.getElementById("g-recaptcha-response").innerHTML="${response}";`
);
console.log(`Submitting....`);
page.click("#Submit");
} catch (error) {
console.log(
"Your request could not be completed at this time, please check your pin number and date of birth. Also make sure your internet connection is working and try again."
);
console.error(error);
}
await page.waitForSelector(
"body > div.container.body-content > div:nth-child(1) > div:nth-child(2) > p"
);
const image = await page.$(
"body > div.container.body-content > div:nth-child(1) > div:nth-child(2) > p"
);
await image.screenshot({
path: "testResults.png",
});
await getImageText();
await page.close(); // Close the website
await browser.close(); //close browser
await deleteImage();
}
main();
//This section grabs the text off the image that was gathered from the web scraper.
async function getImageText() {
// Creates a client
const client = new vision.ImageAnnotatorClient();
console.log(`Looking for text in image`);
// Performs label detection on the image file
const [result] = await client.textDetection("./testResults.png");
const [annotation] = result.textAnnotations;
const text = annotation ? annotation.description : "";
console.log("Extracted text from image:", text);
//Pushed the text into a globally available array.
textArray.push(text);
//Sent a NOTIFICATION ALERT to the client with the text gathered from the image.
var axios = require("axios");
var data = JSON.stringify({
to: "dp8vGNkcYKb-k-72j7t4Mo:APA91bEfrI3_ht89t5X1f3_Y_DACZc9DbWI4VzcYehaQoXtD_IHIFSwm9H1hgXHNq46BQwDTlCKzkWNAHbBGauEXZNQtvhQc8glz4sHQr3JY3KM7OkUEcNB7qMMpCPxRe5GzzHbe3rkE",
notification: {
body: text,
title: "AverHealth Schedule",
},
});
var config = {
method: "post",
url: "https://fcm.googleapis.com/fcm/send",
headers: {
"Content-Type": "application/json",
Authorization: `key=${process.env.FCM_SERVER_KEY}`,
},
data: data,
};
axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});
}
//Captcha Solver for the web scraper
async function initiateCaptchaRequest(apiKey) {
const formData = {
key: apiKey,
method: "userrecaptcha",
googlekey: config.sitekey,
json: 1,
pageurl: config.pageurl,
};
console.log(
`Submitting recaptcha request to 2captcha for ${config.pageurl}`
);
const response = await request.post(config.apiSubmitUrl, {
form: formData,
});
console.log(response);
return JSON.parse(response).request;
}
async function pollForRequestResults(
key,
id,
retries = 90,
interval = 5000,
delay = 1500
) {
console.log(`Waiting for ${delay} milliseconds....`);
await timeout(delay);
return poll({
taskFn: requestCaptchaResults(key, id),
interval,
retries,
});
}
function requestCaptchaResults(apiKey, requestId) {
const url = `${config.apiRetrieveUrl}?key=${apiKey}&action=get&id=${requestId}&json=1`;
console.log(url);
return async function () {
return new Promise(async function (resolve, reject) {
console.log(`Polling for response...`);
const rawResponse = await request.get(url);
console.log(rawResponse);
const resp = JSON.parse(rawResponse);
console.log(resp);
if (resp.status === 0) return reject(resp.request);
console.log("Response received");
console.log(resp);
resolve(resp.request);
});
};
}
// DELETES THE FILE CREATED BY GOOGLEAPI
function deleteImage() {
const path = "./testResults.png";
try {
fs.unlinkSync(path);
console.log("File removed:", path);
} catch (err) {
console.error(err);
}
}
const timeout = (ms) => new Promise((res) => setTimeout(res, ms));
};
app.use(express.urlencoded({ extended: false }));
// Route to results Page
app.get("/results", (req, res) => {
res.sendFile(__dirname + "/src/results.html");
res.send(textArray);
});
app.post("/results", (req, res) => {
// Insert Login Code Here
let username = req.body.username;
let password = req.body.password;
App(username, password);
});
app.listen(port, () => {
console.log(`Scraper app listening at http://localhost:${port}`);
});

I think I got the problem.
In the react app, maybe you are not using e.preventDefault() when you click submit. The browser, by default, redirects to a page where the form action is directing, if the action attribute is empty then the browser reloads the same page. I would recommend you to use e.preventDefault() on form submission and then use fetch API to make the request.
In the express server, on the route POST "results", you are not sending any response back to the user. You should always send a response to the user. In your case you are calling the App function - which has many async functions, but you are not awaiting for App() to complete in the POST route, express is sending default response to the user as soon as it parses App() - it is not waiting for the App() to complete - express will get to this later.
You can make the (req, res) => { ... } function in the route as async function async (req, res) => { ... }, then you can make the App as async function as well. Then you can await App(...) in the route function. Also, you need to await for the main() function as well inside the App() function. Then once App() call has finished, you can send redirect response to the user.

How can i parse this result from a fetch call EXpress NoseJs Javascript

Here is the code of my script, it's in the index.html file, I know it's wrong to be there but first I'm trying to make it work, then I'll move it.
readOperaciones();
async function readOperaciones(){
try{
const listaOr = document.getElementById('listaOrdenada');
const result = await fetch("http://localhost:8080/operaciones", {method: "GET"})
const operaciones = await JSON.parse(result)
//operaciones.forEach(t=>{
for (var i = 0; i < operaciones.length; i++) {
var row = operaciones[i];
console.log(row.codeemp);
}
/*tt = JSON.stringify(t);
const li = document.createElement("li");
li.textContent = tt.text;*/
/*t.forEach(cell=>{
const li = document.createElement("li")
li.textContent = cell.text;
li.id = cell.id;
})*/
//})
}
catch(e){
console.log("Error al leer las operaciones descriptas")
}
}
Here is the connecting with express
const {Client} = require('pg');
const express = require ("express")
const app = express();
app.use(express.json())
const client = new Client({
user: "postgres",
password: "1234",
host: "localhost",
port: 5432,
database: "webaduana",
})
app.get("/", (req, res) => res.sendFile(`${__dirname}/index.html`))
app.get("/operaciones", async (req, res) => {
const rows = await readAll();
res.setHeader("content-type", "application/json")
res.send(JSON.stringify(rows))
})
async function readAll(){
try{
const results = await client.query("select * from operaciones")
return results.rows;
}
catch(e){
console.log(e)
return [];
}
}
I don't know if I need to put more information but my question about all this code is here
I've tried many ways of doing it but I can't get those results in an ol element.
It doesn't give me any error, it just doesn't print anything in the HTML page

Add a .then to the fetch chain and print your results:
fetch('http://example.com/movies.json')
.then(response => {
console.log('response: ' + JSON.stringify(response));
})
...
...
https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch

you can use .json() method :
const fetched = await fetch("/url", {
method: 'GET',
});
const fetchedJson: object = await fetched.json();
console.log(fetchedJson)

There are a few ways to do that.
Using Promises
The fetch response object inherently includes some methods that can help you to get a response in different forms such as .json(), .text(), and .status. Learn more about it here. So, if you simply want to parse the answer into a JSON object, you can do
function doSomethingOnParsedJson(res) {
// Do something on the response here...
}
function readOperacions() {
fetch("http://localhost:8080/operaciones", {
method: "GET",
})
.then(res => res.json())
.then(doSomethingOnParsedJson) // Pass in a function without parentheses
.catch(console.error);
}
It's cleaner if you define a separate function which performs the job you want to do on the parsed response and pass the function (without parentheses) to then but you can also go ahead and give it a function directly like:
function readOperacions() {
fetch("http://localhost:8080/operaciones", {
method: "GET",
})
.then(res => res.json())
.then(parsedResponse => {
// do something...
})
.catch(console.error);
}
Using Async/Await
You can also use the async/await feature to achieve that.
function doSomethingOnParsedJson(res) {
// Do something on the response here...
}
async function readOperacions() {
try {
// Get the response from the server.
const res = await fetch("http://localhost:8080/operaciones", {
method: "GET",
});
// Parse it into a JSON object.
const parsedJson = res.json();
// Do something on it.
doSomethingOnParsedJson(parsedJson);
} catch (error) {
// Show an error if something unexpected happened.
}
}
Side Note
There is a neater way to send a JSON response in Express. You can use the .json method on the Express response object.
app.get("/operaciones", async (req, res) => {
const rows = await readAll();
/* Don't do this!
res.setHeader("content-type", "application/json")
res.send(JSON.stringify(rows))
*/
/* Do this instead ;-) */
res.json(rows);
})
Voila! It's that simple.

Fastify REST-API JWT-Auth Plugin not firing as preHandler

Im setting up a Fastify Rest-Api and wrote a Plugin to encapsulate my authentication logic which is based on JWT. Im using the preHandler Hook on each route that i want to protect but it seems that the preHandler or my plugin just gets ignored since i can just make a request without a token at all and get the data.
I looked up every piece of documentation but still cannot get it running. If i just console.log() my function fastify.authenticate i get an undefined.
This is my plugin customJwtAuth:
const fp = require('fastify-plugin')
async function customJwtAuth(fastify, opts, next) {
//register jwt
await fastify.register(require('fastify-jwt'),
{secret: 'asecretthatsverylongandimportedfromanenvfile'})
fastify.decorate('authenticate', async function(request, reply) {
try {
const tokenFromRequest = request.cookies.jwt
await fastify.jwt.verify(tokenFromRequest, (err, decoded) => {
if (err) {
fastify.log.error(err)
reply.send(err)
}
fastify.log.info(`Token verified: ${decoded}`)
})
} catch (err) {
reply.send(err)
fastify.log.error(err)
}
})
next()
}
module.exports = fp(customJwtAuth, {fastify: '>=1.0.0'})
I register this plugin like this in my main server.js file:
const customJwtAuth = require('./plugin/auth')
fastify.register(customJwtAuth).after(err => {if (err) throw err})
Then i apply my function like this to the routes:
const fastify = require('fastify')
const productHandler = require('../handler/productHandler')
const productRoutes = [
{
method: 'GET',
url: '/api/product',
preHandler: [fastify.authenticate],
handler: productHandler.getProducts
}, ... ]
The api shouldnt return any Data if the request doesnt include a signed jwt or without a jwt at all.

here to you a working example.
Note that you were calling next() when you were registering the decorator that is wrong.
Your main error was due the [fastify.authenticate] line, because you don't have the decorator in that fastify instance.
//### customAuthJwt.js
const fastifyJwt = require('fastify-jwt')
const fp = require('fastify-plugin')
async function customJwtAuth(fastify, opts, next) {
fastify.register(fastifyJwt, { secret: 'asecretthatsverylongandimportedfromanenvfile' })
fastify.decorate('authenticate', async function (request, reply) {
try {
// to whatever you want, read the token from cookies for example..
const token = request.headers.authorization
await request.jwtVerify()
} catch (err) {
reply.send(err)
}
})
}
module.exports = fp(customJwtAuth, { fastify: '>=1.0.0' })
//### server.js
const fastify = require('fastify')({ logger: true })
const customJwtAuth = require('./customAuthJwt')
fastify.register(customJwtAuth)
fastify.get('/signup', (req, reply) => {
// authenticate the user.. are valid the credentials?
const token = fastify.jwt.sign({ hello: 'world' })
reply.send({ token })
})
fastify.register(async function (fastify, opts) {
fastify.addHook('onRequest', fastify.authenticate)
fastify.get('/', async function (request) {
return 'hi'
})
})
fastify.listen(3000)
You get:
curl http://localhost:3000/
{"statusCode":401,"error":"Unauthorized","message":"No Authorization was found in request.headers"}
curl http://localhost:3000/signup
{"token": "eyJhbGciOiJIUzI1NiI..."}
curl 'http://localhost:3000/' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiI...'
hi

if you're using version 2 of fastify you can use PreHandler, if not you need to user beforeHandler
And also, you need to change the routes for something like this
//routes/products.js
const fastify = require('fastify')
const productHandler = require('../handler/productHandler')
module.exports = function (fastify, opts, next) {
fastify.route({
method: 'GET',
url: 'api/product',
beforeHandler: fastify.auth([
fastify.authenticate
]),
handler: productHandler.getProducts
})
......
next()
}
//server.js
....
fastify.register(require('fastify-auth'))
.register(customJwtAuth)
const customJwtAuth = require('./customAuthJwt')
....
fastify.register(
require('./routes/products')
)