I'm getting a 431 (headers fields too large) on some API calls within a fullstack Next JS project. This only occurs on a dynamic API route (/author/get/[slug]), same result with both frontend and Postman. The server is running on local, and other endpoints works fine with exactly the same fetching logic.
The request is not even treated by Next API, no log will appear anywhere.
The database used is mongoDB. The API is pure simple JS.
The objective is to get a single author (will evolve in getStaticProps)
The API call looks like this (no headers whatsoever):
try {
const res = await fetch(`http://localhost:3000/api/author/get/${slug}`, { method: "GET" })
console.log(res)
} catch (error) { console.log(error) }
And the endpoint:
// author/get/[slug].js
import {getClient} from "../../../../src/config/mongodb-config";
export default async function handler(req, res) {
const { query } = req
const { slug } = query
if(req.method !== 'GET') {
return
}
const clientPromise = await getClient()
const author = clientPromise.db("database").collection("authors").findOne({ 'slug': slug })
res.status(200).json(author)
await clientPromise.close()
}
Tried without success:
To remove a nesting level (making the path /author/[slug])
Related
I'm currently attempting to use Supabase's JavaScript API to update a row in my 'profiles' database, which has RLS on, via my backend.
This is being done following Stripe sending me a webhook indicating a payment has been successful.
I won't put the full API call in, but here is my Supabase code:
const supabaseUrl = process.env.REACT_APP_SUPABASE_URL
const supabaseAnonKey = process.env.REACT_APP_SUPABASE_ANON_KEY
const supabase = createClient(supabaseUrl, supabaseAnonKey)
module.exports = async (req, res) => {
if (event.type === "checkout.session.completed") {
const userId = String(event.data.object.client_reference_id)
const { error } = await supabase.from('profiles').update({ premium: 'true' }).eq('id', userId)
if (error) {
console.log(error)
}
}
}
However, every time I try to run this, I get a 404 error. This seems to be because I have RLS on.
As a result, I have two questions:
Is it safe for me to turn RLS off?
How can I adjust my code / apply a new database policy to allow this to be accepted?
I am trying to 1) retrieve hundreds of separate json files from this website https://bioguide.congress.gov/ that contains legislators in the U.S., 2) process them and 3) combine them into a big json that contains all the individual records.
Some of the files I am working with (each individual legislator has a different url that contains their data in a json file format) can be found in these urls:
https://bioguide.congress.gov/search/bio/F000061.json
https://bioguide.congress.gov/search/bio/F000062.json
https://bioguide.congress.gov/search/bio/F000063.json
https://bioguide.congress.gov/search/bio/F000064.json
https://bioguide.congress.gov/search/bio/F000091.json
https://bioguide.congress.gov/search/bio/F000092.json
My approach is to create a for loop to loop over the different ids and combine all the records in an array of objects. Unfortunately, I am stuck trying to access the data.
So far, I have tried the following methods but I am getting a CORS error.
Using fetch:
url = "https://bioguide.congress.gov/search/bio/F000061.json"
fetch(url)
.then((res) => res.text())
.then((text) => {
console.log(text);
})
.catch((err) => console.log(err));
Using the no-cors mode in fetch and getting an empty response:
url = "https://bioguide.congress.gov/search/bio/F000061.json"
const data = await fetch(url, { mode: "no-cors" })
Using d3:
url = "https://bioguide.congress.gov/search/bio/F000061.json"
const data = d3.json(url);
I am getting a CORS related error blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. with all of them.
I would appreciate any suggestions and advice to work around this issue. Thanks.
Following on from what #code says in their answer, here's a contrived (but tested) NodeJS example that gets the range of data (60-69) from the server once a second, and compiles it into one JSON file.
import express from 'express';
import fetch from 'node-fetch';
import { writeFile } from 'fs/promises';
const app = express();
const port = process.env.PORT || 4000;
let dataset;
let dataLoadComplete;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
function getData() {
return new Promise((res, rej) => {
// Initialise the data array
let arr = [];
dataLoadComplete = false;
// Initialise the page number
async function loop(page = 0) {
try {
// Use the incremented page number in the url
const uri = `https://bioguide.congress.gov/search/bio/F00006${page}.json`;
// Get the data, parse it, and add it to the
// array we set up to capture all of the data
const response = await fetch(uri);
const data = await response.json();
arr = [ ...arr, data];
console.log(`Loading page: ${page}`);
// Call the function again to get the next
// set of data if we've not reached the end of the range,
// or return the finalised data in the promise response
if (page < 10) {
setTimeout(loop, 1000, ++page);
} else {
console.log('API calls complete');
res(arr);
}
} catch (err) {
rej(err);
}
}
loop();
});
}
// Call the looping function and, once complete,
// write the JSON to a file
async function main() {
const completed = await getData();
dataset = completed;
dataLoadComplete = true;
writeFile('data.json', JSON.stringify(dataset, null, 2), 'utf8');
}
main();
Well, you're getting a CORS (Cross-Origin Resource Sharing) error because the website you're sending an AJAX request to (bioguide.congress.gov) has not explicitly enabled CORS, which means that you can't send AJAX requests (client-side) to that website because of security reasons.
If you want to send a request to that site, you must send a request from the server-side (such as PHP, Node, Python, etc).
More on the subject
I have running an app in heroku, separate in back and front
back: node + express + mysql
front: vue
the app works fine, but I have an error random: sometimes i have duplicates records inserted from frontend. (I guess the error comes from the front)
from the frontend I use fetch to add the records
const requestOptions = {
method: "POST",
headers: {
"Accept": "application/json",
"Content-Type": "application/json"
},
body: JSON.stringify({
data: data,
...: ...,
}),
};
const response = await fetch(`url_backend_heroku/api/add`, requestOptions);
the records are inserted correctly , but sometimes are inserted duplicated with correct next id
Could it be that fetch is sending 2 requests in some cases?
before deploying in heroku on my local machine I never duplicate records
I've been going around for days and I can't find why this happens
Yeah it is possible you are sending 2 requests somewhere. Put logs in heroku BACK on the specific endpoint to see whats exactly happening.
Also while requesting from Front end check your network tab in developer tools to see if you are actually firing the request twice. Because as you said duplicate records but they have correct ID's , could only mean what you said.
Also, this might or might not be true, but heroku server sleeps on inactivity, so it is possible that might cause an issue but I am not entirely sure on that, will have to check the code and environment for that.
looking at the heroku (back) log it looks like 2 records were inserted normally
looking at the network tab in the browser, only 1 request appears: OPTIONS (204) and POST (200)
the table has an id that is the primary key nothing complicated
on the other hand I am on a Dynos Hobby plan that does not have sleep times (if the free)
put here part or my backend
database.js
const mysql = require('mysql')
const { promisify } = require('util')
const config = { database keys }
const pool = mysql.createPool(config);
pool.getConnection((err: any, connection: any) => {
if (err) {
if (err.code === 'PROTOCOL_CONNECTION_LOST') {
console.error('DATABASE CONNECTION WAS CLOSED')
}
if (err.code === 'ER_CON_COUNT_ERROR') {
console.error('DATABASE HAS TO MANY CONNECTIONS')
}
if (err.code === 'ECONNREFUSED') {
console.error('DATABASE CONNECTION WAS REFUSED')
}
}
if (connection) connection.release()
console.log('DB is Connected')
return
})
pool.query = promisify(pool.query)
export default pool
controller.js
import pool from '../database';
public async insert(req: Request, res: Response) {
let body = req.body
try {
const response = await pool.query('SELECT MAX(id) + 1 as idNew FROM table')
let data = {Id: response[0].idNew, ...body}
//before it had an auto_increment id and it also failed
const result = await pool.query('INSERT INTO table set ?', [data])
res.json({
insertId: response[0].idNew,
message: "Saved OK"
})
} catch (error) {
console.log("error", error)
return res.status(500).send("error")
}
}
can it be a fetch problem? don't try yet to use axios for example
I have some code using GOT querying a graphQL endpoint:
// set up params for call to weather cache
const queryQL = `
query weather {
weather(where: {idLatLong: {_eq: "${latLong}"}}) {
id
idLatLong
updated_at
lat
long
requestedByUserId
data
created_at
}
}
`
const query = {query: queryQL};
const options = {
headers: {
'X-Hasura-Admin-Secret': process.env.HASURA_KEY
},
responseType: 'json'
}
// see if there's an existing record for the lat long
try {
const response = await got.post(process.env.GQL_ENDPOINT, query, options);
console.log('query weather hasura');
console.log(response.body);
} catch(error) {
console.log(error);
}
I am getting a response from Hasura {"errors":[{"extensions":{"path":"$","code":"invalid-headers"},"message":"Missing Authorization header in JWT authentication mode"}]}
How do I see what GOT is sending out to the GQL endpoint? FYI, this call works fine in the GQL console and also in Postman.
The got() library has hooks that allow you to see the headers it's about to send. Here's an example that you can run and then insert the same thing into your code:
const got = require('got');
got("http://www.google.com", {
hooks: {
beforeRequest: [function(options) {
console.log(options);
}]
}
}).then(result => {
let i = 1;
}).catch(err => {
console.log(err);
});
You can also get a network analyzer like Wireshark to put on your client computer and watch the actual network traffic.
I am new to Nuxt.js and I am faced with a strange kind of issue. I have an endpoint in my backend API, allowing the end user to send a token and a new password and reset the user password.
While the request is sent correctly, and the server responding with the correct data:
In the Nuxt.js side, I have an issue, with the response data.
So, to handle all the HTTP requests using the axios, I have a class like that:
class WebAPI {
// $axios is the instance used in the Nuxt.js context
constructor($axios) {
this.$http = $axios;
}
async call(config) {
try {
///////////////////////////////////////////////////////
// HERE IS THE PROBLEM. THE FOLLOWING CONSOLE.LOG
// IT RETURNS undefined WHILE THE NETWORK RESPONSE
// RETURNS WITH DATA
///////////////////////////////////////////////////////
const result = await this.$http(config);
console.log(result);
// ...
} catch( e) {
// ...
}
}
}
And I use this class like:
const data = {
token,
new_password
};
const options = {
method: 'POST',
url : '/reset-password',
data
};
return this.webApi.call(options);
But as you probably see, in the WebAPI service, the response of the axios is undefined.
Also, it worths to mention, that the exact same WebAPI class working perfectly with other API Requests I do throughout the application.
Could you help with that issue? Do you see anything wrong?
I think you are using axios wrong. Try use $request method, like that:
async call(config) {
try {
const result = await this.$http.$request(config);
console.log(result);
// ...
} catch( e) {
// ...
}
}