We Keep Coding

Next js API + MongoDB error 431 on dynamic request

I'm getting a 431 (headers fields too large) on some API calls within a fullstack Next JS project. This only occurs on a dynamic API route (/author/get/[slug]), same result with both frontend and Postman. The server is running on local, and other endpoints works fine with exactly the same fetching logic.
The request is not even treated by Next API, no log will appear anywhere.
The database used is mongoDB. The API is pure simple JS.
The objective is to get a single author (will evolve in getStaticProps)
The API call looks like this (no headers whatsoever):
try {
const res = await fetch(`http://localhost:3000/api/author/get/${slug}`, { method: "GET" })
console.log(res)
} catch (error) { console.log(error) }
And the endpoint:
// author/get/[slug].js
import {getClient} from "../../../../src/config/mongodb-config";
export default async function handler(req, res) {
const { query } = req
const { slug } = query
if(req.method !== 'GET') {
return
}
const clientPromise = await getClient()
const author = clientPromise.db("database").collection("authors").findOne({ 'slug': slug })
res.status(200).json(author)
await clientPromise.close()
}
Tried without success:
To remove a nesting level (making the path /author/[slug])

NodeJS App crashes when wrong parameters are passed to the API

I have created an CRUD API using typescript NodeJS, Express and MongoDB. What I am trying to achieve is that when using the POST method when I send the correct parameter. API works fine. However whenever I send incorrect parameters to the API the whole NodeJS app crashes, I get an error message in console that the parameters passed to the API are wrong and I will have restart the application again.
When as user sends the incorrect parameters to the API. I don't want the NodeJS app to crash. I want to display the useful error message. Keep the app running. I Don't want to restart the application.
This is the code i use to create New providers.
public addProviders(req: Request, res: Response, next: NextFunction) {
var type = req.body.type,
name = req.body.name;
let newProvider = new Provider({
type,
name
})
newProvider.save()
.then((provider: Object) => {
if (provider) {
let statusCode = res.statusCode;
res.json({
statusCode,
provider
})
}
})
}
Below is the code that i have tried so far.
try {
newProvider.save()
.then((provider: Object) => {
if (provider) {
let statusCode = res.statusCode;
res.json({
statusCode,
provider
})
}
})
}
catch (e) {
res.json("enter valid parameters")
}
I am new to NodeJS. Thanks in advance.

You need to add input validation middleware to check inputs before adding to Database.
#1 You can check it manually, like:
var { type, name } = req.body;
if (typeof type !== 'string') {
res.status(400).send("type input is not valid");
} else if (typeof name !== 'string') {
res.status(400).send("name input is not valid");
} else {
let newProvider = new Provider({
type,
name
})
// ...rest of the code
}
#2 Or you can use a package like express-validator.

How do I parse the body of a request using the http module in node.js?

I am trying to write a framework for Node without using any Express code. I want to be able to get the body from a HTTP request using the http module, but nothing I try seems to work. This is my code currently:
import http from "http";
import url from "url";
import Route from "./Route";
import HTTPRequest from "./HTTPRequest";
class HextecCreator {
static createApp = (routes: Array<Route>) => {
return {
getRoutes: () => {
return routes;
},
run: (port: number) => {
http
.createServer(function (req, res) {
let data: any = [];
req
.on("data", (chunk) => data.push(chunk))
.on("end", () => {
data = Buffer.concat(data).toString();
});
var correctRoute;
for (var route of routes) {
if (route.getUrl() === req.url) {
correctRoute = route;
break;
} else {
correctRoute = "Route not found";
}
}
if (typeof correctRoute != "object") {
res.write("Route not found");
res.end();
} else {
res.write(
correctRoute
.getHandlerFunc()(
new HTTPRequest(
req.method,
req.url,
url.parse(req.url as string, true).query,
data
)
)
.getResponse()
);
res.end();
}
})
.listen(port);
},
};
};
}
export default HextecCreator;
This should work, but when I use postman to actually get the body of the request, it is empty. How can I fix this?

The end event gets triggered after your entire function is complete. To to do something with the entire HTTP request body, you will need to trigger this functionality from within the end handler.
This means that the rest of your logic needs to live inside the end event handler, or you need to call a function that "does the rest", but still from within that end event.
I also wrote a framework from scratch, and this is my async/await based version:
https://github.com/curveball/core/blob/master/src/node/request.ts#L36
Note that this still delegates turning an entire stream to a buffer to an external package.

Post Multiple Objects Inside Express Route

I would like to post multiple objects to my mongo database inside of an express route. Currently, everything is working fine when I do it as a single object (ie ONE casino), please see below, but instead of doing this a million times over, can someone help me do it as one giant data dump so I can post ALL my casinos?
Here is my route that works fine for posting a single object:
router.post('/post', async (req, res) => {
console.log(req.body);
const casinoD = new Casino({
casino: req.body.casino,
table_and_other: req.body.table_and_other,
poker: req.body.poker,
slot_machines: req.body.slot_machines,
total_gaming_win: req.body.total_gaming_win,
year: req.body.year,
month: req.body.month,
combined_date: req.body.combined_date
})
try {
const newCasino = await casinoD.save()
res.status(201).json(newCasino)
} catch (err) {
res.status(400).json({ message: err.message})
}
})
I also understand mongoimport is a better way to do this - however that had its own issues in of itself.
Thanks

Like #JDunken said, you can iterate over the POST body as an array and insert in bulk. You'll want to use
insertMany for speed. To insert millions of records, you will probably want to put a sane limit on the number of records per request, and send API requests in batches. Validation is optional, as Mongoose will run validation according to the schema. It depends on how you want to handle validation errors. Make sure to read up on the ordered and rawResult options for that as well.
router.post('/post', async (req, res) => {
// you should sanity check that req.body is an array first, depending on how robust you want error handling to be
const casinos = req.body.filter(input => isValid(input));
try {
const insertedCasinos = await CasinoModel.insertMany(casinos, { ordered: false });
res.status(201).json(insertedCasinos)
} catch (err) {
res.status(400).json({ message: err.message})
}
})
const isValid(input) {
let valid = true;
// implement input validation
return valid;
}

How to check whether request url params are not null in node js app?

I am new to node js programming and trying to develop an API using node js, I am able to retrieve the expected output from the built API but I would like to perform some exception handling. For that I would like to check whether the request params coming from URL are not null. Below is my code:
async function getDetails(input) {
// using knex to execute query
return queries.getbymultiwhere('table_name',{
name:input.name,
id:input.id,
role:input.role
})
}
router.get('/:name/:id/:role',(req,res)=>{
getDetails({
name:req.params.name,
id:req.params.id,
role:req.params.role}).then(Results=>{ Do something with results}
})
In above code I want to check that name, id and role param values are not null.
Any helpful solution will be appreciated.
Thank you!

You can create a middleware which checks those parameters.
function check(fields) {
return (req, res, next) => {
const fails = [];
for(const field of fields) {
if(!req.query[field]) {
fails.push(field);
}
}
if(fails.length > 0){
res.status(400).send(`${fails.join(',')} required`);
}else{
next();
}
};
}
app.get('/api', check(['name', 'id', 'role']), (req, res) => {
getDetails()...
});