Validate URLs in nodejs + express to handle bad request errors - javascript

Well, after reading some things about this issue, i decide to ask about.
The idea is simple in the core: detect incoming invalid URL's/URI's and return a response to the client with the pertinent code/data.
Normally, when treat the errors in node, we use syntax like:
if (err) {
return res.status(500).json(error: {msg: err.message, stack: err.stack});
}
Okay, that is pretty simple but the essential thing is that we detect: "hey, if an error occurs, shoot this thing to the client".
But what if it's not an 500 internal? As i know, bad request errors are send like a client error and have their own stack, type and message in the prompt, rigth?
Reading This, i found the URIError constructor, that points to an instance representing an error that occurs with ivalids URI's. So, it's possible to make some like this:
if (err instanceof URIError) { /* send a handler for bad request */ };
Or i need to validate from scratch the URI and make my own error inheritance?
Any hint? Idea? ... Thanks in advance! <3

I usually do this by defining the routes I want:
app.get('/good' ... );
app.post('/alsogood' ... );
And then to catch unrouted urls, do:
app.all('*', function(req, res) {
throw new Error("Bad request")
})
And then inject an error handling middleware:
app.use(function(e, req, res, next) {
if (e.message === "Bad request") {
res.status(400).json({error: {msg: e.message, stack: e.stack}});
}
});
You can obviously get a lot more complex than this, but this simple idea gets the point across.
Also, for what it's worth, you should probably throw a 404 back to to the client if they're trying to access a URL that doesn't exist.

Related

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to client [duplicate]

I'm facing this weird issue in NodeJS when using with Passport.js, Express and Mongoose. Basically, I get an error saying "Cannot set headers after they are sent to the client" even though I don't send more than one header.
I've read other posts and tried them out as well, and none of them worked.
app.get - is there any difference between res.send vs return res.send
Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
Cannot set headers after they are sent to the client
I've dug through github issues and I can't seem to find a solution. I get the problem that this error is triggered when I send multiple response headers, but the fact is that I am not sending multiple headers. It seems just weird.
This is my stack trace:
(node:9236) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.
Server Running on port 5000
MongoDB Connected Error
[ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the
client
at validateHeader (_http_outgoing.js:503:11)
at ServerResponse.setHeader (_http_outgoing.js:510:3)
at ServerResponse.header (/Users/lourdesroashan/code/github/devlog/node_modules/express/lib/response.js:767:10)
at ServerResponse.json (/Users/lourdesroashan/code/github/devlog/node_modules/express/lib/response.js:264:10)
at Profile.findOne.then.profile (/Users/lourdesroashan/code/github/devlog/routes/api/profile.js:27:30)
at <anonymous>
This is my server code:
router.get("/userprofile", passport.authenticate('jwt', { session: false }), (req, res) => {
Profile.findOne({ user: req.user.id }).then(profile => {
if (!profile) {
return res.status(404).json({ error: "No Profile Found" });
}
else {
res.json(profile);
}
}).catch(err => {
console.log(err);
})
});
I understand what the error means, but from what I know, I don't think I am sending multiple headers, I even checked by console.log that only one of the blocks is run.
Thank you so much in advance! :)
Full Code at: https://github.com/lourdesr/devlog
EDIT:
I figured it out. It was a problem in my passport.js while trying to get the authenticated user. I forgot to use 'return' on the 'done' method, which had caused it. Just added the return statement and it worked!
That particular error occurs whenever your code attempts to send more than one response to the same request. There are a number of different coding mistakes that can lead to this:
Improperly written asynchronous code that allows multiple branches to send a response.
Not returning from the request handler to stop further code in the request handler from running after you've sent a response.
Calling next() when you've already sent a response.
Improper logic branching that allows multiple code paths to execute attempt to send a response.
The code you show in your question does not appear like it would cause that error, but I do see code in a different route here that would cause that error.
Where you have this:
if (!user) {
errors.email = "User not found";
res.status(404).json({ errors });
}
You need to change it to:
if (!user) {
errors.email = "User not found";
res.status(404).json({ errors });
// stop further execution in this callback
return;
}
You don't want the code to continue after you've done res.status(404).json({ errors }); because it will then try to send another response.
In addition, everywhere you have this:
if (err) throw err;
inside an async callback, you need to replace that with something that actually sends an error response such as:
if (err) {
console.log(err);
res.sendStatus(500);
return;
}
throwing inside an async callback just goes back into the node.js event system and isn't thrown to anywhere that you can actually catch it. Further, it doesn't send a response to the http request. In otherwords, it doesn't really do what the server is supposed to do. So, do yourself a favor and never write that code in your server. When you get an error, send an error response.
Since it looks like you may be new here, I wanted to compliment you on including a link to your full source code at https://github.com/lourdesr/devlog because it's only by looking at that that I was able to see this place where the error is occuring.
I was receiving this error because of a foolish mistake on my part. I need to be more careful when referencing my other working code. The truly embarrassing part is how long I spent trying to figure out the cause of the error. Ouf!
Bad:
return res
.send(C.Status.OK)
.json({ item });
Good:
return res
.status(C.Status.OK)
.json({ item });
Use ctrl + F hotkey and find all 'res.' keywords
then replace them with 'return res.',
change all 'res.' to 'return res.'
something like this:
res.send() change to --> return res.send()
maybe you have 'res.' in some block, like if() statement
Sorry for the Late response,
As per the mongoose documentation "Mongoose queries are not promises. They have a .then() function for co and async/await as a convenience. However, unlike promises, calling a query's .then() can execute the query multiple time"
so to use promises
mongoose.Promise = global.Promise //To use the native js promises
Then
var promise = Profile.findOne({ user: req.user.id }).exec()
promise.then(function (profile){
if (!profile) {
throw new Error("User profile not found") //reject promise with error
}
return res.status(200).json(profile) //return user profile
}).catch(function (err){
console.log(err); //User profile not found
return res.status(404).json({ err.message }) //return your error msg
})
here is an nice article about switching out callbacks with promises in Mongoose
and this answer on mongooses promise rejection handling Mongoose right promise rejection handling
There is a simple fix for the node error [ERR_HTTP_HEADERS_SET]. You need to add a return statement in front of your responses to make sure your router exits correctly on error:
router.post("/", async (req, res) => {
let user = await User.findOne({email: req.body.email});
if (!user) **return** res.status(400).send("Wrong user");
});
Because of multiple response sending in your request. if you use return key word in your else condition your code will run properly
if (!profile) {
return res.status(404).json({ error: "No Profile Found" });
}
else {
**return** res.json(profile);
}
This also happen when you tries to send the multiple response for a same request !!
So make sure you always use return keyword to send response to client inorder to stop the further processing !!
Where you have this:
if (!user) { errors.email = "User not found"; res.status(404).json({ errors }); }
You need to change it to:
if (!user) { errors.email = "User not found"; return res.status(404).json({ errors }); }
I got the same error using express and mongoose with HBS template engine. I went to Expressjs and read the docs for res.render, and it says // if a callback is specified, the rendered HTML string has to be sent explicitly. So I wasnt originally sending my html explicitly in the callback,. This is only for a contact form btw, not login info, albeit GET
//Original
let { username, email } = req.query; //My get query data easier to read
res.status(200).render('index', { username, email });
//Solution without error. Second param sending data to views, Third param callback
res.status(200).render('index', { username, email }, (err, html)=>{
res.send(html);
});
In react, if your are calling the function in useEffect hook, make sure to add a dependency to the dependency Array.
I had this error from an if statement not having an else block.
if(someCondition) {
await () => { }
}
await () => { }
I changed the above to this below and solved my issue
if(someCondition) {
await () => { }
} else {
await () => { }
}
For me, I accidentally put a res.status inside of a for loop. So my server would trigger the error the second time a res.status was returned. I needed to put the res.status outside of the for loop so it would only trigger once within the function.
First of all : make sure you didn't miss any asynchronous action without an async/await or use promises/callbacks.
Then attach any res with the return keyword : return res.status(..).json({});
And finally which was my problem: don't use return res.sendStatus if you always have some return res... inside a callback function, but you can always do a retun res.status();
in my case it was :
users.save((err,savedDoc){
if(err) return res.status(501).json({})
res.status(200).json({});
});
return res.status(500); // instead ofdoing return res.sendStatus(500)
you have to enable Promises in your programm, in my case i enabled it in my mongoose schema by using mongoose.Promise = global.Promise .
This enables using native js promises.
other alternatives to this soloution is :
var mongoose = require('mongoose');
// set Promise provider to bluebird
mongoose.Promise = require('bluebird');
and
// q
mongoose.Promise = require('q').Promise;
but you need to install these packages first.
My problem besides not returning, i was forgetting to await an asynchronous function in the handler. So handler was returning and after a bit the async function did its thing. 🤦🏻‍♀️
Before:
req.session.set('x', {...});
req.session.save();
return req.status(200).end();
When i needed to await:
req.session.set('x', {...});
await req.session.save();
return req.status(200).end();
I'm putting this here for anyone else who has the same problem as me- this happened to me because I'm using the next() function without a return preceding it. Just like a lot of the other answers state, not using return with your response will / can cause / allow other code in the function to execute. In my case, I had this:
app.get("/customerDetails", async (req, res, next) => {
// check that our custom header from the app is present
if (req.get('App-Source') !== 'A Customer Header') next();
var customerID = req.query.CustomerID
var rows = await get_customer_details(customerID);
return res.json(rows);
});
In my case, I forgot to include the header in my request, so the conditional statement failed and next() was called. Another middleware function must have then been executed. After the middleware finishes, without a return, the rest of the code in the original middleware function is then executed. So I simply added a return before my next() call:
// serve customer details payload
app.get("/customerDetails", async (req, res, next) => {
// check that our custom header from the app is present
if (req.get('App-Source') !== 'A Customer Header') return next();
var customerID = req.query.CustomerID
var rows = await get_customer_details(customerID);
return res.json(rows);
});

How do I display my catch block message when there is no ID match?

I am getting data from an API and am displaying it on my local server.
Below is my code to get data which matches the ID from the API data:
router.get('/:id', async (req, res) => {
checkString(req.params.id)
try {
const person = await peopleData.getPersonById(req.params.id);
res.json(person);
} catch (e) {
res.status(404).json({ message: 'There is no person with that ID' });
}
If there is no match I want to display the message like in the catch block, but the code does not go there as not getting a match is not an error technically.
So I tried the below code to get this message:
router.get('/:id', async (req, res) => {
checkString(req.params.id)
try {
const person = await peopleData.getPersonById(req.params.id);
if(!person) res.json('There is no person with that ID'); // Added new line here
res.json(person);
} catch (e) {
res.status(404).json({ message: 'There is no person with that ID' });
}
This does the work but it prints the message with quotes around as a string, is there a way I can display the message in the catch block if no match is found?
You can throw an error and the catch will display it.
if(!person) throw new Error("There is no person with that ID");
....
then in the catch...
catch(e){
res.status(404).json({ message: e.message })
}
If you're sending people to a fullscreen "error stack" page, then you may not need to use res.json()! You can also use res.send()
if(!person){ res.send('<p>There is no person with that ID</p>'; return; }
// Or
if(!person){ res.send('There is no person with that ID'; return; }
You are returning Json responses, so it looks like your consumer is not a web page but another app. If so, you should return undefined or null if there is no person found, and let the web page or consumer decide what message to show. Reasons are:
It should be easier to modify web pages than code, and typically the UI or marketing people will always want to fine tune (usually many times) every message on a web page.
Your app is an API app. The place where the user not found message is to be shown can be many steps away. Or it may be inappropriate to show the message at all, for example the consuming app might want to redirect to/show a registration page instead if user is not found.
Your web site may be multi-lingual, and you don't want the back-end to be involved in this.
"User not found" in many situations is not really an error, but it all depends on your application.
The catch block in your case should be used to handle other errors, for example, your database server might be down, or the database request might have timed out, etc etc. Your current code will misleadingly show "user not found" if there is a database error!
I would also let the Express error handler take care of such real errors, instead of coding error handling for every API function you have:
router.get('/:id', async (req, res, next) => {
checkString(req.params.id);
try {
const person = await peopleData.getPersonById(req.params.id);
res.json(person); // assuming getPersonById returns null if user not found
} catch (e) {
next(e);
});
Your Express error handler, where the invocation of the above next function lands, should be something like this (asssuming router is your Express app):
router.use((err, req, res, next) => {
let statusCode = err.status || 500;
// Assuming your app need to return only json responses
res.json(err);
});

Node.js - Serving error 500 pages on unexpected errors

I'm trying to serve 500 pages (some generic HTML that says "500 - internal server error") from my Node.js server to requests that failed to resolve due to developer bugs, but can't find an elegant way to do this.
Lets say we have the following index.js, where a developer innocently made a mistake:
const http = require('http');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
var a = null;
var b = a.c; // this is the mistake
res.end('status: 200');
}
Trying to access property "c" of null throws an error, so "res.end" will never be reached. As a result, the requesting client will eventually get a timeout. Ideally, I my server to have code that can catch errors like this, and return 500 pages to the requesting client (as well as email an administrator and so on).
Using "try catch" in every single block is out of the question. Most Node.js code is async, and a lot of the code relies on external libraries with questionable error handling. Even if I use try-catch everywhere, there's a chance that an error would happen in an external library that didn't have a try-catch block inside of it, in a function that happens asynchronously, and thus my server will crash and the client would never get a response.
Shortest example I can provide:
/* my server's index.js */
const http = require('http');
const poorlyTestedNpmModule = require('some-npm-module');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
try {
poorlyTestedNpmModule(null, onResult);
}
catch(err) {
res.end('status: 500');
}
function onResult(err, expectedResult) {
if(err) {
res.end('status: 400');
}
else {
res.end('status: 200');
}
}
}
/* some-npm-module.js */
module.exports = function poorlyTestedNpmModule(options, callback) {
setTimeout(afterSomething, 100);
function afterSomething() {
var someValue = options.key; // here's the problem
callback(null, someValue);
}
}
Here, the server crashes, due to a function call that led to code that asynchronously throws an error. This code is not code that I control or wish to modify; I want my server to be able to handle all those errors on its own.
Now, I could, for instance, just use the global uncaughtException event, i.e.:
process.on('uncaughtException', doSomething);
but then I have no access to the (req, res) arguments, making it impossible to call res.end for the correct res instance; the only way to have access to them, is to store them in a higher-scope object for each incoming request, and then prune them on successful request resolutions, then mark existing [req, res] stored pairs as "potentially errored" whenever an uncaughtException triggers, and serve 500 pages to those requests whenever the count of currently-active requests matches the count of currently-unresolved-errors (and re-test that count per thrown uncaught expection and per successful res.end call).
Doing that works, but... it's ugly as hell. It means that request objects have to be leaked to the global scope, and it also means that my router module now has a dependency on the uncaughtException global event, and if any other code overwrites that event, everything breaks, or if I ever want to handle other uncaught exceptions for whatever reason, I'll run into cross dependency hell.
The root cause of this problem is that an unexpected error can happen anywhere, but I want to specifically catch whether an unexpected error originated from a stack trace that began from an incoming http request (and not, for example, from some interval I have running in the background, because then I get an unexpected error but obviously don't want to serve a 500 page to anyone, only email an admin with an error log), and on top of needing to know whether the error originated from an http request, I need to have access to the request+response objects that node server objects provide.
Is there no better way?
[Edit] The topic of this question is role distribution in modules.
i.e., one guy is making base code for a server, lets say a "router module". Other people will add new code to the server in the future, handling branches that are routed to.
The guy that writes the base server code has to write it in a way that it will serve 500 pages if any future code is written incorrectly and throws errors. Help him accomplish his goal.
Answers of the format "make sure all future people that add code never make mistakes and always write code that won't throw uncaught errors" will not be accepted.
At first, using uncaughtException in Nodejs is not safe. If you feel that there is no other option in your application, make sure that you exit the process in the handler of 'uncaughtException' and restart the process using pm2 or forever or someother modules. Below link can provide you its reference.
Catch all uncaughtException for Node js app
Coming to the process of error handling, as mentioned, you may always miss to handle errors with callback. To avoid, these we can use an exceptional advantage of promises in nodejs.
/* my server's index.js */
const http = require('http');
const poorlyTestedNpmModule = require('some-npm-module');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
try {
poorlyTestedNpmModule(null)
.then(result => {
res.end('status: 200');
})
.catch(err =>{
console.log('err is', err);
res.end('status: 400');
})
}
catch(err) {
res.end('status: 500');
}
}
/* some-npm-module.js */
module.exports = function poorlyTestedNpmModule(options, callback) {
setTimeout(afterSomething, 100);
afterSomthing = new Promise((resolve, reject)=> {
var someValue = options.key; // here's the problem
resolve(someValue);
})
}
If you see that some of the npm nodemodules are not present with promise, try to write wrappers to convert callback to promise model and use them in your application.

Node.js and https certificate error handling

To make a long story short:
I'm building node app which making a request with https (the secure version of http). Whenever I miss-configure my request options, I'm having this error:
Node.js Hostname/IP doesn't match certificate's altnames
Great... except of the fact that the entire request code is wrapped with a valid try..catch block (which works just fine.. checked that already). The code is basically something like this:
try
{
https.request(options, (response) =>
{
// no way I making it so far this that error
}).end();
}
catch(ex)
{
// for some reason.. I'm not able to get here either
}
What I intend to do is to simply handle that error within my try..catch block
After reading some posts I've learned that this behavior is mainly because the tls module is automatically process the request and therefore making this error - this is a nice piece of information but it doesn't really help me to handle the exception.
Some other suggested to use this option:
rejectUnauthorized: false // BEWARE: security hazard!
But I rather not... so.. I guess my questions are:
Handling an error with a try..catch block should work here..right?
If not - is this behavior is by-design in node?
Can I wrap the code in any other way to handle this error?
Just to be clear - I'm not using any third-party lib (so there is no one to blame)
Any kind of help will be appreciated
Thanks
You need to add an 'error' event handler on the request object returned by https.request() to handle that kind of error. For example:
var req = https.request(options, (response) => {
// ...
});
req.on('error', (err) => {
console.log('request error', err);
});
req.end();
See this section in the node.js documentation about errors for more information.

Redirecting a response to an HTTP 500 error handler, when an error is thrown

If some underlying component throws an error, how do I get to inform the express.js stack about it? I am using an underlying data access framework which occasionally throws specific errors (for instance, when a record is not found in the DB).
Does express.js offer a way to jump straight to a 500 error handler, and redirect the response to a corresponding page? Unfortunately, I do not see a direct way to use the standard express.js way (using a next middleware handler) unless maybe if I use try...catch everywhere, which is also a bit of overhead
Usually you don't redirect to a 500 page. What you do is let the exception propagate and node.js will do a 500 for you if an exception was thrown.
I have some related rules I follow. Maybe you'll find them useful:
Never use try/catch in async code (like node.js). Errors are usually propagated through the first parameter of the supplied callback.
If I need to return an error to the user, I do it using res.send('Error', 500);
Something for those who are still looking for an answer like me:
1- Use express default errorHandler middleware
app.use(express.errorHandler());
Or
2- Supply your own error handling middleware:
app.use(function (error, req, res, next) {
res.status(500);
res.render("500.jade", {});
});
Ideally you would want to use the first option on DEV and the second one on PROD
// development only
if ('development' == app.get('env')) {
app.use(express.errorHandler());
}
app.use((function (error, req, res, next) {
res.status(500);
res.render("500.jade", {});
});
Note: the error handler should be strategically placed below the app.router

Categories