I'm trying to serve 500 pages (some generic HTML that says "500 - internal server error") from my Node.js server to requests that failed to resolve due to developer bugs, but can't find an elegant way to do this.
Lets say we have the following index.js, where a developer innocently made a mistake:
const http = require('http');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
var a = null;
var b = a.c; // this is the mistake
res.end('status: 200');
}
Trying to access property "c" of null throws an error, so "res.end" will never be reached. As a result, the requesting client will eventually get a timeout. Ideally, I my server to have code that can catch errors like this, and return 500 pages to the requesting client (as well as email an administrator and so on).
Using "try catch" in every single block is out of the question. Most Node.js code is async, and a lot of the code relies on external libraries with questionable error handling. Even if I use try-catch everywhere, there's a chance that an error would happen in an external library that didn't have a try-catch block inside of it, in a function that happens asynchronously, and thus my server will crash and the client would never get a response.
Shortest example I can provide:
/* my server's index.js */
const http = require('http');
const poorlyTestedNpmModule = require('some-npm-module');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
try {
poorlyTestedNpmModule(null, onResult);
}
catch(err) {
res.end('status: 500');
}
function onResult(err, expectedResult) {
if(err) {
res.end('status: 400');
}
else {
res.end('status: 200');
}
}
}
/* some-npm-module.js */
module.exports = function poorlyTestedNpmModule(options, callback) {
setTimeout(afterSomething, 100);
function afterSomething() {
var someValue = options.key; // here's the problem
callback(null, someValue);
}
}
Here, the server crashes, due to a function call that led to code that asynchronously throws an error. This code is not code that I control or wish to modify; I want my server to be able to handle all those errors on its own.
Now, I could, for instance, just use the global uncaughtException event, i.e.:
process.on('uncaughtException', doSomething);
but then I have no access to the (req, res) arguments, making it impossible to call res.end for the correct res instance; the only way to have access to them, is to store them in a higher-scope object for each incoming request, and then prune them on successful request resolutions, then mark existing [req, res] stored pairs as "potentially errored" whenever an uncaughtException triggers, and serve 500 pages to those requests whenever the count of currently-active requests matches the count of currently-unresolved-errors (and re-test that count per thrown uncaught expection and per successful res.end call).
Doing that works, but... it's ugly as hell. It means that request objects have to be leaked to the global scope, and it also means that my router module now has a dependency on the uncaughtException global event, and if any other code overwrites that event, everything breaks, or if I ever want to handle other uncaught exceptions for whatever reason, I'll run into cross dependency hell.
The root cause of this problem is that an unexpected error can happen anywhere, but I want to specifically catch whether an unexpected error originated from a stack trace that began from an incoming http request (and not, for example, from some interval I have running in the background, because then I get an unexpected error but obviously don't want to serve a 500 page to anyone, only email an admin with an error log), and on top of needing to know whether the error originated from an http request, I need to have access to the request+response objects that node server objects provide.
Is there no better way?
[Edit] The topic of this question is role distribution in modules.
i.e., one guy is making base code for a server, lets say a "router module". Other people will add new code to the server in the future, handling branches that are routed to.
The guy that writes the base server code has to write it in a way that it will serve 500 pages if any future code is written incorrectly and throws errors. Help him accomplish his goal.
Answers of the format "make sure all future people that add code never make mistakes and always write code that won't throw uncaught errors" will not be accepted.
At first, using uncaughtException in Nodejs is not safe. If you feel that there is no other option in your application, make sure that you exit the process in the handler of 'uncaughtException' and restart the process using pm2 or forever or someother modules. Below link can provide you its reference.
Catch all uncaughtException for Node js app
Coming to the process of error handling, as mentioned, you may always miss to handle errors with callback. To avoid, these we can use an exceptional advantage of promises in nodejs.
/* my server's index.js */
const http = require('http');
const poorlyTestedNpmModule = require('some-npm-module');
const port = 12345;
http.createServer(onHttpRequest).listen(port);
function onHttpRequest(req, res) {
try {
poorlyTestedNpmModule(null)
.then(result => {
res.end('status: 200');
})
.catch(err =>{
console.log('err is', err);
res.end('status: 400');
})
}
catch(err) {
res.end('status: 500');
}
}
/* some-npm-module.js */
module.exports = function poorlyTestedNpmModule(options, callback) {
setTimeout(afterSomething, 100);
afterSomthing = new Promise((resolve, reject)=> {
var someValue = options.key; // here's the problem
resolve(someValue);
})
}
If you see that some of the npm nodemodules are not present with promise, try to write wrappers to convert callback to promise model and use them in your application.
Related
I'm using postgres with knex javascript library to build my sql queries. I want to handle all thrown errors from postgres server, so the way I want to do this is by checking the type of the thrown error.
try {
// knex query here
} catch(error) {
if(error instanceof DatabaseError) {
// handle pg error by checking error.code or something else
// then send an custom error message to the client
}
// handle another error here which is not caused by the postgres server
}
Is there any way to handle the error like this ?
Catching Knex/DB Errors:
You can use the async/await syntax:
async function() {
try {
await knex(/* knex query here*/)
} catch(error) {
if(error instanceof DatabaseError) {
// handle pg error by checking error.code or something else
// then send an custom error message to the client
}Sorry
// handle another error here which is not caused by the postgres server
}
If for some reason you don't want to use that (newer) syntax, you can also chain a .catch ...
knex(/* knex query here*/)
.then(doWhatever)
.catch(error => {
if(error instanceof DatabaseError) { // ...
});
Alternative you can also use Knex's query-error (https://knexjs.org/#Interfaces-query-error) ... but personally I've never seen the point when the built-in promise handlers work just fine.
(EDIT) Distinguishing PG Errors From Knex Ones:
If you want to differentiate Knex and PG-specific errors, you can hook up an error-handler to your PG connection directly (bypassing Knex) like so:
function afterCreate(connection, callback) {
connection.on("error", connectionError);
callback(null, connection);
}
db.client.pool.config.afterCreate = afterCreate;
If you do that you won't need a error instanceof DatabaseError, because all errors caught by the connection.on will be PG errors.
You might also find this issue thread (where I got that code from) useful: https://github.com/knex/knex/issues/522, as it features a discussion of error handling in Knex, and specifically handling of underlying DB errors.
(EDIT) But How Can I Distinguish Errors When I Catch Them?
Unfortunately I don't think PG errors don't have a unique prototype (ie. class) or other distinguishing feature. You can see this by looking at an example one (from that thread I linked):
{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":5432}
As you can see, there's no way to look that and know "that's coming from PostgreSQL", unless you start checking for specific features like code === 'ECONNREFUSED' (there is no isPg: true flag on the error).
And why doesn't Knex just intelligently identify DB errors for us? From that same issue thread:
It is pretty much impossible to create events like redis have for knex because of multitude of different db drivers which doesn't actually support listening for connection errors
-elhigu (Knex team member)
There are various posts dealing with the general issue of timeouts when using http.get(), but non of them seems to address the question of how to deal with timeouts that occur during the stream itself, after a successful response was already received.
Take this code for example. It sends a request to some server, that responds on time, but creates an artificial timeout during the stream:
(async()=>{
//Send request to a dummy server, that creates a timeout, IN THE MIDDLE OF THE STREAM.
const request =http.get('http://localhost/timeout/',async (res)=>{
const write = fs.createWriteStream('.text.txt');
try {
await pipelinePromisified(res,write);
console.log('Everything went fine')//Being that the timeout error during the stream is not caught by pipeline,
//the promise gets resolved..
} catch (error) {
//The error is NOT caught by pipeline!
console.log('Error from pipeline',error)
}
}).on('error',(e)=>{
//Error is caught here
console.log('error from request on error')
})
request.setTimeout(4000,()=>{
request.destroy(new Error('request timed out'));
//This causes the piping of the streams to stop in the middle(resulting a partial file), but the pipeline doesn't treat this is an error.
})
})()
Note the key issue: The timeout during the stream is recognized, the request is destroyed, the IncomingMessage(response) stops pumping data- but pipeline doesn't recognize it as an error.
The outcome is, that the client code is not aware of the fact that file was partially downloaded, being that no error is thrown.
How to handle this situation? In my testing, calling response.emit('error') seems to solve this, but Node's docs clearly state not to do this.
Any help would be greatly appreciated.
Update: It seems that on Node 12(i have 10 and 12 installed via nvm), an error is caught by pipeline, but not in Node 10.
I'm using Firebase-Firestore on Javascript (web) with a Progressive web app. I ran into this error:
INTERNAL ASSERTION FAILED: Got result for empty write pipeline
Because Firebase runs asynchronously with XHR requests, it was difficult to determine the exact source of the error - it seemed like any onSnapshot, set or update was throwing this error for me.
And after that first error came a flurry of other errors:
INTERNAL ASSERTION FAILED: AsyncQueue is already failed: Error: FIRESTORE (5.3.0) INTERNAL ASSERTION FAILED: Got result for empty write pipeline
I thought my operation was pretty normal - just using the API set(), update() , onSnapshot() functions when it happened.
It's not a mission critical error - the code runs fine, but I'm hit with a couple thousand errors when I open debug, so it's prohibitive in that regard.
For my PWA I was using a cache-first, web-reupdate model which returns cachedResponse but also fetch()es the response and caches the fetched response.
Anyone have any insights?
It was the PWA! Using the PWA, I was catching all GET requests, including Firebase's own GET's. Filtering to ensure CORS requests don't return from cache fixed the problem.
To solve this, I added this code to my PWA:
self.addEventListener("fetch", event => {
if (event.request.method == "GET") {
event.respondWith(
(async function() {
const cachedResponse = await cache.match(event.request, {
ignoreSearch: true
});
// Returned the cached response if we have one, otherwise return the network response.
if (cachedResponse && event.request.type!="cors") {
//AVOID CORS FOR THINGS LIKE FIREBASE
updateCache(event);
return cachedResponse;
} else return await updateCache(event);
})()
);
} else {
event.respondWith(fetch(event.request));
}
});
If you're new to the PWA space, want to get a jump start to ANY PWA project, or want to just 'share notes', the repo with the full comprehensive PWA file is here: https://github.com/acenturyandabit/genUI/blob/master/Javascript/pwa.js
I've personally put a lot of time into this so I hope it helps :)
To make a long story short:
I'm building node app which making a request with https (the secure version of http). Whenever I miss-configure my request options, I'm having this error:
Node.js Hostname/IP doesn't match certificate's altnames
Great... except of the fact that the entire request code is wrapped with a valid try..catch block (which works just fine.. checked that already). The code is basically something like this:
try
{
https.request(options, (response) =>
{
// no way I making it so far this that error
}).end();
}
catch(ex)
{
// for some reason.. I'm not able to get here either
}
What I intend to do is to simply handle that error within my try..catch block
After reading some posts I've learned that this behavior is mainly because the tls module is automatically process the request and therefore making this error - this is a nice piece of information but it doesn't really help me to handle the exception.
Some other suggested to use this option:
rejectUnauthorized: false // BEWARE: security hazard!
But I rather not... so.. I guess my questions are:
Handling an error with a try..catch block should work here..right?
If not - is this behavior is by-design in node?
Can I wrap the code in any other way to handle this error?
Just to be clear - I'm not using any third-party lib (so there is no one to blame)
Any kind of help will be appreciated
Thanks
You need to add an 'error' event handler on the request object returned by https.request() to handle that kind of error. For example:
var req = https.request(options, (response) => {
// ...
});
req.on('error', (err) => {
console.log('request error', err);
});
req.end();
See this section in the node.js documentation about errors for more information.
I have a NodeJS/Express web application that allows the user to upload a file, which I then parse using connect-busboy save to my database using Sequelize. Once that's done, I want to redirect the user to a given page. But Express is returning a status of 404 before my Promise resolves, even though I'm never calling next(), which I thought was mandatory in order to call the next handler in the middleware chain and thus result in a 404.
This is my code so far:
function uploadFormFile(req, res, next) {
var documentInstanceID = req.params.documentInstanceID;
// set up an object to hold my data
var data = {
file: null,
documentDate: null,
mimeType: null
};
// call the busboy middleware explicitly
// EDIT: this turned out to be the problem... of course this calls next()
// removing this line and moving it to an app.use() made everything work as expected
busboy(req, res, next);
req.pipe(req.busboy);
req.busboy.on('file', function (fieldName, file, fileName, encoding, mimeType) {
var fileData = [];
data.mimeType = mimeType;
file.on('data', function (chunk) {
fileData.push(chunk);
});
file.on('end', function () {
data.file = Buffer.concat(fileData);
});
});
req.busboy.on('finish', function () {
// api methods return promises from Sequelize
api.querySingle('DocumentInstance', ['Definition'], null, { DocumentInstanceID: documentInstanceID })
.then(function (documentInstance) {
documentInstance.RawFileData = data.file;
documentInstance.FileMimeType = data.mimeType;
// chaining promise
return api.save(documentInstance);
}).then(function () {
res.redirect('/app/page');
});
});
}
I can confirm that my data is being persisted correctly. But due to the race condition, the web page says 'can't POST' due to the 404 status being returned by Express, and the res.redirect is failing with an error setting the headers because it's trying to redirect after the 404 has been sent.
Can anyone help me figure out why Express is returning the 404?
The problem is coming from your internal call to busboy inside your handler. Rather than it executing and simply returning control to your handler, it would be calling the next which is passed to it before it returns control. So you code after the busboy call does execute, but the request has already advanced past that point.
In cases in which you want some middleware to only be executed for certain requests, you can chain middleware into those requests, such as:
router.post('/upload',busboy,uploadFromFile)
You can also separate them with .use() such as:
router.use('/upload', busboy);
router.post('/upload', uploadFromFile);
Either of the above will chain the middleware in the way you intended. In the case of .use() the middleware would also be applied to any applicable .METHOD() as Express refers to it in their documentation.
Also, note that you can pass in an arbitrary number of middleware this way, either as separate parameters or as arrays of middleware functions, such as:
router.post('/example', preflightCheck, logSomeStuff, theMainHandler);
// or
router.post('example', [ preflightCheck,logSomeStuff ], theMainHandler);
The execution behavior of either of the above examples will be equivalent. Speaking only for myself and not suggesting it is a best practice, I normally only use the array-based addition of middleware if i am building the middleware list at runtime.
Good luck with it. I hope you enjoy using Express as much as I have.