Node.js - Domain per Express request, inside another domain - javascript

Error handling in Node. Argh!
I'm trying to layout a basic Node app like this...
Cluster -> Worker -> Server Domain -> Express Request Domain
So, if an error is thrown 18 layers deep into a call stack because someone misspelled their name on a login form, the entire server doesn't crash.
Here's some basic code to simulate the worker part:
var domain, server;
domain = require('domain');
server = domain.create();
server.on('error', function(e) {
console.log('total meltdown...', e.stack);
});
server.run(function() {
var express = require('express')();
express.configure(function() {
// Domain on EVERY request
express.use(function(req, res, next) {
var d = domain.create();
d.on('error', function(e) {
console.log('fired REQUEST error', e.stack);
next(e);
});
d.run(next);
});
// Generic error handler
express.use(function(e, req, res, next) {
res.status(500);
res.end('oops');
});
// Serve the request with a blatent error
express.get('/', function(req, res) {
this_function_does_not_exist();
res.end('we will never get here');
});
});
// Fire 'er up
express.listen(3000);
});
What I'm expecting...
I curl http://localhost:3000/, get a nice little 'oops' error, and see 'fired REQUEST error' and the error stack in the console.
What actually happens...
I get this both as the browser response, and in the console...
ReferenceError: this_function_does_not_exist is not defined
at /Stuff/test.js:38:13
at callbacks (/Stuff/node_modules/express/lib/router/index.js:161:37)
at param (/Stuff/node_modules/express/lib/router/index.js:135:11)
at pass (/Stuff/node_modules/express/lib/router/index.js:142:5)
at Router._dispatch (/Stuff/node_modules/express/lib/router/index.js:170:5)
at Object.router (/Stuff/node_modules/express/lib/router/index.js:33:10)
at next (/Stuff/node_modules/express/node_modules/connect/lib/proto.js:190:15)
at next (/Stuff/node_modules/express/node_modules/connect/lib/proto.js:192:9)
at b (domain.js:183:18)
at Domain.run (domain.js:123:23)
Now why would it go and do something like that?

Ok, solved - Express has a try/catch block, which is getting to my non-existent function call first.
To have the domain catch it, it needs to be taken out of the current call stack, like...
process.nextTick(function() {
this_function_does_not_exist();
res.end('we will never get here');
});
THEN the domain will grab it.

Related

Can't set headers after they are sent, Redirecting from SendFile

Scenario is:
I want to load a webpage (it loads with sendFile) then I want to create a setTimeout timer and redirect the user to another webpage with sendFile also, but I'm getting the headers error.
I've tried a CORS approach like said in other answers (including all headers before each request with app.use)
I've tried to res.end() but it won't just load the first website.
App use
app.use((req, res, next) => {
res.append('Access-Control-Allow-Origin','*');
res.append('Access-Control-Allow-Methods','GET,POST,PUT');
res.append('Access-Control-Allow-Headers','Content-Type');
next();
})
Redirect of first webpage show.
res.redirect(`/pantallas/reloj?tiempo=${Response.rows[0].Tiempo_refresco}&next=${Response.rows[1].Ruta_pantalla}`)
res.end()
Route to show first webpage and then redirecting to second one.
// Being tiempo = 5 and next = '/pantallas/barras'
app.get('/pantallas/reloj', (req, res) => {
res.sendFile(path.join(__dirname + '/pantallas/reloj.html'));
if(req.query.tiempo)
{
setTimeout(() => {
res.sendFile(path.join(__dirname + req.query.next));
}, req.query.tiempo * 1000);
}
})
At this point it loads the first website then after the timer is done it just throws the headers error, any help with it?
This is happening because you are sending the response twice. Hence, the error:
Can't set headers after they are sent.
You cannot return a response twice from the same route, you are sending a file first via the response and then once again inside the set timeout. This is not permitted.
Why is this happening?
The res object in Express is a subclass of Node.js's
http.ServerResponse (read the http.js source). You are allowed to call
res.setHeader(name, value) as often as you want until you call
res.writeHead(statusCode). After writeHead, the headers are baked in
and you can only call res.write(data), and finally res.end(data).
The error "Error: Can't set headers after they are sent." means that you're already in the Body or Finished state, but some function tried to set a header or statusCode. When you see this error, try to look for anything that tries to send a header after some of the body has already been written. For example, look for callbacks that are accidentally called twice, or any error that happens after the body is sent.
EDIT:
You can try something along these lines:
app.use(function(req, res, next) {
if (req.query.tiempo) {
setTimeout(() => {
res.sendFile(path.join(__dirname + req.query.next));
}, req.query.tiempo * 1000);
}
next();
});
app.get('/pantallas/reloj', function(req, res, next) {
res.sendFile(path.join(__dirname + '/pantallas/reloj.html'));
next();
});
More here: Express sendfile and redirect url

Express JS/ MongoDB Post not working

I started working on a MERN App today and am trying to write a restful api. First I am using mlab to store my mongodb database. I have succesfully connected to this database after creating a user. I can manually create a collection and inject some data into this collection. From my server.js file I can then get the data stored in here.
MongoClient.connect(db_url, (err, database) => {
if (err) return console.log(err);
var collection = database.collection('memories'); // Collection called memories
app.listen(3000, () => {
console.log("Listening on 3000");
});
});
Thats all fine and dandy but I want to take it to the next level. I want to write a CRUD api for the collection Memory. Coming from django, I would like to create my model first. Therefore, in my models/memory.js:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var MemorySchema = new Schema({
name: String,
description: String
});
module.exports = mongoose.model('Memory', MemorySchema);
Then I went ahead and started working on my routes/api/api.js:
let router = require('express').Router();
let Memory = require('../../../models/memories');
router.use(function (req, res, next) {
console.log("Something is happening");
next(); // Request stops at middleware without next()
});
router.route('/memory')
.post(function (req, res) {
let memory = new Memory();
memory.name = req.body.name;
memory.description = req.body.description;
memory.save(function (err) {
if (err) {
res.send(err);
}
res.json({message: 'Memory Created'});
});
})
.get(function (req, res) {
res.json({message: 'First memory'});
});
module.exports = router;
And in my server.js I call this module:
const apiRoutes = require('./routes/api/api');
app.use('/api/', apiRoutes);
However, after testing the post api with postman, it the POST request just takes forever before showing up as Could not get any response. However, the GET request works. What am I missing?
EDIT: So the post function is having trouble saving the model instance...
Try adding results as the first parameter in the callback of the save function, then res.json(results, { message: "Memory Created" }) to see if you are returned anything.
The main difference between the post and the get method is that the post method uses Mongoose, while the get doesn't. If you fail to connect to the database then the response can time out due to memory.save(...) not working as it should. And there are no responses sent outside the callback to save, so if your program never enter it, you will never send a response. The request will time out eventually.
In your model file you register a model on the following line:
module.exports = mongoose.model('Memory', MemorySchema);
Mongoose will then look for data in the memorys collection. If you change it to
module.exports = mongoose.model('Memory', MemorySchema, 'memories');
it will use the memories collection instead. This will make it consistent with the connection-to-db snippet you posted. I don't know if that will fix your issue though. I would suggest changing the connection code to
mongoose.connect(dburl, {
useMongoClient: true
});
instead of the native mongo client. You can add these lines too
mongoose.connection.on('connected', function () {
console.log('Mongoose connected');
});
mongoose.connection.on('error', function (err) {
console.log('Mongoose connection error: ' + err);
});
mongoose.connection.on('disconnected', function () {
console.log('Mongoose disconnected');
});
right after the connection code to help with debugging. Make sure you get connected when starting the app.
If you see an error similar to this Error: Can't set headers after they are sent. in the node terminal window, it might be because you are sending two responses in the post function. If an error occurs while saving it will enter the if(err) block, send a response async then go to the res.json(...) response and send that too.
So you have to return after sending the response to exit the function. Either like this
res.send(err);
return;
or like this
return res.send(err);
Same for the json response.
If that doesn't fix the problem you should either fire up the debugger (node --inspect or nodemon --inspect), or insert a console.log('inside post'); inside the post function to see that you're actually entering it.

Nodejs Socket hang up & ECONNRESET - HTTP post request from Meteor to Node js server

I am using a node server to handle all my push notifications services like gcm and apn.
I have 2 different servers. One is running Meteor and another is running Node.JS to handle push notifications. (Both are different servers)
My main app runs on the Meteor server.
I make an HTTP post request to the node.js server to send my notifications.
Usually it works fine, but sometimes on the Meteor server I get this error whenever I call the node.js server:
socket hang up\n at Object.Future.wait (/home/myPc/.meteor/packages/meteor-tool/.1.1.10.ki0ccv++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:398:15)\n at Object.<anonymous> (packages/meteor/helpers.js:119:1)\n at Object.HTTP.call (packages/meteorhacks_kadira/lib/hijack/http.js:10:1)\n at Object.sendPushNotificationsMeteorServer (server/pushNotifications.js:249:1)\n at server/classes/pushNotifications.js:244:1\n at [object Object]._.extend.withValue (packages/meteor/dynamics_nodejs.js:56:1)\n at packages/meteor/timers.js:6:1\n at runWithEnvironment (packages/meteor/dynamics_nodejs.js:110:1)\n - - - - -\n at createHangUpError (http.js:1473:15)\n at Socket.socketOnEnd [as onend] (http.js:1569:23)\n at Socket.g (events.js:180:16)\n at Socket.emit (events.js:117:20)\n at _stream_readable.js:944:16\n at process._tickCallback (node.js:448:13)',
details: { [Error: socket hang up] stack: [Getter] },
data: { [Error: socket hang up] stack: [Getter] },
user: null,
userId: null,
toString: [Function] },
user: null,
userId: null,
toString: [Function] }
OR
Error: read ECONNRESET
at Object.Future.wait (/home/mbm/.meteor/packages/meteor-tool/.1.1.10.12ml1tp++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:398:15)
at Object.call (packages/meteor/helpers.js:119:1)
at Object.sendHttpCall (server/pushNotifications.js:249:1)
at server/pushNotifications.js:244:1
at [object Object]._.extend.withValue (packages/meteor/dynamics_nodejs.js:56:1)
at packages/meteor/timers.js:6:1
at runWithEnvironment (packages/meteor/dynamics_nodejs.js:110:1)
- - - - -
at errnoException (net.js:905:11)
at TCP.onread (net.js:559:19)
Here is my Node.JS server code:
realFs = require('fs');
var gracefulFs = require('graceful-fs');
gracefulFs.gracefulify(realFs);
var http = require('http');
var express = require('express');
var app = express();
var path = require("path");
configClass = require('./classes/config.js').configClass;
helperClass = require('./classes/helper.js').helperClass;
pushNotificationClass = require('./classes/pushNotification.js').pushNotificationClass;
var hostname = 'http://localhost';
var port = 6000;
var bodyParser = require('body-parser');
nodeGcm = require('node-gcm');
apn = require('apn');
apnService = new apn.Connection(helperClass.getAPNOptions());
// -- BODY PARSER -- //
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true}));
process.on('uncaughtException', function (err) {
console.error(err);
console.log("Node NOT Exiting...");
});
// All post requests
app.post('/', function(req, res){
try {
var response = JSON.parse(req.body.pushNotificationApiParams);
var callType = req.body.callType;
switch (callType) {
case 'systemPushNotifications':
return pushNotificationClass.sendPushNotificationsV2(response);
break;
}
}
catch(e){
console.dir(e.stack);
realFs.appendFile('errorLogs/'+helperClass.getCurrentDateFormated()+'.log', helperClass.formatLog('Exception in main Post Method : '+e.stack) , function (err) {
if (err) throw err;
});
}
res.send("OK");
});
app.listen(port, function () {
console.log('Listening at '+hostname+':'+port);
});
And here is my code from Meteor side, where I am make HTTP post request to node js server:
var headers = {
'Content-Type' : 'application/x-www-form-urlencoded'
};
var postFields = {
callType : 'systemPushNotifications',
pushNotificationApiParams : JSON.stringify(pushNotificationApiParams) // contains push notifications data
};
HTTP.call("POST", 'http://localhost:6000', { params:postFields, headers:headers });
Can anyone guide me in the right direction? Also I would really appreciate to know some good practices as well.
Also
There is one more issue I am facing. My node.js server exits after like 24 hours. I don't know why does that happens. It exits without any error or exception in the terminal console. I have to restart it each time.
Considering the ECONNRESET error usually occurs when the *other side of the TCP connection is closed abruptly.
In case of your application
it may be due to the overloading of the server and simply kills the connection as a return which in the similar way blocks the connection to your meteor server
To get more info about the error as mentioned in this thread.
To handle the error you must use an event listener to it to show the full stack traces of it
As mentioned in this thread by Farid Nouri Neshat
To have one listener for a group of calls you can use domains and also catch other errors on runtime. Make sure each async operation related to http(Server/Client) is in different domain context comparing to the other parts of the code, the domain will automatically listen to the error events and will propagate it to it's own handler. So you only listen to that handler and get the error data.
but since the domain have already been deprecated you should use clusters as mentioned here in the docs which uses server.listen(message) and the server.listen(handle)
or you can also use NODE_DEBUG=net or use strace
Update
For the server disconnect i think the error might be in your handling of the bodyparser.For a bad json file the error is uncaught.
Node's default action following an uncaught exception is to exit(crash) on the process.
Handling of bodyparser for a json file can be done in the following way.
var parseJson = bodyPaser.json();
app.use(function (req, res, next) {
req.getBody = function (callback) {
parseJson(req, res,function (err) {
callback(err, req.body);
});
};
next();
});
Reference taken from the GITHUB open issue's here
Update 2
Basically the socket hangup means that the socket doesn't end the connection within the specified time period
According to the source you can see that it occurs if the server never sends the response
.This error should be caught and handled by either
retrying to the request.
handling it later by setting more time period or put res.end() at the end of your function to end the connection.
or you can use the [http.get()][8] with the get requests which will automatically call the req.end() function
Hope it might help you a bit! Cheers!
Ok I found the issue myself here. Its in the node server code. I put return in a switch statement which is not a valid way to return a response in express, so I just removed the return from:
Before:
switch (callType) {
case 'systemPushNotifications':
return pushNotificationClass.sendPushNotificationsV2(response);
break;
}
Now:
switch (callType) {
case 'systemPushNotifications':
pushNotificationClass.sendPushNotificationsV2(response);
break;
}
The above return was terminating the code before the: res.send("OK");

Sinon Mocha Node Error Handling

server.js
var server = http.createServer(function(req, res) {
lib.doSomething(x, y, function(err, data) {
if (err) throw(err);
res.writeHead(200, {
'Content-Type': 'text/plain'
});
res.end(data);
});
});
serverTest.js
var request = require('request');
var server = require('server');
it('throws error when doSomething returns err', function(done){
var expected = 'testError';
doSomething = sinon.stub(lib, 'doSomething', function(x, y, callback){
callback(new Error(expected));
});
try{
request(url, function(err, response, body){
done();
});
} catch(e){
expect(e).to.equal(expected);
};
});
I approached this unit test many ways but whenever I run the tests, I keep getting a Uncaught Error: testError and the test fails. I looked around online and found no luck. What would be the best approach to have this unit test passing with expected errors?
What you have here is a process that is talking to itself through a socket. server.js establishes a service that listens for requests on a socket and responds, and then your test code connects to that socket and makes requests. Now, the thing is that the call stack for the server side and the call stack for the client side are completely different stacks. If the server throws an exception, your client cannot catch it. In other words, your request call won't throw an exception because lib.doSomething's callback does. You could trivially check that the callback threw an exception but then that's not testing much.

how to stop node js server from crashing

I am new to node js. I was trying to create a simple HTTP server. I followed the famous example and created a 'Hello World!' server as follows.
var handleRequest = function(req, res) {
res.writeHead(200);
res1.end('Hello, World!\n');
};
require('http').createServer(handleRequest).listen(8080);
console.log('Server started on port 8080');
Running this code would start the server properly as expected. But trying to access http://127.0.0.1:8080 would crash it by throwing an error that res1 is not defined. I would like to have the server still continue running and gracefully report errors whenever it encounters it.
How do I achieve it? I tried try-catch but that isn't helping me :(
There are a bunch of comments here. First of all, for your example server to work, handleRequest needs to be defined BEFORE using it.
1- What you actually want, which is preventing the process to exit, can be handled by handling uncaughtException (documentation) event:
var handleRequest = function(req, res) {
res.writeHead(200);
res1.end('Hello, World!\n');
};
var server = require('http').createServer(handleRequest);
process.on('uncaughtException', function(ex) {
// do something with exception
});
server.listen(8080);
console.log('Server started on port 8080');
2- I would recomment to use try{} catch(e) {} on your code, such as:
var handleRequest = function(req, res) {
try {
res.writeHead(200);
res1.end('Hello, World!\n');
} catch(e) {
res.writeHead(200);
res.end('Boo');
}
};
3- I guess the example was just an example and not actual code, this is a parsing error that can be prevented. I mention this, since you NEED to NOT have parsing errors on Exception catch handlers.
4- Please note that node process is going to be replaced in the future with domain
5- I'd rather use a framework like express, than doing this stuff.
6- Recommended lecture: StackOverflow - NodeJS best practice for exception handling
I am not targeting your question details but your question's title about preventing Node server from crashing. You can probably use DOMAIN, this will probably stop your server from crashing when an uncaughtException is thrown.
domain = require('domain'),
d = domain.create();
d.on('error', function(err) {
console.error(err);
});
for more details go http://shapeshed.com/uncaught-exceptions-in-node/
beside using this method must try-catch your block.
Maybe you should define handleRequest before you use it:
require('http').createServer(handleRequest).listen(8080);
function handleRequest(req, res) {
res.writeHead(200);
res1.end('Hello, World!\n');
};
Or
var handleRequest = function(req, res) {
res.writeHead(200);
res1.end('Hello, World!\n');
};
require('http').createServer(handleRequest).listen(8080);
And you should be sure that res1 also exists.

Categories