We Keep Coding

How to modularize an express app - with a function, a class and req.pipe?

Here below there are two servers and two gqlServers. All combinations of them work.
The challenge is to extend express with some additional predefined code patterns shared across several apps, exposed through additional methods.
Which combination of a server and gqlServer is considered best practice and best for performance?
server:
server_A is a function that returns a class
server_B is a function that returns a function
gqlServer:
gqlServer_01 uses req.pipe
gqlServer_02 has the original express() passed into it
function gqlServer_01(options) {
let gqlApp = express();
gqlApp.use(options.route, function(req, res, next) {
res.send('gqlServer 01');
// next();
});
gqlApp.listen(8001, err => {
if (err) throw err;
console.log(`>> GQL Server running on 8001`);
});
}
function gqlServer_02(app, options) {
app.use(options.route, function(req, res, next) {
res.send('gqlServer 02');
// next();
});
}
// THIS SERVER ?
function server_A(config = {}) {
config = deepmerge(def_opt, config);
let app = express();
app.get('/', function(req, res, next) {
res.send('root');
// next();
});
class Server {
constructor(opt) {
this.opt = opt;
}
gql(props = {}) {
// THIS GQL SERVER ?
gqlServer_01({ route: '/gql-01' });
app.use('/gql-01', function(req, res) {
req.pipe(request(`http://localhost:8001/gql-01`)).pipe(res);
});
// OR THIS GQL SERVER ?
gqlServer_02(app, { route: '/gql-02' });
}
}
app.listen(8000, err => {
if (err) throw err;
console.log(`>> Server running on 8000`);
});
return new Server(app, config);
}
// OR THIS SERVER ?
function server_B(config = {}) {
config = deepmerge(def_opt, config);
let app = express();
app.get('/', function(req, res, next) {
res.send('root');
// next();
});
app.gql = function(props = {}) {
// THIS GQL SERVER ?
gqlServer_01({ route: '/gql-01' });
app.use('/gql-01', function(req, res) {
req.pipe(request(`http://localhost:8001/gql-01`)).pipe(res);
});
// OR THIS GQL SERVER ?
gqlServer_02(app, { route: '/gql-02' });
};
app.listen(8000, err => {
if (err) throw err;
console.log(`>> Server running on 8000`);
});
return app;
}
The goal is to have the best solution in order to create an npm package out of this and reuse the methods over several projects easily. The project was highly simplified for the sake of clarity.

I don't think you will have performance issues in any of these examples, so the question remains which of them is more modular.
If you are willing to make an npm package out of these, you shouldn't be calling express() inside your server code. Instead you should be passing the app as a parameter. This will allow you to reuse existing express apps initialized elsewhere. For this reason I would go for gqlServer_02
You also want to create a new server each time you call the module function, so I'd go with server_A for this reason. However it needs to receive the express app as parameter, in order to reuse existing express objects. I would also put the app.listen call inside a function in the Server class.

Error cannot be caught with Express.js and gridfs-stream

It's an easy image (picture) download server, Express.js receives the request, gets an image from MongoDB GridFS, and responds with the file.
It's OK when request is valid (when the requested file exists).
The problem is that I cannot catch the MongoError when the query failed (i.e. requested image does not exist).
import Grid from 'gridfs-stream'
const root = 'fs_images'
// This func returns the file stream
export function getReadStream (id) {
const gfs = Grid(mongoose.connection.db, mongoose.mongo)
const options = {
_id: id,
mode: 'r',
root: root
}
const readStream = gfs.createReadStream(options)
readStream.on('error', function (err) {
// throw here
// it performs the same without this on-error hook;
// if comment the `throw err`, nothing will happens
// but I want the caller knows the error
throw err
})
return readStream
}
And this is the router
router.get('/:fileId', function (req, res, next) {
const fileId = req.params.fileId
try {
const imgReadStream = image.getReadStream(fileId)
imgReadStream.pipe(res)
} catch (err) {
// nothing catched here
// instead, the process just crashed
console.log(err)
}
}
And I just cannot catch the err. When I try to request something that doesn't exist the MongoError shows in the console, and the app crashes with errno is 1.
Head of console output:
/.../node_modules/mongodb/lib/utils.js:123
process.nextTick(function() { throw err; });
^
MongoError: file with id 123456123456123456123456 not opened for writing
at Function.MongoError.create (/.../node_modules/mongodb-core/lib/error.js:31:11)
This may be a bit different. If somewhere else throws an Error it will be caught by my error-handler (app.use(function(err, req, res, next){ /* ... */})), or at least by the default handler from Express.js, and returns a 500, without the process crashing.
In short, I want the app to know and catch this MongoError so I can handle it manually (i.e. return a 404 response).

try/catch won't work because the error is happening in a different tick (asynchronously). Perhaps you could listen for the error in the router instead?
const imgReadStream = image.getReadStream(fileId)
imgReadStream.on('error', function(err) {
// Handle error
});
imgReadStream.pipe(res)

How to throw a 404 error in express.js?

In app.js, I have
// catch 404 and forward to error handler
app.use(function(req, res, next) {
var err = new Error('Not Found');
err.status = 404;
next(err);
});
so if I request some not exist url like http://localhost/notfound, above code will execute.
In exist url like http://localhost/posts/:postId, I would like to throw 404 error when access some not exist postId or deleted postId.
Posts.findOne({_id: req.params.id, deleted: false}).exec()
.then(function(post) {
if(!post) {
// How to throw a 404 error, so code can jump to above 404 catch?
}

In Express, a 404 isn't classed as an 'error', so to speak - the reasoning behind this is that a 404 isn't usually a sign that's something's gone wrong, it's just that the server couldn't find anything. Your best bet is to explicitly send a 404 in your route handler:
Posts.findOne({_id: req.params.id, deleted: false}).exec()
.then(function(post) {
if(!post) {
res.status(404).send("Not found.");
}
Or alternatively, if this feels like too much repeated code, you could always pull that code out into a function:
function notFound(res) {
res.status(404).send("Not found.");
}
Posts.findOne({_id: req.params.id, deleted: false}).exec()
.then(function(post) {
if(!post) {
notFound(res);
}
I wouldn't recommend using a middleware in this situation solely because I feel like it makes the code less clear - the 404 is the direct result of the database code not finding anything, so it makes sense to have the response in the route handler.

I have the same app.js structure, and I solved this problem in this way in the route handler:
router.get('/something/:postId', function(req, res, next){
// ...
if (!post){
next();
return;
}
res.send('Post exists!'); // display post somehow
});
The next() function will call the next middleware which is the error404 handler if it is right after your routes in the app.js.

You can use this and the end of your routers.
app.use('/', my_router);
....
app.use('/', my_router);
app.use(function(req, res, next) {
res.status(404).render('error/404.html');
});

you're probably looking for something like https://github.com/expressjs/api-error-handler
or just https://github.com/jshttp/http-errors

Even though 404 pages are not considered an error in Express as written here, its really damn handy if you DO handle them like so. For instance when you are developing an API that wants consistent JSON output. The following code should help you with that:
Define a helper function abort to create status errors that can be easily used in your code to pass to the next function:
// Use the `statuses` package which is also a dependency of Express.
const status = require('statuses');
const abort = (code) => {
const err = new Error(status[code]);
const err.status = code;
return err;
};
Define the catch-all middleware for 404 pages which should be defined at the bottom of your stack (after all routes have been added). This forwards the 404 as an error:
app.use((req, res, next) => {
next(abort(404));
});
Lastly, the final error handler will now consistently send all errors in JSON format:
app.use((err, req, res, next) => {
if(!res.headersSent) {
// You can define production mode here so that the stack trace will not be sent.
const isProd = false;
res.status(err.status || 500).json({
error: err.toString(),
...(!isProd && {stack: err.stack.split('\n').map(i => i.trim())}),
});
}
next(err);
});

Unable to receive upstream GCM messages with Node.js + XMPP

I'm new to node.js and XMPP but not Javascript or GCM. I'm unable to receive upstream messages using node-xmpp and none of the callbacks are called, not even error. I've looked through the other SO threads but none of the solutions have worked. Here is my entire route:
var express = require('express');
var router = express.Router();
var xmpp = require('node-xmpp');
router.get('/', function(req, res, next) {
var options = {
type: 'client',
jid: 'project-12345#gcm.googleapis.com',
password: 'apiKey12345',
port: 5235,
host: 'gcm.googleapis.com',
legacySSL: true,
preferredSaslMechanism : 'PLAIN'
};
// this prints correctly
console.log('Creating xmpp app');
var cl = new xmpp.Client(options);
cl.connection.socket.setTimeout(0);
cl.connection.socket.setKeepAlive(true, 10000);
// None of these callbacks are called
cl.on('online', function() {
console.log('online');
});
cl.on('connection', function() {
console.log('online');
});
cl.on('authenticate', function(opts, cb) {
console.log('authenticated');
});
cl.on('error',function(e) {
console.error(e);
});
cl.on('stanza', function(stanza) {
console.log(stanza);
});
res.render('index', { title: 'GCM upstream test' });
});
module.exports = router;
Thanks

OP here: The issue was because the route was terminating the XMPP action upon reaching res.render. Upon removing the XMPP code from the route, I get an XMPP authentication failure, which is most likely due to an incorrect jid/password. The project requirements have changed and I no longer need upstream messaging, so I will not attempt to fix the auth failure.
Thanks for the responses

It is possible to enhance the express.js req and res variables without using a middleware function?

I'm working in a restful service using express.js and i want to enhance the req and res variables so for example you could write something like
app.use(function (req, res, next) {
res.Ok = function (data) {
res.status(200).send(data);
};
res.InternalError = function (err) {
res.status(500).send(err);
};
});
And later
router.get('/foo', function (req, res) {
res.Ok('foo');
})
This will send 'foo' in the body of the response and set the status code to 200 and is working perfectly.
My first question is if it is possible to add such functionality without a middleware function, lets say in a property or the prototype of the app variable?
The second question is if there are performance issues if you add many functionality with middleware functions at the app level. Are this functions attached to the request and response object per request or once on the application startup?
I know the Sails framework already do this but I'm wondering if they use middleware functions as well.

I keep digging and turns out that the request and response object are exposed in express using the __proto__ property.
var express = require('express'),
app = express();
app.response.__proto__.foo = function (data) {
this.status(200).send(data);
};
And later in the router
router.get('/foo', function (req, res, next) {
res.foo('test');
});
This will print test in your browser so it is possible to add functionality without using any middleware.
Note: I'm sure there are some drawbacks to this approach (overwriting express predefined properties, for example) but for testing purposes and adding very simple functionality I think is slightly better in terms of performance.

I'm not aware of any other way than using middleware. But in my opinion you could do the following to achieve nearly the same thing.
// Some Route
router.get('/foo', function(req, res, next) {
// ...
if(err) {
res.status(500);
return next(err);
}
return res.send('ok');
});
// Another route
router.get('/bar', function(req, res, next) {
// ...
if(badUserId) {
res.status(400);
return next('Invalid userId.');
}
req.result = 'hello';
return next();
});
router.use(function(req, res) {
// I prefer to send the result in the route but an
// approach like this could work
return res.send(req.result);
});
// Error Middleware
router.use(function(err, req, res, next) {
if(res.statusCode === 500) {
// Log the error here
return res.send('Internal server error');
} else {
return res.send(err);
}
});