I'm working with an express application. There are some express routes, as
server.get('*' , ... )
etc. which perform some common operations: authentication, validation... etc.
they also decorates the response with meaningful information: i.e. in every request to the server it gives not only the expected json/html, but also information regarding the user, some app metadata that the front-end consumes etc. etc.
Let's say all this extra metadata cames in a field called extradata in every request to the server.
Now, there is a bug that is causing a problem: instead of returning its expected response (a json with a bunch of system logs), is sending only this extradata field.
I'm pretty confident the problem is in one of the middlewares, because that code that sends the response in this case is really simple, it's just a res.send() of a json. So I believe this part of the app is requiring some module that sets a middleware which causes the error. There are a lot of global vars and implicit parameters in the app so is really difficult to debug it manualluy.
I attempted to bypass such middlewares programmatically, like:
delete server._router.stack[2];
but is causing an TypeError: Cannot read property 'route' of undefined and thus preventing my app to build: sure this is not the way to go.
so, is there a way to programmatically ignore or bypass express routes that are yet set?
Even better, is there a way to programmatically tap into express middlewares and log every request and response?
(afaik, there are libreries like morgan that logs every request, but I don't think they apply to this case since I need to discriminate between middlewares).
What I generally do is simply use the next method. You can access it by simply passing it to the callback function. Something like:
app.use(function(req, res, next) {
if(...) {
next();
} else {
...
}
}
What this is going to do is go to the next middleware.
So if I understand correctly, you can check what you exactly need in the if-statement and do things accordingly.
What I would suggest is you read the Express API documentation, especially the section about middleware, which you can find here. Moreover, try to isolate the suspects and solve the issue by removing the problem, rather than deleting handlers and trying to solve the problem the easy way.
Related
I am working on an application that uses an express server to reach out to an API to fetch data. In our organisation outbound traffic requires a proxy which I have supplier to axios like below (not the real one):
let response = await axios.get(endpointUrl, {
proxy: {
host: "123.45.678.90",
port: 0000,
},
})
Passing various URLs into the axios get function returns varied results, with the following URLs returning a result:
https://www.boredapi.com/api/activity
https://api.ipify.org?format=json
https://jsonplaceholder.typicode.com/todos/1
Whereas the following URLs are returning an ECONNRESET error almost instantly:
https://api.publicapis.org/entries
https://randomuser.me/api/
https://reqres.in/api/users
I can't see any pattern between the URLs that are/are not working so wondered if a fresh set of eyes could spot the trait in them? It's important to note that all these URLs return successfully in the browser, just through this axios call being the problem.
To add to the mystery, the URLs that do work work on my machine, do work on a machine outside our organisation - so potentially a clue there?
Any help/guidance of course would be appreciated, thank you.
This error simply means that the other party closed the connection in a way that was probably not normal (or perhaps in a hurry).
For example, a socket connection may be closed by the other party abruptly for various reasons or you may have lost your wifi signal while running your application. You will then see this error/exception on your end.
What could also be the case: at random times, the other side is overloaded and simply kills the connection as a result. If that's the case, depends on what you're connecting to exactly…
Solution - This is happening because you are not listening to/handling the 'error' event. To fix this, you need to implement a listener that can handle such errors.
If the URL that work on your machine also work outside your organization and the other don't, it is most likely a problem with your proxy.
Some proxies might have configurations that makes them remove headers or change the request in a way that the target does not receive it as intended.
I also encountered a problem with axios and proxies once. I had to switch libs to make it work. To be sure, I would recommand using a lib like "request" (deprecated) juste to make sure it is not a problem with axios. There are multiple open issues on the axios repository for proxy issues.
ECONNRESET is likely occurring either because the proxy runs into some sort of error and drops the connection or the target host finds something wrong with the incoming connection and decides to immediately drop it.
That target host may either be finding a problem because of the proxy or it may be expecting something in the request that it finds is missing.
Since you have evidence that all the requests work fine when running from a different location (not through your proxy) and I can confirm that your code works fine from my location (also not running through your proxy), it definitely seems like the evidence points at your proxy as causing some problem in some requests.
One way to debug proxy issues like this is to run a request through the proxy that ends up going to some server you can debug on and see exactly what the proxy has done to the incoming request, compared to a request to that same host that doesn't go through the proxy. That will hopefully highlight some difference that you can then test to see if that's causing the problem and then eventually work on the configuration of the proxy to correct.
I am using a res.redirect('page.ejs'); and on my browser I get the message:
Cannot GET /page.ejs
I have not declared this in my routes file in the style of :
app.get('/page', function(req, res) {
res.render('page.ejs');
});
Should this be included in order for the res.redirect() to work?
When I do not use res.redirect() but res.render(), even if I have not the app.get() code, it still works.
so to understand this, let's look at what each of these methods do.
res.redirect('page.ejs');
// or, more correctly, you're redirecting to an *endpoint*
// (not a page. the endpoint will render a *page*) so it should be:
res.redirect('/page');
this will tell express to redirect your request to the GET /page.ejs endpoint. An endpoint is the express method you described above:
app.get('/page', function(req, res) {
res.render('page.ejs');
});
since you don't have that endpoint defined it will not work. If you do have it defined, it will execute the function, and the res.render('page.ejs') line will run, which will return the page.ejs file. You could return whatever you want though, it can be someOtherPage.ejs or you can even return json res.json({ message: 'hi' });
res.render('page.ejs');
this will just respond to the client (the front-end / js / whatever you want to call it) with the page.ejs template, it doesn't need to know whether the other endpoint is present or not, it's returning the page.ejs template itself.
so then, it's really up to you what you want to use depending on the scenario. Sometimes, one endpoint can't handle the request so it defers the request to another endpoint, which theoretically, knows how to handle the request. In that case redirect is used.
hope that makes sense and clarifies your confusion
(I'm not an expert on the inner-workings of express, but this is a high-level idea of what it's doing)
You should do res.redirect('/page')
I am currently building an API thanks to Express.js, and I still can not believe how amazing it is. I figured out how to use middlewares, handling requests and responses, how to go to the next middleware ... But there is something that triggers me a lot, which is next().
I know what next() is, I just can't figure out what kind of parameters can next() take, and what they will do. At first I thought it was to pass data to the next middleware, turned out I was wrong, there is req.locals for this.
Can someone enlighten me on this ?
You have three choices for calling next():
1. Continue routing. If you just want to continue routing to the next route handler in the chain that matches this route, then you just call it with no parameters:
next();
This is most often used in middleware:
app.use((req, res, next) => {
// set an item in the session and then continue routing
req.session.xxx = "foo";
next();
});
2. Abort routing with an error to your centralized error handler. If you want to set an error and have routing skip to your generalized error handler, then you can pass an error:
next(new Error("timeout contacting database"));
This will then invoke your general error handling route and pass it that specific error where your general error handling code can decide what to do. This allows you to centralize your error handling response code in one place. Examples of using next(err) are here.
You can see the doc for generalized error handling in Express for examples on how that works.
3. Skip route handlers in this router, continue routing in other routers. If you want to skip all route handlers in the current router, but continue with any other routes that might be registered, then you can do this:
next('route');
From the doc:
You can provide multiple callback functions that behave just like middleware, except that these callbacks can invoke next('route') to bypass the remaining route callback(s). You can use this mechanism to impose pre-conditions on a route, then pass control to subsequent routes if there is no reason to proceed with the current route.
If you repeatedly search for "next(" on this page of the doc, you will find lots of examples. There's an example of this usage here in the doc.
FYI, Express 5 may add some new uses for next() because it may return a promise that lets you know when routing is now completely done.
I am having a NodeJS based REST service exposed using Express (4.0.0) where I have two different routes like this:
router.get('/buckets/:bucketId/entities/bulk', getEntitiesInBulk);
router.get('/buckets/:bucketId/entities/:key', getEntityByKey);
When I send a request like this:
http://<server:port>/buckets/responses/entities/k3
The request is being handled by getEntityByKey() which I have defined there, but strangely when I bombard it with many requests it sometimes get handled by getEntitiesInBulk() and gets some error in response which is only thrown by getEntitiesInBulk().
I am totally confused about how is this possible.
Express is confused because your routes are not unique. "bulk" will sometimes be used as a :key in the first route. Simply change the signature a bit, like
router.get('/buckets/:bucketId/entities/bulk', getEntitiesInBulk);
router.get('/buckets/:bucketId/entity/:key', getEntityByKey);
I am writing a web app in node.js. Now every processing on the server is always in the context of a session which is either retrieved or created at the very first stage when the request hits the server. After this the execution flows through multiple modules and callbacks within them. What I am struggling with is in creating a programming pattern so that at any point in the code the session object is available without the programmer requiring it to pass it as an argument in each function call.
If all of the code was in one single file I could have had a closure but if there are function calls to other modules in other files how do I program so that the session object is available in the called function without passing it as an argument. I feel there should be some link between the two functions in the two files but how to arrange that is where I am getting stuck.
In general I would like to say there is always a execution context which could be a session or a network request whose processing is spread across multiple files and the execution context object is to be made available at all points. There can actually be multiple use cases like having one Log object for each network request or one Log object per session. And the plumbing required to make this work should be fitted sideways without the application programmer bothering about it. He just knows that that execution context is available at all places.
I think it should fairly common problem faced by everyone so please give me some ideas.
Following is the problem
MainServer.js
app = require('express').createServer();
app_module1 = require('AppModule1');
var session = get_session();
app.get('/my/page', app_module1.func1);
AppModule1.js
app_module2 = require('AppModule2');
exports.func1 = function(req,res){
// I want to know which the session context this code is running for
app_module2.func2(req,res);
}
AppModule2.js
exports.func2 = function(req,res){
// I want to know where the session context in which this code is running
}
You can achieve this using Domains -- a new node 0.8 feature. The idea is to run each request in it's own domain, providing a space for per-request data. You can get to the current request's domain without having to pass it all over via process.domain.
Here is an example of getting it setup to work with express:
How to use Node.js 0.8.x domains with express?
Note that domains in general are somewhat experimental and process.domain in particular is undocumented (though apparently not going away in 0.8 and there is some discussion on making it permanent). I suggest following their recommendation and adding an app-specific property to process.domain.data.
https://github.com/joyent/node/issues/3733
https://groups.google.com/d/msg/nodejs-dev/gBpJeQr0fWM/-y7fzzRMYBcJ
Since you are using Express, you can get session attached to every request. The implementation is following:
var express = require('express');
var app = express.createServer();
app.configure('development', function() {
app.use(express.cookieParser());
app.use(express.session({secret: 'foo', key: 'express.sid'}));
});
Then upon every request, you can access session like this:
app.get('/your/path', function(req, res) {
console.log(req.session);
});
I assume you want to have some kind of unique identifier for every session so that you can trace its context. SessionID can be found in the 'express.sid' cookie that we are setting for each session.
app.get('/your/path', function(req, res) {
console.log(req.cookies['express.sid']);
});
So basically, you don't have to do anything else but add cookie parser and enable sessions for your express app and then when you pass the request to these functions, you can recognize the session ID. You MUST pass the request though, you cannot build a system where it just knows the session because you are writing a server and session is available upon request.
What express does, and the common practice for building an http stack on node.js is use http middleware to "enhance" or add functionality to the request and response objects coming into the callback from your server. It's very simple and straight-forward.
module.exports = function(req, res, next) {
req.session = require('my-session-lib');
next();
};
req and res are automatically passed into your handler, and from their you'll need to keep them available to the appropriate layers of your architecture. In your example, it's available like so:
AppModule2.js
exports.func2 = function(req,res){
// I want to know where the session context in which this code is running
req.session; // <== right here
}
Nodetime is a profiling tool that does internally what you're trying to do. It provides a function that instruments your code in such a way that calls resulting from a particular HTTP request are associated with that request. For example, it understands how much time a request spent in Mongo, Redis or MySQL. Take a look at the video on the site to see what I mean http://vimeo.com/39524802.
The library adds probes to various modules. However, I have not been able to see how exactly the context (url) is passed between them. Hopefully someone can figure this out and post an explanation.
EDIT: Sorry, I think this was a red-herring. Nodetime is using the stack trace to associate calls with one another. The results it presents are aggregates across potentially many calls to the same URL, so this is not a solution for OP's problem.