Is there a way to restrict allowed uri characters in Expressjs. like set a variable of characters:
allowed_uri_chars = 'a-zA-Z0-9'; etc
The most logical and easiest way to go would be to use a regex in some middleware:
var url = require("url");
app.use(function(req, res, next) {
var pathname = url.parse(req.url).pathname;
if(!pathname.match(/^[a-z0-9\/]+$/i)) return res.send(403);
next();
});
Or, as the URL is already parsed by Express, but this is not exposed publicly, you may want to save one call to parse and rely on Express internals (which I wouldn't advise):
app.use(function(req, res, next) {
if(!req._parsedUrl.pathname.match(/^[a-z0-9\/]+$/i)) return res.send(403);
next();
});
This ensures that your pathname (for http://example.com/foo/bar?id=1334, pathname would be /foo/bar) only contains letters, numbers, and slashes. It raises a 403 if not.
Can't you use a regex to parse the authorized chars?
Or do you mean in the routing directly?
Related
I have a middleware function that takes a query string value and turns it into
I am using Express 4 and Node.js. I have a route defined for /:module/:page.aspx and am using req.params and req.query values across a number of middleware functions.
app.get('/:module/:page.aspx', catchEncodedQuery, doSomeStuff, doOtherStuff);
I have one middleware function that appears first, which catches certain requests which have a query string value that can be transformed into a new req.url that I set for use further on in the middleware chain. Basically, I want to proceed as though the un-encoded form of this query string value was the request URL the whole time.
function catchEncodedQuery(req, res, next) {
if (
// req meets some specific criteria
) {
req.url = myUnencodeFunction(req.query.url_str);
next();
} else {
next();
}
};
But I would also like req.params and req.query to reflect this new URL. I could just parse the new URL myself and reassign these values to req.params and req.query, but I'm wondering if there's a better way to do this that leverages Express's existing URL-parsing functionality.
I'll start with some context:
I have a RESTful API Server that includes routes for managing users; e.g.
PUT .../users/:id/profile
As part of our authentication flow we verify the user's identity and compare the id in the url with the id retrieved from our IDP.
What I'm trying to do is implement a 'me' replacement approach; where a user, instead of providing their id in the url, can simply provide 'me' instead, and the id is reassigned with id retrieved from the IDP based on their authentication. I've seen Google do this for some of their APIs. E.g.
.../users/me/profile
So far I've been attempting a simple replacement with the assumption that a modified req.params will propagate forward:
req.params.id = req.params.id === 'me'
? session.id
: req.params.id
This wasn't working so I did a bit of reading up on it and apparently req.params gets built with each middleware function (when .use() is called), so reassigning req.params.id doesn't propagate throughout the app as I had hoped.
Currently all our business logic pulls the id from req.params so I was wondering if there's a way to get the above behaviour to work to avoid refactoring all my business logic?
I understand that req.params is built from parsing the url; so would modifying the url be an option?; to replace 'me' with the id; so that req.params is populated as intended? If so, how could this be achieved?
Or, if you have a nicer alternative to getting the above 'me' replacement behaviour to work then I'm all ears!
Global middleware (that you add using app.use() or router.use()) is typically not even aware of any route handlers that declare parameters, so req.params usually isn't even populated.
A solution would be to "inline" the modifying middleware:
app.put('/users/:id/profile', middleware, ...)
But that would require rewriting all your route handlers.
As you already point out correctly, you can modify the URL:
app.use(function(req, res, next) {
if (req.url === '/users/me/profile') {
req.url = '/users/1234/profile';
}
next();
});
app.put('/users/:id/profile', ...)
Instead of having to match each URL specifically, you can perform substring matches, or use a list of all URL's that may contain an :id parameter (and therefore, may contain the me identifier).
I just realized that this should work (using req.param()):
router.param('id', function(req, res, next, id) {
if (id === 'me') {
req.params.id = '1234';
}
next();
});
This should be added to routers that declare parameters, either in route handlers directly, or in mountpoints.
A convoluted example:
let childRouter = express.Router({ mergeParams : true });
childRouter.put('/profile', ...);
let parentRouter = express.Router({ mergeParams : true });
parentRouter.use('/:id', childRouter);
parentRouter.param('id', function(req, res, next, id) {
if (id === 'me') {
req.params.id = '1234';
}
next();
});
app.use('/users', parentRouter);
Because parentRouter declares a mountpoint /:id, it should get the resolving function.
The question is related to: when I refresh the page, not the page loads. Angularjs + Nodejs
But in this case I disable html5mode and , therefore the url the symbol appears #. I want to have control access to certain sections of the code that I show below, but do not know how to handle the #
app.get(['/homepage', '/recoveryPassword', '/manageAccount', '/shop/:itemId/cat/:categoryId', '/shop/:shopName', '/manageMyShops', '/manageMyShops/id/:shopId', '/manageWeb'], isLoggedIn, function (req, res) {
res.end();
});
I am reluctant to post this because I think this is NOT a real solution to OP's problem. To me this is a hack, so use it at your own risk.
To start with - it is unconventional for REST URL to have special characters like '#' in them. While I believe there is no standard way of naming the endpoints but it is recommended to make sure the URLs follow these rules;
all lower-case (because url are not case-sensitive)
no special characters
no camel case
Anyway, my approach is.
On the server code;
Assuming you are defining a path /config# then replace the '#' char with something unique e.g. _bbb Unique enough? you judge, just make sure it doesn't clash with other endpoints.
Add an express layer to decode the URL's special character and replace the special character with the unique word you have given in point 1. (This will be more clear when you see the code).
Client code:
Encode the URL. Why? Because URL can't have '#', because it is a reserved character. So it has to be encoded as '%23' instead.
Example:
Client:
"use strict";
let request = require("request");
let req = {
url: `localhost:4444/${encodeURIComponent('config#')}`,
proxy: 'http://localhost:4444',
method: 'GET',
headers: {
'Content-Type': 'application/json'
}
};
request(req, function (err, res, body) {
this.config = JSON.parse(body);
console.log("response => " + this.config);
});
Server:
"use strict";
var express = require("express");
var bodyParser = require("body-parser");
var app = express();
var config = require('config');
app.use(bodyParser.json());
app.use((req, res, next) => {
let decoded_url = decodeURIComponent(req.url);
req.url = decoded_url.replace('#', '_bbb');
next();
});
app.get('/config_bbb', function(req, res){
res.json('{name: test}');
});
// Start the server
app.set('port', 4444);
app.listen(app.get('port'), "0.0.0.0", function() {
console.log('started');
});
Output:
response => {name: test}
The idea here is. When the client calls the endpoint, yes the URL is still with /config# but you encode it so that it looks like /config%23.
See:
http://www.w3schools.com/tags/ref_urlencode.asp
Then when the request comes through into the server, the layer you have added in expressJS would decode the URL from /config%23 to /config# and you replace the # char with something unique. Let say _bbb, so that the final URL is /config_bbb which gets routed to the actual endpoint.
Unfortunately express doesn't like endpoints to have special characters in them, if you don't replace the # to be something recognizable then you will find that it doesn't get routed it properly even thou the URL matches.
I am working with node/express/passport/ looking at code that attempts to use a request like:
req._parsedUrl.pathname;
I cannot figure out where this variable is coming from. Is this a canonical variable name that is set in a common .js library? It doesn't seem exposed in any headers.
req._parsedUrl is created by the parseurl library which is used by Express' Router when handling an incoming request.
The Router doesn't actually intend to create req._parsedUrl. Instead parseurl creates the variable as a form of optimization through caching.
If you want to use req._parsedUrl.pathname do the following instead in order to ensure that your server doesn't crash if req._parsedUrl is missing:
var parseUrl = require('parseurl');
function yourMiddleware(req, res, next) {
var pathname = parseUrl(req).pathname;
// Do your thing with pathname
}
parseurl will return req._parsedUrl if it already exists or if not it does the parsing for the first time. Now you get the pathname in a save way while still not parsing the url more than once.
You can write a middleware to handle then set properties for req.
var myMiddleWare = function () {
return function (req, res, next) {
req._parsedUrl = 'SOME_THING';
next()
}
};
app.get('/', myMiddleWare, function (req, res) {
console.log(req._parsedUrl); // SOME_THING
res.end();
})
Express middleware document in here
There are many ways to mock requests using things like supertest and nock but what I'm looking to do is to be able to create fake request objects, as if the request was going to a different URL, that can be passed on to other processing functions.
app.get("/render", function(req, res) {
// how do I do this?
var fake = createFakeRequest("/bar/?baz=qux", req);
// I want doStuff to believe the request came to /bar/?baz=qux
doStuff(fake, function(err, result) {
res.send(result);
});
});
I'm aware I could modify all of he variables such as req.path, req.query, req.url but I'm worried I may miss something, and it seems like someone out there must have already solved this problem.
The reason I need this behavior is that for any given URL in my CMS, there can be multiple drafts. Each draft will have different content elements, but those individual content elements may have logic based on the URL or query parameters. So even though the request came in as /render/?draft=1&url=/foo/bar/, I want the content element processors to believe the request came in to /foo/bar/ and be oblivious to the version system which actually handled the initial HTTP request.
Not sure to understand but seems like url rewriting, so using a middleware could work
function urlRewrite(req, res, next){
req.url ="...";
next();
}
and so
app.use(urlRewrite);
Be sure to use it at the right place (depending on your server goal)
Cause we maybe need params before the rewrite... and if rewrite, you may need it after...
EDIT
In my framework:
server.before(extractPath);
server.before(urlParams);
server.before(pathParams);
server.get("/foo", route.foo);
So I could write
server.before(extractPath);
=> server.before( function(req, res, next){
urlRewrite(req, res, function(){
extractPath(req, res, next);
}
});
server.before(urlParams);
server.before(pathParams);
server.get("/foo", route.foo);
If urlRewrite depends on urlParams, I could write
server.before(extractPath);
server.before(urlParams);
=> server.before( function(req, res, next){
urlRewrite(req, res, function(){
extractPath(req, res, function(){
urlParams(req, res, next);
});
});
});
server.before(pathParams);
server.get("/foo", route.foo);
As I said, it depends on your framework