I would like to break down the URL of the website with regex. The URL is similar as follows:
https://product.testing.com/intro/index.aspx?source=newsletter&product=watch&brand=rolex
The regex that I used is as follows:
(http:\/\/www\.|https:\/\/www\.|http:\/\/|https:\/\/)?(product\.)(testing\.)(com\/)(.*)(\?|\?)([^=]\w+(?=&))
The first question is, I would like to chop the part after aspx? into pieces i.e source=newsletter, product=watch etc, and the code does not work for the last part, What did I do wrong and how should I change?
The second question is, The domain name part is a kind of hard coding...how can I make it better and more flexible e.g can be applied to https://contact.testing.com/contactoursales/index.aspx?
Thank you for your help in advance!
I recommend using the url package, instead of a regex to parse an URL.
const URL = require('url');
const url = 'https://product.testing.com/intro/index.aspx?source=newsletter&product=watch&brand=rolex';
// Pass true to parse the querystring too
const parsed = URL.parse(url, true);
Which will output:
Url {
protocol: 'https:',
slashes: true,
auth: null,
host: 'product.testing.com',
port: null,
hostname: 'product.testing.com',
hash: null,
search: '?source=newsletter&product=watch&brand=rolex',
query: { source: 'newsletter', product: 'watch', brand: 'rolex' },
pathname: '/intro/index.aspx',
path: '/intro/index.aspx?source=newsletter&product=watch&brand=rolex',
href: 'https://product.testing.com/intro/index.aspx?source=newsletter&product=watch&brand=rolex' }
I would like to chop the part after aspx? into pieces i.e
source=newsletter, product=watch etc, and the code does not work for
the last part, What did I do wrong and how should I change?
Passing true as second parameter to url.parse will parse the querystring for you.
console.log(params.query);
/* {
source: 'newsletter',
product: 'watch',
brand: 'rolex'
} */
If you're not using node.js you can use webpack to use url package on the browser.
webpack url-parser.js -o url-parser.min.js
Related
I am currently trying to pass my password reset generated token inside my unprotected route but whenever I execute my GET request, I receive an 401 Unauthorized request.
I've tried including the package Path-to-RegExp and constructing a separate array route but it didn't work:
let tokens = [];
const unprotected = [
pathToRegexp('/user/reset/:token', tokens),
];
My password-reset token is generated in a separated service and called in a controller:
const token = crypto.randomBytes(20).toString('hex');
user.update({
resetPasswordToken: token,
resetPasswordExpires: Date.now() + 360000,
});
Here is how I've structured my expressJwt with unless:
app.use(expressJwt({
secret: process.env.SECRET_BEARER,
getToken: req => {
MY TOKEN AUTHORISATION CODE IS PLACED HERE.
}
}).unless({ path: ['/images/', '/user/password-reset', unprotected ]}));
My issue is that whenever I try to create a unauthenticated route such as .unless({path: ['/images/', '/user/password-reset', '/user/reset/:token' ]})); the route /user/reset/:token is only parsed as a string a the value of :token is not actually passed.
I've read some similar questions about passing it with regex or functions but I couldn't figure it out myself. This and this question have been particularly useful on how to approach the problem.
You can pass a regex to unless, which you may have already realized since you tried to use Path-to-RegExp. You could also just try to write the regex yourself and pass it directly to unless. In your case your unless would look like this:
.unless({ path: [/\/images\//, /\/user\/password-reset\//, /^\/user\/reset\/[a-z0-9_-]*/]}));
EDIT: this SO answer suggest that you cannot combine regex and strings in the same array, so I've converted all paths to regex expressions.
You have an array within an array for the value of path passed into unless.
Change:
}).unless({ path: ['/images/', '/user/password-reset', unprotected ]}));
To:
}).unless({ path: unprotected }));
And change unprotected to include the other paths:
const unprotected = [
'/images/',
'/user/password-reset',
pathToRegexp('/user/reset/:token', tokens),
];
Ensure you test with a path that includes a token e.g:
/user/reset/123-some-real-looking-value
I am curious about the NodeJS URL API. Specifically, the url.format(urlObject) method. I would like to create a function that validates an urlObject before calling format, and throws a TypeError if any of the object key/value pairs are invalid, or "extra" (not present in the spec).
Is there some way, outside of TypeScript typings, to achieve this?
You will have a hard time doing that because when you create a URL instance:
u = new URL('http://example.com/');
both Object.keys(u) and Object.getOwnPropertyNames(u) will return an empty array and even JSON.stringify(u) will not help you here as it results in a single top level string, not a JSON representation of the object.
So you will not be able to able to check for any extra fields easily, unless you manually parse the output of util.format(u) or something like that - but you would still need to check for object keys (including non-enumerable properties) to validate other objects that are not an instance of URL.
So the most reasonable solution would be to check if u instanceof URL and if it's true then trust that it's fine, but if it's false then check for the specific properties.
But keep in mind that the url.format() doesn't need an actual URL instance, e.g. those two lines have the same effect:
a = url.format(new URL('http://example.com/'));
b = url.format({ protocol: 'http', host: 'example.com', pathname: '/' });
// a === b
To validate if the URL instance (or any other object) has the following properties and they are strings, like it should be in the URL instance, you can use something like:
const k = [
'href',
'origin',
'protocol',
'username',
'password',
'host',
'hostname',
'port',
'pathname',
'search',
'hash',
];
const validUrl = u => k.filter(k => typeof u[k] === 'string').length === k.length;
This will return true for normal URL instances:
u = new URL('http://example.com/');
console.log(validUrl(u));
but also for other objects, if they have all of the required fields.
But keep in mind that the url.format() doesn't require all of the fields, as you can see on the example above, so that way you would reject some of the valuse that would be perfectly fine for url.format() like:
u = { protocol: 'http', host: 'example.com', pathname: '/' };
To sum it up:
Real URL instances don't show you what properties do they have so you will not easily test for extra properties - but I would not advice to do that anyway because your code will break if the URL ever gets extended with new properties which can happen in the future.
To see if it has all the required (in your opinion - not necessarily in the opinion of url.format()!) properties, you can write a simple validator function but then you will not be able to omit empty properties like username if there is none and you would have to pass empty strings for all empty properties explicitly, if you want to be able to use URL-like objects that are otherwise compatible with url.format().
Maybe you want to test just some of the fields like protocol, host and pathname but then you have to handle that url.format() will take either host or hostname, preferring host if both are available etc.
You can also try some hack like this:
const testUrl = u => {
try {
new URL(url.format(u));
} catch (e) {
throw new TypeError('your error message');
}
};
and have a function that experimentally test if a given value works with url.format() giving a URL parsable by new URL() like this:
testUrl(new URL('http://example.com')); // ok
testUrl({host: 'example.com', protocol: 'http'}); // ok
testUrl({host: 'example.com'}); // throws a TypeError
Now in your function that calls url.format(urlObject) all you have to do is add: testUrl(urlObject) in the line before and it will throw exceptions that you describe.
All in all, this is a tricky question. You can do it in many ways but all have some drawbacks. The URL instances are actually quite strange and even Joi breaks on them!
For example:
!joi.object({ host: joi.string().required() }).validate({ host: 'example.com' }).error;
// returns true
!joi.object({ host: joi.string().required() }).validate({ nohost: 'example.com' }).error
// returns false
!joi.object({ host: joi.string().required() }).validate(new URL('http://example.com/')).error;
// this throws an exception!
// TypeError: Cannot read property 'host' of undefined
// WTF???
If even Joi cannot validate URL instances then they are not easy to validate.
Hopefully some of the examples above will do what you need.
I am using vue-router on a multipage app; we are adding history mode because our requirements insist our urls not contain special characters (#) and want to ensure from within our frontend code that we are always prepending our clientside routes with /v/. (to avoid having to update all our routes in the whole app, and to centralize this functionality).
I want to add a route similar to:
routes: [
// add `v/` to paths that don't start with `v/`
{
path: `(^\/(?!v))*`,
redirect: to => `/v/${to}`
},
]
Based on using regex inside path-to-regexp, but I'm not entirely clear from their documentation how to properly pass in a whole route as a regex (or is it possible?), especially without getting caught in a redirect loop.
I was able to use 2 redirects to achieve what I needed (in this case falling back to '/' as a sort of 404.
// Fallback to baseRoute for prefixed unmatched routes
{ path: `${prefix}/*`, redirect: '/' },
// Fallback to baseRoute for unprefixed, unmatched routes (try again with prefix)
{ path: `*`, redirect: to => prefix + to.path }
(An interesting gotcha is that in a redirect function the to value remains unchanged if you get redirected, so I originally had an infinite loop. with to => to.path.startsWith(prefix) ? '/' : prefix + to.path)
My current Problem is: i need to be able to perform the following call:
https://{Net-IP}:{Net-Port}/rest/devices/{deviceName}.{deviceID}
First of all defining a host variable such as:
host: {Net-IP}:{Net-Port} is not possible.
Second, if i then try to implement the Path-Parameter in the paths such as:
/devices/{deviceName}.{deviceID}/measurePoints:
get:
summary: Method returns list of measure points
parameters:
- name: deviceName
in: path
type: string
required: true
description: device Name
- name: deviceID
in: path
type: string
required: true
description: device ID
then it says Path templating is not allowed.
I need this type of calls to be translated for Postman, i appreciate your help!
Why don't you make use of environmental variables/constants etc.
I'm building a server in Node.js that receives data from Javascript on a website. The data is being sent using jQuery's getJSON method, like so:
$.getJSON(
url,
{
id: track.id,
artist: track.artist,
title: track.title,
album: track.album,
filename: track.filename,
user: track.user,
userId: track.userId,
room: track.room,
timestamp: track.timestamp
}
);
I'm then trying to get at the data using Node's url module like this:
var urlObj = url.parse(request.url, true);
var jsonCallback = urlObj.query["callback"];
This works fine most of the time, but it fails when one of the parameters contains an apostrophe. It looks like it's stopping parsing the query string at the apostrophe. Here's what console.log prints for two different query objects:
{ callback: 'jQuery15105242477038409561_1304925579219',
id: '6c91c74db064c93f1f020000',
artist: 'Radiohead',
title: 'Everything In Its Right Place',
album: 'Kid A',
filename: '01 Everything In Its Right Place.m4a',
user: 'Cowrelish',
userId: '82a89b4df7a9120305000000',
room: 'qwantz',
timestamp: '1304924972555',
_: '1304925611362' }
{ callback: 'jQuery15105242477038409561_1304925579221',
id: '798cc74dfcce4337f7010000',
artist: 'Toy Dolls',
title: 'The Final Countdown',
album: 'HOLY SHIT!!! IT' }
The album for the second one is "HOLY SHIT!! IT'S THE FINAL COUNTDOWN!", as you can see it's been truncated at the apostrophe.
I've tried escaping the apostrophe in the client code, but all I get then is "album: 'HOLY SHIT!!! IT\\'". It's escaping the slash, but not the apostrophe.
Is this a bug or a feature? Should I be escaping apostrophes in a different way on the client, or looking to fix a bug in Node.js?
You shouldn't have to escape on the jquery side, and it doesn't appear to be a node bug (at least with latest release).
I tried to duplicate your problem and wasn't able to using node 0.4.7.
I did this:
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.5.1/jquery.min.js"></script>
<script type="text/javascript">
$(document).ready(function () {
$.getJSON('/', {test:"it's a test"});
});
</script>
with node code that just did this on the incoming request:
console.log(require('url').parse(req.url, true));
which gave me:
{ test: 'it\'s a test' }
So, doesn't seem to be a general problem with either jQuery or node.
Have you looked at the full url before it is parsed? You should be able to just copy and paste that into the node interactive shell and do something like this to test:
require('url').parse('http://www.example.org/?q=it\'s working', true);
Anything between the browser and the web server that might be touching the url?