I am curious about the NodeJS URL API. Specifically, the url.format(urlObject) method. I would like to create a function that validates an urlObject before calling format, and throws a TypeError if any of the object key/value pairs are invalid, or "extra" (not present in the spec).
Is there some way, outside of TypeScript typings, to achieve this?
You will have a hard time doing that because when you create a URL instance:
u = new URL('http://example.com/');
both Object.keys(u) and Object.getOwnPropertyNames(u) will return an empty array and even JSON.stringify(u) will not help you here as it results in a single top level string, not a JSON representation of the object.
So you will not be able to able to check for any extra fields easily, unless you manually parse the output of util.format(u) or something like that - but you would still need to check for object keys (including non-enumerable properties) to validate other objects that are not an instance of URL.
So the most reasonable solution would be to check if u instanceof URL and if it's true then trust that it's fine, but if it's false then check for the specific properties.
But keep in mind that the url.format() doesn't need an actual URL instance, e.g. those two lines have the same effect:
a = url.format(new URL('http://example.com/'));
b = url.format({ protocol: 'http', host: 'example.com', pathname: '/' });
// a === b
To validate if the URL instance (or any other object) has the following properties and they are strings, like it should be in the URL instance, you can use something like:
const k = [
'href',
'origin',
'protocol',
'username',
'password',
'host',
'hostname',
'port',
'pathname',
'search',
'hash',
];
const validUrl = u => k.filter(k => typeof u[k] === 'string').length === k.length;
This will return true for normal URL instances:
u = new URL('http://example.com/');
console.log(validUrl(u));
but also for other objects, if they have all of the required fields.
But keep in mind that the url.format() doesn't require all of the fields, as you can see on the example above, so that way you would reject some of the valuse that would be perfectly fine for url.format() like:
u = { protocol: 'http', host: 'example.com', pathname: '/' };
To sum it up:
Real URL instances don't show you what properties do they have so you will not easily test for extra properties - but I would not advice to do that anyway because your code will break if the URL ever gets extended with new properties which can happen in the future.
To see if it has all the required (in your opinion - not necessarily in the opinion of url.format()!) properties, you can write a simple validator function but then you will not be able to omit empty properties like username if there is none and you would have to pass empty strings for all empty properties explicitly, if you want to be able to use URL-like objects that are otherwise compatible with url.format().
Maybe you want to test just some of the fields like protocol, host and pathname but then you have to handle that url.format() will take either host or hostname, preferring host if both are available etc.
You can also try some hack like this:
const testUrl = u => {
try {
new URL(url.format(u));
} catch (e) {
throw new TypeError('your error message');
}
};
and have a function that experimentally test if a given value works with url.format() giving a URL parsable by new URL() like this:
testUrl(new URL('http://example.com')); // ok
testUrl({host: 'example.com', protocol: 'http'}); // ok
testUrl({host: 'example.com'}); // throws a TypeError
Now in your function that calls url.format(urlObject) all you have to do is add: testUrl(urlObject) in the line before and it will throw exceptions that you describe.
All in all, this is a tricky question. You can do it in many ways but all have some drawbacks. The URL instances are actually quite strange and even Joi breaks on them!
For example:
!joi.object({ host: joi.string().required() }).validate({ host: 'example.com' }).error;
// returns true
!joi.object({ host: joi.string().required() }).validate({ nohost: 'example.com' }).error
// returns false
!joi.object({ host: joi.string().required() }).validate(new URL('http://example.com/')).error;
// this throws an exception!
// TypeError: Cannot read property 'host' of undefined
// WTF???
If even Joi cannot validate URL instances then they are not easy to validate.
Hopefully some of the examples above will do what you need.
Related
I am making a get request with additional params options, since I am using that request on a filter, so the params are filters for what to get back:
const res = await axios.get("http://localhots:3000/getsomedata", {
params: {
firstFilter: someObject,
secondFilter: [someOtherObject, someOtherObject]
}
});
The request goes through just fine, on the other end, when I console.log(req.query); I see the following:
{
firstFilter: 'someObject',
'secondFilter[]': ['{someOtherObject}', '{someOtherObject}'],
}
If I do req.query.firstFilter that works just fine, but req.query.secondFilter does not work and in order for me to get the data, I have to do it with req.query["secondFilter[]"], is there a way to avoid this and be able to get my array of data with req.query.secondFilter?
My workaround for now is to do:
const filter = {
firstFilter: req.query.firstFilter,
secondFilter: req.query["secondFilter[]"]
}
And it works of course, but I don't like it, I am for sure missing something.
Some tools for parsing query strings expect arrays of data to be encoded as array_name=1&array_name=2.
This could be a problem if you have one or more items because it might be an array or might be a string.
To avoid that problem PHP required arrays of data to be encoded as array_name[]=1&array_name[]=2 and would discard all but the last item if you left the [] out (so you'd always get a string).
A lot of client libraries that generated data for submission over HTTP decided to do so in a way that was compatible with PHP (largely because PHP was and is very common).
So you need to either:
Change the backend to be able to parse PHP style
Change your call to axios so it doesn't generate PHP style
Backend
The specifics depend what backend you are using, but it looks like you might be using Express.js.
See the settings.
You can turn on Extended (PHP-style) query parsing by setting it to "extended" (although that is the default)
const app = express()
app.set("query parser", "extended");
Frontend
The axios documentation says:
// `paramsSerializer` is an optional function in charge of serializing `params`
// (e.g. https://www.npmjs.com/package/qs, http://api.jquery.com/jquery.param/)
paramsSerializer: function (params) {
return Qs.stringify(params, {arrayFormat: 'brackets'})
},
So you can override that
const res = await axios.get("http://localhots:3000/getsomedata", {
params: {
firstFilter: someObject,
secondFilter: [someOtherObject, someOtherObject]
},
paramsSerializer: (params) => Qs.stringify(params, {arrayFormat: 'repeat'})
});
My example requires the qs module
This has to do with params not being serialized correctly for HTTP GET method. Remember that GET has no "body" params similar to POST, it is a text URL.
For more information I refer to this answer, which provides more detailed info with code snippets.
I am using this: https://www.npmjs.com/package/youtube-notification to get new uploads for my discord bot.
I'm not sure what to put for the urlCallback section?
This is the example code:
const notifier = new YouTubeNotifier({
hubCallback: 'https://example.com/youtube',
port: 8080,
secret: 'Something',
path: '/youtube'
});
notifier.setup();
I think, the documentation for the package makes it pretty clear
Quoting it
YouTubeNotifier.constructor(options)
options is an object that you may write your own properties to. The following properties are read by YouTubeNotifier:
hubCallback - Your ip/domain name that will be used as a callback URL by Pubsubhubbub. It must be in a URL format, ex: 'https://example.com/'. This is a required property as the default is undefined.
So basically your callback will be your localhost kinda thing, where the server will get data from. If you have let's say on a Repl in Repl.it, then it will have to get the data from the Repl hosting URL
Ok, this might need a little bit of explaination up front.
I am currently working on an automation project using node-red.
I want to upload and download files from an remote server using ssh. For this tasks I use this node-red package called node-red-contrib-sftpc. I rewrote the library a little bit, so that I can hand over some credentials for the sftp connection, via the payload which is handed over to the node.
To establish an connection the sftp.connect method of the ssh2-sftp-client is used:
await sftp.connect({
host: node.server.host,
port: node.server.port,
username: node.server.username,
password: node.server.password});
There you can find in the documentation that you can provide connect with the parameters hostHash and hostVerifier. The documentation of the ssh2 model, on which the ssh2-sftp-client is based, states that:
hostHash - string - Any valid hash algorithm supported by node. The host's key is hashed using this algorithm and passed to the hostVerifier function as a hex string. Default: (none)
hostVerifier - function - Function with parameters (hashedKey[,
callback]) where hashedKey is a string hex hash of the host's key for
verification purposes. Return true to continue with the handshake or
false to reject and disconnect, or call callback() with true or false
if you need to perform asynchronous verification. Default:
(auto-accept if hostVerifier is not set)
So here is my problem: How do I write the hostVerifier function? I want to pass hashedKey and also fingerprint, so that I can return true or false, when the handshake worked out or not.
I want to check, if the given server key fingerprint is the "right" one and that I connect to the correct server.
So far as I understood the second parameter, will be a callback function, but I do not know how to use that, so that it will verify the handshake.
This was my try, or at least how I tried to do it.
node.server.hostVerifier = function (hashedKey, (hashedKey, msg.fingerprint)=> {
if (hashedKey = msg.fingerprint) return true;
else return false
}){};
await sftp.connect({
host: node.server.host,
port: node.server.port,
username: node.server.username,
password: node.server.password,
hostHash: 'someHashAlgo',
hostVerifier: node.server.hostVerifier,});
I know that this is completely wrong, but I am about to get crazy, because I have no idea, how to proper check the ssh host key fingerprint.
So I found a solution myself and want to share it with you.
I defined an arrow function as the hostVerifier function, which takes implicite the value of the fingerprint through the msg.fingerprint variable. I only do this, if node.server.fingerprint has an value. So if I do not have an fingerprint at hand, the connection will still established.
node.server.fingerprint = msg.fingerprint;
if(!!node.server.fingerprint){
node.server.hostHash = 'md5';
node.server.hostVerifier = (hashedKey) => {
return (hashedKey === msg.fingerprint) ;};
node.server.algorithms = {serverHostKey: ['ssh-rsa'],};
};
For that I also declare my node.server.alogrithms. With that it was a little bit of try and error.
So I put everything together here:
await sftp.connect({
host: node.server.host,
port: node.server.port,
username: node.server.username,
password: node.server.password,
hostHash: node.server.hostHash,
hostVerifier: node.server.hostVerifier,
algorithms: node.server.algorithms,
});
I've searched around for ways to query a graphQl endpoint, but so far they've all had a pretty similar and clunky solution as I have: string building. Here's a few links:
querying graphql with node
https://blog.apollographql.com/4-simple-ways-to-call-a-graphql-api-a6807bcdb355
https://www.npmjs.com/package/graphql-request
Ultimately, the code I'm running right now (and do not truly like) is:
// replace anything that looks like this => %someVar
// with the value at vars[someVar]
function injectVars(query, vars) {
return query.replace(/%([\d\w]+)/g, (_, varName) => {
if (!vars.hasOwnProperty(varName)) {
throw new Error(`Undefined variable: ${varName}`);
}
return vars[varName];
});
}
// a query with the injection point "%userId"
const userQuery = `query getUser{
user(
id: "%userId"
) {
id
name
}
}`;
// successful injection
const query = injectVars(userQuery, {
userId: "some-id-123"
});
console.log(query);
// failed usage
const willFail = injectVars(userQuery, {
nonVar: "missingUserId"
});
Problems I have with the code above:
Appears to be open to injection attacks
Not great type checking
Kind of a hassle with ENUMS, or type conversion (ie, take a date and make it string)
Not very extensible. Would have to copy paste the same stuff a lot for different iterations.
I'm new to this, and inherently don't trust my solutions yet.
Anyways, how do I query a GraphQl endpoint from node js without clunky string manipulation? Are there any tools/suggestions which would make this interaction more object oriented?
Use variables as shown in the official tutorial.
A GraphQL query can be parameterized with variables, maximizing query reuse, and avoiding costly string building in clients at runtime... Variables must be defined at the top of an operation and are in scope throughout the execution of that operation.
An example:
const query = `query getUser($id: ID!) {
user(
id: $id
) {
id
name
}
}`
const variables = {
id: 'someId'
}
fetch('/graphql', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Accept': 'application/json',
},
body: JSON.stringify({
query,
variables,
})
})
The JSON values you provide are parsed into the types you specify in your document (in the above example, an ID scalar). The variables will be validated by the GraphQL service so you could also skip checking if they are defined or the correct type. String values will be parsed into the appropriate enum value as long as the variable is declared as an enum type. IDs can be either Intergers or Strings. Input object types will be parsed and handled appropriately as well.
I am currently trying to pass my password reset generated token inside my unprotected route but whenever I execute my GET request, I receive an 401 Unauthorized request.
I've tried including the package Path-to-RegExp and constructing a separate array route but it didn't work:
let tokens = [];
const unprotected = [
pathToRegexp('/user/reset/:token', tokens),
];
My password-reset token is generated in a separated service and called in a controller:
const token = crypto.randomBytes(20).toString('hex');
user.update({
resetPasswordToken: token,
resetPasswordExpires: Date.now() + 360000,
});
Here is how I've structured my expressJwt with unless:
app.use(expressJwt({
secret: process.env.SECRET_BEARER,
getToken: req => {
MY TOKEN AUTHORISATION CODE IS PLACED HERE.
}
}).unless({ path: ['/images/', '/user/password-reset', unprotected ]}));
My issue is that whenever I try to create a unauthenticated route such as .unless({path: ['/images/', '/user/password-reset', '/user/reset/:token' ]})); the route /user/reset/:token is only parsed as a string a the value of :token is not actually passed.
I've read some similar questions about passing it with regex or functions but I couldn't figure it out myself. This and this question have been particularly useful on how to approach the problem.
You can pass a regex to unless, which you may have already realized since you tried to use Path-to-RegExp. You could also just try to write the regex yourself and pass it directly to unless. In your case your unless would look like this:
.unless({ path: [/\/images\//, /\/user\/password-reset\//, /^\/user\/reset\/[a-z0-9_-]*/]}));
EDIT: this SO answer suggest that you cannot combine regex and strings in the same array, so I've converted all paths to regex expressions.
You have an array within an array for the value of path passed into unless.
Change:
}).unless({ path: ['/images/', '/user/password-reset', unprotected ]}));
To:
}).unless({ path: unprotected }));
And change unprotected to include the other paths:
const unprotected = [
'/images/',
'/user/password-reset',
pathToRegexp('/user/reset/:token', tokens),
];
Ensure you test with a path that includes a token e.g:
/user/reset/123-some-real-looking-value