Encoding uri component, do not automatically decode - javascript

I have users with slahes in their usernames. I want to give them easy urls such as /user/username even if their username is problematic. ie /user/xXx/superboy.
I'm using client side routing and I don't think there's any wildcard support. One obvious way to fix this would be to encode their username. href="/user/xXx%2Fsuperboy". But the browser automatically decodes the url when going to the link and then my router ends up not matching anyway. Is there some way to keep the browser from automatically decoding the url or any other way to solve my problem (perhaps a different decoding scheme?). Thanks.
I'm using angularjs with angular ui-router for routing.

Part 1.
Automatic decoding of URIs can be encounted in many situations, such as it being interpreted once then the interpretation passed on (to be re-interpreted).
Part 2.
In a path in a URI, / has a special meaning, so you can't use it as the name of a file or directory. This means if you're mapping something that isn't a real path to a path, you may end up with unexpected characters causing problems. To solve this the characters need to be encoded.
As you want to map usernames to a URI, you have to consider this might happen, so you have to encode in a way that allows for this. From your question, it looks like this happens once, so you'll need to double encode any part of the URI that isn't a "real URI path".
Also maybe you can explain how reliable this is and whether it's advisable
If you always have it used in the same way, it should be reliable. As for advisable, it would be much better to use the query part, rather than the path for this. href="/user?xXx/superboy" is a valid URI and you can get the query string easily (everything after first ?, or an inbuilt method). The only character you'd have to watch for is #, which has special meaning again.

Related

Should we enclose filename with encodeURIComponent in Javascript?

I'm accepting files to be uploaded to my site. So, is it a safe practice to encodeURIComponent the filename? Or should I use escape()? OR is it necessary at all?
You should never use escape for anything (unless forced to because you're sending information to something that will use unescape [which it shouldn't]).
Whether you need to use encodeURIComponent depends entirely on whether you're going to use the filename directly as a URI component¹. If you are, yes, you should use it. If you aren't, no, you probably shouldn't.
¹ for instance, as a query string parameter when you're creating the query string manually rather than via URLSearchParams (which is generally better practice)
encodeURIComponent takes a string and escapes it to make it safe to insert into a URI, typically used for query string data.
If you are inserting a string into a URI then you can use it, but should probably use URLSearchParams to construct the whole query string instead.
If you aren't inserting a string into a URI then you probably should not use it.
escape is deprecated and should not be used. It doesn't work property with Unicode.
Considerations for accepting files are typically more along the lines of "Will this accidentally overwrite an existing file?" and "Are the characters in this filename allowed by my filesystem?".
Some people prefer to generate a completely new file name (e.g. with a guid library) to ensure it is safe. You could store the original name in a database (at which point your escaping should be handled by parametrised queries).

javascript encodeURIComponent and escape?

I use JS to sent encodeURIComponent string to a PHP file write and has been working fine for years; until recently I met with a strange effect that the text need to be further encoded with escape in order to get it to work! The sympton start to show only when I use an open source wysiwyg editor !
What could be the offending characters in URI that need escape to fix it? I used to think URI only reserve ? & = for its syntax to work.
The situation you describe could possibly be explained--although there's no way of knowing without you telling us what the string is, and how it's being used--by a URL which involves two levels of nested URL-like values.
Consider a URL taking a query parameter which is another URL:
http://me.com?url=http://you.com?qp=1
That URL is subject to misinterpretation, so we would normally URL-encode the you.com URL, giving us:
http://me.com?url=http%3A%2F%2Fyou.com%3Fqp%3D1
Whoever is working with this URL can now extract the query parameter named url with the value http%3A%2F%2Fyou.com%3Fqp%3D1, decode it (often a framework or library will decode it for you), and then use it to jump to or call that URL.
Consider, however, the case where the you.com URL itself has a query parameter, not ?qp=1 as given in the first example, but rather something that itself needs to be URL-encoded. To keep things simple, we'll just use "cat?pictures". We'd need to encode that, making the query parameter
In other words, the URL in question is going to be
?qp=cat%3Fpictures
If we just use that as is, then our entire URL becomes
http://me.com?url=http%3A%2F%2Fyou.com%3Fqp=cat%3Fpictures
Unfortunately, if we now decode that in a naive way, we get
http://me.com?url=http://you.com?qp=cat?pictures
In other words, the nested URL has been decoded as well, meaning that it will think the URL has two query paramters, namely url and qp. To successfully deal with this problem, we need to encode the second query parameter a second time, yielding
http://me.com?url=http%3A%2F%2Fyou.com%3Fqp%3Dcat%253Fpictures
Please note, however, that if you use your language or environment's built-in tools and libraries for handling query parameters, most of this will happen automatically and prevent you from having to worry about it.
The symptom start to show only when I use an open source wysiwyg editor
An editor merely places characters in a file. It's very hard to imagine that an editor is causing the problem you refer to, unless perhaps one editor is configured to use smart quotes, for example, which would pretty much break everything that involved quotes.

XSS preventing for parameters passed to JS code file

I have the following flow: A URL with query parameters, that runs some logic on the server side, and then generates using a template engine a stub HTML page with
A javascript file included, that does the main logic.
<script> tag that includes a JS object, that has parameters to this JS code, partially taken from the query parameters before.
Now I want to sanitize the parameters I receive, to prevent XSS injection. The issue, that one of the parameters there is a token, so I don't want to be too strict on the validations (simply not allowing all possible XSS characters sounds too strict), yet most of the libraries I've found dealing with pure HTML, and not a JS code (within <script> tag). I also feeling a bit uneasy, when I read all the regex solution, because I'm used to trust open source libraries when dealing with security (that have unit tests and not a bunch of regex).
Any advice on libraries & possible approach? We run in JVM environment.
The easiest, simplest, and therefore more secure approach is to use data attributes to represent the dynamic, user supplied values.
This way you only need to worry about HTML encoding, none of the complex hex entity encoding (\x00) that OWASP recommend.
For example, you could have:
<body data-token="#param.token" />
Where #param.token will output an HTML encoded version of the query string parameter. e.g. page?token=xyz" would output
<body data-token="xyz"" />
This will mitigate your XSS vulnerability concern.
Then you can use something like JQuery to easy retrieve the data attribute values in your JavaScript:
var token = $("body").data("token");
Simple and secure.
Imagining you want to assign your parameter as a string, as such:
{
...
x: '[PARAMETER]'
}
You want to make sure that [PARAMETER] does not break out of the quoted string.
In this case what you need to escape is the ' character and the closing </script>tag. Note: take into consideration "escape-the-escape" attacks, where the attacker sends the string \', which is escaped as \\', which turns back to ' (and you are back from where you started).
It's generally simply safer, as OWASP notes, to
escape all characters less than 256 with the \xHH format
I invite you to read the OWASP page on XSS attacks, and in particular https://www.owasp.org/index.php/XSS_%28Cross_Site_Scripting%29_Prevention_Cheat_Sheet#RULE_.233_-_JavaScript_Escape_Before_Inserting_Untrusted_Data_into_JavaScript_Data_Values

Should I worry that using GET in a form element doesn't automatically URL-encode angle brackets?

So I decided to use GET in my form element, point it to my cshtml page, and found (as expected) that it automatically URL encodes any passed form values.
I then, however, decided to test if it encodes angle brackets and surprisingly found that it did not when the WebMatrix validator threw a server error warning me about a potentially dangerous value being passed.
I said to myself, "Okay, then I guess I'll use Request.Unvalidated["searchText"] instead of Request.QueryString["searchText"]. Then, as any smart developer who uses Request.Unvalidated does, I tried to make sure that I was being extra careful, but I honestly don't know much about inserting JavaScript into URLs so I am not sure if I should worry about this or not. I have noticed that it encodes apostrophes, quotations, parenthesis, and many other JavaScript special characters (actually, I'm not even sure if an angle bracket even has special meaning in JavaScript OR URLs, but it probably does in one, if not both. I know it helps denote a List in C#, but in any event you can write script tags with it if you could find a way to get it on the HTML page, so I guess that's why WebMatrix's validator screams at me when it sees them).
Should I find another way to submit this form, whereas I can intercept and encode the user data myself, or is it okay to use Request.Unvalidated in this instance without any sense of worry?
Please note, as you have probably already noticed, my question comes from a WebMatrix C#.net environment.
Bonus question (if you feel like saving me some time and you already know the answer off the top of your head): If I use Request.Unvalidated will I have to URL-decode the value, or does it do that automatically like Request.QueryString does?
---------------------------UPDATE----------------------------
Since I know I want neither a YSOD nor a custom error page to appear simply because a user included angle brackets in their "searchText", I know I have to use Request.Unvalidated either way, and I know I can encode whatever I want once the value reaches the cshtml page.
So I guess the question really becomes: Should I worry about possible XSS attacks (or any other threat for that matter) inside the URL based on angle brackets alone?
Also, in case this is relevant:
Actually, the value I am using (i.e. "searchText") goes straight to a cshtml page where the value is ran through a (rather complex) SQL query that queries many tables in a database (using both JOINS and UNIONS, as well as Aliases and function-based calculations) to determine the number of matches found against "searchText" in each applicable field. Then I remember the page locations of all of these matches, determine a search results order based on relevance (determined by type and number of matches found) and finally use C# to write the search results (as links, of course) to a page.
And I guess it is important to note that the database values could easily contain angle brackets. I know it's safe so far (thanks to HTML encoding), but I suppose it may not be necessary to actually "search" against them. I am confused as to how to proceed to maximum security and functional expecations, but if I choose one way or the other, I may not know I chose the wrong decision until it is much too late...
URL and special caracters
The url http://test.com/?param="><script>alert('xss')</script> is "benign" until it is read and ..
print in a template : Hello #param. (Potential reflected/persisted XSS)
or use in Javascript : divContent.innerHTML = '<a href="' + window.location.href + ... (Potential DOM XSS)
Otherwise, the browser doesn't evaluate the query string as html/script.
Request.Unvalidated/Request.QueryString
You should use Request.Unvalidated["searchText"] if you are expecting to receive special caracters.
For example : <b>User content</b><p>Some text...</p>
If your application is working as expected with QueryString["searchText"], you should keep it since it validate for potential XSS.
Ref: http://msdn.microsoft.com/en-us/library/system.web.httprequest.unvalidated.aspx

Why is character encoding important for URLs?

I'm currently learning JavaScript, and I don't understand why it is important to encode URLs.
>>> var url = 'http://www.packtpub.com/scr ipt.php?q=this and that';
>>> encodeURI(url);
"http://www.packtpub.com/scr%20ipt.php?q=this%20and%20that"
For instance, in this example what purpose would it serve to change the first URL to the latter one.
It depends on what you're going to be doing with that URL.
When you just use a document.location = url, you don't want it encoded.
If you plan on passing that URL as a variable, then yes you want it encoded or it will confuse the browser. For instance:
http://www.someurl.com?myFavwebsite=http://www.stackoverflow.com?someParam=test.
See how that could be confusing to the browser?
By the way, never use a space in a url or php file. i've always found that to cause unnecessary stress. :)
Only a limited number of characters are allowed in URLs, according to the RFC 3986 standard. If you have a space in a URL, for example, this will make the URL invalid unless you encode it.
Often, browsers can deal with URLs that are not properly encoded by doing the encoding themselves, but that's not something you should rely on as a web developer.
URL encoding is also critical when using URLs as parameters of another URL. In this case the reserved characters of the URL need to be encoded, not just the non-permitted characters. For this, however, you don't use encodeURI, but encodeURIComponent.

Categories