We Keep Coding

Stripping filename from HTML Encoded UNIX Path

I am writing a NodeJS application using Express and Google Datastore. I am trying to get the filename from a UNIX path. The path is stored in an HTML encoded format in the database.
Here's the path un-encoded:
/toplevel/example/text123.txt
Here's how the path is stored in the database HTML encoded format:
/toplevel/example/test123.txt
Since the path is HTML encoded, this line is not working.
let filename_only = requested_filepath_unescaped.split('/').pop().toString();
I also tried splitting by the encoded characters but that does not work either (perhaps because split doesn't work with multiple characters?)
let filename_only = requested_filepath_unescaped.split('&#x2F').pop().toString();
What is the best way to either split the string as-is, or de-code the HTML back into an unencoded string?

Well, split works with multiple characters, so I don't know what goes wrong when you tried it.
However if you can use jQuery, you can also decode the html like this:
var htmlDecoded = $('<div />').html(htmlEncoded).text()
After that you can split on '/'.
(The code I gave creates a div tag in memory (it is not added to the DOM, the web page), after that it sets the html of it, which automatically decodes the html entities.
EDIT:
As I am unsure what the problem of the OP is and I can't comment due to low reputation, I give some more suggestions here.
Maybe the variable you call split on is not really a string object. Try converting to string first:
var filename = filepath.toString().split('/');
Other option is to use regex, but I don't know what exactly solves that, but might be worth trying.
var filename = filepath.toString().split(/&#2F;/);
EDIT2: Tested and working in Chrome v62 and Node v6.11.4.

What is the right way to deal with special characters in resource URIs? How to properly escape?

Our application implements APIs to access entity details via AJAX GET requests - it once used POST but we decided to change our standards.
Unfortunately we hit a design flaw recently.
Suppose our API is http://localhost/app/module/entity/detail/{id} where {id} is a Spring #PathVariable that matches that entity's primary key.
If the primary key is a numeric surrogate key (auto-increment) there is no problem then. But we thought that this all worked with String primary keys.
It happened that we found some valid production data contain slashes, backslashes and semicolons for primary key. Our tables cannot use auto-increment surrogate keys because of their gargantuar size.
More in general, we discovered that we were unprepared to handle non-alphanumeric characters.
What the defective code was originally
Here is how our application used to reach to display entity data in a single page form the entity list:
User navigates table.jsp where an AJAX list is retrieved via POST
An Angular expression constructs the link to the detail page of that entity by means of detail?entityId={{::row:idAttribute}}
A valid link is generated by Angular
Examples:
detail?entityId=5903475934 //numeric case, we have several details page with same navigation pattern
detail?entityID=AAABBBCCC
detail?entityID=DO0000099101\test
The user clicks on the link and the browser points to the corresponding address
http://localhost/blablabla/detail?entityId=DO0000099101\test //Could be escaped.... read more later
The page needs the ID code from the parameters to issue the correct AJAX call
Need to retrieve the Entity ID from the query string. The Angular controller is in a separate file that doesn't see the query string (and is included dynamically, having the same page name)
<script type="text/javascript">
var ____entityId = '${param.entityId}';
</script>
Which gets translated to
<script type="text/javascript">
var ____entityId = 'DO0000099101\test'; //Yes, I know it's incorrect because now we have a TAB
</script>
Angular fetches the Entity ID into the page scope
In the Angular controller, we do the following
$scope.entityId = ___entityId;
$http.get($const.urlController+'detail/'+$scope.entityId)
Ajax call is issued
http://localhost/blablablabla/entity/detail/DO0000099101est //URL is bad
Spring MVC decodes the #PathVariable
Unfortunately, the parameter is treated as DO0000099101est
What we tried to fix
We have tried to fix the mistake by escaping the \t as it is clear that its presence in the HTML content constitutes a bug. Javascript interprets it as a tab.
Trying to URL-escape the ID
We tried to manually navigate to http://localhost/blablabla/detail?entityId=DO0000099101%5Ctest by escaping the backslash to its URL entity. The purpose is that if this worked we could modify Angular code in the table page
The result is that the \t appeared again as it is in the Javascript fragment. From that point, the sequence is the same
<script type="text/javascript">
var ____entityId = 'DO0000099101\test';
</script>
Trying to Java-escape + URL-Escape the ID in the URL
What if the Angular table page escaped the URL by this way?
http://localhost/blablabla/detail?entityId=DO0000099101%5C%5Ctest
<script type="text/javascript">
var ____entityId = 'DO0000099101\\test'; //Looks great
</script>
Unfortunately Angular now reverses the backslash into a forward slash when performing the Ajax request
http://localhost/blablabla/detail/DO0000099101/test
Trying to perform the Ajax call manually with escaped URL
So know that the REST url is http://localhost/app/module/entity/detail/{id}, let's try to see how Spring expects the backslash to be escaped
http://localhost/app/module/entity/detail/DO0000099101%5Ctest //results in 400 error
http://localhost/app/module/entity/detail/DO0000099101\\test //Chrome reversed the backslash into a forward slash and the result is 404 as expected for http://localhost/app/module/entity/detail/DO0000099101//test
Using encodeURIComponent
We already tried this (but I didn't mention in the original post). To #Thomas comment, we tried again to encode the primary key by Javascript escaping in the controller itself.
$http.get($const.urlController+'detail/'+encodeURIComponent($scope.entityId))
The problem with this approach is that the backslash-T sequence in the URL is encoded to %09, which results in a tab decoded on the server side
Using both encodeURIComponent and a Java escape
Now we tried to use both approach #4 and fixing the hardcoded Java-escaped (i.e. \\t) into the Javascript for testing purposes.
<script type="text/javascript">
var ____entityId = 'DO0000099101\\test';
</script>
$http.get($const.urlController+'detail/'+encodeURIComponent($scope.entityId))
This time the Ajax call resulted in 400 error as case #3
http://localhost/app/module/entity/detail/DO0000099101%5Ctest //results in 400 error
Question time
I want to ask this question in two ways, one specific and one more general.
Specific: how can I properly format a #PathVariable so that it can preserve special characters?
General: in the REST world, what care is to be done when dealing with resource IDs that may contain special characters?
For the second, let me be more clear. If you control your REST application and generation of IDs, you can choose to allow only alphanumeric identifier (user-generated or random or whatever) and sleep happy.
But in our case we need to browse entities that come supplied from external systems that have broader restrictions on character sets allowed for entity primary identifiers.
And again, we cannot use auto-increment surrogate keys.

Problems with passing Euro Sign as URL parameter

I am trying to pass the Euro ( € ) sign as url parameter in my spring jsp. What is the right way to do so ? I tried the following with no avail. Problem is the character is getting encoded properly but not getting decoded from my destination jsp.
I am using
<%#page contentType="text/html;charset=UTF-8" %>
Here is the calling jsp:
<script>
...
// params contains the euro sign
document.location='dest.jsp?p='+escape(params);
In the dest.jsp
<input type="hidden" id="par" value="${param.p}">
and in a script in the same page
console.log($('#par').val())
when I use escape(params) I get the url as %u20AC . But no (empty) values in the dest.jsp
when I use encodeURI(params) or encodeURIComponent I get url as € . But the value in dest.jsp as ⬠- something which I can't use to render as euro sign

I'm going to assume you are using Tomcat because that's what I tested with and we get the same result.
What you will want to do is open up your Tomcat servlet.xml file and find the HTTP connector and add the useBodyEncodingForURI attribute with the value true.
<Connector connectionTimeout="20000" port="8080" protocol="HTTP/1.1"
redirectPort="8443" useBodyEncodingForURI="true">
</Connector>
Then, you will want to register a CharacterEncodingFilter to set the HttpServletRequest character encoding.
You can read more about this behavior in my answer here:
Character encoding in query string, hebrew

You need indeed to encode the € sign which should give %E2%82%AC using UTF-8. You need to be careful with the encoding you use on both ends.
Something like URLEncoder.encode(url, "UTF-8") on the client would do.
If you are using Spring, org.springframework.web.util.UriUtils has also nice utilities you can use.
If the decoding issue is on the server, you need first to make sure that your web container decodes the URI with the proper encoding.
Tomcat decodes URI with ISO-8859-1 by default so you need to update your connector configuration
<Connector port="8080" ...
URIEncoding="UTF-8"/>

See the following answers
Spring MVC: How to store € character?
Getting question mark instead accented letter using spring MVC 3
I think that org.springframework.web.filter.CharacterEncodingFilter should help here.
Try it with and without your encodeURI(params)

Not opening the url in window.open

I have a link in the page.On clicking the link the pop up should render with required url.I have the window.open code as follows
Click here
on clicking above link the popup is opening with url upto http://astrik.com/click?id=613*B and offerValue is not showing up in the url of pop up.I need the offerValue to be shown in the url.Any reason for not showing up the offerValue.

The provided URL contains a ? that indicates the start of an argument list. Each argument is separated by an &. Thus, for the parser the provided argument list look like this:
name=aravind, age=19, url=http://astrik.com/click?id=613*B and offerValue=2.9. They all belong to the request to /action.do.
You need to escape the last amp with its URL escape code %26 like this:
/action.do?name=aravind&age=19&url=http://astrik.com/click?id=613*B%26offerValue=2.9
to "bind" the last argument to the astrik URL. Maybe it is also required to escape the second ? with %3F.

You should escape the data parts in url's before using it. In this case the data part url is bugged by the ampersand(&) in the url.
var url = encodeURIComponent('http://astrik.com/click?id=613*B&offerValue=2.9');
Click here
(or just replace the ampersand, not sure now)

Try the following:
Click here
You must URL encode your parameters for the "url" value, otherwise they will get appended to the original URL.

Newline problem when writing to HTML using JavaScript

I'm using regular textbox as a text input where the users wrties their comments. I then use JQuery and JSON to send data to the server and then insert it into the database.
When I want to display this text I use jQuery to download it prepare HTML and display it in the browser, but there are no new lines.
How can I keep any newlines entered by the user so that they are displayed in the browser?
EDIT:
The problem is that when I do alert $('.detailsCommentContent').val() I can see line breaks in the alert window, but when I then pass it as a GET argument:
insertComment.aspx?id=10&content= " + $('.detailsCommentContent').val() "
then in the url there are no signs of newLine :(

Just do like this answer: keep formatting entered in asp.net textbox (carriage return, new line, etc)
theStringYouWantToFormat.Replace(char.ConvertFromUtf32(13),"<br/>")

Before writing out the HTML using javascript to the page, make sure to replace all the newlines with <br /> tags. Here is a simple extension for string that will allow you to do it using javascript (source):
String.prototype.NewlineToBR = function() {
return this.replace( /\r\n|\r|\n/g, br || '');
}
Usage:
var htmlString = newlineString.NewlineToBR();
Then just insert the new string into you HTML.

where do you want to display the text? in a textarea or directly on the page?
if on the page you'll have to convert the newlines to <br/> tags when getting the text from the db and printing it to the page.

I beleive this is down to the encoding you are using. Difference between unicode and ascii or something similar. It's been a while since I worked on something like this but I think it boiled down to two options.
match up the encoding on save and on load (we found that we had ascii on one and unicode on another).
replace all new line character with an arbituary value when saving and swap it back when you load it.