We Keep Coding

What is the right way to safely and accurately insert user-provided URL data into an HTML5 document?

Given an arbitrary customer input in a web form for a URL, I want to generate a new HTML document containing that URL within an href. My question is how am I supposed to protect that URL within my HTML.
What should be rendered into the HTML for the following URLs that are entered by an unknown end user:
http://example.com/?file=some_19%affordable.txt
http://example.com/url?source=web&last="f o o"&bar=<
https://www.google.com/url?source=web&sqi=2&url=https%3A%2F%2Ftwitter.com%2F%3Flang%3Den&last=%22foo%22
If we assume that the URLs are already uri-encoded, which I think is reasonable if they are copying it from a URL bar, then simply passing it to attr() produces a valid URL and document that passes the Nu HTML checker at validator.w3.org/nu.
To see it in action, we set up a JS fiddle at https://jsfiddle.net/kamelkev/w8ygpcsz/2/ where replacing the URLs in there with the examples above can show what is happening.
For future reference, this consists of an HTML snippet
<a>My Link</a>
and this JS:
$(document).ready(function() {
$('a').attr('href', 'http://example.com/request.html?data=>');
$('a').attr('href2', 'http://example.com/request.html?data=<');
alert($('a').get(0).outerHTML);
});
So with URL 1, it is not possible to tell if it is URI encoded or not by looking at it mechanically. You can surmise based on your human knowledge that it is not, and is referring to a file named some_19%affordable.txt. When run through the fiddle, it produces
My Link
Which passes the HTML5 validator no problem. It likely is not what the user intended though.
The second URL is clearly not URI encoded. The question becomes what is the right thing to put into the HTML to prevent HTML parsing problems.
Running it thru the fiddle, Safari 10 produces this:
My Link
and pretty much every other browser produces this:
My Link
Neither of these passes the validator. Three complaints are possible: the literal double quote (from un-escaping HTML), the spaces, or the trailing < character (also from un-escaping HTML). It just shows you the first of these it finds. This is clearly not valid HTML.
Two ways to try to fix this are a) html-escape the URL before giving it to attr(). This however results in every & becoming & and the entities such as & and < become double-escaped by attr(), and the URL in the document is entirely inaccurate. It looks like this:
My Link
The other is to URI-encode it before passing to attr(), which does result in a proper validating URL which actually clicks to the intended destination. It looks like this:
My Link
Finally, for the third URL, which is properly URI encoded, the proper HTML that validates does come out.
My Link
and it does what the user would expect to happen when clicked.
Based on this, the algorithm should be:
if url is encoded then
pass as-is to attr()
else
pass encodeURI(url) to attr()
however, the "is encoded" test seems to be impossible to detect in the affirmative based on these two prior discussions (indeed, see example URL 1):
How to find out if string has already been URL encoded?
How to know if a URL is decoded/encoded?
If we bypass the attr() method and forcibly insert the HTML-escaped version of example URL 2 into the document structure, it would look like this:
My Link
Which seemingly looks like valid HTML, yet fails the HTML5 validator because it unescapes to have invalid URL characters. The browsers, however, don't seem to mind it. Unfortunately, if you do any other manipulation of the object, the browser will re-escape all the &'s anyway.
As you can see, this is all very confusing. This is the first time we're using the browser itself to generate the HTML, and we are not sure if we are getting it right. Previously, we did it server side using templates, and only did the HTML-escape filter.
What is the right way to safely and accurately insert user-provided
URL data into an HTML5 document (using JavaScript)?

If you can assume the URL is either encoded or not encoded, you may be able to get away with something along the lines of this. Try to decode the URL, treat an error as the URL not being encoded and you should be left with a decoded URL.
<script>
var inputurl = 'http://example.com/?file=some_19%affordable.txt';
var myurl;
try {
myurl = decodeURI(inputurl);
}
catch(error) {
myurl = inputurl;
}
console.log(myurl);
</script>

What type of regexp would I need to extract image url from markdown?

Given the following
Some post content here
![Habits Are Hard to Build and Break](https://i.imgur.com/vv0UfA5l.png)
Yad ayaday dyaya
* Hi
* There
How could I extract the first image in markdown? E.g I basically want to parse it then save the first image to a variable like extractedImage "https://i.imgur.com/vv0UfA5l.png"
extractImageFromMarkdown(text) {
// somehow parse text to grab first image_url
return extractedImageUrl;
}

text.match(/!\[.*?\]\((.*?)\)/)[1]
This assumes that you can't have any escaped brackets though, like ![\]] which is probably a safe assumption. This will also get only the first image as requested.

htmlparser2 convert xml object into string

I'm having a little trouble getting the htmlparser2 module (using node.js) to output an xml string. Basically I'm parsing it in like so:
var htmlparser=require('htmlparser2');
function(xmlString,cb){
var handler=new htmlparser.DomHandler(cb);
var parser = new htmlparser.Parser(handler);
parser.write(xmlString);
parser.done();
}
Then I get an object called "dom", which I do some work on. After that work is done, I want to export it back into an XML string. I know that htmlparser.DomUtils.getOuterHTML(dom) works for HTML objects, but doesn't work for XML (at least not by default). I get back <undefined></undefined> when I call htmlparser.DomUtils.getOuterHTML(dom) on an xml dom.
Thanks in advance for any help you're able to offer!
-Dylan

Fixed. Basically you need to use a second option in getOuterHTML...
htmlparser.DomUtils.getOuterHTML(dom,{xmlMode:true})
If that doesn't work, try calling on the inner elems in the array like:
htmlparser.DomUtils.getOuterHTML(dom[0],{xmlMode:true})
You'll have to wrap a for loop around it to get the whole document, but it works for me!

Javascript - search for HTML elements in string

I have string with html elements. There are tables with captions. I need to find table which has caption with certain text and then return this table - as a string.
What is the best way to do this with simple javascript, without any libraries ?
F.e. this is an initial string
<table border="1"><caption><strong>First</strong></caption><tbody><tr><td>...</td></tr></tbody></table><table border="1"><caption><strong>Result</strong></caption><tbody><tr><td>...</td></tr></tbody></table><table border="1"><caption><strong>Last</strong></caption><tbody><tr><td>...</td></tr></tbody></table>
I want to get this string :
<table border="1"><caption><strong>Result</strong></caption><tbody><tr><td></td></tr></tbody></table>
Any advice or algorithm how to effeciently resolve this problem ? The challenge is to resolve it with javascript without using any third-party libraries and also without converting text into xml or something similar (because some of html code is not well formatted and it causes errors).

I have not had time to completely test this, but you might be able to try using a regular expression and the match() function. Assuming your table string is in a variable called str, then something along the lines of
var res = str.match(\b<table\.\w+_</table>\b);
res will be an array of matches of strings that begin with '', which you could then check to see which string contains the caption that you need.
Hope that helps!

Fix JS object replacing/encoding of strings?

I have an object with a value that has spaces in it, and it gets replaced with an encoded string, like:
alldata["test"] will return "Long+name"
or something like
alldata["test"] will return "%BLong+name%B"
when it's set by using
alldata["test"] = "Long name" (or "[Long name]") via a series of code.
Am I missing something? I don't think using $.toEvalJSON is the right way to go because I haven't transformed the object into JSON. I'd rather not do a string.replace either because I'd have to capture every possible type of input that is encoded.
Thank you!

If your question is how to remove the encoding, you could always use
unescape(s)
See Escape and Unescape Functions

The issue is related to the fact that I failed to mention that the object was being assigned the string as a result of a .serialize() command. Hence a urldecode() will work perfectly.