We Keep Coding

What is the right way to safely and accurately insert user-provided URL data into an HTML5 document?

Given an arbitrary customer input in a web form for a URL, I want to generate a new HTML document containing that URL within an href. My question is how am I supposed to protect that URL within my HTML.
What should be rendered into the HTML for the following URLs that are entered by an unknown end user:
http://example.com/?file=some_19%affordable.txt
http://example.com/url?source=web&last="f o o"&bar=<
https://www.google.com/url?source=web&sqi=2&url=https%3A%2F%2Ftwitter.com%2F%3Flang%3Den&last=%22foo%22
If we assume that the URLs are already uri-encoded, which I think is reasonable if they are copying it from a URL bar, then simply passing it to attr() produces a valid URL and document that passes the Nu HTML checker at validator.w3.org/nu.
To see it in action, we set up a JS fiddle at https://jsfiddle.net/kamelkev/w8ygpcsz/2/ where replacing the URLs in there with the examples above can show what is happening.
For future reference, this consists of an HTML snippet
<a>My Link</a>
and this JS:
$(document).ready(function() {
$('a').attr('href', 'http://example.com/request.html?data=>');
$('a').attr('href2', 'http://example.com/request.html?data=<');
alert($('a').get(0).outerHTML);
});
So with URL 1, it is not possible to tell if it is URI encoded or not by looking at it mechanically. You can surmise based on your human knowledge that it is not, and is referring to a file named some_19%affordable.txt. When run through the fiddle, it produces
My Link
Which passes the HTML5 validator no problem. It likely is not what the user intended though.
The second URL is clearly not URI encoded. The question becomes what is the right thing to put into the HTML to prevent HTML parsing problems.
Running it thru the fiddle, Safari 10 produces this:
My Link
and pretty much every other browser produces this:
My Link
Neither of these passes the validator. Three complaints are possible: the literal double quote (from un-escaping HTML), the spaces, or the trailing < character (also from un-escaping HTML). It just shows you the first of these it finds. This is clearly not valid HTML.
Two ways to try to fix this are a) html-escape the URL before giving it to attr(). This however results in every & becoming & and the entities such as & and < become double-escaped by attr(), and the URL in the document is entirely inaccurate. It looks like this:
My Link
The other is to URI-encode it before passing to attr(), which does result in a proper validating URL which actually clicks to the intended destination. It looks like this:
My Link
Finally, for the third URL, which is properly URI encoded, the proper HTML that validates does come out.
My Link
and it does what the user would expect to happen when clicked.
Based on this, the algorithm should be:
if url is encoded then
pass as-is to attr()
else
pass encodeURI(url) to attr()
however, the "is encoded" test seems to be impossible to detect in the affirmative based on these two prior discussions (indeed, see example URL 1):
How to find out if string has already been URL encoded?
How to know if a URL is decoded/encoded?
If we bypass the attr() method and forcibly insert the HTML-escaped version of example URL 2 into the document structure, it would look like this:
My Link
Which seemingly looks like valid HTML, yet fails the HTML5 validator because it unescapes to have invalid URL characters. The browsers, however, don't seem to mind it. Unfortunately, if you do any other manipulation of the object, the browser will re-escape all the &'s anyway.
As you can see, this is all very confusing. This is the first time we're using the browser itself to generate the HTML, and we are not sure if we are getting it right. Previously, we did it server side using templates, and only did the HTML-escape filter.
What is the right way to safely and accurately insert user-provided
URL data into an HTML5 document (using JavaScript)?

If you can assume the URL is either encoded or not encoded, you may be able to get away with something along the lines of this. Try to decode the URL, treat an error as the URL not being encoded and you should be left with a decoded URL.
<script>
var inputurl = 'http://example.com/?file=some_19%affordable.txt';
var myurl;
try {
myurl = decodeURI(inputurl);
}
catch(error) {
myurl = inputurl;
}
console.log(myurl);
</script>

Ignore commas in a text input field when submitting

So I've made this search that does what its supposed to do front-end wise. However, when submitting I'd like the query to ignore commas.
Right now I'm using commas to make a comma separated search. The whole thing is, when I submit; the comma's are included and thus messes up my search values.
Is there any way to ignore comma's upon submit?
Example: Searching [Example][Test] will actually return Example,Test.
I've made a fiddle here
Any suggestions and help is greatly appreciated.

var firster = true;
//capture form submit
$('form.nice').submit(function(e){
if(firster){
// if its the first submit prevent default
e.preventDefault();
// update input value to have no commas
var val = $('input').val();
val = val.replace(/,/g, ' ');
$('input').val(val);
// let submit go through and submit
firster = false;
$(this).submit();
}
});
DEMO

Looking at your profile, I'm guessing you're using python as a server-side language. The issue you're trying to solve is best dealt with server-side: never rely on front-end code to escape or format data that is being used in a query... check Bobby Tables for more info
Anyhow, in python, you could try this:
ajaxString.replace(",","\", \"")
Thiis will replace all commas witIh " OR ", so a string like some, keywords is translated into some", "keywords, just add some_field IN (" and the closing ") to form a valid query.
Alternatively, you can split the keywords, and deal with them separately (which could come in handy when sorting the results depending on how relevant the results might be.
searchTerms = ajaxString.split(",")
>>>['some','keywords']
That should help you on your way, I hope.
Lastly, I'd suggest just not bothering with developing your own search function at all. Just add a google search to your site, they're the experts. There is just no way you, by yourself, can do better. Or even if you could, just imagine how long it'd take you!
Yes, sometimes a company will create their own search-engine, but only if they have a good reason to do so, and have the resources such an endevour requires. Programming is often all about being "cleverly lazy": Don't reinvent the wheel.

quick Jquery .load chat not working

I have the following jquery:
var msg = $("#newmessage").val();
var user = $("#userchat").val();
var filename = "/inc/chat.php?msg="+msg+"&user="+user;
alert(filename);
$("#chatData").load(filename);
when 'msg' does not have a space in it, the #chatData loads fine and posts the variable.
When it does have a space in it, I just get a blank div. With no information in it whatsoever.
if I load up the php file that inserts the data into the DB, and manually type the same GET data, it works fine.
Whats going on?

Try using
encodeURIComponent(msg)

Also consider:
$("#chatData").load('/inc/chat.php',
{ 'msg' : $("#newmessage").val(), 'user' : $("#userchat").val() }
);
URI encoding is done, if needed, by jQuery.
You don't have to worry about URI encoding as the POST method is used since data is provided as an object (source).
In this case POST may be better than GET anyways.
If you were using $_GET in your php file you will need to use either $_REQUEST or $_POST.

you have to encode your message before sending using encodeURIComponent() and decode on server-site using urldecode().
doing this will escape/encode special characters that aren't allowed in an url or that will break your query-string otherwise (like a & in your message that would otherwise start a new argument).

You can use either escape, encodeURI or encodeURIComponent, but escape is the only method supported by every browser, although most modern browsers support the latter.
Reference
Take a look at this document, which does a good job of explaining all three.

The space could be causing an issue - try javascript's encodeURIComponent():
var msg = encodeURIComponent($("#newmessage").val());
var user = encodeURIComponent($("#userchat").val());

Passing 2 URL Parameters?

I need to pass 2 URL parameters in a URL. The URL originates in an email and the user will click the link directing them to my site. The first parameter triggers a script on the page the second parameter is for a module my CMS will render from the parameter.
First Parameter is : message=1 (This parameter triggers the javascript)
The second Parameter is: name={tag_recipientfirstname} (My CMS will render the module)
The script that is called for the first looks like this:
<script type="text/javascript">
var $j = jQuery.noConflict();
$j(document).ready(function() {
var url = window.location.href;
url = url.toLowerCase();
if (url.indexOf('message=1') != -1) {
$j("a.message").colorbox({
open:true
});
}
$j("a.message").colorbox(); //not related to URL parameter
});
</script>
The second parameter is used on the page as:
<div>
<p>{ module_url,name} (again CMS will render this module)</p>
</div>
EDIT
I realize I left a couple things out:
First: How do I pass both parameters so they will both function as listed above?
And the CMS I am using is Business Catalyst.

//split the `location.search` string at the ampersands
var search_arr = window.location.search.replace('?', '').split('&'),
len = search_arr.length,
get_vars = {},
tmp = [];
//iterate through the key/value pairs and add them to the `get_vars` object
for (var i = 0; i < len; i++) {
tmp = search_arr[i].split('=');
get_vars[tmp[0]] = tmp[1];
}
//you can now access your GET variables through the `get_vars` object like: `get_vars.name`
//you can check for the existence of a certain GET variable like this
if (typeof(get_vars['message-1']) != 'undefined') {
$j("a.message").colorbox({
open:true
});
}
Here is a demo:http://jsfiddle.net/aBH8K/1/ (http://jsfiddle.net/aBH8K/1/show/?message-1=3 to see with get var)
Some related documentation:
window.location: https://developer.mozilla.org/en/DOM/window.location

Your question is not so much about generic development, rather a very specific commercial product; I do not know which plan you subscribed (free o pay-for?) with them but in any case it would be best to go through their support (see also my conclusion)
Nevertheless I'll try to put you on the right track.
Your questions
First,
the url in the email
In the email you will have somehow to build a link with the two parameters you want as #Jasper is explaining.
this means something like:
http://yourwebsite.com/destination/path/?message=1&name={tag_recipientfirstname}
Everything after the question mark is a GET query string.
Parameters are separated by the "&" symbol.
I definitely don't know how properly build urls in BC emails, but I feel like it should be an automated somewhere allowing you to specify additional parameters if you need.
the javascript
What you got will still work. It's not very nice, and you can use Jasper's solution or any other such as How can I get query string values in JavaScript?
Nothing to do then unless you want to make it better and more robust.
Business Catalyst (the page)
You usually have ways in a CMS to retrieve get parameters. Often something like
{ GET.param_name }
One step back
I am no expert with BC, but I have the feeling that you are taking a complicate path for something that is probably already baked in.
Again I suggest you go into their support section (though it's rather confusing I must say!) and try to understand what's the best way to achieve your objective. There are always many ways to skin a poor cat.
If you are getting support in your plan, definitely go that way and try to explain what you objectives are rather then how to achieve the technical solution that you think is the good one!

Add more GET data for Javascript

I am using the javascript below to send information from one website to another, but I don't know how to add more data. ?url is fine. I'm not sure how to add more data to the javascript. I would like to add an ?img, a ?title ... i tried a few times but no luck. Help please.
JavaScript
onclick="window.open('http://mysite.com/submit.?url='+(document.location.href));return false;"
PHP
$url = $_GET['url'];

Separate the parameters with &.
http://mysite.com/submit?param1=value1&param2=value2&param3=value3
You should also encode your values with encodeURI().

You wouldn't add ?moreParm...etc, you use an ampersand (&) to add additional parameters.
var url = `http://mysite.com/submit.?url=' + document.location.href;
url += '&anotherParam=foo`;
// etc.
You need to escape all parameter values accordingly.
I'd also recommend moving it into a function so your embedded JS doesn't become impossible to read.