how do I properly encode a URL in JavaScript? - javascript

I am working on a browser plugin that takes the URL of the current page or any selected link as a parameter and send it to our server.
When characters of the basic latin alphabet are present in the url (like http://en.wikipedia.org/wiki/Vehicle), the plugin works fine. However, when the URL contains characters from another alphabet such as http://ru.wikipedia.org/wiki/Коляска the plugin does not work, I do use the encodeURIComponentmethod but that does not seem to solve the issue. Any idea?
Thanks,
Olivier.

You probably want to use encodeURI/decodeURI, if you are trying to take a full URI with non-ASCII characters and translate it into its encoded form. These preserve special URI characters, such as : and /, instead of escaping them; it only escapes non-ASCII characters and characters that are invalid in URIs. Thus, they do essentially the same thing as typing in the address bar or putting a URI in an <a href="..."> (though the behavior might vary somewhat between browser, and isn't exactly the same).
encodeURIComponent is intended for encoding only a single component of a URI, replacing special characters that are meaningful in URIs, so that you can use the component as a query parameter or path component in a longer URI.

This one says it covers UTF-8. http://www.webtoolkit.info/javascript-url-decode-encode.html. Might solve your problem

Related

Correct way to escape SVG in `data:` URIs?

Chrome has recently started blocking URLs with new line and < characters.
I maintain an application which heavily relies on SVG images within a data URI (data:image/svg+xml;utf8,<svg>......). The reason for using the data: URI instead of a traditional image or SVG tag is that the SVGs are loaded at run time from an SVG attribute within a larger JSON document.
Will it be possible to use SVGs in data: URLs after this change takes place in Chrome 60? What are my alternatives if not?
UPDATE: User Álvaro González recommends using encodeURIComponent which indeed makes the deprecation warning go away.
A data URI is still a URI so encodeURIComponent() should be the right tool:
The encodeURIComponent() function encodes a Uniform Resource
Identifier (URI) component by replacing each instance of certain
characters by one, two, three, or four escape sequences representing
the UTF-8 encoding of the character (will only be four escape
sequences for characters composed of two "surrogate" characters).
Don't forget to apply it to the image data, not the entire URI.
If the URI is used in CSS that's all you'd need. If you want to inject it in HTML you should also apply HTML encoding on top of that (this time for the complete URI).
encodeURIComponent() does the job, but escapes more than it needs to. Here's a blog post detailing what seems to work best, such as minimizing escape sequences and fixing things like using double-quotes where single-quotes would be more efficient:
https://codepen.io/tigt/post/optimizing-svgs-in-data-uris
It includes an tool to encode SVG for CSS, which can be used locally or online. Here's another online tool:
https://yoksel.github.io/url-encoder/

URL encoding inside browsers

I was reading about URL encode inside browser. I know it is done to send URL over internet. But when I see this encoding inside address bar, it seems that only some characters being encoded. However, when I copy and paste inside notepad, all SPECIAL CHARACTERS seem to be encoded. What we see in address browser is fake? Actually all characters are encoded? Do browsers use javascript function encodeURI() for this purpose?
Yes, what the address bar shows is often not the actual URL. Most browsers decode encoded characters to display a more friendly looking URL, and to allow you to read the URL as actual characters, which may or may not be useful.
However, some characters look extremely similar to other characters, like the cyrillic letter А (U+0410) which is virtually indistinguishable from the latin letter A (U+0041). To avoid confusion and/or phishing attacks, letters which are easily confused may not be decoded.

When is it necessary to escape a JavaScript bookmarklet?

Bookmarklets I've made seem to always work well even though I don't escape them, so why do bookmarklet builders like John Gruber's escape special characters like space to %20?
If you copy/paste code into a bookmark using the browsers bookmark properties editor, you don't need to encode anything except newlines and percent signs. But, in fact, Firefox (and maybe other browsers) will encode spaces for you. If you check the properties again after saving, you'll see them encoded. You need to encode newlines because a bookmark can be only 1 line. You need to encode percent signs because the browser expects percent sights to always represent the first character in an encoded set. If you put the bookmarklet on a website in the HREF property of an A tag, the same rules apply, but you must also either HTML encode or URL encode any double quotes.
A lot of bookmarklet makers will just use encodeURIComponent, which encodes more than what is strictly necessary, but it might be considered more "correct".

Why is character encoding important for URLs?

I'm currently learning JavaScript, and I don't understand why it is important to encode URLs.
>>> var url = 'http://www.packtpub.com/scr ipt.php?q=this and that';
>>> encodeURI(url);
"http://www.packtpub.com/scr%20ipt.php?q=this%20and%20that"
For instance, in this example what purpose would it serve to change the first URL to the latter one.
It depends on what you're going to be doing with that URL.
When you just use a document.location = url, you don't want it encoded.
If you plan on passing that URL as a variable, then yes you want it encoded or it will confuse the browser. For instance:
http://www.someurl.com?myFavwebsite=http://www.stackoverflow.com?someParam=test.
See how that could be confusing to the browser?
By the way, never use a space in a url or php file. i've always found that to cause unnecessary stress. :)
Only a limited number of characters are allowed in URLs, according to the RFC 3986 standard. If you have a space in a URL, for example, this will make the URL invalid unless you encode it.
Often, browsers can deal with URLs that are not properly encoded by doing the encoding themselves, but that's not something you should rely on as a web developer.
URL encoding is also critical when using URLs as parameters of another URL. In this case the reserved characters of the URL need to be encoded, not just the non-permitted characters. For this, however, you don't use encodeURI, but encodeURIComponent.

should encodeURI ever be used?

Is there any valid use for javascript's encodeURI function?
As far as I can tell, when you are trying to make a HTTP request you should either have:
a complete URI
some fragment you want to put in a URI, which is either a unicode string or UTF-8 byte sequence
In the first case, obviously nothing needs to be done to request it. Note: if you actually want to pass it as a parameter (e.g ?url=http...) then you actually have an instance of the second case that happens to look like a URI.
In the second case, you should always convert a unicode string into UTF-8, and then call encodeURIComponent to escape all characters before adding it to a URI. (If you have a UTF-8 byte sequence instead of a unicode string you can skip the convert-to-utf8 step).
Assuming I havent missed anything, I can't see a valid use for encodeURI. If you use it, it's likely you've constructed an invalid URI and then attempted to "sanitize" it after the fact, which is simply not possible since you don't know which characters were intended literally, and which were intended to be escaped.
I have seen a lot of advice against using escape(), but don't see anybody discouraging encodeURI. Am I missing a valid use?
I have a blog post which answers this question in a lot of detail.
You should never use encodeURI to construct a URI programmatically, for the reasons you say -- you should always use encodeURIComponent on the individual components, and then compose them into a complete URI.
Where encodeURI is almost useful is in "cleaning" a URI, in accordance with Postel's Law ("Be liberal in what you accept, and conservative in what you send.") If someone gives you a complete URI, it may contain illegal characters, such as spaces, certain ASCII characters (such as double-quotes) and Unicode characters. encodeURI can be used to convert those illegal characters into legal percent-escaped sequences, without encoding delimiters. Similarly, decodeURI can be used to "pretty-print" a URI, showing percent-escaped sequences as technically-illegal bare characters.
For example, the URL:
http://example.com/admin/login?name=Helen Ødegård&gender=f
is illegal, but it is still completely unambiguous. encodeURI converts it into the valid URI:
http://example.com/admin/login?name=Helen%20%C3%98deg%C3%A5rd&gender=f
An example of an application that might want to do this sort of "URI cleaning" is a web browser. When you type a URL into the address bar, it should attempt to convert any illegal characters into percent-escapes, rather than just having an error. Software that processes URIs (e.g., an HTML scraper that wants to get all the URLs in hyperlinks on a page) may also want to apply this kind of cleaning in case any of the URLs are technically illegal.
Unfortunately, encodeURI has a critical flaw, which is that it escapes '%' characters, making it completely useless for URI cleaning (it will double-escape any URI that already had percent-escapes). I have therefore borrowed Mozilla's fixedEncodeURI function and improved it so that it correctly cleans URIs:
function fixedEncodeURI(str) {
return encodeURI(str).replace(/%25/g, '%').replace(/%5B/g, '[').replace(/%5D/g, ']');
}
So you should always use encodeURIComponent to construct URIs internally. You should only never use encodeURI, but you can use my fixedEncodeURI to attempt to "clean up" URIs that have been supplied from an external source (usually as part of a user interface).
encodeURI does not encode the following: , / ? : # & = + $ # whereas encodeURIComponent does.
There are a myriad of reasons why you might want to use encodeURI over encodeURIComponent, such as assigning a URL as a variable value. You want to maintain the URL but encode paths, query string and hash values. Using encodeURIComponent would make the URL invalid.

Categories