We Keep Coding

How can I replace some calls to JavaScript's eval() with Ext.decode()?

We are trying to get rid of all of our eval() calls in our JavaScript. Unfortunately, I am not much of a JavaScript programmer, and I need some help.
Many of our eval() calls operate on strings, outputs from a web service, that are very JSON-like, for example, we might eval the following string:
ClassMetaData['Asset$Flex'] = {
fields: {
}
,label: 'Flex Fields'
};
I've seen various suggestions on the Internet suggesting Ext.decode(). The documentation for it says - "Decodes (parses) a JSON string to an object. If the JSON is invalid, this function throws a SyntaxError unless the safe option is set." The string that I am supplying as an argument isn't legitimate JSON as I understand it (the field names aren't quoted), but Ext.decode() nearly works for me anyway. If I decode the above string, I get an error (why?) - "Uncaught SyntaxError: Unexpected token ;". However, if I remove the trailing semi-colon, and decode, everything seems to be fine.
I am using the following code to determine whether the decode call and the eval call do the same thing:
var evaled = eval(inputString);
var decoded = Ext.decode(inputString.replace(";", "")); // remove trailing ";", if any
console.log("Equal? - " + (JSON.stringify(decoded) == JSON.stringify(evaled)));
Unfortunately, this is not a very good solution. For example, some of the input strings to eval are fairly complex. They may have all sorts of embedded characters - semicolons, HTML character encodings, etc. Decode may complain about some other syntax problem, besides semicolons at the end, and I haven't found a good way to determine where the problem is that decode objects to. (It doesn't say "illegal character in position 67", for example.)
My questions:
Could we, with a small amount of work, create a generic solution
using decode?
Is there an easy way to convert our JSON-like input
into true JSON?
Is there a better way of comparing the results of
eval and decode than JSON.stringify(decoded) == JSON.stringify(evaled)?

How to insert arbitrary JSON in HTML's script tag

I would like to store a JSON's contents in a HTML document's source, inside a script tag.
The content of that JSON does depend on user submitted input, thus great care is needed to sanitise that string for XSS.
I've read two concept here on SO.
1. Replace all occurrences of the </script tag into <\/script, or replace all </ into <\/ server side.
Code wise it looks like the following (using Python and jinja2 for the example):
// view
data = {
'test': 'asdas</script><b>as\'da</b><b>as"da</b>',
}
context_dict = {
'data_json': json.dumps(data, ensure_ascii=False).replace('</script', r'<\/script'),
}
// template
<script>
var data_json = {{ data_json | safe }};
</script>
// js
access it simply as window.data_json object
2. Encode the data as a HTML entity encoded JSON string, and unescape + parse it in client side. Unescape is from this answer: https://stackoverflow.com/a/34064434/518169
// view
context_dict = {
'data_json': json.dumps(data, ensure_ascii=False),
}
// template
<script>
var data_json = '{{ data_json }}'; // encoded into HTML entities, like < > &
</script>
// js
function htmlDecode(input) {
var doc = new DOMParser().parseFromString(input, "text/html");
return doc.documentElement.textContent;
}
var decoded = htmlDecode(window.data_json);
var data_json = JSON.parse(decoded);
This method doesn't work because \" in a script source becames " in a JS variable. Also, it creates a much bigger HTML document and also is not really human readable, so I'd go with the first one if it doesn't mean a huge security risk.
Is there any security risk in using the first version? Is it enough to sanitise a JSON encoded string with .replace('</script', r'<\/script')?
Reference on SO:
Best way to store JSON in an HTML attribute?
Why split the <script> tag when writing it with document.write()?
Script tag in JavaScript string
Sanitize <script> element contents
Escape </ in script tag contents
Some great external resources about this issue:
Flask's tojson filter's implementation source
Rail's json_escape method's help and source
A 5 year long discussion in Django ticket and proposed code

Here's how I dealt with the relatively minor part of this issue, the encoding problem with storing JSON in a script element. The short answer is you have to escape either < or / as together they terminate the script element -- even inside a JSON string literal. You can't HTML-encode entities for a script element. You could JavaScript-backslash-escape the slash. I preferred to JavaScript-hex-escape the less-than angle-bracket as \u003C.
.replace('<', r'\u003C')
I ran into this problem trying to pass the json from oembed results. Some of them contain script close tags (without mentioning Twitter by name).
json_for_script = json.dumps(data).replace('<', r'\u003C');
This turns data = {'test': 'foo </script> bar'}; into
'{"test": "foo \\u003C/script> bar"}'
which is valid JSON that won't terminate a script element.
I got the idea from this little gem inside the Jinja template engine. It's what's run when you use the {{data|tojson}} filter.
def htmlsafe_json_dumps(obj, dumper=None, **kwargs):
"""Works exactly like :func:`dumps` but is safe for use in ``<script>``
tags. It accepts the same arguments and returns a JSON string. Note that
this is available in templates through the ``|tojson`` filter which will
also mark the result as safe. Due to how this function escapes certain
characters this is safe even if used outside of ``<script>`` tags.
The following characters are escaped in strings:
- ``<``
- ``>``
- ``&``
- ``'``
This makes it safe to embed such strings in any place in HTML with the
notable exception of double quoted attributes. In that case single
quote your attributes or HTML escape it in addition.
"""
if dumper is None:
dumper = json.dumps
rv = dumper(obj, **kwargs) \
.replace(u'<', u'\\u003c') \
.replace(u'>', u'\\u003e') \
.replace(u'&', u'\\u0026') \
.replace(u"'", u'\\u0027')
return Markup(rv)
(You could use \x3C instead of \u003C and that would work in a script element because it's valid JavaScript. But might as well stick to valid JSON.)

First of all, your paranoia is well founded.
an HTML-parser could be tricked by a closing script tag (better assume by any closing tag)
a JS-parser could be tricked by backslashes and quotes (with a really bad encoder)
Yes, it would be much "safer" to encode all characters that could confuse the different parsers involved. Keeping it human-readable might be contradicting your security paradigm.
Note: The result of JSON String encoding should be canoncical and OFC, not broken, as in parsable. JSON is a subset of JS and thus be JS parsable without any risk. So all you have to do is make sure the HTML-Parser instance that extracts the JS-code is not tricked by your user data.
So the real pitfall is the nesting of both parsers. Actually, I would urge you to put something like that into a separate request. That way you would avoid that scenario completely.
Assuming all possible styles and error-corrections that could happen in such a parser it might be that other tags (open or close) might achieve a similar feat.
As in: suggesting to the parser that the script tag has ended implicitly.
So it is advisable to encode slash and all tag braces (/,<,>), not just the closing of a script-tag, in whatever reversible method you choose, as long as long as it would not confuse the HTML-Parser:
Best choice would be base64 (but you want more readable)
HTMLentities will do, although confusing humans :)
Doing your own escaping will work as well, just escape the individual characters rather than the </script fragment
In conclusion, yes, it's probably best with a few changes, but please note that you will be one step away from "safe" already, by trying something like this in the first place, instead of loading the JSON via XHR or at least using a rigorous string encoding like base64.
P.S.: If you can learn from other people's code encoding the strings that's nice, but you should not resort to "libraries" or other people's functions if they don't do exactly what you need.
So rather write and thoroughly test your own (de/en)coder and know that this pitfall has been sealed.

Escape with Unicode Code Points similar to PHP in Javascript

When using PHP to generate JSON, it encodes higher characters using the \u0123 code-point notation.
(I know this not necessary, but for unnamed reasons I want that.)
I am trying to achieve the same in JavaScript. I searched high and low and found nothing. The encodeUri function does nothing for me (even though many suggested that it would).
Any helpful hints? I hope that I do not have to use some big external library but rather something build-in or a few lines of nice code - this can not be this hard, can it...?!
I have an input string in the form of:
var stringVar = 'Hällö Würld.';
My desired conversion would give me something like:
var resultStringVar = 'H\u00e4ll\u00f6 W\u00fcrld.';

I’ve made a library for just that called jsesc. From its README:
This is a JavaScript library for escaping JavaScript strings while generating the shortest possible valid ASCII-only output. Here’s an online demo.
This can be used to avoid mojibake and other encoding issues, or even to avoid errors when passing JSON-formatted data (which may contain U+2028 LINE SEPARATOR, U+2029 PARAGRAPH SEPARATOR, or lone surrogates) to a JavaScript parser or an UTF-8 encoder, respectively.
For your specific example, use it as follows:
var stringVar = 'Hällö Würld.';
var resultStringVar = jsesc(stringVar, { 'json': true, 'wrap': false });

How to retain escape sequences while extracting strings from json objects in javascript. (Strings are actually regex patterns)

I have a JSON object as follows:
var jsonObject = {"regex":"<span class=\"Value\">\\$(.+?)<\\/span>"};
My target is to use this regular expression to scrape a value from a html document.
var match = html.match(new RegExp(jsonObject.regex, 'i'));
This however returns an error. The problem seems to be that the escape sequences in the regex string are lost in the string jsonObject.regex
A call to jsonObject.regex returns
< span class="Value">\$(.+?)<\ /span>
(The escape sequences like \" and \\ are lost)
I could replace the respective characters using javascript, but it seems the inefficient thing to do since I already have the correct format in the json object.
Any clues or workarounds are appreciated. Thanks!

You are doing two things wrong here.
First and foremost, you are trying to build a program that uses arbitrary regular expressions on HTML. Don't do that. You have a DOM at your disposal on the client side, you should use one of the selector engines available. Examples include the browser built-in document.querySelectorAll(), Sizzle (which is also part of jQuery), NWMatcher, or an XPath-based selector engine like XPath.js.
Then, you obviously do not use a JSON serializer to build your JSON string on the server side, or things like messed-up escaping would not happen on the client side.
Lastly, what you have in your first code sample is not JSON. It's a JavaScript object literal. JSON is always a string:
'{"regex":"<span class=\"Value\">\\$(.+?)<\\/span>"}'
Selecting what you seem to want in jQuery would become as simple as
var value = $("span.value").text();
But as I said, you are not bound to use jQuery, there are lighter-weight alternatives if HTML-scraping is your main goal.

Can you create a JavaScript string without using ' or " quotes?

I have a JS file with some XML in it, where the XML is supposed to get converted to a word by the server.
E.g.
var ip = "<lang:cond><lang:when test="$(VAR{'ip_addr'})">$(VAR{'ip_addr'})</lang:when></lang:cond>";
This gets converted to:
var ip = "192.168.0.0";
However, in case the server doesn't work as intended, I don't want there to be a syntax error, and this is VERY important. Currently there would be a syntax error because the language uses both types of quotes. I can't think of a way to get around this, but perhaps there's another way to do quotes in JavaScript? Or to create a string?
For example, in Python I'd use triple quotes:
ip = """<lang:cond><lang:when test="$(VAR{'ip_addr'})">$(VAR{'ip_addr'})</lang:when></lang:cond>"""
Anyone have a bright idea?

I have had to create strings without quotes for a project as well. We were delivering executable client javascript to the browser for an internal website. The receiving end strips double and single quotes when displayed. One way I have found to get around quotes is by declaring my string as a regular expression.
var x = String(/This contains no quotes/);
x = x.substring(1, x.length-1);
x;

Using String prototype:
String(/This contains no quotes/).substring(1).slice(0,-1)
Using String.fromCharCode
String.fromCharCode(72,69,76,76,79)
Generate Char Codes for this:
var s = "This contains no quotes";
var result = [];
for (i=0; i<s.length; i++)
{
result.push(s.charCodeAt(i));
}
result

In JavaScript, you can escape either type of quote with a \.
For example:
var str = "This is a string with \"embedded\" quotes.";
var str2 = 'This is a string with \'embedded\' quotes.';
In particular, your block of JavaScript code should be converted to:
var ip = "<lang:cond><lang:when test=\"$(VAR{'ip_addr'})\">$(VAR{'ip_addr'})</lang:when></lang:cond>";
In general, I always prefer to escape the quotes instead of having to constantly switch quote types, depending upon what type of quotes may be used within.

I was looking for a solution to the same problem. Someone suggested looking at https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/template_strings which proved helpful. After reading about half the article, it stated that you can create strings with the backward tick character. (`)
Try this :)
document.getElementById('test').innerHTML = `'|'|'|"|"`
<div id="test" style="font-size:3em;"></div>

You can't create a string without using a single or double quote, as even calling the String() prototype object directly still requires you to pass it the string.
Inside XML you would use CDATA, but inside JS you'll have to just escape the '\"strings\"' "\'appropriately\'"