What I'm doing is creating a truth table generator. Using a function provided by the user (e.g. a && b || c), I'm trying to make JavaScript display all combinations of a, b and c with the result of the function.
The point is that I'm not entirely sure how to parse the function provided by the user. The user could namely basically put everything he wants in a function, which could have the effect of my website being changed etc.
eval() is not secure at all; neither is new Function(), as both can make the user put everything to his liking in the function. Usually JSON.parse() is a great alternative to eval(), however functions don't exist in JSON.
So I was wondering how I can parse a custom boolean operator string like a && b || c to a function, whilst any malicious code strings are ignored. Only boolean operators (&&, ||, !) should be allowed inside the function.
Even if you check for boolean expressions, I could do this:
(a && b && (function() { ruinYourShit(); return true; })())
This is to illustrate that your problem is not solvable in the general case.
To make it work for your situation, you'd have to impose severe restrictions on variable naming, like requiring that all variables are a single alphabet letter, and then use a regular expression to kick it back if anything else is found. Also, to "match" a boolean expression, you'd actually have to develop a grammar for that expression. You are attempting to parse a non-regular language (javascript) and therefore you cannot write a regular expression that could match every possible boolean expression of arbitrary complexity. Basically, the problem you are trying to solve is really, really hard.
Assuming you aren't expecting the world to come crashing down if someone ruins your shit, you can develop a good enough solution by simply checking for the function keyword, and disallowing any logical code blocks contained in { and }.
I don't see the problem with using eval() and letting the user do whatever (s)he wants, as long as you have proper validation/filters on any of your server-side scripts that expect input. Sure, the user could do something that ruins your page, but (s)he can do that already. The user can easily enough run any javascript (s)he wants on your page already, using any number built-in browser features or add-ons.
What I would do would be to first split the user input into tokens (variable names, operators and possibly parentheses for grouping), then build an expression tree from that and then generate the relevant output from the expression tree (possibly after running some simplification on it).
So, say, break the string "a & !b" into the token sequence "a" "&" "!" "b", then go through and (eventually) build something like: booland(boolvar("a"), boolnot(boolvar("b"))) and you then have a suitable data structure to run your (custom, hopefully injection-free) evaluator over.
I ended up with a regular expression:
if(!/^[a-zA-Z\|\&\!\(\)\ ]+$/.test(str)) {
throw "The function is not a combination of Boolean operators.";
return;
}
Related
Once there was a search input.
It was responsible for filtering data in a table based on user input.
But this search input was special: it would not do anything unless a minimum of 3 characters was entered.
Not because it was lazy, but because it didn't make sense otherwise.
Everything was good until a new and strange (compared to English) language came to town.
It was Japanese and now the minimum string length of 3 was stupid and useless.
I lost the last few pages of that story. Does anyone remember how it ends?
In order to fix the issue, you obviously need to determine if user's input belongs to certain script(s). The most obvious way to do this is to use Unicode Regular Expressions:
var regexPattern = "[\\p{Katakana}\\p{Hiragana}\\p{Han}]+";
The only issue would be, that JavaScript does not support this kind of regular expressions out of the box. Anyway, you are lucky - there is a JS library called XRegExp and its Scripts add-on seems to exactly what you need. Now, the question is, whether you want to require at least three characters for non-Japanese or non-Chinese users, or do it otherwise - require at least three characters for certain scripts (Latin, Common, Cyrillic, Greek and Hebrew) while allowing any other to be searched on one character. I'd suggest the second solution:
if (XRegExp('[\\p{Latin}\\p{Common}\\p{Cyrillic}\\p{Greek}\\p{Hebrew}]+').test(input)) {
// test for string length and call AJAX if the string is long enough
} else {
// call AJAX search method
}
You might want to pre-compile the regular expression for better performance, but that's basically it.
I guess it mainly depends on where you get that min length variable from. If it's hardcoded, you'd probably better use a dynamic internationalization module:
int.getMinStringLength(int.getCurrentLanguage())
Either you have a dynamic bindings framework such as AngularJS, or you update that module when the user changes the language.
Now maybe you'd want to sort your supported languages by using grouping attributes such as "verbose" and "condensed".
I'm not sure why Microsoft hasn't allowed an Intellisense-friendly environment for using native regex with JavaScript in WebMatrix, but here is what I see when I attempt an otherwise normal JavaScript function:
As you can see, while this is perfectly valid JavaScript (using native regex), WebMatrix's Intellisense for my .js file is showing more random colors than a kaleidoscope.
I probably shouldn't be complaining, since it works, but I would like to restore human-readability if I can. I have noticed that this hasn't been mentioned anywhere else before (that I can find), and I was wondering if there were a more aesthetically pleasing way to handle this, given the environment.
I've tried using something like new RegEx(/&/g) for the first argument in the replace function, but of course, it produces the same glitch.
I've also tried storing the regex in a string, but I don't think that is in the format the first argument expects, so no dice there either.
I am not a master at regular expressions, by any means, so I apologize if I am overlooking a simple workaround here.
Is there anything I can do to retain this function in a more human-readable way?
--------------------------UPDATE-----------------------------
I just noticed that the line input = input.replace(/'/g, ",") is actually replacing an apostrophe with a comma. Rather than reload the picture, I will just mention that here (the proper hex code for apostrophe should be ').
new RegExp(/&/g) does not really make sense as the /&/g already creates a RegExp object. You can use new RegExp('&', 'g') instead.
By the way, this is not the only problem there is with JavaScript Intellisense.
I'm trying to understand how JS is actually parsed. But my searches either return some ones very vaguely documented project of a "parser/generator" (i don't even know what that means), or how to parse JS using a JS Engine using the magical "parse" method. I don't want to scan through a bunch of code and try all my life to understand (although i can, it would take too long).
i want to know how an arbitrary string of JS code is actually turned into objects, functions, variables etc. I also want to know the procedures, and techniques that turns that string into stuff, gets stored, referenced, executed.
Are there any documentation/references for this?
Parsers probably work in all sorts of ways, but fundamentally they first go through a stage of tokenisation, then give the result to the compiler, which turns it into a program if it can. For example, given:
function foo(a) {
alert(a);
}
the parser will remove any leading whitespace to the first character, the letter "f". It will collect characters until it gets something that doesn't belong, the whitespace, that indicates the end of the token. It starts again with the "f" of "foo" until it gets to the "(", so it now has the tokens "function" and "foo". It knows "(" is a token on its own, so that's 3 tokens. It then gets the "a" followed by ")" which are two more tokens to make 5, and so on.
The only need for whitespace is between tokens that are otherwise ambiguous (e.g. there must be either whitespace or another token between "function" and "foo").
Once tokenisation is complete, it goes to the compiler, which sees "function" as an identifier, and interprets it as the keyword "function". It then gets "foo", an identifier that the language grammar tells it is the function name. Then the "(" indicates an opening grouping operator and hence the start of a formal parameter list, and so on.
Compilers may deal with tokens one at a time, or may grab them in chunks, or do all sorts of weird things to make them run faster.
You can also read How do C/C++ parsers work?, which gives a few more clues. Or just use Google.
While it doesn't correspond closely to the way real JS engines work, you might be interested in reading Douglas Crockford's article on Top Down Operator Precedence, which includes code for a small working lexer and parser written in the Javascript subset it parses. It's very readable and concise code (with good accompanying explanations) which at least gives you an outline of how a real implementation might work.
A more common technique than Crockford's "Top Down Operator Precedence" is recursive descent parsing, which is used in Narcissus, a complete implementation of JS in JS.
maybe esprima will help you to understand how JS parses the grammar. it's online
a="79 * 2245 + (79 * 2 - 7)";
b="";
c=["1","2","3","4","5","6","7","8","9","0","+","-","/","*"];
for (i=1;i<a.length;i++){
for (ii=1;i<c.length;i++){
b=(a.substring(0,i))+(c[ii])+(a.substring(i+1,a.length));
alert(eval(b.replace(" ","")));
}
}
I need to find out how to make it so that when I use eval, I know that the input will not stop the script, and if it would normally crash the script to just ignore it. I understand that eval is not a good function to use, but I want a quick and simple method by which I can solve this. The above code tries to output all of the answers with all of the possible replacements for any digit, sign or space in the above. i represents the distance through which it has gone in the string and ii represents the symbol that it is currently checking. a is the original problem and b is the modified problem.
Try catching the exception eval might throw, like this:
try{
alert(eval(b.replace(" ","")));
} catch (e){
//alert(e);
}
You can check for a few special cases and avoid some behaviors with regex or the like, but there is definitely no way to 'if it would normally crash just ignore it'
That is akin to the halting problem, as mellamokb refers to. And theres no way to know ipositively f a script runs to completion besides running it.
One should be very careful to vet any strings that go to eval, and keep user input out of them as much as possibl except for real simple and verifiable things like an integer value. If you can find a way around eval altogether than all the better.
For the calculation example you show its probably best to parse it properly into tokens and go from there rather than evaluate in string form.
PS - if you really want to check out these one-off's to the expression in a, it is a somewhat interesting use of eval eespite its faults. cam you explain why you are trimming the whitespace imediately before evaluation? i dont believe i can think of a situation where it effects the results. for (at least most) valid expressions it makes no difference, and while it might alter some of the invalid cases i cant think of a case where it does so meaningfully
Hey folks, thanks for reading
I am currently attempting to do a Google-style calculator. You input a string, it determines if it can be calculated and returns the result.
I began slowly with the basics : + - / * and parenthesis handling.
I am willing to improve the calculator over time, and having learned a bit about lexical analysis a while ago, I built a list of tokens and associated regular expression patterns.
This kind of work is easily applicable with languages such as Lex and Yacc, except I am developping a Javascript-only application.
I tried to transcript the idea into Javascript but I can't figure out how to handle everything in a clean and beautiful way, especially nested parenthesis.
Analysis
Let's define what a calculator query is:
// NON TERMINAL EXPRESSIONS //
query -> statement
query -> ε // means end of query
statement -> statement operator statement
statement -> ( statement )
statement -> prefix statement
statement -> number
number -> integer
number -> float
// TERMINAL EXPRESSIONS //
operator -> [+*/%^-]
prefix -> -
integer -> [0-9]+
float -> [0-9]+[.,][0-9]+
Javascript
Lexical Analysis consists in verifying there is nothing that doesn't look like one of the terminal expressions : operator, prefixes, integer and float. Which can be shortened to one regular expression:
(I added spaces to make it more readable)
var calcPat =
/^ (\s*
( ([+/*%^-]) | ([0-9]+) | ([0-9]+[.,][0-9]+) | (\() | (\)) )
)+ \s* $/;
If this test passes, query is lexically correct and needs to be grammar-checked to determine if it can be calculated. This is the tricky part
I am not going to paste code because it is not clean nor easily understandable, but I am going to explain the process I followed and why I'm stuck:
I created a method called isStatement(string) that's supposed to call itself recursively. The main idea is to split the string into 'potential' statements and check if they really are statements and form one altogether.
Process is the following:
-If the first two tokens are a number followed by an operator:
-Then,
-- If the remaining is just one token and it is a number:
--- Then this is a statement.
--- Else, check if the remaining tokens form a statement (recursive call)
-Else, If the first token is a parenthesis
-Then, Find matching closing parenthesis and check if what's inside is a statement (recursion)
-- Also check if there is something after closing parenthesis and if it forms a statement when associated with the parenthesis structure.
What's the problem ?
My problem is that I cannot find matching parenthesis when there is nested structures. How can I do that ? Also, as you can see, this is not a particurlarly generic and clean grammar-checking algorithm. Do you have any idea to improve this pattern ?
Thank you so much for having taken the time to read everything.
Gael
(PS: As you probably noticed, I am not a native english speaker ! Sorry for mistakes and all !)
You've got the right idea about what lexical analysis is, but you seem to have gotten confused about the distinction between the token grammar and the language grammar. Those are two different things.
The token grammar is the set of patterns (usually regular expressions) that describe the tokens for the language to be parsed. The regular expressions are expressions over a character set.
The language grammar (or target grammar, I suppose) is the grammar for the language you want to parse. This grammar is expressed in terms of tokens.
You cannot write a regular expression to parse algebraic notation. You just can't. You can write a grammar for it, but it's not a regular grammar. What you want to do is recognize separate tokens, which in your case could be done with a regular expression somewhat like what you've got. The trick is that you're not really applying that expression to the overall sentence to be parsed. Instead, you want to match a token at the current point in the sentence.
Now, because you've got Javascript regular expressions to work with, you could come up with a regular expression designed to match a string of tokens. The trick with that will be coming up with a way to identify which token was matched out of the list of possibilities. The Javascript regex engine can give you back arrays of groups, so maybe you could build something on top of that.
edit — I'm trying to work out how you could put together a (somewhat) general-purpose tokenizer builder, starting from a list of separate regular expressions (one for each token). It's possibly not very complicated, and it'd be pretty fun to have around.