For example, given the string "2009/11/12" I want to get the regex ("\d{2}/d{2}/d{4}"), so I'll be able to match "2001/01/02" too.
Is there something that does that? Something similar? Any idea' as to how to do it?
There is text2re, a free web-based "regex by example" generator.
I don't think this is available in source code, though. I dare to say there is no automatic regex generator that gets it right without user intervention, since this would require the machine knowing what you want.
Note that text2re uses a template-based, modularized and very generalized approach to regular expression generation. The expressions it generates work, but they are much more complex than the equivalent hand-crafted expression. It is not a good tool to learn regular expressions because it does a pretty lousy job at setting examples.
For instance, the string "2009/11/12" would be recognized as a yyyymmdd pattern, which is helpful. The tool transforms it into this 125 character monster:
((?:(?:[1]{1}\d{1}\d{1}\d{1})|(?:[2]{1}\d{3}))[-:\/.](?:[0]?[1-9]|[1][012])[-:\/.](?:(?:[0-2]?\d{1})|(?:[3][01]{1})))(?![\d])
The hand-made equivalent would take up merely two fifths of that (50 characters):
([12]\d{3})[-:/.](0?\d|1[0-2])[-:/.]([0-2]?\d|3[01])\b
It's not possible to write a general solution for your problem. The trouble is that any generator probably wouldn't know what you want to check for, e.g. should "2312/45/67" be allowed too? What about "2009.11.12"?
What you could do is write such a generator yourself that is suited for your exact problem, but a general solution won't be possible.
I've tried a very naive approach:
class RegexpGenerator {
public static Pattern generateRegexp(String prototype) {
return Pattern.compile(generateRegexpFrom(prototype));
}
private static String generateRegexpFrom(String prototype) {
StringBuilder stringBuilder = new StringBuilder();
for (int i = 0; i < prototype.length(); i++) {
char c = prototype.charAt(i);
if (Character.isDigit(c)) {
stringBuilder.append("\\d");
} else if (Character.isLetter(c)) {
stringBuilder.append("\\w");
} else { // falltrought: literal
stringBuilder.append(c);
}
}
return stringBuilder.toString();
}
private static void test(String prototype) {
Pattern pattern = generateRegexp(prototype);
System.out.println(String.format("%s -> %s", prototype, pattern));
if (!pattern.matcher(prototype).matches()) {
throw new AssertionError();
}
}
public static void main(String[] args) {
String[] prototypes = {
"2009/11/12",
"I'm a test",
"me too!!!",
"124.323.232.112",
"ISBN 332212"
};
for (String prototype : prototypes) {
test(prototype);
}
}
}
output:
2009/11/12 -> \d\d\d\d/\d\d/\d\d
I'm a test -> \w'\w \w \w\w\w\w
me too!!! -> \w\w \w\w\w!!!
124.323.232.112 -> \d\d\d.\d\d\d.\d\d\d.\d\d\d
ISBN 332212 -> \w\w\w\w \d\d\d\d\d\d
As already outlined by others a general solution to this problem is impossible. This class is applicable only in few contexts
Excuse me, but what you all call impossible is clearly an achievable task. It will not be able to give results for ALL examples, and maybe not the best results, but you can give it various hints, and it will make life easy. A few examples will follow.
Also a readable output translating the result would be very useful.
Something like:
"Search for: a word starting with a non-numeric letter and ending with the string: "ing".
or: Search for: text that has bbb in it, followed somewhere by zzz
or: *Search for: a pattern which looks so "aa/bbbb/cccc" where "/" is a separator, "aa" is two digits, "bbbb" is a word of any length and "cccc" are four digits between 1900 and 2020 *
Maybe we could make a "back translator" with an SQL type of language to create regex, instead of creating it in geekish.
Here's are a few examples that are doable:
class Hint:
Properties: HintType, HintString
enum HintType { Separator, ParamDescription, NumberOfParameters }
enum SampleType { FreeText, DateOrTime, Formatted, ... }
public string RegexBySamples( List<T> samples,
List<SampleType> sampleTypes,
List<Hint> hints,
out string GeneralRegExp, out string description,
out string generalDescription)...
regex = RegExpBySamples( {"11/November/1999", "2/January/2003"},
SampleType.DateOrTime,
new HintList( HintType.NumberOfParameters, 3 ));
regex = RegExpBySamples( "123-aaaaJ-1444",
SampleType.Format, HintType.Seperator, "-" );
A GUI where you mark sample text or enter it, adding to the regex would be possible too.
First you mark a date (the "sample"), and choose if this text is already formatted, or if you are building a format, also what the format type is: free text, formatted text, date, GUID or Choose... from existing formats (which you can store in library).
Lets design a spec for this, and make it open source... Anybody wants to join?
Loreto pretty much does this. It's an open source implementation using the common longest substring(s) to generate the regular expressions. Needs multiple examples of course, though.
No, you cannot get a regex that matches what you want reliably, since the regex would not contain semantic information about the input (i.e. it would need to know it's generating a regex for dates). If the issue is with dates only I would recommend trying multiple regular expressions and see if one of them matches all.
I'm not sure if this is possible, at least not without many sample strings and some learning algorithm.
There are many regex' that would match and it's not possible for a simple algorithm to pick the 'right' one. You'd need to give it some delimiters or other things to look for, so you might as well just write the regex yourself.
sounds like a machine learning problem. You'll have to have more than one example on hand (many more) and an indication of whether or not each example is considered a match or not.
I don't remember the name but if my theory of computation cells serve me right its impossible in theory :)
I haven't found anything that does it , but since the problem domain is relatively small (you'd be surprised how many people use the weirdest date formats) , I've able to write some kind of a "date regular expression generator".
Once I'm satisfied with the unit tests , I'll publish it - just in case someone will ever need something of the kind.
Thanks to everyone who answered (the guy with the (.*) excluded - jokes are great , but this one was sssssssssoooo lame :) )
In addition to feeding the learning algorithm examples of "good" input, you could feed it "bad" input so it would know what not to look for. No letters in a phone number, for example.
Related
The question is as simple as the title: How do I make sure a string contains a calculation.
I have a string which should contain a calculation. The string could be something like this: "10 * 10 / 2 * 3". But there are always people who want to break things, so I want to check if it's a calculation. The user could fill in something like this: "alert('hi!')".
When I use eval(), it will still alert the user (if the user filled in: "alert()").
Does anyone know how I can check if the string contains a calculation and nothing else?
If you just want to check if a string contains an arithmetic operation, you coul match it with a regexp like this:
([-+]?[0-9]*\.?[0-9]+[\/\+\-\*])+([-+]?[0-9]*\.?[0-9]+)
BUT, i strongly suggest to use a javascript library as mathjs, or the JavaScript Expression Evaluator library, with which you can do, for example:
Parser.evaluate("3 ^ x", { x: 7 });
These libraries will take care to ignore code injection, for sure... :-)
In this case, I'd recommend a different solution.
Rather than trying to white list your string to ensure it is just math expressions, I'd recommend you try to use library specifically for handling math in the first place.
I have no experience with it, but a quick google of "javascript math parsers" brought me to this one: http://mathjs.org/docs/expressions/parsing.html
You could just check the input sting for characters. If the sting contains any characters like (a,b,c ... z) then you do not eval it.
I have a large valid JavaScript file (utf-8), from which I need to extract all text strings automatically.
For simplicity, the file doesn't contain any comment blocks in it, only valid ES6 JavaScript code.
Once I find an occurrence of ' or " or `, I'm supposed to scan for the end of the text block, is where I got stuck, given all the possible variations, like "'", '"', "\'", '\"', '", `\``, etc.
Is there a known and/or reusable algorithm for detecting the end of a valid ES6 JavaScript text block?
UPDATE-1: My JavaScript file isn't just large, I also have to process it as a stream, in chunks, so Regex is absolutely not usable. I didn't want to complicate my question, mentioning joint chunks of code, I will figure that out myself, If I have an algorithm that can work for a single piece of code that's in memory.
UPDATE-2: I got this working initially, thanks to the many advises given here, but then I got stuck again, because of the Regular Expressions.
Examples of Regular Expressions that break any of the text detection techniques suggested so far:
/'/
/"/
/\`/
Having studied the matter closer, by reading this: How does JavaScript detect regular expressions?, I'm afraid that detecting regular expressions in JavaScript is a whole new ball game, worth a separate question, or else it gets too complicated. But I appreciate very much if somebody can point me in the right direction with this issue...
UPDATE-3: After much research I found with regret that I cannot come up with an algorithm that would work in my case, because presence of Regular Expressions makes the task incredibly more complicated than was initially thought. According to the following: When parsing Javascript, what determines the meaning of a slash?, determining the beginning and end of regular expressions in JavaScript is one of the most complex and convoluted tasks. And without it we cannot figure out when symbols ', '"' and ` are opening a text block or whether they are inside a regular expression.
The only way to parse JavaScript is with a JavaScript parser. Even if you were able to use regular expressions, at the end of the day they are not powerful enough to do what you are trying to do here.
You could either use one of several existing parsers, that are very easy to use, or you could write your own, simplified to focus on the string extraction problem. I hardly imagine you want to write your own parser, even a simplified one. You will spend much more time writing it and maintaining it than you might think.
For instance, an existing parser will handle something like the following without breaking a sweat.
`foo${"bar"+`baz`}`
The obvious candidates for parsers to use are esprima and babel.
By the way, what are you planning to do with these strings once you extract them?
If you only need an approximate answer, or if you want to get the string literals exactly as they appear in the source code, then a regular expression can do the job.
Given the string literal "\n", do you expect a single-character string containing a newline or the two characters backslash and n?
In the former case you need to interpret escape sequences exactly like a JavaScript interpreter does. What you need is a lexer for JavaScript, and many people have already programmed this piece of code.
In the latter case the regular expression has to recognize escape sequences like \x40 and \u2026, so even in that case you should copy the code from an existing JavaScript lexer.
See https://github.com/douglascrockford/JSLint/blob/master/jslint.js, function tokenize.
Try code below:
txt = "var z,b \n;z=10;\n b='321`1123`321321';\n c='321`321`312`3123`';"
function fetchStrings(txt, breaker){
var result = [];
for (var i=0; i < txt.length; i++){
// Define possible string starts characters
if ((txt[i] == "'")||(txt[i] == "`")){
// Get our text string;
textString = txt.slice(i+1, i + 1 + txt.slice(i+1).indexOf(txt[i]));
result.push(textString)
// Jump to end of fetched string;
i = i + textString.length + 1;
}
}
return result;
};
console.log(fetchStrings(txt));
I need to split a string to one or more substrings each of which contains no more or less than two dots. For example, if the string is foo.boo.coo.too" then what would be the regex to get the following array?: ["foo.boo.coo", "boo.coo.too"]. I hope there will be someone to answer this question - I will really admire you, as I've been programming for several years and have not still be used to regular expressions well enough to solve this particular problem by myself. Thank you very much in advance. Let me know your identity so that I can credit you as a contributor of the program I am creating.
RegEx is for this Problem not the best solution a similar problem was discussed here: split-a-sting-every-3-characters-from-back-javascript
A good javascript solution would be a javascript function like this
function splitter(text){
var parts = text.split(".");
var times = parts.length - 2;
var values = [];
for(var index = 0; index<times;index++)
{
values.push(parts.slice(index,index+3).join("."));
}
return values;
}
splitter("too.boo.coo.too")
//=> Result tested on Chrome 25+ ["too.boo.coo", "boo.coo.too"]
I hope this helps
If you want to Use Regex try the Lookhead Stuff, this could help http://www.regular-expressions.info/lookaround.html
Regex by its nature will return non-intersecting results, so if you want "all matches" from a single regex - it's not possible.
So basically you will need to find first match, and then start from next position to find next match and so on; something like this technique described here regex matches with intersection in C# (it's not JavaScript but idea is the same)
You can use the following regex for example:
(?<=^|\.)((?:[^.]*\.){2}[^.]*?)(?=$|\.)
It ensures that it starts and ends with dot, or at begin/end of line, and contains exactly two dots inside, and captures result in first capture. You can replace * with + to make sure at least one symbol exists between dots, if it is required.
But you need to understand that such approach has really bad performance for the task you are solving, so may be using other way (like split + for) will be better solution.
Hi need to extract ONE letter from a string.
The string i have is a big block of html, but the part where i need to search in is this text:
Vahvistustunnus M :
And I need to get the M inside the nbsp's
So, who is the quickest regex-guru out there? :)
Ok, according to this page in the molybdenum api docs, the results will be all of the groups concatenated together. Given that you just want the char between the two 's then it's not good enough to match the whole thing and then pull out the group. Instead you'll need to do something like this:
(?<=Vahvistustunnus )[a-zA-Z](?= )
Warning
This might not work for you because lookbehinds (?<=pattern) are not available in all regex flavors. Specifically, i think that because molybdenum is a firefox extension, then it's likely using ECMA (javascript) regex flavor. And ECMA doesn't support lookbehinds.
If that's the case, then i'm gonna have to ask someone else to answer your question as my regex ninja (amateur) skills don't go much further than that. If you were using the regex in javascript code, then there are ways around this limitation, but based on your description, it sounds like you have to solve this problem with nothing but a raw regex?
Looks like it uses JavaScript and if so
var str = "Vahvistustunnus M :";
var patt = "Vahvistustunnus ([A-Z]) :";
var result = str.match(patt)[1];
should work.
I'm trying to parse apart the different filter strings for an sql server WHERE clause, and I thought that regular expressions would be the way to go. Fortunately, the filter will always be flat, there will be no "sub-filters". Each filter statement will always be surrounded in parenthesis. Here's an example of a string I'd like to parse:
([IsActive]=(1)) AND ([NoticeDate] >= GETDATE()) AND (NoticeDate <= DATEADD(day, 90, GETDATE()))
The result I'd like is an array with the following items:
[0] = ([IsActive]=(1)) AND
[1] = ([NoticeDate] >= GETDATE()) AND
[2] = (NoticeDate <= DATEADD(day, 90, GETDATE()))
The closest I've come is the following regex:
/\(.+?\) (and|or)/i
but this only returns
[0] = ([IsActive]=(1)) AND
[1] = AND
So basically I'd like to return anything surrounded in parenthesis, followed by a space, followed by the string "and" or "or". The last statement will not be followed by and/or, though I could concatenate a " and" string if that would make it easier. This is being done using classic asp JScript. I've pretty much exhausted my (weak) regular expression abilities, any help would be greatly appreciated. Thanks in advance
Try this regex:
(?i)\(.+?\)(?:\s+(?:and|or)|$)
Output:
If your WHERE clauses are free-form, /\(.+?\) (and|or)/i will spuriously match inside tokens like strings and comments.
Consider
SELECT resting_place FROM Nation
WHERE date_founded LIKE "Four score and seven years ago"
AND conceived IN (liberty)
ORDER BY the_people
The and in "Four score and seven years" is not a SQL keyword, but a single regular expression approach is not going to be able to distinguish keyword uses from non-keyword uses.
A more robust way would probably be to do a proper parser. This should be doable with a classic LL(1) parser. There are plenty of parser generators around. In fact, one can easily consider regular expressions to be a kind of parser generator, too.
Anyway, for this kind of tasks, a proper LL(1) parser is likely the better idea. It would also be able to support nested terms, if you want.
https://en.wikipedia.org/wiki/LL_parser