Regular Expression to match URLs / web addresses - javascript

I have a JS function which is passed a string that a RegEx is run against, and returns any matches:
searchText= // some string which may or may not contain URLs
Rxp= new RegExp("([a-zA-Z\d]+://)?(\w+:\w+#)?([a-zA-Z\d.-]+\.[A-Za-z]{2,4})(:\d+)?(/.*)?/ig")
return searchText.match(Rxp);
The RegExp should return matches for any of the following (and similar derivations):
google.com
www.google.com
http://www.google.com
http://google.com
google.com?querystring=value
www.google.com?querystring=value
http://www.google.com?querystring=value
http://google.com?querystring=value
However, no such luck. Any suggestions?

In a string, \ has to be escaped: \\.
First, the string is interpreted. \w turns in w, because it has no significant meaning.
Then, the parsed string is turned in a RegEx. But \ is lost during the string parsing, so your RegEx breaks.
Instead of using the RegExp constructor, use RegEx literals:
Rxp = /([a-zA-Z\d]+:\/\/)?(\w+:\w+#)?([a-zA-Z\d.-]+\.[A-Za-z]{2,4})(:\d+)?(\/.*)?/ig;
// Note: I recommend to use a different variable name. Variables starting with a
// capital usually indicate a constructor, by convention.
If you're not 100% sure that the input is a string, it's better to use the exec method, which coerces the argument to a string:
return Rxp.exec(searchText);
Here's a pattern which includes the query string and URL fragment:
/([a-zA-Z\d]+:\/\/)?(\w+:\w+#)?([a-zA-Z\d.-]+\.[A-Za-z]{2,4})(:\d+)?(\/[^?#\s]*‌)?(\?[^#\s]*)?(#\S*)?/ig

Firstly, there's no real need to create your pattern via the RegExp constructor since it doesn't contain anything dynamic. You can just use the literal /pattern/ instead.
If you do use the constructor, though, you have to remember your pattern is declared as a string, not a literal REGEXP, so you'll need to double-escape special characters, e.g. \\d, not \d. Also, there were several forward slashes you weren't escaping at all.
With the constructor, modifiers (g, i) are passed as a second argument, not appended to the pattern.
So to literally change what you have, it would be:
Rxp= new RegExp("([a-zA-Z\\d]+:\\/\\/)?(\\w+:\\w+#)?([a-zA-Z\\d.-]+\\.[A-Za-z]{2,4})(:\\d+)?(\\/.*)?", "ig")
But better would be:
Rxp = /([a-zA-Z\d]+:\/\/)?(\w+:\w+#)?([a-zA-Z\d.-]+\.[A-Za-z]{2,4})(:\d+)?(\/.*)?/gi;

Related

Regular expresion for RegExp in Dart

I have the following in JavaScript:
function escape(text)
{
var tx = text.replace(/[&<>"']/g);
}
Im having problems trying to do the same on Dart:
var reg = new RegExp("/[&<>"']/g"); -->this throws error.
How can I get an equivalent expression?
The Dart RegExp source does not use / to delimit regular expressions, they're just strings passed to the RegExp constructor.
It's usually recommended that you use a "raw string" because backslashes mean something in RegExps as well as in non-raw string literals, and the JavaScript RegExp /\r\n/ would be RegExp("\\r\\n") in Dart without raw strings, but RegExp(r"\r\n") with a raw string, much more readable.
In this particular case, where the string contains both ' and ", that becomes harder, but you can use a "multiline string" instead - it uses tripple quote characters as delimiters, so it can contain single quote characters unescaped (it doesn't have to actually span multiple lines).
Dart doesn't have something similar to the g flag of JavaScript regexps. Dart regexps are stateless, it's the functions using them which need to care about remembering where it matched, not the RegExp itself. So, no need for the g.
So:
RegExp(r"""[&<>"']""");
// or
RegExp(r'''[&<>"']''');
That gets a little crowded with all those quotes, and you can choose to use a non-raw string instead so you can escape the quote which matches the string (which is easier because your RegExp does not contain any backslashes itself):
RegExp("[&<>\"']");
// or
RegExp('[&<>"\']');
If you do that when your regexp uses a RegExp backslash, then you'll need to double the backslash, something which is easy to forget, which is why raw strings are recommended.
You forgot to escape double quotes
new RegExp("/[&<>\"']", 'g');

How to make \ a literal back slash using Regexp in JS

I am pretty new to Regexp and it seems that the \ is used for meta characters. My problem is I want to search this string exactly \"mediaType\":\"img\"
Now I also want to dynamically put a variable in for img. So I want it to be something like this
new RegExp(`\"mediaType\":\"${variable}\"`)
How do I write this to make it work?
Short answer:
function escapeRegEx(s) {
return s.replace(/[-\/\\^$*+?.()|[\]{}]/g, '\\$&');
}
var expression = new RegExp('\\\\"mediaType\\\\":\\\\"' + escapeRegEx(variable) + '\\\\"');
// or, using a template literal:
var expression = new RegExp(`\\\\"mediaType\\\\":\\\\"${escapeRegEx(variable)}\\\\"`);
Long answer:
Besides being used for meta characters, backslash in regular expressions can be used to escape characters that would otherwise have meaning (like *, $, parentheses, and \). So the way to match a backslash in a regular expression is to add another one as an escape character: \\.
Taking that into account, the regular expression you want to end up with is \\"mediaType\\":\\"img\\", and if you were using a regular expression literal that would be it. Unfortunately it gets a little more involved because you need to create an expression dynamically, you need to provide the expression as a string, which also needs the backslashes escaped. That adds a second layer of escaping, so you need to double up each of the \ characters again, and you end up with new RegExp('\\\\"mediaType\\\\":\\\\"img\\\\"').
Another complication is that you want the contents of variable to be matched literally, not interpreted as a regular expression. Unfortunately, there's no built-in way to automatically escape regular expressions in JavaScript, so you'll need to use one of the solutions in Is there a RegExp.escape function in Javascript?. I used a slightly modified version of the accepted answer that defines it as a standalone function instead of adding it to the RegExp object. The exact solution doesn't matter, as long as you escape the dynamic part.
You just want to use String.raw
const variable = 'text'
const regexp = new RegExp(String.raw `\"mediaType\":\"${variable}\"`)
console.log(regexp)

Create regular expression dynamically with regex chars

Say I want to match any of these files:
/foo/bar/baz/x
/foo/bar/baz/y/z
so I create a regex like so:
new RegExp('^/foo/bar/baz/.*')
however my question is - how do I tell the RegExp constructor to view . * and ^ as regular expression characters, not literal characters?
As this is node.js related, my solution would be to use quotemeta
https://github.com/substack/quotemeta
as if it was perl related my solution would be to use \Q \E ;-)
new RegExp('^/foo/bar/baz/.*')
...and
/^\/foo\/bar\/baz\/.*/
...are equivalent. Everything in the new RegExp string are treated as regular expression characters, not literal characters (luckily / does not need to be escaped).
If you do want to make something a literal character using the constructor syntax, make sure you use a double backslash since \ within a string literal has its own meaning. For example, if you want to capture a literal . character, this:
new RegExp('\.')
...won't work right because that is interpreted the same as '.', making the RegExp the same as /./. You'd need to do this instead:
new RegExp('\\.')

Regex Removing Hash from Beginning

I am running up against a weird problem here. When I run:
"#/a/b/c/d".replace("#\/","")
I get what I would expect: a/b/c/d.
But When I precede this regex with a start of string character ^, To get:
"#/a/b/c/d".replace("^#\/","")
This returns the original string "#a/b/c/d".
Could anybody explain why the hash isn't removed and possibly suggest an alternative that would remove the hash only if it appears at the beginning of the string?
The first argument to replace can be either a string or a regular expression.
You are passing a string, so it is looking for an exact match for that string.
Pass a regular expression instead.
"#/a/b/c/d".replace(/^#\//,"")
When you pass a string as the first argument to .replace() method, the string is used as a literal and only 1 replacement can be made.
See the MDN replace reference:
substr (pattern)
A String that is to be replaced by newSubStr. It is treated as a verbatim string and is not interpreted as a regular expression.
What you need is to pass the regex object as the first argument:
"#/a/b/c/d".replace(/^#\//,"")
^^^^^^
There is no point in using a constructor notation (RegExp("^#/") since the pattern is not built dynamically from variables.
There are no double slashes in the pattern! The outer /.../ are called regex delimiters. Then, what is insde is a pattern. The pattern is ^#/, but since the / are used as regex delimiters, the / in the pattern must be escaped to match a literal / (only in the regex literal notation).
String.replace takes as its first argument either a string or a regexp. If you give it a string, it is not interpreted as or converted to a regexp; it is treated literally. Therefore, when you add ^ to your string, it replaces nothing, since there is no ^ in the input.
As explained in the comments, you can either pass a regexp literal (/.../) as the first argument, or you could construct a regexp with new RegExp (but why would you do that?).

Javascript RegExp anomaly [duplicate]

I am trying to build a regexp from static text plus a variable in javascript. Obviously I am missing something very basic, see comments in code below. Help is very much appreciated:
var test_string = "goodweather";
// One regexp we just set:
var regexp1 = /goodweather/;
// The other regexp we built from a variable + static text:
var regexp_part = "good";
var regexp2 = "\/" + regexp_part + "weather\/";
// These alerts now show the 2 regexp are completely identical:
alert (regexp1);
alert (regexp2);
// But one works, the other doesn't ??
if (test_string.match(regexp1))
alert ("This is displayed.");
if (test_string.match(regexp2))
alert ("This is not displayed.");
First, the answer to the question:
The other answers are nearly correct, but fail to consider what happens when the text to be matched contains a literal backslash, (i.e. when: regexp_part contains a literal backslash). For example, what happens when regexp_part equals: "C:\Windows"? In this case the suggested methods do not work as expected (The resulting regex becomes: /C:\Windows/ where the \W is erroneously interpreted as a non-word character class). The correct solution is to first escape any backslashes in regexp_part (the needed regex is actually: /C:\\Windows/).
To illustrate the correct way of handling this, here is a function which takes a passed phrase and creates a regex with the phrase wrapped in \b word boundaries:
// Given a phrase, create a RegExp object with word boundaries.
function makeRegExp(phrase) {
// First escape any backslashes in the phrase string.
// i.e. replace each backslash with two backslashes.
phrase = phrase.replace(/\\/g, "\\\\");
// Wrap the escaped phrase with \b word boundaries.
var re_str = "\\b"+ phrase +"\\b";
// Create a new regex object with "g" and "i" flags set.
var re = new RegExp(re_str, "gi");
return re;
}
// Here is a condensed version of same function.
function makeRegExpShort(phrase) {
return new RegExp("\\b"+ phrase.replace(/\\/g, "\\\\") +"\\b", "gi");
}
To understand this in more depth, follows is a discussion...
In-depth discussion, or "What's up with all these backslashes!?"
JavaScript has two ways to create a RegExp object:
/pattern/flags - You can specify a RegExp Literal expression directly, where the pattern is delimited using a pair of forward slashes followed by any combination of the three pattern modifier flags: i.e. 'g' global, 'i' ignore-case, or 'm' multi-line. This type of regex cannot be created dynamically.
new RegExp("pattern", "flags") - You can create a RegExp object by calling the RegExp() constructor function and pass the pattern as a string (without forward slash delimiters) as the first parameter and the optional pattern modifier flags (also as a string) as the second (optional) parameter. This type of regex can be created dynamically.
The following example demonstrates creating a simple RegExp object using both of these two methods. Lets say we wish to match the word "apple". The regex pattern we need is simply: apple. Additionally, we wish to set all three modifier flags.
Example 1: Simple pattern having no special characters: apple
// A RegExp literal to match "apple" with all three flags set:
var re1 = /apple/gim;
// Create the same object using RegExp() constructor:
var re2 = new RegExp("apple", "gim");
Simple enough. However, there are significant differences between these two methods with regard to the handling of escaped characters. The regex literal syntax is quite handy because you only need to escape forward slashes - all other characters are passed directly to the regex engine unaltered. However, when using the RegExp constructor method, you pass the pattern as a string, and there are two levels of escaping to be considered; first is the interpretation of the string and the second is the interpretation of the regex engine. Several examples will illustrate these differences.
First lets consider a pattern which contains a single literal forward slash. Let's say we wish to match the text sequence: "and/or" in a case-insensitive manner. The needed pattern is: and/or.
Example 2: Pattern having one forward slash: and/or
// A RegExp literal to match "and/or":
var re3 = /and\/or/i;
// Create the same object using RegExp() :
var re4 = new RegExp("and/or", "i");
Note that with the regex literal syntax, the forward slash must be escaped (preceded with a single backslash) because with a regex literal, the forward slash has special meaning (it is a special metacharacter which is used to delimit the pattern). On the other hand, with the RegExp constructor syntax (which uses a string to store the pattern), the forward slash does NOT have any special meaning and does NOT need to be escaped.
Next lets consider a pattern which includes a special: \b word boundary regex metasequence. Say we wish to create a regex to match the word "apple" as a whole word only (so that it won't match "pineapple"). The pattern (as seen by the regex engine) needs to be: \bapple\b:
Example 3: Pattern having \b word boundaries: \bapple\b
// A RegExp literal to match the whole word "apple":
var re5 = /\bapple\b/;
// Create the same object using RegExp() constructor:
var re6 = new RegExp("\\bapple\\b");
In this case the backslash must be escaped when using the RegExp constructor method, because the pattern is stored in a string, and to get a literal backslash into a string, it must be escaped with another backslash. However, with a regex literal, there is no need to escape the backslash. (Remember that with a regex literal, the only special metacharacter is the forward slash.)
Backslash SOUP!
Things get even more interesting when we need to match a literal backslash. Let's say we want to match the text sequence: "C:\Program Files\JGsoft\RegexBuddy3\RegexBuddy.exe". The pattern to be processed by the regex engine needs to be: C:\\Program Files\\JGsoft\\RegexBuddy3\\RegexBuddy\.exe. (Note that the regex pattern to match a single backslash is \\ i.e. each must be escaped.) Here is how you create the needed RegExp object using the two JavaScript syntaxes
Example 4: Pattern to match literal back slashes:
// A RegExp literal to match the ultimate Windows regex debugger app:
var re7 = /C:\\Program Files\\JGsoft\\RegexBuddy3\\RegexBuddy\.exe/;
// Create the same object using RegExp() constructor:
var re8 = new RegExp(
"C:\\\\Program Files\\\\JGsoft\\\\RegexBuddy3\\\\RegexBuddy\\.exe");
This is why the /regex literal/ syntax is generally preferred over the new RegExp("pattern", "flags") method - it completely avoids the backslash soup that can frequently arise. However, when you need to dynamically create a regex, as the OP needs to here, you are forced to use the new RegExp() syntax and deal with the backslash soup. (Its really not that bad once you get your head wrapped 'round it.)
RegexBuddy to the rescue!
RegexBuddy is a Windows app that can help with this backslash soup problem - it understands the regex syntaxes and escaping requirements of many languages and will automatically add and remove backslashes as required when pasting to and from the application. Inside the application you compose and debug the regex in native regex format. Once the regex works correctly, you export it using one of the many "copy as..." options to get the needed syntax. Very handy!
You should use the RegExp constructor to accomplish this:
var regexp2 = new RegExp(regexp_part + "weather");
Here's a related question that might help.
The forward slashes are just Javascript syntax to enclose regular expresions in. If you use normal string as regex, you shouldn't include them as they will be matched against. Therefore you should just build the regex like that:
var regexp2 = regexp_part + "weather";
I would use :
var regexp2 = new RegExp(regexp_part+"weather");
Like you have done that does :
var regexp2 = "/goodweather/";
And after there is :
test_string.match("/goodweather/")
Wich use match with a string and not with the regex like you wanted :
test_string.match(/goodweather/)
While this solution may be overkill for this specific question, if you want to build RegExps programmatically, compose-regexp can come in handy.
This specific problem would be solved by using
import {sequence} from 'compose-regexp'
const weatherify = x => sequence(x, /weather/)
Strings are escaped, so
weatherify('.')
returns
/\.weather/
But it can also accept RegExps
weatherify(/./u)
returns
/.weather/u
compose-regexp supports the whole range of RegExps features, and let one build RegExps from sub-parts, which helps with code reuse and testability.

Categories