Matching multiple quotes in a sentence - javascript

I am trying to match multiple quotes inside of a single sentence, for example the line:
Hello "this" is a "test" example.
This is the regex that I am using, but I am having some problems with it:
/[^\.\?\!\'\"]{1,}[\"\'\“][^\"\'\“\”]{1,}[\"\'\“\”][^\.\?\!]{1,}[\.\?\!]/g
What I am trying achieve with this regex is to find everything from the start of the last sentence until I hit quotes, then find the closing set and continue until either a .?!
The sample text that I am using to test with is from Call of Cthulhu:
What seemed to be the main document was headed “CTHULHU CULT” in characters painstakingly printed to avoid the erroneous reading of a word so unheard-of. The manuscript was divided into two sections, the first of which was headed “1925—Dream and Dream Work of H. A. Wilcox, 7 Thomas St., Providence, R.I.”, and the second, “Narrative of Inspector John R. Legrasse, 121 Bienville St., New Orleans, La., at 1908 A. A. S. Mtg.—Notes on Same, & Prof. Webb’s Acct.” The other manuscript papers were all brief notes, some of them accounts of the queer dreams of different persons, some of them citations from theosophical books and magazines.
The issue comes on the line The manuscript was.... Does anyone know how to account for repeats like this? Or is there a better way?

This one ignores [.?!] inside quotes. But cases like Acct.” The nth will be considered as a single sentence in this case. Probably a . is missing over there.
var r = 'What seemed to be the main document was headed “CTHULHU.?! CULT” in characters painstakingly printed to avoid the erroneous reading of a word so unheard-of. The manuscript was divided into two sections, the first of which was headed “1925—Dream and Dream Work of H. A. Wilcox, 7 Thomas St., Providence, R.I.”, and the second, “Narrative of Inspector John R. Legrasse, 121 Bienville St., New Orleans, La., at 1908 A. A. S. Mtg.—Notes on Same, & Prof. Webb’s Acct.” The other manuscript papers were all brief notes, some of them accounts of the queer dreams of different persons, some of them citations from theosophical books and magazines.'
.split(/[“”]/g)
.map((x,i)=>(i%2)?x.replace(/[.?!]/g,''):x)
.join("'")
.split(/[.?!]/g)
.filter(x => x.trim()).map(x => ({
sentence: x,
quotescount: x.split("'").length - 1
}));
console.log(r);

You can use this naive pattern:
/[^"'“.!?]*(?:"[^"*]"[^"'“.!?]*|'[^']*'[^"'“.!?]*|“[^”]*”[^"'“.!?]*)*[.!?]/
details:
/
[^"'“.!?]* # all that isn't a quote or a punct that ends the sentence
(?:
"[^"*]" [^"'“.!?]*
|
'[^']*' [^"'“.!?]*
|
“[^”]*” [^"'“.!?]*
)*
[.!?]
/
If you want something more strong, you can emulate the "atomic grouping" feature, in particular if you are not sure that each opening quote has a closing quote (to prevent catastrophic backtracking):
/(?=([^"'“.!?]*))\1(?:"(?=([^"*]))\2"[^"'“.!?]*|'(?=([^']*))\3'[^"'“.!?]*|“(?=([^”]*))\4”[^"'“.!?]*)*[.!?]/
An atomic group forbids backtracking once closed. Unfortunately this feature doesn't exist in Javascript. But there's a way to emulate it using a lookahead that is naturally atomic, a capture group and a backreference:
(?>expr) => (?=(expr))\1

Related

Replace words in a paragraph using Javascript

I have a paragraph of some texts. I want to replace some words in that using wildcard.
The below is my Paragraph.
0.7% lower on the prospect of fresh restrictions that would deal a blow to hopes of a swift economic
recovery. <Origin Href=\"StoryRef\">urn:newsml:reuters.com:*:nL4N2EC04Z</Origin>\n The 2,000-plus cases reported on Sunday was a shocker ,said Nicholas Mapa, ING
In this Para, I want to remove <Origin Href=\"StoryRef\">urn:newsml:reuters.com:*:nL4N2EC04Z</Origin>\n
There are multiple paragraphs. But only the uncommon one is nL4N2EC04Z
All other words are common in those paragraphs .
<Origin Href=\"StoryRef\">urn:newsml:reuters.com:*:(need_to_use_wild_card_here)</Origin>\n
I tried to replace one half.
My code
storyRef="<Origin Href=\"StoryRef\">urn:newsml:reuters.com:*:";
storyRef.replace(storyRef," ")
But am stuck in replacing other parts.
It seems encoding problem. Try to use JSON.stringify to ensure that characters like < and some others does not read decoded.
You can improve Regex too to something like this 👇
storyRef.replace(/<Origin.*Origin>\\n/gm, ' ');
This example will start in <Origin, get all content between until Origin>\n.
const p = JSON.stringify('0.7% lower on the prospect of fresh restrictions that would deal a blow to hopes of a swift economic recovery. <Origin Href=\"StoryRef\">urn:newsml:reuters.com:*:nL4N2EC04Z</Origin>\n The 2,000-plus cases reported on Sunday was a shocker ,said Nicholas Mapa, ING');
const storyRef = JSON.parse(p.replace(/(<Origin.*Origin>\\n)+/gm, ' '));
console.log(storyRef);

Regex javascript tuning

First steps in regex and been trying to get some values out of emails that are being fetched. So far I've achieved some (even if it's not with the best approach) but in need of some more values and can't figure out how to get them.
This is the email template that is more or less always the same:
London, 06/20/09
Mr. Tom Waits
Process ref.: CR // 1943061
Your reference: 338256
Clients' names: Mary Lamb, John Snow
We return to your contact regarding the complaint on behalf of the clients mentioned above.
We inform you that the refund process has already started, so you should receive the respective amount (375EUR) within 4/6 weeks.
Payment ref.: 2500062960.
Our compliments,(...)
WHAT I NEED:
Date after "London,"
Process ref. (2 letters + // + digits)
Your ref. number
Clients' names
Amount
Payment reference number
Notes the amount not always comes between "( )", sometimes it's preceeded by "amount" others by "amount of", sometimes "EUR" is separated by a space, but the needed value is always the first digits combination on the paragraph
WHAT I HAVE SO FAR:
(?:London,)(.*)\s+(?:Mr. Tom Waits)\s+(?:Process ref.:)\s+(....\d+)\s+(?:Your reference:)\s+(\d+)\s+(?:Clients' names:)\s+(.*)\s+
WHAT IT RETRIEVES:
Date after "London,"
Process ref. (2 letters + // + digits)
Your reference number
Clients' names
WHAT'S MISSING:
Amount
Payment reference number
Other Concerns:
I tried to exclude the "Mr" parapraph but I think there may be a problem when they write it a little different like with an extra space or something
The same problem may arise if they also write the items a little different like, for example, "Process reference" instead of "Process ref."
Thanks in advance.
This one's gonna be one hefty regex. I do hope you don't try this regex on a massive file but rather one singular entry (like the one you've shown in your post).
Anyway, here's the regex-
London, ([\d\/]+)[\n\w\W]*?Process ref(?:\.|erence): ([A-Z \/\d]+)[\n\w\W]*?Your ref(?:\.|erence): (\d+)[\n\w\W]*?Client(?:s'|'s) name(?:s)?: ([\w ,]+)[\n\w\W]*?amount.*?(\d+)[\n\w\W]*?Payment ref(?:\.|erence): (\d+)
This should be quite permissive, it doesn't depend on many variable (seemingly) things, apart from London, that one's a bit of a hard coding but I'm assuming it's always London.
Now, let's walk through this-
London, ([\d\/]+) - This basically matches London, DATE - where date is, well, a date, where each element of the date is separated by a /.
In this case, it matches 06/20/09 from London, 06/20/09
[\n\w\W]*? - Try to keep up with this one - I'm using it a A LOT.
This will match pretty much all new lines, word characters and non word characters in a non-greedy way. In this particular case, this will match pretty much everything and that includes the newlines. This is used to just skip over anything and everything until we reach the desired spot.
Process ref(?:\.|erence): ([A-Z \/\d]+) - Captures the process reference, which can consist of capital alphabets (I assume, you can change that), slashes (/), and digits
Works with both ref. and reference
In this case, it matches CR // 1943061, from Process ref.: CR // 1943061
[\n\w\W]*? - Ignore everything up until the next token
Your ref(?:\.|erence): (\d+) - Captures "your reference", which can consist of digits
Works with both ref. and reference
In this case, it matches 338256
[\n\w\W]*? - Ignore everything up until the next token
Client(?:s'|'s) name(?:s)?: ([\w ,]+) - Captures the client name(s) - modified so it supports single client names too. (check the demo). The name list can consist of word characters, spaces and a comma.
In this case, it captures Mary Lamb, John Snow, from Clients' names: Mary Lamb, John Snow
(\d+) - Capture the digits - this is a big assumption, I'm assuming that the only digits that appear after client name list, are the ones for the amount. If they aren't
[\n\w\W]*? - Ignore everything up until the next token
amount.*?(\d+) - Captures the first group of digits that appear after amount. This is a bit of an assumption, I'm assuming that amount word is actually present in that paragraph.
In this case, it captures 375, from amount (375EUR)
[\n\w\W]*? - Ignore everything up until the next token
Payment ref(?:\.|erence): (\d+) - Capture the Payment reference number, which can consist of digits
Works with both ref. and reference
In this case, it captures 2500062960, from Payment ref.: 2500062960.
Check out the demo!

Regular expression to group prices and item name

Given the following examples:
100k melon 8
200 knife 7k
1.2m maple logs 19
I need to be able to take the first string as one group, the middle parts as another group, and the last part as the final group.
The current expression I have is this but regular expressions really throw me for a whirl:
([\d+|k])
Do any of you veterans have an idea of where I should go next or an explanation of how I can reach a solution to this?
Unfortunately, Reference - What does this regex mean? doesn't really solve my problem since it's just a dump of all of the different tokens you can use. Where I'm having trouble is putting all of that together in a meaningful way.
Here's what I came up with:
([0-9\.]+[a-z]?)\s([a-z\ ]+)\s([0-9\.]+[a-z]?)
and a quick overview of the groups:
([0-9\.]+[a-z]?)
matches any number or dot any number of times, plus an optional 1-character unit, like "k" or "m"
([a-z\ ]+)
matches letter and/or spaces. This may include a trailing or leading space, which is annoying, but I figured this was good enough.
([0-9\.]+[a-z]?)
same as the first group.
The three groups are separate by a space each.
Solution
This regex does the job
^(\S+)\s+(.*)\s+(\S+)$
See a demo here (Regex101)
Explanation
Explicitly match non-whitespace sequences at the start and the end of the string, resp.
The remainder are the middle parts surrounded by ws sequences.
Okay, if I understand your question correctly, you have the following String '100k melon 8 200 knife 7k 1.2m maple logs 19'. You should make a function that returns a .match():
function thrice(str){
return str.match(/(\d+\.\d+|\d+)\w?\D+(\d+\.\d+|\d+)\w?/g);
}
console.log(thrice('100k melon 8 200 knife 7k 1.2m maple logs 19'));
Otherwise, if you just want to test each String separately, you may consider:
function goodCat(str){
let g = str.match(/^(\d+\.\d+|\d+)\w?\D+(\d+\.\d+|\d+)\w?$/) ? true : false;
return g;
}
console.log(goodCat('100000k is 777 not enough 5'));
console.log(goodCat('100k melon 8'));
console.log(goodCat('200 knife 7k'));
console.log(goodCat('1.2m maple logs 19'));

how to compare two strings by meaning?

I want the user of my node.js application to write down ideas, which then get stored in a database.
So far so good, but I don't want redundant entrys in that table, so I decided to check for similarity, using this one:
https://www.npmjs.com/package/string-similarity-js
Do you know a way, in which I can compare two strings by meaning? In like getting a high similarity score for "using public transport" vs "driving by train" which performs very poor in the above one.
To compare two strings by meaning, the strings would need to be convert first to a tensor and then evalutuate the distance or similarity between the tensors. Many algorithm can be used to convert strings to tensors - all related to the domain of interest. But the Universal Sentence Encoder is a wide broad sentence encoder that will project all words in one dimensional space. The cosine similarity can be used to see how closed some words are in meaning.
Example
Though king and kind are closed in hamming distance (difference of only one character), they are very different. Whereas queen and king though they seems not related (because all characters are different) are close in meaning. Therefore the distance (in meaning) between king and queen should be smaller than between king and kind as demonstrated in the following snippet.
<script src="https://cdn.jsdelivr.net/npm/#tensorflow/tfjs"></script>
<script src="https://cdn.jsdelivr.net/npm/#tensorflow-models/universal-sentence-encoder"></script>
<script>
(async() => {
const model = await use.load();
const embeddings = (await model.embed(['queen', 'king', 'kind'])).unstack()
tf.losses.cosineDistance(embeddings[0], embeddings[1], 0).print() // 0.39812755584716797
tf.losses.cosineDistance(embeddings[1], embeddings[2], 0).print() // 0.5585797429084778
})()
</script>
Comparing the meaning of two string is still an ongoing research. If you really want to solve the problem (or to get really good performance of your language modal) you should consider get a PhD.
For out of box solution at the time: I found this Github repo that implement google's BERT modal and use it to get the embedding of two sentences. In theory, the two sentence share the same meaning if there embedding is similar.
https://github.com/UKPLab/sentence-transformers
# the following is simplified from their README.md
embedder = SentenceTransformer('bert-base-nli-mean-tokens')
# Corpus with example sentences
S1 = ['A man is eating a food.']
S2 = ['A man is eating pasta.']
s1_embedding = embedder.encode(S1)
s2_embedding = embedder.encode(S2)
dist = scipy.spatial.distance.cdist([s1_embedding], [s2_embedding], "cosine")[0]
Example output (copied from their README.md)
Query: A man is eating pasta.
Top 5 most similar sentences in corpus:
A man is eating a piece of bread. (Score: 0.8518)
A man is eating a food. (Score: 0.8020)
A monkey is playing drums. (Score: 0.4167)
A man is riding a horse. (Score: 0.2621)
A man is riding a white horse on an enclosed ground. (Score: 0.2379)

Easy one: REGEX to select everything after ' - ' in multi line text

Apparently this is not easy! There's got to be a way using pure regex?? I just know there is....
I have found a way to select the text after the first occurrence of a hyphen in a text file
Unique Thing - Some Text
Another Thing - Some Text again
Some Thing - Some more text
But I only want the right side of the hyphen..
Anyone know a quick regex to accomplish this?
To be clear, given text above i want
Some Text
Some Text again
Some more text
Thanks ya'll
UPDATE:
Maybe it would help with an actual chunk of text. This is from the most recent live stream chat for the whitehouse press briefing Aug 2, 2017.
Hernando Arce - build the wall with solar panels,
Christmas Girl - Let's do our own quick internet poll on live chat. Ready........Good with new immigrating into the US policy he is talking about. YES or NO,
ART - AMEN,
coffeefish - Stop H1B visa corruption!,
CarollDelMuro .Arbonne - Red,
Legion - BUILD THE WALL!,
wass sabi - MAGA,
Yokoshima - I live in Florida. Speaking English isn't racist. If you've ever been to Miami, you would know why it's needed.,
Home O'DFree - NO the campaign was BUILD THE WALL,
Melissa Renee - is he on benzos,
Paid Observer - kim jung un vs Trump in basketball,
Selina Serrano - polling data,
zonnekat - aliens....,
Farrah - NFL ,
Selina Serrano - massive,
Glenda Greene - MAGA,
Christoph Schneider - who would ever come to USA when they get lower pays? Russians?,
Carolyn Hall - MAGA MAGA MAGA ,
Sandra Honeyman - Isn't limiting immigration to skilled workers going to displace more skilled American workers?,
Mike Hancock - AMERICA FIRST,
Adnan Khan - Send them back to Mars,
Paid Observer - wtf is that,
GDotcom - THIS BETTER PASS OR THERE SHALL BE HANGINGS,
Null_Mage - This man is more attractive than Sarah,
monkeygraborange - FUCK CONGRESS,
Selina Serrano - personal,
This is the text i'm testing in regex101.
^[^-]*[^ -] does not seem to work here.
I do like the few suggestions about splitting line by line then matching however, the chat stream is many thousands of lines. The end result of all this is counting occurrences of words. For anyone whos interested the repo is https://github.com/archae0pteryx/yt-live-chat-scraper I just pushed the logs from the latest press briefing.
/[\s\S]- (.*)/g - Should do it
[\s\S] - For matching new lines
/g - Continues matching
🍻
You can use a capturing group if you want to use a regex:
const r = /- (.*)/
console.log('Unique Thing - Some Text'.match(r)[1]) //'Some Text'
Try this:
.*-[ ]*
Choose everything before the hyphen and white spaces after the hyphen.
With this patter you can remove all the text that match and leave the right side that you want.
UPDATE:
But, if you want the right side you can use it:
-(.*)
and choose the group 1:

Categories