We Keep Coding

Regex javascript tuning

First steps in regex and been trying to get some values out of emails that are being fetched. So far I've achieved some (even if it's not with the best approach) but in need of some more values and can't figure out how to get them.
This is the email template that is more or less always the same:
London, 06/20/09
Mr. Tom Waits
Process ref.: CR // 1943061
Your reference: 338256
Clients' names: Mary Lamb, John Snow
We return to your contact regarding the complaint on behalf of the clients mentioned above.
We inform you that the refund process has already started, so you should receive the respective amount (375EUR) within 4/6 weeks.
Payment ref.: 2500062960.
Our compliments,(...)
WHAT I NEED:
Date after "London,"
Process ref. (2 letters + // + digits)
Your ref. number
Clients' names
Amount
Payment reference number
Notes the amount not always comes between "( )", sometimes it's preceeded by "amount" others by "amount of", sometimes "EUR" is separated by a space, but the needed value is always the first digits combination on the paragraph
WHAT I HAVE SO FAR:
(?:London,)(.*)\s+(?:Mr. Tom Waits)\s+(?:Process ref.:)\s+(....\d+)\s+(?:Your reference:)\s+(\d+)\s+(?:Clients' names:)\s+(.*)\s+
WHAT IT RETRIEVES:
Date after "London,"
Process ref. (2 letters + // + digits)
Your reference number
Clients' names
WHAT'S MISSING:
Amount
Payment reference number
Other Concerns:
I tried to exclude the "Mr" parapraph but I think there may be a problem when they write it a little different like with an extra space or something
The same problem may arise if they also write the items a little different like, for example, "Process reference" instead of "Process ref."
Thanks in advance.

This one's gonna be one hefty regex. I do hope you don't try this regex on a massive file but rather one singular entry (like the one you've shown in your post).
Anyway, here's the regex-
London, ([\d\/]+)[\n\w\W]*?Process ref(?:\.|erence): ([A-Z \/\d]+)[\n\w\W]*?Your ref(?:\.|erence): (\d+)[\n\w\W]*?Client(?:s'|'s) name(?:s)?: ([\w ,]+)[\n\w\W]*?amount.*?(\d+)[\n\w\W]*?Payment ref(?:\.|erence): (\d+)
This should be quite permissive, it doesn't depend on many variable (seemingly) things, apart from London, that one's a bit of a hard coding but I'm assuming it's always London.
Now, let's walk through this-
London, ([\d\/]+) - This basically matches London, DATE - where date is, well, a date, where each element of the date is separated by a /.
In this case, it matches 06/20/09 from London, 06/20/09
[\n\w\W]*? - Try to keep up with this one - I'm using it a A LOT.
This will match pretty much all new lines, word characters and non word characters in a non-greedy way. In this particular case, this will match pretty much everything and that includes the newlines. This is used to just skip over anything and everything until we reach the desired spot.
Process ref(?:\.|erence): ([A-Z \/\d]+) - Captures the process reference, which can consist of capital alphabets (I assume, you can change that), slashes (/), and digits
Works with both ref. and reference
In this case, it matches CR // 1943061, from Process ref.: CR // 1943061
[\n\w\W]*? - Ignore everything up until the next token
Your ref(?:\.|erence): (\d+) - Captures "your reference", which can consist of digits
Works with both ref. and reference
In this case, it matches 338256
[\n\w\W]*? - Ignore everything up until the next token
Client(?:s'|'s) name(?:s)?: ([\w ,]+) - Captures the client name(s) - modified so it supports single client names too. (check the demo). The name list can consist of word characters, spaces and a comma.
In this case, it captures Mary Lamb, John Snow, from Clients' names: Mary Lamb, John Snow
(\d+) - Capture the digits - this is a big assumption, I'm assuming that the only digits that appear after client name list, are the ones for the amount. If they aren't
[\n\w\W]*? - Ignore everything up until the next token
amount.*?(\d+) - Captures the first group of digits that appear after amount. This is a bit of an assumption, I'm assuming that amount word is actually present in that paragraph.
In this case, it captures 375, from amount (375EUR)
[\n\w\W]*? - Ignore everything up until the next token
Payment ref(?:\.|erence): (\d+) - Capture the Payment reference number, which can consist of digits
Works with both ref. and reference
In this case, it captures 2500062960, from Payment ref.: 2500062960.
Check out the demo!

how to compare two strings by meaning?

I want the user of my node.js application to write down ideas, which then get stored in a database.
So far so good, but I don't want redundant entrys in that table, so I decided to check for similarity, using this one:
https://www.npmjs.com/package/string-similarity-js
Do you know a way, in which I can compare two strings by meaning? In like getting a high similarity score for "using public transport" vs "driving by train" which performs very poor in the above one.

To compare two strings by meaning, the strings would need to be convert first to a tensor and then evalutuate the distance or similarity between the tensors. Many algorithm can be used to convert strings to tensors - all related to the domain of interest. But the Universal Sentence Encoder is a wide broad sentence encoder that will project all words in one dimensional space. The cosine similarity can be used to see how closed some words are in meaning.
Example
Though king and kind are closed in hamming distance (difference of only one character), they are very different. Whereas queen and king though they seems not related (because all characters are different) are close in meaning. Therefore the distance (in meaning) between king and queen should be smaller than between king and kind as demonstrated in the following snippet.
<script src="https://cdn.jsdelivr.net/npm/#tensorflow/tfjs"></script>
<script src="https://cdn.jsdelivr.net/npm/#tensorflow-models/universal-sentence-encoder"></script>
<script>
(async() => {
const model = await use.load();
const embeddings = (await model.embed(['queen', 'king', 'kind'])).unstack()
tf.losses.cosineDistance(embeddings[0], embeddings[1], 0).print() // 0.39812755584716797
tf.losses.cosineDistance(embeddings[1], embeddings[2], 0).print() // 0.5585797429084778
})()
</script>

Comparing the meaning of two string is still an ongoing research. If you really want to solve the problem (or to get really good performance of your language modal) you should consider get a PhD.
For out of box solution at the time: I found this Github repo that implement google's BERT modal and use it to get the embedding of two sentences. In theory, the two sentence share the same meaning if there embedding is similar.
https://github.com/UKPLab/sentence-transformers
# the following is simplified from their README.md
embedder = SentenceTransformer('bert-base-nli-mean-tokens')
# Corpus with example sentences
S1 = ['A man is eating a food.']
S2 = ['A man is eating pasta.']
s1_embedding = embedder.encode(S1)
s2_embedding = embedder.encode(S2)
dist = scipy.spatial.distance.cdist([s1_embedding], [s2_embedding], "cosine")[0]
Example output (copied from their README.md)
Query: A man is eating pasta.
Top 5 most similar sentences in corpus:
A man is eating a piece of bread. (Score: 0.8518)
A man is eating a food. (Score: 0.8020)
A monkey is playing drums. (Score: 0.4167)
A man is riding a horse. (Score: 0.2621)
A man is riding a white horse on an enclosed ground. (Score: 0.2379)

Easy one: REGEX to select everything after ' - ' in multi line text

Apparently this is not easy! There's got to be a way using pure regex?? I just know there is....
I have found a way to select the text after the first occurrence of a hyphen in a text file
Unique Thing - Some Text
Another Thing - Some Text again
Some Thing - Some more text
But I only want the right side of the hyphen..
Anyone know a quick regex to accomplish this?
To be clear, given text above i want
Some Text
Some Text again
Some more text
Thanks ya'll
UPDATE:
Maybe it would help with an actual chunk of text. This is from the most recent live stream chat for the whitehouse press briefing Aug 2, 2017.
Hernando Arce - build the wall with solar panels,
Christmas Girl - Let's do our own quick internet poll on live chat. Ready........Good with new immigrating into the US policy he is talking about. YES or NO,
ART - AMEN,
coffeefish - Stop H1B visa corruption!,
CarollDelMuro .Arbonne - Red,
Legion - BUILD THE WALL!,
wass sabi - MAGA,
Yokoshima - I live in Florida. Speaking English isn't racist. If you've ever been to Miami, you would know why it's needed.,
Home O'DFree - NO the campaign was BUILD THE WALL,
Melissa Renee - is he on benzos,
Paid Observer - kim jung un vs Trump in basketball,
Selina Serrano - polling data,
zonnekat - aliens....,
Farrah - NFL ,
Selina Serrano - massive,
Glenda Greene - MAGA,
Christoph Schneider - who would ever come to USA when they get lower pays? Russians?,
Carolyn Hall - MAGA MAGA MAGA ,
Sandra Honeyman - Isn't limiting immigration to skilled workers going to displace more skilled American workers?,
Mike Hancock - AMERICA FIRST,
Adnan Khan - Send them back to Mars,
Paid Observer - wtf is that,
GDotcom - THIS BETTER PASS OR THERE SHALL BE HANGINGS,
Null_Mage - This man is more attractive than Sarah,
monkeygraborange - FUCK CONGRESS,
Selina Serrano - personal,
This is the text i'm testing in regex101.
^[^-]*[^ -] does not seem to work here.
I do like the few suggestions about splitting line by line then matching however, the chat stream is many thousands of lines. The end result of all this is counting occurrences of words. For anyone whos interested the repo is https://github.com/archae0pteryx/yt-live-chat-scraper I just pushed the logs from the latest press briefing.

/[\s\S]- (.*)/g - Should do it
[\s\S] - For matching new lines
/g - Continues matching
🍻

You can use a capturing group if you want to use a regex:
const r = /- (.*)/
console.log('Unique Thing - Some Text'.match(r)[1]) //'Some Text'

Try this:
.*-[ ]*
Choose everything before the hyphen and white spaces after the hyphen.
With this patter you can remove all the text that match and leave the right side that you want.
UPDATE:
But, if you want the right side you can use it:
-(.*)
and choose the group 1:

Matching multiple quotes in a sentence

I am trying to match multiple quotes inside of a single sentence, for example the line:
Hello "this" is a "test" example.
This is the regex that I am using, but I am having some problems with it:
/[^\.\?\!\'\"]{1,}[\"\'\“][^\"\'\“\”]{1,}[\"\'\“\”][^\.\?\!]{1,}[\.\?\!]/g
What I am trying achieve with this regex is to find everything from the start of the last sentence until I hit quotes, then find the closing set and continue until either a .?!
The sample text that I am using to test with is from Call of Cthulhu:
What seemed to be the main document was headed “CTHULHU CULT” in characters painstakingly printed to avoid the erroneous reading of a word so unheard-of. The manuscript was divided into two sections, the first of which was headed “1925—Dream and Dream Work of H. A. Wilcox, 7 Thomas St., Providence, R.I.”, and the second, “Narrative of Inspector John R. Legrasse, 121 Bienville St., New Orleans, La., at 1908 A. A. S. Mtg.—Notes on Same, & Prof. Webb’s Acct.” The other manuscript papers were all brief notes, some of them accounts of the queer dreams of different persons, some of them citations from theosophical books and magazines.
The issue comes on the line The manuscript was.... Does anyone know how to account for repeats like this? Or is there a better way?

This one ignores [.?!] inside quotes. But cases like Acct.” The nth will be considered as a single sentence in this case. Probably a . is missing over there.
var r = 'What seemed to be the main document was headed “CTHULHU.?! CULT” in characters painstakingly printed to avoid the erroneous reading of a word so unheard-of. The manuscript was divided into two sections, the first of which was headed “1925—Dream and Dream Work of H. A. Wilcox, 7 Thomas St., Providence, R.I.”, and the second, “Narrative of Inspector John R. Legrasse, 121 Bienville St., New Orleans, La., at 1908 A. A. S. Mtg.—Notes on Same, & Prof. Webb’s Acct.” The other manuscript papers were all brief notes, some of them accounts of the queer dreams of different persons, some of them citations from theosophical books and magazines.'
.split(/[“”]/g)
.map((x,i)=>(i%2)?x.replace(/[.?!]/g,''):x)
.join("'")
.split(/[.?!]/g)
.filter(x => x.trim()).map(x => ({
sentence: x,
quotescount: x.split("'").length - 1
}));
console.log(r);

You can use this naive pattern:
/[^"'“.!?]*(?:"[^"*]"[^"'“.!?]*|'[^']*'[^"'“.!?]*|“[^”]*”[^"'“.!?]*)*[.!?]/
details:
/
[^"'“.!?]* # all that isn't a quote or a punct that ends the sentence
(?:
"[^"*]" [^"'“.!?]*
|
'[^']*' [^"'“.!?]*
|
“[^”]*” [^"'“.!?]*
)*
[.!?]
/
If you want something more strong, you can emulate the "atomic grouping" feature, in particular if you are not sure that each opening quote has a closing quote (to prevent catastrophic backtracking):
/(?=([^"'“.!?]*))\1(?:"(?=([^"*]))\2"[^"'“.!?]*|'(?=([^']*))\3'[^"'“.!?]*|“(?=([^”]*))\4”[^"'“.!?]*)*[.!?]/
An atomic group forbids backtracking once closed. Unfortunately this feature doesn't exist in Javascript. But there's a way to emulate it using a lookahead that is naturally atomic, a capture group and a backreference:
(?>expr) => (?=(expr))\1

Bizarre JavaScript newline splitting behaviour

So, I've got a remotely retrieved text document that I want to split by two newline (\n) characters. For example, the popular hymn All Things Bright And Beautiful.
All Things Bright And Beautiful
Cecil F. Alexander
All things bright and beautiful,
All creatures great and small,
All things wise and wonderful,
The Lord God made them all.
Each little flower that opens,
Each little bird that sings,
He made their glowing colours,
He made their tiny wings.
The purple-headed mountain,
The river running by,
The sunset and the morning,
That brightens up the sky;
The cold wind in the winter,
The pleasant summer sun,
The ripe fruits in the garden,
He made them every one;
The tall trees in the greenwood,
The meadows for our play,
The rushes by the water,
To gather every day;
He gave us eyes to see them,
And lips that we might tell
How great is God Almighty,
Who has made all things well.
Taking a look in TextMate with invisibles on shows me two newline characters in between verses. So far so good. When I copy and paste this into irb everything works absolutely fine:
ruby-1.8.7-p334 :034 > content.split "\n\n"
=> ["All Things Bright And Beautiful\nCecil F. Alexander", "All things bright and beautiful,\nAll creatures great and small,\nAll things wise and wonderful,\nThe Lord God made them all. ", "Each little flower that opens,\nEach little bird that sings,\nHe made their glowing colours,\nHe made their tiny wings.", "The purple-headed mountain,\nThe river running by,\nThe sunset and the morning,\nThat brightens up the sky;", "The cold wind in the winter,\nThe pleasant summer sun,\nThe ripe fruits in the garden,\nHe made them every one;", "The tall trees in the greenwood,\nThe meadows for our play,\nThe rushes by the water,\nTo gather every day;", "He gave us eyes to see them,\nAnd lips that we might tell\nHow great is God Almighty,\nWho has made all things well."]
ruby-1.8.7-p334 :035 >
The problem comes the moment I go to JavaScript. In the console:
If I split by one \n it works absolutely fine. I've tried using carriage returns, newlines, passing a regex through as the delimiter and nothing seems to be working. I just cannot demystify this behaviour.
Any help would be thoroughly appreciated.

Javascript's split works the same as Ruby's. Most likely the \ns changed into something else because of how you imported the text to your script.
Do a this.element_content.split('')
var s = this.element_content;
var chars = [];
for(i=0; i<s.length; i++){ chars.push(s.charAt(i)); }
console.log(chars)
to convert your string into an array of characters and check what is actually being used in the line breaks.

Try, this.element_content.split(/[\n\r]{1,2}/m) using m for multiline