1

For example I have s="I REALLY don't want to talk about it, not at all!"

I want re.findall(reg, s) to return "I" "don't" "want" "to" "talk" "about" "it" "," "not" "at" "all" "!"

So far I got reg=r'[^\w\s]+|\w+|\n' which can not filter out the word "REALLY"

thanks

runnerpaul
  • 5,942
  • 8
  • 49
  • 118
Jack2019
  • 275
  • 2
  • 10

2 Answers2

2

The \w+ pattern matches 1 or more any word chars, including words in ALLCAPS.

Note that I, a pronoun, is also ALLCAPS. Thus, assuming you want to skip all ALLCAPS words of 2 or more letters, you may consider fixing your current pattern as

r'[^\w\s]+|\b(?![A-Z]{2,}\b)\w+|\n'

See the regex demo

The \b(?![A-Z]{2,}\b)\w+ pattern matches

  • \b - word boundary
  • (?![A-Z]{2,}\b) - a negative lookahead that fails the match if, immediately to the right of the current location, there are 2 or more ASCII uppercase letters followed with a word boundary
  • \w+ - 1 or more word chars (if you only want to match letters, replace with [^\W\d_]+).

To support all Unicode uppercase letters, you may use PyPi regex with r'[^\w\s]+|\b(?!\p{Lu}{2,}\b)\w+|\n' pattern, or build the class using pLu = '[{}]'.format("".join([chr(i) for i in range(sys.maxunicode) if chr(i).isupper()])) (Python 3) or pLu = u'[{}]'.format(u"".join([unichr(i) for i in xrange(sys.maxunicode) if unichr(i).isupper()])) (Python 2). See Python regex for unicode capitalized words. Note I'd recommend sticking to the latest Python versions or the latest PyPi regex modules.

Wiktor Stribiżew
  • 607,720
  • 39
  • 448
  • 563
  • thanks! is there any recommended book/document on reg expression with very nice explanations just like the one you wrote here ? – Jack2019 Nov 18 '18 at 02:20
  • @SoManyProblems No idea which part of the answer you refer to. To see a specific regex description, go to regex101.com. To learn regex in general, there is a nice ["Mastering Regular Expressions" book written by J. Friedl](http://ww2.ii.uj.edu.pl/~tabor/prII09-10/perl/master.pdf). – Wiktor Stribiżew Nov 18 '18 at 10:02
  • @SoManyProblems If this solution worked for you please consider accepting the answer. – Wiktor Stribiżew Nov 19 '18 at 09:24
1

This quote by Brian Kernighan is especially true for regular expressions.

Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?

So if something is difficult to do in a single regular expression, you might want to split it into two steps. Find all words first, and then filter out the all uppercase words. Easier to understand and easier to test.

>>> import re
>>> s="I REALLY don't want to talk about it, not at all!"
>>> words = re.findall(r"[\w']+", s)
>>> words = [w for w in words if w.upper() != w]
>>> print(words)
["don't", 'want', 'to', 'talk', 'about', 'it', 'not', 'at', 'all']
Håken Lid
  • 22,318
  • 9
  • 52
  • 67