Regex to match capital/special/unicode/vietnamese characters

Question

I'm facing an issue. Indeed, I work with vietnamese texts and I want to find every word containing uppercase(s) (capital letter). When I use the 're' module, my function (temp) does not catch word like "Đà". The other way (temp2) is to check each character at a time, it works but it is slow since I have to split the sentences into words.

Hence I would like to know if there is a way of the "re" module to catch all the special capital letter.

I have 2 ways :

def temp(sentence):
    return re.findall(r'[a-z]*[A-Z]+[a-z]*', sentence)


lis=word_tokenize(sentence)
def temp2(lis):
    proper_noun=[]
    for word in lis:
        for letter in word:
            if letter.isupper():
                proper_noun.append(word)
                break
    return proper_noun

Input:

'nous avons 2 Đồng et 3 Euro'

Expected output :

['Đồng','Euro']

Thank you!

Answer 1

You may use this regex:

\b\S*[AĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴA-Z]+\S*\b

Regex Demo

Answer 2

The answer of @Rizwan M.Tuman is correct. I want to share with you the speed of execution of the three functions for 100,000 sentences.

lis=word_tokenize(sentence)
def temp(lis):
    proper_noun=[]
    for word in lis:
        for letter in word:
            if letter.isupper():
                proper_noun.append(word)
                break
    return proper_noun

def temp2(sentence):
    return re.findall(r'[a-z]*[A-Z]+[a-z]*', sentence)

def temp3(sentence):
    return re.findall(capital_letter,sentence)

By this way:

start_time = time.time()
for k in range(100000):
    temp2(sentence)
print("%s seconds" % (time.time() - start_time))

Here are the results:

>>Check each character of a list of words if it is a capital letter (.isupper())
(sentence has already been splitted into words)
0.4416656494140625 seconds

>>Function with re module which finds normal capital letters [A-Z] :
0.9373950958251953 seconds

>>Function with re module which finds all kinds of capital letters :
1.0783331394195557 seconds

Answer 3

To match only 1+ letter chunks that contain at least 1 uppercase Unicode letter you may use

import re, sys, unicodedata

pLu = '[{}]'.format("".join([chr(i) for i in range(sys.maxunicode) if chr(i).isupper()]))
p = re.compile(r"[^\W\d_]*{Lu}[^\W\d_]*".format(Lu=pLu))

sentence = 'nous avons 2 Đồng et 3 Ęułro.+++++++++++++++Next line'
print(p.findall(sentence))
# => ['Đồng', 'Ęułro', 'Next']

The pLu is a Unicode letter character class pattern built dynamically using unicodedata. It is dependent on the Python version, use the latest to include as many Unicode uppercase letters as possible (see this answer for more details, too). The [^\W\d_] is a construct matching any Unicode letter. So, the pattern matches any 0+ Unicode letters, followed with at least 1 Unicode uppercase letter, and then having any 0+ Unicode letters.

Note that your original r'[a-z]*[A-Z]+[a-z]*' will only find Next in this input:

print(re.findall(r'[a-z]*[A-Z]+[a-z]*', sentence)) # => ['Next']

See the Python demo

To match the words as whole words, use \b word boundary:

p = re.compile(r"\b[^\W\d_]*{Lu}[^\W\d_]*\b".format(Lu=pLu))

In case you want to use Python 2.x, do not forget to use re.U flag to make the \W, \d and \b Unicode aware. However, it is recommended to use the latest PyPi regex library and its [[:upper:]] / \p{Lu} constructs to match uppercase letters since it will support the up-to-date list of Unicode letters.