How to extract a HTML tag pattern multiple times from a string?

Question

I've got this pattern based on which I want to search a string to find all matches. After using findall(), only the last one matched is printed.

The string which I want to process is below:

'<inventor sequence="001" designation="us-only"><addressbook><last-name>Li</last-name><first-name>Shuo</first-name><address><city>Beijing</city><country>CN</country></address></addressbook></inventor><inventor sequence="002" designation="us-only"><addressbook><last-name>Liu</last-name><first-name>Xin Peng</first-name><address><city>Beijing</city><country>CN</country></address></addressbook></inventor><inventor sequence="003" designation="us-only"><addressbook><last-name>Sun</last-name><first-name>Sheng Yan</first-name><address><city>Beijing</city><country>CN</country></address></addressbook></inventor><inventor sequence="004" designation="us-only"><addressbook><last-name>Wang</last-name><first-name>Hua</first-name><address><city>Littleton</city><state>MA</state><country>US</country></address></addressbook></inventor><inventor sequence="005" designation="us-only"><addressbook><last-name>Wang</last-name><first-name>Jun</first-name><address><city>Littleton</city><state>MA</state><country>US</country></address></addressbook></inventor>'

I try to use the following code to extract all inventors from the string.

INVENTORS_CONTENT_PATTERN = re.compile('<inventor sequence=".*" designation=".*">(.*?)</inventor>')

re.findall(INVENTORS_CONTENT_PATTERN, data)

The result I get is the last one matched, not all the inventors from data:

['<addressbook><last-name>Wang</last-name><first-name>Jun</first-name><address><city>Littleton</city><state>MA</state><country>US</country></address></addressbook>']

The wildcard in `designation=".*"` ends up matching the middle 90% of the text. — John Gordon, Aug 24 '19 at 02:00
Using regex to parse a nested structure such as HTML is a recipe for heartbreak. Consider using a purpose-built xml/html parser. — John Gordon, Aug 24 '19 at 02:01
See https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags — The fourth bird, Aug 24 '19 at 08:19

score 0 · Accepted Answer · answered Aug 24 '19 at 01:59

This expression might be closer to what you have in mind:

<inventor sequence="[^"]*" designation="[^"]*">(.*?)<\/inventor>

Test

import re

regex = r'<inventor sequence="[^"]*" designation="[^"]*">(.*?)<\/inventor>'
test_str = """
<inventor sequence="001" designation="us-only"><addressbook><last-name>Li</last-name><first-name>Shuo</first-name><address><city>Beijing</city><country>CN</country></address></addressbook></inventor><inventor sequence="002" designation="us-only"><addressbook><last-name>Liu</last-name><first-name>Xin Peng</first-name><address><city>Beijing</city><country>CN</country></address></addressbook></inventor><inventor sequence="003" designation="us-only"><addressbook><last-name>Sun</last-name><first-name>Sheng Yan</first-name><address><city>Beijing</city><country>CN</country></address></addressbook></inventor><inventor sequence="004" designation="us-only"><addressbook><last-name>Wang</last-name><first-name>Hua</first-name><address><city>Littleton</city><state>MA</state><country>US</country></address></addressbook></inventor><inventor sequence="005" designation="us-only"><addressbook><last-name>Wang</last-name><first-name>Jun</first-name><address><city>Littleton</city><state>MA</state><country>US</country></address></addressbook></inventor>

"""
print(re.findall(regex, test_str))

Output

['<addressbook><last-name>Li</last-name><first-name>Shuo</first-name><address><city>Beijing</city><country>CN</country></address></addressbook>', '<addressbook><last-name>Liu</last-name><first-name>Xin Peng</first-name><address><city>Beijing</city><country>CN</country></address></addressbook>', '<addressbook><last-name>Sun</last-name><first-name>Sheng Yan</first-name><address><city>Beijing</city><country>CN</country></address></addressbook>', '<addressbook><last-name>Wang</last-name><first-name>Hua</first-name><address><city>Littleton</city><state>MA</state><country>US</country></address></addressbook>', '<addressbook><last-name>Wang</last-name><first-name>Jun</first-name><address><city>Littleton</city><state>MA</state><country>US</country></address></addressbook>']

If you wish to explore/simplify/modify the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.

How to extract a HTML tag pattern multiple times from a string?

1 Answers1

Test

Output