0

I know that beautiful soup has a function to match classes based on regex that contains certain strings, based on a post here. Below is a code example from that post:

regex = re.compile('.*listing-col-.*')
for EachPart in soup.find_all("div", {"class" : regex}):
        print EachPart.get_text()

Now, is it possible to do the opposite? Basically, find classes that do not contain a certain regex. In SQL language, it's like:

where class not like '%test%'

Thanks in advance!

Wiktor Stribiżew
  • 607,720
  • 39
  • 448
  • 563
mangarapaul
  • 69
  • 5

3 Answers3

1

This actually can be done by using Negative Lookahead

Negative Lookahead has the following syntax (?!«pattern») and matches if pattern does not match what comes before the current location in the input string.

In your case, you could use the following regex to match all classes that don’t contain listing-col- in their name:

regex = re.compile('^((?!listing-col-).)*$')

Here’s the pretty simple and straightforward explanation of this regex ^((?!listing-col-).)*$:

  • ^ asserts position at start of a line
  • Capturing Group ((?!listing-col-).)*
    • * matches the previous token between zero and unlimited times, as many times as possible, giving back as needed
    • Negative Lookahead (?!listing-col-). Assert that the Regex below does not match. listing-col- matches the characters listing-col- literally (case sensitive)
    • . matches any character
  • $ asserts position at the end of a line

Also, you may find the https://regex101.com site useful

It will help you test your patterns and show you a detailed explanation of each step. It's your best friend in writing regular expressions.

andylvua
  • 312
  • 2
  • 7
  • 1
    thank you so much! This works perfectly and I can even create a list of tags to exclude using your approach as well! – mangarapaul Jul 22 '22 at 15:24
0

One possible solution is utilizing regex directly. You can refer to Regular expression to match a line that doesn't contain a word.

Or you can introduce a function to implement the logic and pass it to find_all as a parameter. You can refer to https://beautiful-soup-4.readthedocs.io/en/latest/index.html?highlight=find_all#find-all

kuriboh
  • 141
  • 5
0

You can use css selector syntax with :not() pseudo class and * contains operator

data = [i.text() for i in soup.select('div[class]:not([class*="listing-col-"])')]
QHarr
  • 83,427
  • 12
  • 54
  • 101