3

It's not really scraping, I'm just trying to find the URLs in a web page where the class has a specific value. For example:

<a class="myClass" href="/url/7df028f508c4685ddf65987a0bd6f22e">

I want to get the href value. Any ideas on how to do this? Maybe regex? Could you post some example code? I'm guessing html scraping libs, such as BeautifulSoup, are a bit of overkill just for this...

Huge thanks!

Charles Stewart
  • 11,661
  • 4
  • 46
  • 85
pns
  • 413
  • 1
  • 8
  • 19

7 Answers7

16

Regex is usally a bad idea, try using BeautifulSoup

Quick example:

html = #get html
soup = BeautifulSoup(html)
links = soup.findAll('a', attrs={'class': 'myclass'})
for link in links:
    #process link
Yacoby
  • 54,544
  • 15
  • 116
  • 120
9

Aargh, not regex for parsing HTML!

Luckily in Python we have BeautifulSoup or lxml to do that job for us.

Community
  • 1
  • 1
Daniel Roseman
  • 588,541
  • 66
  • 880
  • 895
2

Regex would be a bad choice. HTML is not a regular language. How about Beautiful Soup?

Mark Byers
  • 811,555
  • 193
  • 1,581
  • 1,452
1

Regex should not be used to parse HTML. See the first answer to this question for an explanation :)

+1 for BeautifulSoup.

Community
  • 1
  • 1
John Keyes
  • 5,479
  • 1
  • 29
  • 48
1

If your task is just this simple, just use string manipulation (without even regex)

f=open("htmlfile")
for line in f:
    if "<a class" in line and "myClass" in line and "href" in line:
        s = line [ line.index("href") + len('href="') : ]
        print s[:s.index('">')]
f.close()

HTML parsers is not a must for such cases.

ghostdog74
  • 327,991
  • 56
  • 259
  • 343
0

read Parsing Html The Cthulhu Way https://blog.codinghorror.com/parsing-html-the-cthulhu-way/

Cœur
  • 37,241
  • 25
  • 195
  • 267
George Godik
  • 1,716
  • 1
  • 14
  • 19
0

The thing is I know the structure of the HTML page, and I just want to find that specific kind of links (where class="myclass"). BeautifulSoup anyway?

pns
  • 413
  • 1
  • 8
  • 19