How to find and get the text from html file using python?

Question

Hi i have used this tag to find the content of tag from html file.

def everything_between(text,begin,end):
    idx1=content.find(begin)
    idx2=content.find(end,idx1)
    return content[idx1+len(begin):idx2].strip()

content=open('page.html').read()
title=everything_between(content,'<ul class="members">','</ul>')
interesting=everything_between(content,'INTERESTING:','bodystuff')
print(title)

but in tag <ul class="member"> there are multiple <ahref> tags, i want to get the content between <a href> that has <a href="/history/member/">

the script should get the values between <a href="/history/member/"></a>.

how can i do this?

Answer 1

http://www.crummy.com/software/BeautifulSoup/

soup.title
# <title>The Dormouse's story</title>

soup.title.name
# u'title'

soup.title.string
# u'The Dormouse's story'

soup.title.parent.name
# u'head'

soup.p
# <p class="title"><b>The Dormouse's story</b></p>

soup.p['class']
# u'title'

soup.a
# <a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>

soup.find_all('a')
# [<a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>,
#  <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>,
#  <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>]

soup.find(id="link3")
# <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>