Is there a way to use readability (text extraction algorithm) and a custom algorithm in python to extract links from text?

Question

I'd like to figure out a way of extracting links that are in the body of text.

2.) I'd like to somehow compare the extracted text to the original html text in order to extract links in the actual body of an article.

score 2 · Accepted Answer · answered Jan 04 '11 at 00:05

2

Well, it looks like it returns a BeautifulSoup tree. So you should be able to do something like:

article = page.summary()   # Extract article using readability
article.findAll("a")       # Return a list of all links in the article

answered Jan 04 '11 at 00:05

Thomas K

BeautifulSoup is definitely the way to go. – Sri Raghavan Jan 04 '11 at 00:51
1

@Sri: Readability already uses BeautifulSoup. It's designed to pare a page down to the content, minus advertising, navigation and so forth. – Thomas K Jan 04 '11 at 01:09

1 Answers1