0

Here's what I have:

r = requests.get("http://www.cnn.com/")
htmlelement = lxml.html.fromstring(r.text)
html = lxml.html.tostring(htmlelement)
tree = lxml.etree.fromstring(html)
print tree.xpath('//*[@id="cnn_maintt1imgbul"]/div/div[2]/div/h1/a')

I thought xml.html corrected the broken html?

The error is:

XMLSyntaxError: Opening and ending tag mismatch: link line 32 and head, line 75, column 8

Thanks!

User
  • 23,729
  • 38
  • 124
  • 207
  • http://stackoverflow.com/questions/1922032/parsing-html-in-python-lxml-or-beautifulsoup-which-of-these-is-better-for-wha – dstromberg Apr 27 '14 at 02:39

1 Answers1

1

I don't understand why you're trying to reparse the content after getting this far:

>>> htmlelement = lxml.html.fromstring(r.text)

Because at this point you can simply apply your xpath expression:

>>> results = htmlelement.xpath('//*[@id="cnn_maintt1imgbul"]/div/div[2]/div/h1/a')
>>> results
[<Element a at 0x1113a1f50>]
>>> print lxml.html.tostring(results[0])
'<a href="/2014/04/26/world/asia/south-korea-ship-sinking/index.html?hpt=hp_t1" target="">SOUTH KOREAN PRIME MINISTER RESIGNS</a>'

I believe your problem is that lxml.html.tostring() still generates HTML, which you then try to parse with the XML parser.

larsks
  • 277,717
  • 41
  • 399
  • 399