Convert ascii characters to normal text

Question

I have text like this:

&#8216;The zoom animations everywhere on the new iOS 7 are literally making me nauseous and giving me a headache,&#8217;wroteforumuser Ensorceled.

I understand that #8216 is an ASCII character.How can i convert it to normal characters without using .replace which is cumbersome.

Is [this](http://stackoverflow.com/questions/730299/replace-html-entities-with-the-corresponding-utf-8-characters-in-python-2-6) relevant? — rlms, Sep 28 '13 at 15:37
@user2784753: And what difference that *that* make? The answer still applies. — Martijn Pieters, Sep 28 '13 at 15:44
That particular numeric entity is not an ASCII character. ASCII only covers codes up to 127; above that is ISO 8859-1 (aka Latin 1) up to 255 and then ISO 10646 (pretty much Unicode). — Yann Vernier, Sep 28 '13 at 16:19
@YannVernier: ASCII and Latin-1 are part of Unicode *too*. But that character is not ASCII, indeed. — Martijn Pieters, Sep 28 '13 at 16:23
Yes, but only the early parts. I apologize if I made that unclear. What I wanted to point out is that ASCII isn't sufficient for this character (opening single quote, the with ’ being closing single quote); this may make the question of converting it to "normal" characters rather complicated. Unicode is sufficient, so Python produces a unicode object (prefixed with u) in your answer. — Yann Vernier, Sep 28 '13 at 16:32

score 3 · Answer 1 · answered Sep 28 '13 at 15:37

You have an HTML escape there. Use the HTMLParser.HTMLParser() class to unescape these:

from HTMLParser import HTMLParser

parser = HTMLParser()
unescaped = parser.unescape(escaped)

Demo:

>>> from HTMLParser import HTMLParser
>>> parser = HTMLParser()
>>> escaped = '&#8216;The zoom animations everywhere on the new iOS 7 are literally making me nauseous and giving me a headache,&#8217;wroteforumuser Ensorceled.'
>>> parser.unescape(escaped)
u'\u2018The zoom animations everywhere on the new iOS 7 are literally making me nauseous and giving me a headache,\u2019wroteforumuser Ensorceled.'
>>> print parser.unescape(escaped)
‘The zoom animations everywhere on the new iOS 7 are literally making me nauseous and giving me a headache,’wroteforumuser Ensorceled.

In Python 3, the HTMLParser module has been renamed to html.parser; adjust the import accordingly:

from html.parser import HTMLParser

Convert ascii characters to normal text

1 Answers1