4

I parsed a HTML document and have Russian text in it. When I'm trying to print it in Python, I get this:

ÐлÑбниÑнÑй новогодний пÑнÑ

I tried to decode it and I get ISO-8859-1 encoding. I'm trying to decode it like that:

print drink_name.decode('iso8859-1')

But I get an error. How can I print this text, or encode it in Unicode?

hazestalker
  • 15
  • 1
  • 6
aaaapppp
  • 51
  • 1
  • 5
  • You don't encode it to unicode, you decode it to unicode – wim Nov 11 '14 at 16:44
  • Please include the code that you use to parse the HTML document in the first place, so we can help you not make this mistake in the first place. – Martijn Pieters Nov 11 '14 at 16:46
  • The answer can be [here](https://stackoverflow.com/questions/9233027/unicodedecodeerror-charmap-codec-cant-decode-byte-x-in-position-y-character). It helped me. – Rashevskii Mar 22 '22 at 07:43

1 Answers1

4

You have a Mojibake; UTF-8 bytes decoded as Latin-1 or CP1251 in this case.

You can repair it by reversing the process:

>>> print u'ÐлÑбниÑнÑй новогодний пÑнÑ'.encode('latin1').decode('utf8')
Клубничный новогодний пунш

(I had to copy the string from the original post source to capture all the non-printable bytes in the Mojibake).

The better method would be to not incorrectly decoding in the first place. You decoded the original text with the wrong encoding, use UTF-8 as the codec instead.

If you used requests to download the page, do not use response.text in this case; if the server failed to specific codec then the HTTP RFC default is to use Latin-1, but HTML documents often embed the encoding in a <meta> header instead. Leave decoding in such cases to your parser, like BeautifulSoup:

response = requests.get(url)
soup = BeautifulSoup(response.content)  # pass in undecoded bytes
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343