How can you print a string using raw_unicode_escape encoding in python 3?

Question

The following code with fail in Python 3.x with TypeError: must be str, not bytes because now encode() returns bytes and print() expects only str.

#!/usr/bin/python
from __future__ import print_function
str2 = "some unicode text"
print(str2.encode('raw_unicode_escape'))

How can you print a Unicode string escaped representation using print()? I'm looking for a solution that will work with Python 2.6 or newer, including 3.x

Update

Below line will work with 3.x but it will not work with 2.6, generating AttributeError: 'file' object has no attribute 'buffer'

sys.stdout.buffer.write(str2.encode('raw_unicode_escape'))

score 5 · Accepted Answer · answered Jun 14 '10 at 18:11

5

I'd just use:

print(str2.encode('raw_unicode_escape').decode('ascii'))

if you want identical code in Python 3 and Python 2.6 (otherwise you could use repr in 2.6 and ascii in Python 3, but that's not really "identical";-).

answered Jun 14 '10 at 18:11

Alex Martelli

854,459
170
1,222
1,395

Thanks Alex, I'm currently looking to build a set of function overrides for Python 2.6+/3.x for making it more Unicode friendly. I hope I will succeed. Any idea on how to override file.write function to make it accept bytes? it is related to http://stackoverflow.com/questions/984014/python-3-is-using-sys-stdout-buffer-write-good-style – sorin Jun 14 '10 at 19:04
1

@Sorin, you typically do type-checks if you want to accept both unicode and byte strings and treat them differently; sometimes you can get away with adaptation (e.g. a method that accepts a unicode string and returns it unchanged, **or** a byte string and returns the unicode string obtained by decoding it) -- but that's a bit far afield of what can easily be discussed in a comment, and really very different from the original question, so you may want to ask another, separate question about this! – Alex Martelli Jun 14 '10 at 19:22
ps ```python -c "print(chr(128).encode('raw_unicode_escape').decode('ascii'));"``` will crash with ```UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)``` – hanshenrik Sep 18 '22 at 18:58

score 1 · Answer 2 · edited May 23 '17 at 11:48

I can't reproduce your issue, please see previous revisions of this answer for a log of my attempts (which explains my link in the comments).

However:

It seems like you are trying to force an encoding while writing to a file by doing all the legwork yourself. However in Python 3, open() accepts an encoding parameter that does all the magic for you.

badp@delta:~$ python3
Python 3.1.2 (r312:79147, Apr 15 2010, 12:35:07) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> foo = open("look mah, utf-32", "w", encoding="utf-32")
>>> foo.write("bar")
3
>>> foo.close()
>>> foo = open("look mah, utf-32", "rb")
>>> foo.read()
b'\xff\xfe\x00\x00b\x00\x00\x00a\x00\x00\x00r\x00\x00\x00'

If you are looking for a Python 2 equivalent, it seems like you really want to use io.open().

Try to do this running from inside a file, it will not work. Running from the console seams to work but not from files. Also I added a new comment regarding usage of buffer. — sorin, Jun 14 '10 at 16:08

JAB · Answer 3 · 2010-06-14T19:01:00.617

http://docs.python.org/py3k/library/functions.html#ascii

As repr(), return a string containing a printable representation of an object, but escape the non-ASCII characters in the string returned by repr() using \x, \u or \U escapes. This generates a string similar to that returned by repr() in Python 2.

And the resulting string will indeed be of type str rather than bytes.

Example:

>>> a = '''Ⴊ ⇠ ਐ ῼ இ ╁ ଠ ୭ ⅙ ㈣'''
>>> ascii(a)
"'\\u10aa \\u21e0 \\u0a10 \\u1ffc \\u0b87 \\u2541 \\u0b20 \\u0b6d \\u2159 \\u3223'"
>>> print(ascii(a))
'\u10aa \u21e0 \u0a10 \u1ffc \u0b87 \u2541 \u0b20 \u0b6d \u2159 \u3223'

And if you wanted to trim off the extra quotes, you could just do print(ascii(a)[1:-1]).

EDIT: As Alex states, you'd have to use repr in Python 2.6 instead of ascii. His solution does indeed work for both Python 2 and 3, but if you plan on doing the conversion a lot (and thus would prefer something easier to type multiple times), one possibility is to put a conditional at the start of your program as follows:

import sys
if sys.version_info[0] == 3:
    unic = ascii
else:
    unic = repr

And then you just use unic (or whatever you want to call it) wherever you'd use repr in Python 2 and ascii in Python 3.

...Though I suppose you could use elif sys.version_info[0] == 2: instead of else: if you wanted to be a bit more careful.

@Alex: True, but as shown in the quote in my answer, `repr` is. — JAB, Jun 14 '10 at 18:21
`repr` in Python 3 does not replace non-ascii characters with escapes -- using two different functions depending on language level (`repr` in Python 2, `ascii` in Python 3) is not wht the OP requires, "a solution that will work with Python 2.6 or newer, including 3.x". — Alex Martelli, Jun 14 '10 at 18:32

How can you print a string using raw_unicode_escape encoding in python 3?

Update

3 Answers3