329

Like in:

u'Hello'

My guess is that it indicates "Unicode", is that correct?

If so, since when has it been available?

OscarRyz
  • 196,001
  • 113
  • 385
  • 569

5 Answers5

235

You're right, see 3.1.3. Unicode Strings.

It's been the syntax since Python 2.0.

Python 3 made them redundant, as the default string type is Unicode. Versions 3.0 through 3.2 removed them, but they were re-added in 3.3+ for compatibility with Python 2 to aide the 2 to 3 transition.

Nick T
  • 25,754
  • 12
  • 83
  • 121
Stefan Kendall
  • 66,414
  • 68
  • 253
  • 406
  • 1
    Combining unicode + raw (regex) strings (e.g. `ur"string"`) is valid in Python 2, but it is unfortunately invalid syntax in Python 3. – cowlinator Feb 18 '20 at 09:07
155

The u in u'Some String' means that your string is a Unicode string.

Q: I'm in a terrible, awful hurry and I landed here from Google Search. I'm trying to write this data to a file, I'm getting an error, and I need the dead simplest, probably flawed, solution this second.

A: You should really read Joel's Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) essay on character sets.

Q: sry no time code pls

A: Fine. try str('Some String') or 'Some String'.encode('ascii', 'ignore'). But you should really read some of the answers and discussion on Converting a Unicode string and this excellent, excellent, primer on character encoding.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Andrew
  • 9,090
  • 8
  • 46
  • 59
  • 7
    This works if the string contains *ASCII text only*. In all other cases you'll have to explicitly encode. – Martijn Pieters Sep 13 '14 at 13:55
  • 3
    This treats the u'' as something "to get rid of". This tells me that you don't actually understand what it is. You generally do not just want to "get rid of" it, and the correct way to make a byte string from a Unicode string depends on what that string contains and in which context. – Lennart Regebro Dec 09 '14 at 12:32
  • 3
    @LennartRegebro totally agreed - this was a throwaway answer that was meant to be tongue in cheek, but it accumulated a sort of horrifying number of upvotes. edited to try to steer folks in the right direction. – Andrew Feb 05 '15 at 22:28
  • 2
    That was a fun read! Thanks! Article is 17 years old and it's still accurate. Wow. – Kerwin Sneijders Feb 28 '20 at 11:08
57

My guess is that it indicates "Unicode", is it correct?

Yes.

If so, since when is it available?

Python 2.x.

In Python 3.x the strings use Unicode by default and there's no need for the u prefix. Note: in Python 3.0-3.2, the u is a syntax error. In Python 3.3+ it's legal again to make it easier to write 2/3 compatible apps.

Jacinda
  • 4,932
  • 3
  • 26
  • 37
kennytm
  • 510,854
  • 105
  • 1,084
  • 1,005
  • 4
    It's even a Syntax Error in Python 3 to use the `u` prefix. – Tim Pietzcker Mar 17 '10 at 18:53
  • 14
    @TimPietzcker: Only in 3.0-3.2; in 3.3+ it's legal (and meaningless), to make it easier to write 2.6+/3.3+ single-codebase libraries and apps. – abarnert Sep 11 '14 at 23:59
  • @abarnert: Well, that comment is now four-and-a-half years old :) – Tim Pietzcker Sep 12 '14 at 07:59
  • 3
    @TimPietzcker: Sure, but just as your comment was a useful addendum for anyone finding this useful answer by search in 2010, I think it's useful to mention the change in 3.3 to anyone finding it in 2014. It might arguably be better to edit the answer, but I think it's a minor point that most people won't run into (because unless you're still using 3.0-3.2 in 2014, "no need for the prefix" is all you need to know). – abarnert Sep 12 '14 at 17:36
  • If you're writing code for arbitrary users to download and run, and want to cover the most possible cases without making assumptions, it's helpful to know 3.0-3.2 will break. Because you need to decide if you care to use `six.text_type()` everywhere for the (hopefully miniscule) number of people still using 3.[012] - at least the information is there so you can choose. – dwanderson Aug 22 '18 at 00:31
8

I came here because I had funny-char-syndrome on my requests output. I thought response.text would give me a properly decoded string, but in the output I found funny double-chars where German umlauts should have been.

Turns out response.encoding was empty somehow and so response did not know how to properly decode the content and just treated it as ASCII (I guess).

My solution was to get the raw bytes with 'response.content' and manually apply decode('utf_8') to it. The result was schöne Umlaute.

The correctly decoded

für

vs. the improperly decoded

fĂźr

Chris
  • 5,788
  • 4
  • 29
  • 40
2

All strings meant for humans should use u"".

I found that the following mindset helps a lot when dealing with Python strings: All Python manifest strings should use the u"" syntax. The "" syntax is for byte arrays, only.

Before the bashing begins, let me explain. Most Python programs start out with using "" for strings. But then they need to support documentation off the Internet, so they start using "".decode and all of a sudden they are getting exceptions everywhere about decoding this and that - all because of the use of "" for strings. In this case, Unicode does act like a virus and will wreak havoc.

But, if you follow my rule, you won't have this infection (because you will already be infected).

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Frank Krueger
  • 69,552
  • 46
  • 163
  • 208
  • `bash -c "echo Shouldn\\'t you use b\\\"...\\\" for byte arrays?"` – kennytm Mar 17 '10 at 19:06
  • @KennyTM Sounds good! Simply meant to say all strings meant for humans should use `u""`. – Frank Krueger Mar 17 '10 at 19:38
  • 1
    If you want to religiously use Unicode everywhere—which, for many applications (but not all), is a good thing—you almost certainly want Python 3.x, not 2.x. That may not have been true in 2010 when this was written, but in 2014, most libraries or platforms that prevent you from upgrading to 3.x will also prevent you from using Unicode properly… – abarnert Sep 12 '14 at 00:01