How to print symbols like ● to files in Python

Question

I'm trying to write the symbol ● to a text file in python. I think it has something to do with the encoding (utf-8). Here is the code:

# -*- coding: utf-8 -*-
outFile = open('./myFile.txt', 'wb')
outFile.write("●")
outFile.close()

Instead of the black "●" I get "â—". How can I fix this?

Answer 1

Open the file using the io package for this to work with both python2 and python3 with encoding set to utf8 for this to work. When printing, When writing, write as a unicode string.

import io
outFile = io.open('./myFile.txt', 'w', encoding='utf8')
outFile.write(u'●')
outFile.close()

Tested on Python 2.7.8 and Python 3.4.2

Answer 2

If you are using Python 2, use codecs.open instead of open and unicode instead of str:

# -*- coding: utf-8 -*-
import codecs
outFile = codecs.open('./myFile.txt', 'wb', 'utf-8')
outFile.write(u"●")
outFile.close()

In Python 3, pass the encoding keyword argument to open:

# -*- coding: utf-8 -*-
outFile = open('./myFile.txt', 'w', encoding='utf-8')
outFile.write("●")
outFile.close()

Answer 3

>>> ec = u'\u25cf' # unicode("●", "UTF-8")
>>> open("/tmp/file.txt", "w").write(ec.encode('UTF-8'))

Answer 4

What your program does is to produce an output file in the same encoding as your program editor (the coding at the top does not matter, unless your program editor uses it for saving the file). Thus, if you open myFile.txt with a program that uses the same encoding as your program editor, everything looks fine.

This does not mean that your program works for everybody.

For this, you must do two things. You must first indicate the encoding used for text files on your machine. This is a little hard to detect, but the following should often work:

# coding=utf-8  # Put your editor's encoding here

import codecs
import locale
import sys

# Selection of the first non-None, reasonable encoding:
out_encoding = (locale.getlocale()[1]
                or locale.getpreferredencoding()
                or sys.stdin.encoding or sys.stdout.encoding
                # Default:
                or "UTF8")

outFile = codecs.open('./myFile.txt', 'w', out_encoding)

Note that it is very important to specify the right coding on top of the file: this must be your program editor's encoding.

If you know the encoding you want for your output file, you can directly put it in open(). Otherwise, the more general and portable out_encoding expression above should work for most users on most computers (i.e., whatever their encoding of choice is, they should be able to read "●" in the resulting file—assuming their computer's encoding can represent it).

Then you must print a string, not bytes:

outFile.write(u"●")

(note the leading u, meaning "unicode string").

For a deeper understanding of the issues at hand, one of my previous answers should be very helpful: UnicodeDecodeError when redirecting to file.

Answer 5

This should do the trick

# -*- coding: utf-8 -*-
outFile = open('./myFile.txt', 'wb')
outFile.write(u"\u25CF".encode('utf-8'))
outFile.close()

have a look at this

Answer 6

I'm very sorry, but writing a symbol to a text file without saying what the encoding of the file should be is simply non-sense.

It may not be evident at first sight, but text files are indeed encoded and may be encoded in different ways. If you have only letters (upper and lower case, but not accented oned), digits and simple symbols (everything that has an ASCII code below 128), all should be fine, because ASCII 7 bits is now a standard and in fact those characters have same representation in major encodings.

But as soon as you get true symbols, or accented chars, their representation vary from one encoding to the other. For example, the symbol ● has a UTF-8 representation of (Python coding) : \xe2\x97\x8f. What is worse, it cannot be represented in latin1 (ISO-8859-1) encoding.

Another example is the french e accent aigu : é it is represented in UTF8 as \xc3\xa9 (note 2 bytes), but is represented in Latin1 as \x89 (one single byte)

So I tested your code in my Ubuntu box using a UTF8 encoding and the command cat myFile.txt ... correctly showed the bullet !

sba@sba-ubuntu:~/stackoverflow$ cat myFile.txt 
●sba@sba-ubuntu:~/stackoverflow$ 

(as you didn't add any newline after the bullet, the prompt immediately follows it)

In conclusion :

Your code correctly writes the bullet to the file in UTF8 encoding. If your system uses natively another encoding (ISO-8859-1 or its variant Windows-1252) you cannot natively convert it because this character simply does not exist in this encodings.

But you can always see it in a text editor that supports different encoding like the excellent vim that exists on all major systems.

---

Proof of above :

On a Windows 7 computer, I opened a vim window and instructed it to accept utf8 with :set encoding='utf8'. I then pasted original code from OP and saved it to a file foo.py.

I opened a cmd.exe window and executed python foo.py (using a Python 2.7) : it created a file myFile.txt containing the 3 bytes (hexa) : e2 97 8f that is the utf8 representation of the bullet ● (I could confirm it with vim Tools/Hexa convert).

I could even open myFile.txt in idle and actually saw the bullet. Even notepad.exe could show the bullet !

So even on a Windows 7 computer that does not natively accept utf-8, the code from OP correctly generates a text file that when opened with a text editor accepting UTF-8 contains the bullet ●.

Of course, if I try to open myFile.txt with vim in latin1 mode, I get : â—, on a cmd windows with codepage 850, type myFile.txt shows ÔùÅ, and with codepage 1252 (variant of latin1) : â—.

In conclusion original OP code creates a correct utf8 encoded file - it is up to the reading part to interpret correctly utf8.