MySQL special chars lost when queried in Python. (But works with Django.)

Question

I have a MySQL database that is utf8 encoded. It contains a row with a value "sauté". Note the accent over the e.

When I select this data using a Django application then it correctly detects the é. However, when I execute it as a Python program then this information (the special e) seems to be lost.

I've tried various combinations of encode and decode and "from future import unicode_literals" and putting a "u" in front of the sauté string and putting a u in front of the query string. No luck. How can I get this info correctly out of my database (with Python) and test for it?

# Connection when excute as .py:  (Django is usual in settings.py)
cursor = MySQLdb.connect(host='<xxx>',user="<xxx>", passwd="<xxx>", db="<xxx>",
          unix_socket = 'path/mysql.sock'
          ).cursor() 

# Same code in Django and Python execution:

cursor.execute(select line from my_table where id = 27)
results = cursor.fetchall()
for r in results:
    line = r[0]
    if re.search("sauté", line):
        do_something() # Should get here, but only with Django

If `line` isn't `"sauté"` then what is it? Please include the entirety of `line`. — metatoaster, Sep 30 '15 at 03:47
Check this post. http://stackoverflow.com/questions/8365660/python-mysql-unicode-and-encoding — JD Vangsness, Sep 30 '15 at 03:47
@JDVangsness Thank you!! Connect using charset='utf8' and then use the 'u': u"sauté" Do you want to write it as an answer? — user984003, Sep 30 '15 at 03:53
This is probably a duplicate of that same question if that is the case. — metatoaster, Sep 30 '15 at 03:58
That is why i posted the link to the other question. vote those answers — JD Vangsness, Sep 30 '15 at 04:02
I searched and didn't find the other question. Sometimes it's nice to have the same question worded differently. If you don't know the exact problem then you won't always know the right words to search for. — user984003, Sep 30 '15 at 04:05

score 0 · Accepted Answer · answered Sep 30 '15 at 06:31

Thanks to JD Vangsness (in the comments) for the answer.

I needed to add charset='utf8' to the connection parameters.

cursor = MySQLdb.connect(host='<xxx>',user="<xxx>", charset='utf8', passwd="<xxx>", db="<xxx>", unix_socket = 'path/mysql.sock').cursor()

And then I also needed to make the comparison string unicode:

u"sauté"

MySQL special chars lost when queried in Python. (But works with Django.)

1 Answers1