0

How can I prevent Flask from returning Latin characters as Unicode representation? For example, for this sort of characters: http://graphemica.com/%C3%91

My FlaskApp performs a SELECT inside a MySQL database, retrieves row by row, adding them to a list, storing the list into a dictionary and finally, returning a JSON.

FlaskAPP

@app.route("/json")
def show_json():
    avisos_dict = {}
    records_list = []
    query = "SELECT * FROM tjs_stage.avisos"
    cur.execute(query)
    con.commit()
    records = cur.fetchall()
    for row in records:
        records_list.append(row)
    avisos_dict['avisos'] = records_list
    return jsonify(avisos_dict)

MySQL data

mysql> SELECT * FROM tjs_stage.avisos;
+----+---------------------+---------------------------------+---------------------+---------------------+
| Id | Titulo              | Aviso                           | Creacion            | Modificacion        |
+----+---------------------+---------------------------------+---------------------+---------------------+
|  1 | PRIMERO AVISO       | MAÑANA EMPIEZAM VACACIONES     | 2016-04-06 18:12:52 | 2016-04-06 18:12:52 |
| 64 | Aviso de Vacaciones | Mañana empezam las vacaciones! | 2016-05-30 17:19:17 | 2016-05-30 17:19:17 |
+----+---------------------+---------------------------------+---------------------+---------------------+
2 rows in set (0.00 sec)

JSON

enter image description here

Does anyone knows how to solve this?

Thanks in advance ;)

ivanleoncz
  • 9,070
  • 7
  • 57
  • 49
  • Possible duplicate of [How to get string objects instead of Unicode ones from JSON in Python?](http://stackoverflow.com/questions/956867/how-to-get-string-objects-instead-of-unicode-ones-from-json-in-python) –  May 30 '16 at 17:37
  • 1
    The JSON being generated is not only legal, but less vulnerable to encoding problems than an unescaped version would be. JSON is UTF-8, but not all software handles it correctly. – Kevin May 30 '16 at 18:03
  • @Kevin: JSON is a *text* format. It is defined in terms of text (a sequence of Unicode codepoints). It has nothing to do with utf-8 *character encoding* that could be used to *encode* text into a byte sequence. utf-8 is common and it is the default for application/json content-type that is used to transfer a json text over the internet but nothing stops you from saving json text using any character encoding you like on your local disk. – jfs May 31 '16 at 10:25

1 Answers1

4

"Ñ" and "\u00d1" json strings represent the same U+00D1 Unicode character:

>>> import json
>>> json.loads(u'"Ñ"') == json.loads(u'"\\u00d1"')
True

Note: the backslash is escaped because it is special inside Python string literals too.

To avoid ascii-encoded json, set JSON_AS_ASCII to False:

By default Flask serialize object to ascii-encoded JSON. If this is set to False Flask will not encode to ASCII and output strings as-is and return unicode strings. jsonify will automatically encode it in utf-8 then for transport for instance.

Example:

#!/usr/bin/env python
from flask import Flask, jsonify

app = Flask(__name__)
app.config['JSON_AS_ASCII'] = False


@app.route('/')
def unicode_json():
    return jsonify({'text': u'\xd1'})


if __name__ == '__main__':
    app.debug = True
    app.run()

To run the server (assuming you saved it to app.py):

$ python -mpip install flask -U
$ python app.py

To test it:

$ python -mpip install httpie
$ http --pretty=none :5000

Output:

HTTP/1.0 200 OK
Content-Length: 18
Content-Type: application/json
Date: Tue, 31 May 2016 14:54:20 GMT
Server: Werkzeug/0.10.4 Python/2.7.9

{
    "text": "Ñ"
}

@Hareendra Chamara Philips suggests in the comment:

Alternatively, if some one is using config.py and doing application.config.from_pyfile('config.py') in __init__.py, you can use JSON_AS_ASCII = False in the config.py.

jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • It makes sense, but it didn't worked... I still don't have the character as it is from the database. – ivanleoncz May 31 '16 at 14:29
  • @ivanlmj: *"I still don't have the character as it is from the database"* — what does it mean? 1- If you understand the answer then you should understand that the json text (as it is shown in your question) is *already correct* as it is. 2- the second part (about `JSON_AS_ASCII`) is for your convenience only (so that you could see utf-8 encoded text while debugging instead of Unicode escapes). The representation does not affect the final result (the Unicode object you would get if you parse the json text). 3- try the example from the answer – jfs May 31 '16 at 14:55
  • Sorry, Sebastian.. Maybe I didn't made myself clear... I have a MySQL database, with records that contain characters like "ñ","í", for example. I want to display characters like these in a JSON document (character as it is from the database). Now that you putted an example, I figured out that I didn't had configured JSON_AS_ASCII properly (thanks for the example!). With the right configuration of the parameter, I'm receiving something like this: ["Mañana empezam las vacaciones!"].. Can I retrieve the character as it is from the database (ñ, for example), instead of represent it? – ivanleoncz May 31 '16 at 15:13
  • @ivanlmj: for the third time: you are already receiving the correct characters. Your issue is how they are displayed. If `JSON_AS_ASCII` is `False` then your environment must understand utf-8 character encoding. Look at the output at the end of the answer—I've copy-pasted it as is from my terminal (that understands utf-8). If your environment does not understand utf-8 then do not set `JSON_AS_ASCII` to `False`, use unicode escapes instead. Again, if you feed the json text to a json parser; the result is *the same* either way. Try it `print(requests.get('http://localhost:5000').json()['text'])` – jfs May 31 '16 at 15:25
  • I discovered that the browser was displaying "ñ" instead of "ñ"... Thank you, Sebastian! – ivanleoncz May 31 '16 at 21:01
  • Alternatively, if some one is using config. py and doing application.config.from_pyfile('config.py') in __init__.py, you can use JSON_AS_ASCII = False in the config.py. Works like a charm. Thanks for the answer @jfs – Hareendra Chamara Philips Sep 11 '17 at 14:51