Python Variable Encoding

Question

I have written a program that reads csv files written in various languages (i.e. German) and uploads them to a MySQL database for searching.

A cell containing "Kreuter Bůch. Strasbourg: W. Rihel (Strasbourg)" gets uploaded properly to the database.

mysql> select * from metadata where id = 484203\G
*************************** 1. row ***************************
        id: 484203
element_id: 98049
  field_id: 1005
     value: Kreuter Bůch. Strasbourg: W. Rihel (Strasbourg)
1 row in set (0.16 sec)

The API returns the correct value:

curl http://server/metadata/api/484203
{
  "id" : 484203,
  "elementId" : 98049,
  "fieldId" : 1005,
  "name" : "publisher",
  "value" : "Kreuter Bůch. Strasbourg: W. Rihel (Strasbourg)"
}

The assigned value in python is incorrect.

{'id': 484203, 'elementId': 98049, 'fieldId': 1005, 'name': 'publisher', 'value': 'Kreuter BÅ¯ch. Strasbourg: W. Rihel (Strasbourg)'}

Sample Code

    url = f"http://server/metadata/api/"
    payload = json.dumps({
        "elementId": elementId,
        "fieldId": fieldId,
        "value": value
    })
    headers = {
        'Content-Type': 'application/json'
    }
    response = requests.request("POST", url, headers=headers, data=payload)
    data = json.loads(response.text)
    print(data)

Thanks in Advance

Why is data = response.json() better than json.loads(response.text)? — jjohnson, Feb 09 '23 at 16:59
more readable, no need to `import json`. It's a convenience method of `Response` object — buran, Feb 09 '23 at 17:00
The same way, instead of using `json.dumps` and then `data=` you can use `json=` and pass the dict direectly — buran, Feb 09 '23 at 17:02

score 1 · Answer 1 · answered Feb 09 '23 at 17:22

1

    response = requests.request("Get", url, headers=headers, data=payload)
    response.encoding = "utf8"
    data = response.json()

Python assumed that the returned response was encoded in "ascii".

answered Feb 09 '23 at 17:22

jjohnson

41
5

Python Variable Encoding

1 Answers1