0

I have a field to get from database which contains string with this part één and while getting this i get error:

"UnicodeEncodeError: 'ascii' codec can't encode characters in position 12-15: ordinal not in range(128)"

I have search this error, and other people were having issue due to unicodes which start something like this u'\xa0, etc. But in my case, i think its due to special characters. I can not do changes in database as its not under my access. I can just access it.

The code is here: (actually its call to external url)

req = urllib2.Request(url)
req.add_header("Content-type", "application/json")
res = urllib2.urlopen(req,timeout = 50)         #50 secs timeout
clientid = res.read()
result = json.loads(clientid)

Then I use result variable to get the above mentioned string and I get error on this line:

updateString +="name='"+str(result['product_name'])+"', "
kenorb
  • 155,785
  • 88
  • 678
  • 743
Shaibi Rana
  • 25
  • 1
  • 7
  • Show the code where you "use result variable ... and get error" – Janne Karila Jun 05 '14 at 06:02
  • Where is the call to the database and which line shows the error? – Matthias Jun 05 '14 at 06:02
  • it crashes here when i try to concat it with my update query i.e. updateString +="name='"+str(result['product_name'])+"', " on this line gives me error – Shaibi Rana Jun 05 '14 at 06:17
  • i donot have a database call, i just call the API, it gives me json data and i use it further , but gives me error as mentioned in above comment – Shaibi Rana Jun 05 '14 at 06:18
  • Possible duplicate of [Python - 'ascii' codec can't decode byte](https://stackoverflow.com/questions/9644099/python-ascii-codec-cant-decode-byte) – kenorb May 28 '17 at 16:26

2 Answers2

3

You need to find the encoding for which is used for your data before it's inserted into the database. Let's assume it's UTF-8 since that's the most common.

In that case you will want to UTF-8 decode instead of ascii decode. You didn't provide any code, so I'm assuming you have "data".decode(). Try "data".decode("utf-8"), and if your data was encoded using this encoding, it will work.

kenorb
  • 155,785
  • 88
  • 678
  • 743
Martin Konecny
  • 57,827
  • 19
  • 139
  • 159
  • I tried your above mentioned solution, it gave a different error this time i.e. s = obj['product_name'].decode("utf8") File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeEncodeError: 'ascii' codec can't encode characters in position 12-15: ordinal not in range(128) – Shaibi Rana Jun 05 '14 at 05:57
  • So it crashes on json.loads? Or is there another spot? – Martin Konecny Jun 05 '14 at 06:01
  • it crashes here when i try to concat it with my update query i.e. updateString +="name='"+str(result['product_name'])+"', " on this line gives me error – Shaibi Rana Jun 05 '14 at 06:15
  • Then perhaps your encoding is not `utf-8`. Try to find what encoding is used. – Martin Konecny Jun 05 '14 at 06:33
  • my apologies its "decoding Unicode is not supported" error now. – Shaibi Rana Jun 05 '14 at 06:34
  • That must be another line then. We aren't "decoding Unicode" on the line I provided you. – Martin Konecny Jun 05 '14 at 06:36
  • I used your code like this updateString +="name='"+unicode(result['product_name'], 'utf8')+"', " gives me this error: TypeError: decoding Unicode is not supported – Shaibi Rana Jun 05 '14 at 06:40
  • So it sounds to me like the string already was unicode then. So remove str() and unicode functions on that line. – Martin Konecny Jun 05 '14 at 06:51
1

So it sounds to me like the string already was unicode then. So remove str() and unicode functions on that line.

Martin Konecny
  • 57,827
  • 19
  • 139
  • 159