3

I am trying to use json.loads() in python

I get the error:

the JSON object must be str, not 'bytes'

a = {'h': '123', 'w': '12345', 'data': "b'eyJod2lkIjpwomfcwpvepovnepovqrepniLLKJAMSNDMSNDMAWEFMOEDAad='"}


a.update(json.loads(base64.b64decode(a['data'])))

Here the 'data' portion of a was being loaded in as a json dump with b64encoding.

'data':base64.b64encode(json.dumps(test).encode()); where test = some string eg('epovqrepniLLKJAMSNDMSNDMAWEFMOEDAad=')

I have tried using:

a.update(json.loads(base64.b64decode(a['data']).decode('utf-8')))

Giving me a 'utf-8' codec can't decode bytes in position: invalid continuation byte

I have also tried using decodebytes instead of b64decode to no avail.

I'd really appreciate any help!

LemonSnippet
  • 105
  • 3
  • 12
  • You don't need to json.loads(data=a) as it is already a python dict, you just need to output this dict directly. And outputing json string from a dict is json.dumps()'s job – min2bro Jun 05 '18 at 07:16
  • The `data` value looks pretty corrupted. Do you know what `"b'eyJ...` was before it was encoded with JSON and B64? By removing the outer quotes and applying `b64decode` to the byte string `b'eyJ...'`, I get something that looks like JSON in the beginning (`b'{"hwid":`), but the remainder is clearly not valid JSON. – lenz Jun 05 '18 at 19:24
  • @lenz you are correct, these were dummy values. The actual encoded data was valid. I need to be able to remove the outer quotes in the dict. I am however, unable to use json,loads on the dict even though I used b64decode – LemonSnippet Jun 05 '18 at 19:35
  • Please show some actual (possibly abbreviated) values, so one can test a recovery strategy. – lenz Jun 05 '18 at 20:58
  • @lenz Apologies. There was sensitive data and I couldn't put it here. – LemonSnippet Jun 07 '18 at 06:13

1 Answers1

3

Thank you all for your help.

After lots of searching on Stackoverflow coupled with testing on my local machine I was able to drill it down to this.

The object (a['data']) that was being passed in had some values that were not utf-8 decodable.

It was in the form of b'xxxsknoen'

I ended up deleting the b and the quotes in the front and end and then converting it to an str.

var = base64.b64decode(str(a['data'])[2:-1]).decode('utf-8')
a.update(json.loads(var))
LemonSnippet
  • 105
  • 3
  • 12