How can I serialize this Python dict?

Question

I'm trying to serialize a dict (assembled from other dicts) and no matter what I do, I can't seem to serialize this sucker. Json.dumps() throws a type error and says it's unserializable.

Is this a unicode issue or a data type issue? Everything should be a string or int

edits:

updated the JSON snippet - no formatting, so hopefully no bracket omissions this time)
the dict is built using context local variables from the Flask-Ask extension for Alexa Skills Kit. So, I mean, if there are unicode / syntax errors in the {u'foo': u'bar'} structure, it should be from there.

    {'event': {'session': {u'application': {u'applicationId': u'amzn1.ask.skill.9633f809-4141-4cf8-b5f4-5b605e597758'}, u'sessionId': u'amzn1.echo-api.session.68d0c695-bb6c-4929-96d0-6eed0ed9c413', u'user': {u'userId': u'amzn1.ask.account.AGN4D7NM4VKGNCCTCMD3OWT2GR6YKIQF7TODTS4LXUW7UMEVKK6TTKC3JUM25EX3AT4W6WQYCHINGWKWV6TTTNH72U4KL4FH3LXSH3666E3YWPJMYHBDMVDMQZ5SNCIRSTL4BSOMF2V6TNH7I6T4E2NY7IMDPDIIZX3GXDYHZX2NKYELYZE5NE63LJMBAYFE3ZOMRA3F66QYMCY', u'accessToken': u'Atza|IwEBIBFtege3DE13oOO5xiztl3aHcq0mE5dno1lo0p6579-WzNTCNaCvjZicNU-GR5BbDmNWGxjQIk-wnuopeywFQ7vJx-utWTIQrJMXh0AUWJExktH2CiwQmy1sdf9mAvaUTYtTpfxB3J5NGVcE7aeYAMenPiDtSq6yvyjLyjF6V8HqQBxRNiStieHG-YJLCW1oOWs84V143L6wcrgQv5BK-siZAJDcBK_6jiXDFE9cgfIjVpLiSyAH1IqfhJ4-9Gc5CvOAmRVsWCGgxaV-aMntA7FlOnG1eP6ilk81HWRgVusdfy7x7JDxMhEzkHF6l11g9jUFwtAkVIa-9wMedGG6XPFbyNlEbqa6XJuOiawHs9LYDVuAIYRrPyVrF_MANbKApRSrzAX-GFjVC3MSQU48-ySbCxSfo4D5dRlHS8c_3_cFxr8etnEX17HY5G9oC1uYGjEsdfHuDVBUJ3FP65flxRye9zL7S3UBJBwRkxXLiC8iBifb-5sMLnKn7gw8RRe-y3cNCZnCxtmEGzSpHWscxd6v9xsdfdxzf4BXiUaTgpI2oW6JFhXDKotdf_WY-mSsix0URJrwwicC8bQ'}, 'attributes': {'name': u'Foo Bar', 'zip': u'11295', 'userId': u'amzn1.account.AHARAFOFWsdfT7QWD6FXOQRLZR577OA', 'tzid': u'America/New_York', 'lat': 40.6945036, 'lng': -73.9565551, 'email': u'foo@bar.com'}, u'new': 'True'}, 'version': '1.0', 'request': {u'locale': u'en-US', u'timestamp': u'2017-10-20T18:22:42Z', u'type': u'LaunchRequest', u'requestId': u'amzn1.echo-api.request.770f8588-aa1d-443a-b7f6-39abe079e474'}, 'response': {u'version': u'1.0', u'response': {u'outputSpeech': {u'text': u'Welcome to Date Night! Are you staying in or going out?', u'type': u'PlainText'}, u'shouldEndSession': 'False', u'reprompt': {u'outputSpeech': {u'text': u'Hey, Welcome to Date Night! Are you staying in or going out?', u'type': u'PlainText'}}}, u'sessionAttributes': {u'name': u'Foo Bar', u'zip': u'11295', u'userId': u'amzn1.account.AHARAFOsdfsdfsdfFWT7QWD6FXOQRLZR577OA', u'tzid': u'America/New_York', u'lat': 40.6945036, u'lng': -73.9565551, u'email': u'foo@bar.com'}}}}

code:

Here's my code for this function:

print ask_version
print ask_session
print ask_request 
print resp # resp is None in this case
ask_session.pop('dialogState', None)
#ask_session['new'] = str(ask_session['new'])

payload = {
    "event" : {
        "version": str(ask_version).encode('utf8'),
        "session": ask_session,
        "request": ask_request
    } 
}

if resp is not None:
    payload['event']['response'] = json.loads(str(resp))
    #payload['event']['response']['response']['shouldEndSession'] = str(payload['event']['response']['response']['shouldEndSession'])

print payload # works fine
print json.dumps(payload) # throws an error

console output

1.0

{'dialogState': None, u'application': {u'applicationId': u'amzn1.ask.skill.9633f809-4141-4cf8-b5f4-5b605e597758'}, u'sessionId': u'amzn1.echo-api.session.354294b7-78ce-4c47-8673-f671ecb750a6', u'user': {u'userId': u'amzn1.ask.account.AGN4D7NM4VKGNCCTCMD3OWT2GR6YKIQF7TODTS4LXUW7UMEVKK6TTKC3JUM25EX3AT4W6WQYCHINGWKWV6TTTNH72U4KL4FH3LXSH3666E3YWPJMYHBDMVDMQZ5SNCIRSTL4BSOMF2V6TNH7I6T4E2NY7IMDPDIIZX3GXDYHZX2NKYELYZE5NE63LJMBAYFE3ZOMRA3F66QYMCY', u'accessToken': u'Atza|E5dno1lo0p6579-WzNTCNaCvjZicNU-GR5BbDmNWGxjQIk-wnuopeywFQ7vJx-utWTIQrJMXh0AUWJExktH2CiwQmy19mAvaUTYtTpfxB3J5NGVcE7aeYAMenPiDtSq6yvyjLyjF6V8HqQBxRNiStieHG-YJLCW1oOWs84V143L6wcrgQv5BK-siZAJDcBK_6jiXDFE9cgfIjVpLiSyAH1IqfhJ4-9Gc5CvOAmRVsWCGgxaV-aMntA7FlOnG1eP6ilk81HWRgVuy7x7JDxMhEzkHF6l11g9jUFwtAkVIa-9wMedGG6XPFbyNlEbqa6XJuOiawHs9LYDVuAIYRrPyVrF_MANbKApRSrzAX-GFjVC3MSQU48-ySbCxSfo4D5dRlHS8c_3_cFxr8etnEX17HY5G9oC1uYGjEHuDVBUJ3FP65flxRye9zL7S3UBJBwRkxXLiC8iBifb-5sMLnKn7gw8RRe-y3cNCZnCxtmEGzSpHWscxd6v9xdxzf4BXiUaTgpI2oW6JFhXDKotdf_WY-mSsix0URJrwwicC8bQ'}, 'attributes': {'name': u'Foo Bar', 'zip': u'11205', 'userId': u'amzn1.account.AHARAFOFWT7QWD6FXOQRLZR577OA', 'tzid': u'America/New_York', 'lat': 40.99999, 'lng': -73.99999, 'email': u'foo@bar.com'}, u'new': True}

{u'locale': u'en-US', u'timestamp': u'2017-10-20T18:38:32Z', u'type': u'LaunchRequest', u'requestId': u'amzn1.echo-api.request.99b6181b-7e28-4c1d-a49d-ed1f9d0d8c8d'}

None

{'event': {'session': {u'application': {u'applicationId': u'amzn1.ask.skill.9633f809-4141-4cf8-b5f4-5b605e597758'}, u'sessionId': u'amzn1.echo-api.session.354294b7-78ce-4c47-8673-f671ecb750a6', u'user': {u'userId': u'amzn1.ask.account.AGN4D7NM4VKGNCCTCMD3OWT2GR6YKIQF7TODTS4LXUW7UMEVKK6TTKC3JUM25EX3AT4W6WQYCHINGWKWV6TTTNH72U4KL4FH3LXSH3666E3YWPJMYHBDMVDMQZ5SNCIRSTL4BSOMF2V6TNH7I6T4E2NY7IMDPDIIZX3GXDYHZX2NKYELYZE5NE63LJMBAYFE3ZOMRA3F66QYMCY', u'accessToken': u'Atza|E5dno1lo0p6579-WzNTCNaCvjZicNU-GR5BbDmNWGxjQIk-wnuopeywFQ7vJx-utWTIQrJMXh0AUWJExktH2CiwQmy19mAvaUTYtTpfxB3J5NGVcE7aeYAMenPiDtSq6yvyjLyjF6V8HqQBxRNiStieHG-YJLCW1oOWs84V143L6wcrgQv5BK-siZAJDcBK_6jiXDFE9cgfIjVpLiSyAH1IqfhJ4-9Gc5CvOAmRVsWCGgxaV-aMntA7FlOnG1eP6ilk81HWRgVuy7x7JDxMhEzkHF6l11g9jUFwtAkVIa-9wMedGG6XPFbyNlEbqa6XJuOiawHs9LYDVuAIYRrPyVrF_MANbKApRSrzAX-GFjVC3MSQU48-ySbCxSfo4D5dRlHS8c_3_cFxr8etnEX17HY5G9oC1uYGjEHuDVBUJ3FP65flxRye9zL7S3UBJBwRkxXLiC8iBifb-5sMLnKn7gw8RRe-y3cNCZnCxtmEGzSpHWscxd6v9xdxzf4BXiUaTgpI2oW6JFhXDKotdf_WY-mSsix0URJrwwicC8bQ'}, 'attributes': {'name': u'Foo Bar', 'zip': u'11205', 'userId': u'amzn1.account.AHARAFOFWT7QWD6FXOQRLZR577OA', 'tzid': u'America/New_York', 'lat': 40.99999, 'lng': -73.99999, 'email': u'foo@bar.com'}, u'new': True}, 'version': '1.0', 'request': {u'locale': u'en-US', u'timestamp': u'2017-10-20T18:38:32Z', u'type': u'LaunchRequest', u'requestId': u'amzn1.echo-api.request.99b6181b-7e28-4c1d-a49d-ed1f9d0d8c8d'}}}

2017-10-20 11:38:35,551 INFO 127.0.0.1 - - [20/Oct/2017 11:38:35] "POST /alexa/ HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python2.7/site-packages/flask_ask/core.py", line 663, in _flask_view_func
    result = self._launch_view_func()
  File "/Users/Neal/Documents/repos/nealrs_heroku/application.py", line 738, in launch
    dashbot("incoming")
  File "/Users/Neal/Documents/repos/nealrs_heroku/application.py", line 665, in dashbot
    print json.dumps(payload)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 244, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: {u'application': {u'applicationId': u'amzn1.ask.skill.9633f809-4141-4cf8-b5f4-5b605e597758'}, u'sessionId': u'amzn1.echo-api.session.354294b7-78ce-4c47-8673-f671ecb750a6', u'user': {u'userId': u'amzn1.ask.account.AGN4D7NM4VKGNCCTCMD3OWT2GR6YKIQF7TODTS4LXUW7UMEVKK6TTKC3JUM25EX3AT4W6WQYCHINGWKWV6TTTNH72U4KL4FH3LXSH3666E3YWPJMYHBDMVDMQZ5SNCIRSTL4BSOMF2V6TNH7I6T4E2NY7IMDPDIIZX3GXDYHZX2NKYELYZE5NE63LJMBAYFE3ZOMRA3F66QYMCY', u'accessToken': u'Atza|E5dno1lo0p6579-WzNTCNaCvjZicNU-GR5BbDmNWGxjQIk-wnuopeywFQ7vJx-utWTIQrJMXh0AUWJExktH2CiwQmy19mAvaUTYtTpfxB3J5NGVcE7aeYAMenPiDtSq6yvyjLyjF6V8HqQBxRNiStieHG-YJLCW1oOWs84V143L6wcrgQv5BK-siZAJDcBK_6jiXDFE9cgfIjVpLiSyAH1IqfhJ4-9Gc5CvOAmRVsWCGgxaV-aMntA7FlOnG1eP6ilk81HWRgVuy7x7JDxMhEzkHF6l11g9jUFwtAkVIa-9wMedGG6XPFbyNlEbqa6XJuOiawHs9LYDVuAIYRrPyVrF_MANbKApRSrzAX-GFjVC3MSQU48-ySbCxSfo4D5dRlHS8c_3_cFxr8etnEX17HY5G9oC1uYGjEHuDVBUJ3FP65flxRye9zL7S3UBJBwRkxXLiC8iBifb-5sMLnKn7gw8RRe-y3cNCZnCxtmEGzSpHWscxd6v9xdxzf4BXiUaTgpI2oW6JFhXDKotdf_WY-mSsix0URJrwwicC8bQ'}, 'attributes': {'name': u'Foo Bar', 'zip': u'11205', 'userId': u'amzn1.account.AHARAFOFWT7QWD6FXOQRLZR577OA', 'tzid': u'America/New_York', 'lat': 40.99999, 'lng': -73.99999, 'email': u'foo@bar.com'}, u'new': True} is not JSON serializable

You are mixing `str` and `unicode` types, if this is Python 2.. — juanpa.arrivillaga, Oct 20 '17 at 18:21
i dont think that matters @juanpa.arrivillaga (str and unicode mix) ... it should still be json serializeable — Joran Beasley, Oct 20 '17 at 18:23
`u 'application'` (and tons of others) looks like `SyntaxError` to me. — CristiFati, Oct 20 '17 at 18:23
you have a space after every `u` ... that is a syntax error ... — Joran Beasley, Oct 20 '17 at 18:23
Yes, actually, this simply isn't a valid Python literal. How is this being generated? Are you creating this by hand? — juanpa.arrivillaga, Oct 20 '17 at 18:24
@JoranBeasley you're right, that was my first thought, but it's not surprising that this is handled as one would hope. Clearly, though, this is simply invalid as you've pointed out. — juanpa.arrivillaga, Oct 20 '17 at 18:26
`json.dumps` has no problem with the new data. Tested on Python 2.6.6 and Python 3.6.0 — PM 2Ring, Oct 20 '17 at 18:33
@PM2Ring hunh, still having trouble on my end i added my code + console output. thanks for taking a look! — nealrs, Oct 20 '17 at 18:48
Is it possible that one of these values is an instance of a `dict`-*like* class that is not actually `dict` subclass? E.g. what does `print(type(ask_session))` return? — Iguananaut, Oct 20 '17 at 19:06
@Iguananaut looks like they are `` (i really thought they were dicts. — nealrs, Oct 20 '17 at 19:13

score 2 · Answer 1 · answered Oct 20 '17 at 19:21

Sometimes you'll encounter objects that look like a normal dict when printed, but really aren't. This is especially common in web frameworks. Try converting it to a normal dict that the json module knows how to serialize like:

ask_session = dict(ask_session)

One problem that may occur is if you have a nested data structure. In that case it might be good to add a custom encoder for that type. See for example: https://stackoverflow.com/a/3768975/982257

It may not be necessary though. Flask has a built-in utility function called jsonify that knows how to handle a lot of types that show up in Flask. I haven't tested it in this case, but I almost always use it when I'm writing Flask apps.

looked through the context local doc and it seems like `ask_version._get_current_object()` is the right way to do this. Finally generated valid json! thanks a ton for suggesting that maybe my dicts weren't actually dicts. — nealrs, Oct 20 '17 at 19:29
Try `jsonify` though, if you haven't. It might already handle this. — Iguananaut, Oct 20 '17 at 19:30

score 1 · Accepted Answer · answered Oct 20 '17 at 19:52

Looks like jsonify & dict() aren't smart enough to deal with Flask's context locals (<class 'werkzeug.local.LocalProxy'>). Only thing that worked for me was using ._get_current_object() to access the proxy as a dict. Plug that into json.dumps() and you'll be good.

How can I serialize this Python dict?

code:

console output

2 Answers2