Nested Dictionaries to JSON for the POST request Python

Question

I am having trouble converting the payload data in the form of nested dictionaries to pass it as a data for the POST request using Python requests module. The form data is as below:

payload = {'request':  {
                'appkey': "936725A4-7D9A-11E5-81AC-86EC8D89CD5A"},
            'formdata':{
                    'currency':'US',
                    'dataview':'store_default',
                    'distinct':'_distance, clientkey',
                    'geolocs':{
                            'geoloc':[{
                                    '0':{
                                            'address1':'',
                                            'addressline':'19128, PA',
                                            'city':'Philadelphia',
                                            'country':'US',
                                            'latitude':'40.0532987',
                                            'longitude':'-75.23040379999998',
                                            'postalcode':'19128',
                                            'province':'',
                                            'state':'PA'}}]
                            },
                    'google_autocomplete':'true',
                    'limit':'250',
                    'nobf':'1',
                    'searchradius':'15|25|50|100|250|350|450|550|650|750|850|950',
                    'true':'1',
                    'where':{'partner_reseller': {'eq':'1'}}}                    
          }

r = requests.post(url,data=simplejson.dumps(payload),headers=header)
result = simplejson.loads(str(r.content))

Can somebody please assist me with structure and can point out the mistake in what I have written. I keep getting the following error:

{'code': 1008,
 'response': {'message': 'The submitted XML is not properly formed'}}

I'll appreciate your help a lot. Thank you.

What API is this for? Is there documentation? The url you are hitting would help, if it is a public API — heinst, Apr 25 '17 at 01:34
You got 'submitted XML' in message, so the api request probably XML data ? Or you should set in your header, the `Content-Type` to `application/json` for that. As said heinst, please give us the documentation or the name of the API if you can. — rsz, Apr 25 '17 at 01:42
@rsz: I have set the 'content-type' to 'application/json' already in the header — Krishnang Dalal, Apr 25 '17 at 01:56

score 14 · Answer 1 · answered Sep 18 '19 at 19:45

I had similar problem, frustrating but I solved it. Python requests do not work with nested json, they expect one layer json. It processes like form(application/x-www-form-urlencoded)

# Wrong
data = {'param1': {'a':[100, 200]},
        'param2': 'value2',
        'param3': False}

# You have to convert values into string:
data = {'param1': json.dumps({'a':[100, 200]}),
        'param2': 'value2',
        'param3': json.dumps(False)}

In your case:

import json
params = {
        'appkey': "936725A4-7D9A-11E5-81AC-86EC8D89CD5A"},
        'formdata':{
            'currency':'US',
            'dataview':'store_default',
            'distinct':'_distance, clientkey',
            'geolocs':{
                    'geoloc':[{
                            '0':{
                                    'address1':'',
                                    'addressline':'19128, PA',
                                    'city':'Philadelphia',
                                    'country':'US',
                                    'latitude':'40.0532987',
                                    'longitude':'-75.23040379999998',
                                    'postalcode':'19128',
                                    'province':'',
                                    'state':'PA'}}]
                    },
            'google_autocomplete':'true',
            'limit':'250',
            'nobf':'1',
            'searchradius':'15|25|50|100|250|350|450|550|650|750|850|950',
            'true':'1',
            'where':{'partner_reseller': {'eq':'1'}}}                    
          }
payload = {'request':  json.dumps(params) }

r = requests.post(url,data=payload) # important to keep payload as json!!!
result = r.text # or depends what the return is..

Thank you, saved me from more than a week of frustration. I was not aware of the nested json limitation in requests. — Eduardo Matsuoka, Aug 17 '21 at 18:05

score 12 · Answer 2 · answered Apr 25 '17 at 01:47

12

My suggestion is to use the JSON parameter and let requests both encode the object to JSON and let requests set the Content-Type header to application/json.

It's very possible that the web service assumes you're passing it XML, unless you specify that you're passing JSON, via setting the Content-Type to application/json. (It's also possible this web API really wants XML too, the docs for the service would tell you)

requests.post(url,json=payload,headers=header)

answered Apr 25 '17 at 01:47

RyanWilcox

13,890
1
36
60

@RyanWicox: Thank you for your suggestion. So should I use the payload in the same form as I have mentioned here or do I need to change the structure? – Krishnang Dalal Apr 25 '17 at 01:58
I'm getting an following error: TypeError: request() got an unexpected keyword argument 'json'. – Krishnang Dalal Apr 25 '17 at 02:01
The json parameter was added in requests 2.4.2, are you using a version before that maybe???/ – RyanWilcox Apr 25 '17 at 02:03
same error. I have the updated the package to version 2.13.0 – Krishnang Dalal Apr 25 '17 at 02:17

score 0 · Answer 3 · answered Apr 23 '23 at 15:39

Taking inspiration from both the previous answers, another approach that I personally find cleaner is to serialize the body into string and set Content-Type header to application/json

import json, requests

#Other stuff

headers = { "Content-Type": "application/json" }
data = json.dumps(payload)

r = requests.post(url,data=data,headers=headers)

Nested Dictionaries to JSON for the POST request Python

3 Answers3

Linked