71

can anyone point out a tutorial that shows me how to do a POST request using urllib2 with the data being in JSON format?

Paul Podgorsek
  • 2,416
  • 3
  • 19
  • 22
pup
  • 719
  • 1
  • 5
  • 3

6 Answers6

138

Messa's answer only works if the server isn't bothering to check the content-type header. You'll need to specify a content-type header if you want it to really work. Here's Messa's answer modified to include a content-type header:

import json
import urllib2
data = json.dumps([1, 2, 3])
req = urllib2.Request(url, data, {'Content-Type': 'application/json'})
f = urllib2.urlopen(req)
response = f.read()
f.close()
Bob Van Zant
  • 2,101
  • 1
  • 14
  • 6
  • 4
    Working with python3 and urllib, additionally I had to encode my json parsed data in order to get this working `data = json.dumps([1, 2, 3]).encode("utf8)`. See http://stackoverflow.com/a/5441022/3757139 – Samuel Feb 23 '17 at 11:46
  • Should `data = urllib.parse.urlencode(data)` also be applied? – Hans Ginzel Jan 22 '19 at 12:05
  • urlencoding the json payload would not be normal for a post request like this where the json object is going into the post body.If you were putting the json data into the querystring of a url you would need to encode it. However, urlencoding the json string isn't what urlencode is designed for. Something seems weird about how or why you're asking this question, you may want to initiate a new post and ask your question more fully – Bob Van Zant Jan 23 '19 at 13:46
38

Whatever urllib is using to figure out Content-Length seems to get confused by json, so you have to calculate that yourself.

import json
import urllib2
data = json.dumps([1, 2, 3])
clen = len(data)
req = urllib2.Request(url, data, {'Content-Type': 'application/json', 'Content-Length': clen})
f = urllib2.urlopen(req)
response = f.read()
f.close()

Took me for ever to figure this out, so I hope it helps someone else.

Daniel Serodio
  • 4,229
  • 5
  • 37
  • 33
CloudMarc
  • 381
  • 3
  • 2
  • 2
    Thank you veeery much for this!! 5 years later and this is still broken!! – GuyMatz Oct 11 '16 at 20:31
  • 1
    I can verify that in older Python at least (specifically 2.6) nothing will work unless you set Content-Length as in this answer! – Wingware Apr 20 '17 at 19:57
17

Example - sending some data encoded as JSON as a POST data:

import json
import urllib2
data = json.dumps([1, 2, 3])
f = urllib2.urlopen(url, data)
response = f.read()
f.close()
Messa
  • 24,321
  • 6
  • 68
  • 92
5

To read json response use json.loads(). Here is the sample.

import json
import urllib
import urllib2

post_params = {
    'foo' : bar
}

params = urllib.urlencode(post_params)
response = urllib2.urlopen(url, params)
json_response = json.loads(response.read())
amarnath
  • 785
  • 3
  • 19
  • 23
eseceve
  • 330
  • 2
  • 4
3

You certainly want to hack the header to have a proper Ajax Request :

headers = {'X_REQUESTED_WITH' :'XMLHttpRequest',
           'ACCEPT': 'application/json, text/javascript, */*; q=0.01',}
request = urllib2.Request(path, data, headers)
response = urllib2.urlopen(request).read()

And to json.loads the POST on the server-side.

Edit : By the way, you have to urllib.urlencode(mydata_dict) before sending them. If you don't, the POST won't be what the server expect

Uli Köhler
  • 13,012
  • 16
  • 70
  • 120
Stan
  • 8,710
  • 2
  • 29
  • 31
2

This is what worked for me:

import json
import requests
url = 'http://xxx.com'
payload = {'param': '1', 'data': '2', 'field': '4'}
headers = {'content-type': 'application/json'}
r = requests.post(url, data = json.dumps(payload), headers = headers)
kgr
  • 9,750
  • 2
  • 38
  • 43
arcana
  • 37
  • 1
  • 7
    I don't think that the requests package is part of the standard library. – Dana the Sane Oct 25 '12 at 15:14
  • Although 'requests' package is not part of the standard lib, it is easily installed using pip. Perfect for those who are working within a virtual environment. – Will Oct 01 '14 at 20:49