228

I need to upload some data to a server using HTTP PUT method in Python. From my brief reading of the urllib2 docs, it only does HTTP POST.

Is there any way to do an HTTP PUT in Python?

informatik01
  • 16,038
  • 10
  • 74
  • 104
Amandasaurus
  • 58,203
  • 71
  • 188
  • 248

14 Answers14

327

I've used a variety of python HTTP libs in the past, and I've settled on requests as my favourite. Existing libs had pretty useable interfaces, but code can end up being a few lines too long for simple operations. A basic PUT in requests looks like:

payload = {'username': 'bob', 'email': 'bob@bob.com'}
>>> r = requests.put("http://somedomain.org/endpoint", data=payload)

You can then check the response status code with:

r.status_code

or the response with:

r.content

Requests has a lot synactic sugar and shortcuts that'll make your life easier.

Jaroslav Bezděk
  • 6,967
  • 6
  • 29
  • 46
John Carter
  • 6,752
  • 2
  • 32
  • 53
  • 11
    Even though the above code looks extremely simple, don't infer that 'requests' is in any way lacking or under-powered. It is extremely capable, just with a highly tidy interface. – Jonathan Hartley Feb 08 '12 at 18:17
  • 5
    I wonder how long it'll take this answer to gradually accumulate votes until it becomes the new highest-voted answer? – Jonathan Hartley Feb 09 '12 at 12:27
  • 2
    You don't understand how much this is awesome!!! I was struggling with a crummy java library! ... I think I kinda love you for pointing to "Requests"! – Shehaaz Apr 26 '13 at 05:13
  • Didn't see your answer till now but yeah I agree this is the simplest approach. I posted an answer with slightly different details. Cheers – radtek Sep 25 '14 at 18:11
  • 4
    Use `json=payload` parameter if you want the data to be in the body. – ManuelSchneid3r Mar 31 '17 at 21:06
  • If I want to put file then ? filname is in variable say tailf ! – Ashish Karpe Aug 18 '17 at 07:19
  • bty what is difference between python requests.post & requests.put can both be used to upload file to server same like curl -X POST -d @$myfilename http://lsd-qa-web-3.56m.vgtf.net/nba/$myfilename – Ashish Karpe Aug 18 '17 at 07:46
  • @AshishKarpe How the server treats it. You're talking to the server, and telling it what you want it to do, but the server chooses how to interpret it. – wizzwizz4 Apr 09 '18 at 10:53
  • So with your example you replace the `username` and `email` value with the one you just defined in the dict? Is that what the put method does? And what happens if the key `username` does not exist? Does it create a new one? @JohnCarter – Blinxen Apr 17 '18 at 12:28
  • 1
    I want to quickly point out that this interface is better than the other answer, because it's functional programming. Why create an object when a function with simple data structures as parameters will do. I wish other python libraries would follow suit. – Marc Nov 01 '18 at 18:31
246
import urllib2
opener = urllib2.build_opener(urllib2.HTTPHandler)
request = urllib2.Request('http://example.org', data='your_put_data')
request.add_header('Content-Type', 'your/contenttype')
request.get_method = lambda: 'PUT'
url = opener.open(request)
Florian Bösch
  • 27,420
  • 11
  • 48
  • 53
  • 4
    Looks like a bit of a dirty hack, but it seems to work elegantly and completly – Amandasaurus Sep 22 '08 at 09:14
  • 6
    It would be less of a hack if you were to subclass urllib2.Request instead of monkey-patching it. – Jason R. Coombs Oct 18 '11 at 11:59
  • 23
    This answer was brilliant when it was written, but nowadays it's a lot easier to use the 'requests' package instead, see John Carter's answer. 'Requests' is in no way a toy - it is extremely capable. – Jonathan Hartley Feb 08 '12 at 18:15
  • 5
    This answer is canonical, but outdated. Please, consider using `requests` library instead. – jsalonen Mar 25 '12 at 17:11
  • 13
    Thank you, I didn't want to use anything outside of the standard python library and this works perfect. I don't quite understand why the urllib2 is designed to only cater for GET and POST as standard, but this work around is champion. Can the urllib2.build_opener(urllib2.HTTPHandler) be reused in multiple calls? – WeNeedAnswers Nov 16 '12 at 01:12
  • 4
    You could use `urllib2.urlopen` instead of `opener.open` if you do not need to use an opener. – Xiao Oct 10 '14 at 06:52
  • Can this work with https Auth key too? i.e. using urlib2 with a put – fkl Feb 18 '15 at 19:39
  • Since I asked this question, I have also discovered the excellent requests library. I have changed the accepted answer to the one that had that. – Amandasaurus Apr 29 '15 at 09:03
  • Warning... Doesn't seem to work well for use with the webHDFS API. This answer looked promising, as I can only use the standard python libraries for my task, however, I couldn't get it to work well when redirects are involved. Specifically when trying to write a new file using webHDFS APIs, a two-put process is required with the first PUT command returning a redirect URL which isn't handled well by this solution. All I would get back was an HTTP 307 redirect error without any header information to grab the new redirect URL from. I'm also an idiot and could be missing something here too. – Phaxmohdem Jul 27 '17 at 16:10
  • 2
    For any folks not wanting to use a separate library... the last line should be as follows if you want to get back the body: response = opener.open(request) return response.read() – Edwin Evans Aug 23 '17 at 20:17
46

Httplib seems like a cleaner choice.

import httplib
connection =  httplib.HTTPConnection('1.2.3.4:1234')
body_content = 'BODY CONTENT GOES HERE'
connection.request('PUT', '/url/path/to/put/to', body_content)
result = connection.getresponse()
# Now result.status and result.reason contains interesting stuff
Spooles
  • 785
  • 6
  • 16
  • Don't use httplib if you (or any of your potential users) need proxy support. See [this article](http://www.velocityreviews.com/forums/t325113-httplib-and-proxy.html) for details. – Jason R. Coombs Oct 18 '11 at 11:55
  • Why would you say that? Your article clearly states that it works. See Rolf Wester's answer, he says that urllib fails but httplib works. – Spooles Oct 20 '11 at 14:41
  • httplib only works when the user explicitly connects to the proxy and modifies the request to include the full URL in the GET parameter. The reason urllib failed was because the http_proxy wasn't set properly. urllib uses httplib under the scenes, but also handles redirects, proxies, etc. – Jason R. Coombs Oct 21 '11 at 14:54
  • While this solution would work, the one using requests is much simpler, elegant, and in my opinion better. – tgrosinger Jun 29 '12 at 20:21
  • 1
    @tgrosinger I agree. This answer was posted before requests existed. As of now, the only case where this solution would be better is if only Python Standard Library would be available. Otherwise, I'd also recommend to use requests. – Spooles Aug 22 '12 at 13:59
13

You can use the requests library, it simplifies things a lot in comparison to taking the urllib2 approach. First install it from pip:

pip install requests

More on installing requests.

Then setup the put request:

import requests
import json
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}

# Create your header as required
headers = {"content-type": "application/json", "Authorization": "<auth-key>" }

r = requests.put(url, data=json.dumps(payload), headers=headers)

See the quickstart for requests library. I think this is a lot simpler than urllib2 but does require this additional package to be installed and imported.

radtek
  • 34,210
  • 11
  • 144
  • 111
10

This was made better in python3 and documented in the stdlib documentation

The urllib.request.Request class gained a method=... parameter in python3.

Some sample usage:

req = urllib.request.Request('https://example.com/', data=b'DATA!', method='PUT')
urllib.request.urlopen(req)
anthony sottile
  • 61,815
  • 15
  • 148
  • 207
8

You should have a look at the httplib module. It should let you make whatever sort of HTTP request you want.

John Montgomery
  • 8,868
  • 4
  • 33
  • 43
8

I needed to solve this problem too a while back so that I could act as a client for a RESTful API. I settled on httplib2 because it allowed me to send PUT and DELETE in addition to GET and POST. Httplib2 is not part of the standard library but you can easily get it from the cheese shop.

Mike
  • 3,663
  • 3
  • 20
  • 12
  • 2
    httplib2 is borderline abandonware. It has a long list of bugs that go unfixed despite community contributions (patches). I suggest thinking twice before using httplib2 in any production environments. – Jason R. Coombs Oct 18 '11 at 11:58
6

I also recommend httplib2 by Joe Gregario. I use this regularly instead of httplib in the standard lib.

jbochi
  • 28,816
  • 16
  • 73
  • 90
Corey Goldberg
  • 59,062
  • 28
  • 129
  • 143
3

Have you taken a look at put.py? I've used it in the past. You can also just hack up your own request with urllib.

Community
  • 1
  • 1
William Keller
  • 5,256
  • 1
  • 25
  • 22
  • I don't really wanna use some random guys http library – Amandasaurus Sep 22 '08 at 09:10
  • trying import put getting # pip install put Downloading/unpacking put Could not find any downloads that satisfy the requirement put Cleaning up... No distributions at all found for put – Ashish Karpe Aug 18 '17 at 07:58
  • # tail -f /root/.pip/pip.log Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main status = self.run(options, args) File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1178, in prepare_files – Ashish Karpe Aug 18 '17 at 08:00
  • url = finder.find_requirement(req_to_install, upgrade=self.upgrade) File "/usr/lib/python2.7/dist-packages/pip/index.py", line 277, in find_requirement raise DistributionNotFound('No distributions at all found for %s' % req) DistributionNotFound: No distributions at all found for put – Ashish Karpe Aug 18 '17 at 08:00
2

You can of course roll your own with the existing standard libraries at any level from sockets up to tweaking urllib.

http://pycurl.sourceforge.net/

"PyCurl is a Python interface to libcurl."

"libcurl is a free and easy-to-use client-side URL transfer library, ... supports ... HTTP PUT"

"The main drawback with PycURL is that it is a relative thin layer over libcurl without any of those nice Pythonic class hierarchies. This means it has a somewhat steep learning curve unless you are already familiar with libcurl's C API. "

wnoise
  • 9,764
  • 37
  • 47
2

If you want to stay within the standard library, you can subclass urllib2.Request:

import urllib2

class RequestWithMethod(urllib2.Request):
    def __init__(self, *args, **kwargs):
        self._method = kwargs.pop('method', None)
        urllib2.Request.__init__(self, *args, **kwargs)

    def get_method(self):
        return self._method if self._method else super(RequestWithMethod, self).get_method()


def put_request(url, data):
    opener = urllib2.build_opener(urllib2.HTTPHandler)
    request = RequestWithMethod(url, method='PUT', data=data)
    return opener.open(request)
Wilfred Hughes
  • 29,846
  • 15
  • 139
  • 192
1

You can use requests.request

import requests

url = "https://www.example/com/some/url/"
payload="{\"param1\": 1, \"param1\": 2}"
headers = {
  'Authorization': '....',
  'Content-Type': 'application/json'
}

response = requests.request("PUT", url, headers=headers, data=payload)

print(response.text)
SuperNova
  • 25,512
  • 7
  • 93
  • 64
0

A more proper way of doing this with requests would be:

import requests

payload = {'username': 'bob', 'email': 'bob@bob.com'}

try:
    response = requests.put(url="http://somedomain.org/endpoint", data=payload)
    response.raise_for_status()
except requests.exceptions.RequestException as e:
    print(e)
    raise

This raises an exception if there is an error in the HTTP PUT request.

Adam Erickson
  • 6,027
  • 2
  • 46
  • 33
0

Using urllib3

To do that, you will need to manually encode query parameters in the URL.

>>> import urllib3
>>> http = urllib3.PoolManager()
>>> from urllib.parse import urlencode
>>> encoded_args = urlencode({"name":"Zion","salary":"1123","age":"23"})
>>> url = 'http://dummy.restapiexample.com/api/v1/update/15410' + encoded_args
>>> r = http.request('PUT', url)
>>> import json
>>> json.loads(r.data.decode('utf-8'))
{'status': 'success', 'data': [], 'message': 'Successfully! Record has been updated.'}

Using requests

>>> import requests
>>> r = requests.put('https://httpbin.org/put', data = {'key':'value'})
>>> r.status_code
200
Ransaka Ravihara
  • 1,786
  • 1
  • 13
  • 30