7

I want to upload a file to an url. The file I want to upload is not on my computer, but I have the url of the file. I want to upload it using requests library. So, I want to do something like this:

url = 'http://httpbin.org/post'
files = {'file': open('report.xls', 'rb')}
r = requests.post(url, files=files)

But, only difference is, the file report.xls comes from some url which is not in my computer.

Rishi
  • 1,987
  • 6
  • 32
  • 49
  • 1
    That does not seem like a good solution. Does it? – Rishi Aug 29 '13 at 17:56
  • 3
    How else are you going to do it? A form that takes a file requires you to send the file's data in the `POST` body, which means you have to fetch the URL's data to send it. (Of course someone could write a form that takes a URL and fetches it for you, but that would be a different request than the one that takes a file.) – abarnert Aug 29 '13 at 18:00
  • 1
    As a side note, except in quick&dirty scripts, you should never `open` a file and just leak it like that, because there's no guarantee as to when the file will get closed, and that's a great way to get mysterious and hard-to-debug `EMFILE` errors and the like. – abarnert Aug 29 '13 at 18:09
  • 1
    The CPython (default) interpreter will actually `close` any `file` instance as soon as it has zero references, I suppose out of customer service. But better to use a `with open(...) as f:`) block, which explicitly closes the file upon exiting the block even if an exception occurs inside the block. – wberry Aug 29 '13 at 18:40
  • 1
    related: [Using Python Requests to 'bridge' a file without loading into memory?](http://stackoverflow.com/q/15973204/4279) – jfs Aug 31 '13 at 07:19

3 Answers3

13

The only way to do this is to download the body of the URL so you can upload it.

The problem is that a form that takes a file is expecting the body of the file in the HTTP POST. Someone could write a form that takes a URL instead, and does the fetching on its own… but that would be a different form and request than the one that takes a file (or, maybe, the same form, with an optional file and an optional URL).

You don't have to download it and save it to a file, of course. You can just download it into memory:

urlsrc = 'http://example.com/source'
rsrc = requests.get(urlsrc)
urldst = 'http://example.com/dest'
rdst = requests.post(urldst, files={'file': rsrc.content})

Of course in some cases, you might always want to forward along the filename, or some other headers, like the Content-Type. Or, for huge files, you might want to stream from one server to the other without downloading and then uploading the whole file at once. You'll have to do any such things manually, but almost everything is easy with requests, and explained well in the docs.*

* Well, that last example isn't quite easy… you have to get the raw socket-wrappers off the requests and read and write, and make sure you don't deadlock, and so on…

abarnert
  • 354,177
  • 51
  • 601
  • 671
3

There is an example in the documentation that may suit you. A file-like object can be used as a stream input for a POST request. Combine this with a stream response for your GET (passing stream=True), or one of the other options documented here.

This allows you to do a POST from another GET without buffering the entire payload locally. In the worst case, you may have to write a file-like class as "glue code", allowing you to pass your glue object to the POST that in turn reads from the GET response.

(This is similar to a documented technique using the Node.js request module.)

wberry
  • 18,519
  • 8
  • 53
  • 85
  • [last time I've checked](http://stackoverflow.com/a/16247129/4279), `requests` couldn't do it "without buffering the entire payload locally". Could you provide a complete example that shows otherwise? – jfs Aug 31 '13 at 07:11
  • You can do it if you don't multipart-encode the file. This involves passing the raw file-like object to the data argument, as detailed in the answer. =) – Lukasa Sep 01 '13 at 07:16
0
import requests

img_url = "http://...."

res_src = requests.get(img_url)


payload={}

files=[
  ('files',('image_name.jpg', res_src.content,'image/jpeg'))
]

headers = {"token":"******-*****-****-***-******"}

response = requests.request("POST", url, headers=headers, data=payload, files=files)

print(response.text)

above code is working for me.

Mohd Ahshan Danish
  • 125
  • 1
  • 8