Uploading multiple files in a single request using python requests module

Question

The Python requests module provides good documentation on how to upload a single file in a single request:

 files = {'file': open('report.xls', 'rb')}

I tried extending that example by using this code in an attempt to upload multiple files:

 files = {'file': [open('report.xls', 'rb'), open('report2.xls, 'rb')]}

but it resulted in this error:

 File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py",      line 1052, in splittype
 match = _typeprog.match(url)
 TypeError: expected string or buffer

Is it possible to upload a list of files in a single request using this module, and how?

why has there not been an accepted answer? Does the high vote answer below not suffice? — Wade Anderson, Oct 29 '14 at 20:05

score 61 · Answer 1 · edited Dec 27 '15 at 21:38

61

To upload a list of files with the same key value in a single request, you can create a list of tuples with the first item in each tuple as the key value and the file object as the second:

files = [('file', open('report.xls', 'rb')), ('file', open('report2.xls', 'rb'))]

edited Dec 27 '15 at 21:38

Kinsa

686
8
14

answered Dec 25 '13 at 06:51

Ryan Fau

1,096
9
12

Lets say I have a list of file names, is there a way to do this with a list comprehension? – Maciek Semik Mar 22 '19 at 02:37
4

@MatthewSemik file_names = ["abc.txt", "pqr.txt" ] files = [('file', open(f, 'rb')) for f in file_names] – Swapnil Suryawanshi Dec 10 '19 at 06:50

score 32 · Answer 2 · edited Dec 27 '15 at 21:41

32

Multiple files with different key values can be uploaded by adding multiple dictionary entries:

files = {'file1': open('report.xls', 'rb'), 'file2': open('otherthing.txt', 'rb')}
r = requests.post('http://httpbin.org/post', files=files)

edited Dec 27 '15 at 21:41

Kinsa

686
8
14

answered Aug 12 '13 at 10:37

Lukasa

14,599
4
32
34

Interesting. Will try your approach. The reason I tried list was because Flask (python web framework) says files is a multidict and the way to access all the files upload would be to do: request.files.getall('file') – user462455 Aug 12 '13 at 18:36
2

Do I need to close the file descriptors by myself? or will they be automatically closed like with `open('file', 'r') as f`..? – kaki gadol Oct 30 '19 at 17:07
@Lukasa, is there a solution similar approach in R ? – user5249203 Mar 25 '21 at 20:50

score 25 · Answer 3 · edited Feb 07 '20 at 15:27

25

The documentation contains a clear answer.

Quoted:

You can send multiple files in one request. For example, suppose you want to upload image files to an HTML form with a multiple file field ‘images’:

To do that, just set files to a list of tuples of (form_field_name, file_info):

url = 'http://httpbin.org/post'
multiple_files = [('images', ('foo.png', open('foo.png', 'rb'), 'image/png')),
                      ('images', ('bar.png', open('bar.png', 'rb'), 'image/png'))]
r = requests.post(url, files=multiple_files)
r.text

# {
#  ...
#  'files': {'images': 'data:image/png;base64,iVBORw ....'}
#  'Content-Type': 'multipart/form-data; boundary=3131623adb2043caaeb5538cc7aa0b3a',
#  ...
# }

edited Feb 07 '20 at 15:27

C.Nivs

12,353
2
19
44

answered Oct 29 '14 at 20:15

Wade Anderson

2,461
1
20
22

What form can file_info take? Can I omit the content-type? What else can be part of file_info? the documentation doesn't go into detail. – A.R. Jan 29 '15 at 13:50
@AmauryRodriguez I recommend you look at the source for all those details. – Wade Anderson Oct 28 '15 at 19:11

score 4 · Answer 4 · answered May 06 '21 at 14:34

In case you have the files from a form and want to forward it to other URL or to API. Here is an example with multiple files and other form data to forward to other URL.

images = request.files.getlist('images')
files = []
for image in images:
    files.append(("images", (image.filename, image.read(), image.content_type)))
r = requests.post(url="http://example.com/post", data={"formdata1": "strvalue", "formdata2": "strvalue2"}, files=files)

Exactly what I needed thank you so much!! – omercotkd Apr 03 '22 at 11:07 — omercotkd, Apr 03 '22 at 11:07

score 3 · Answer 5 · edited Apr 23 '19 at 21:28

You need to create a file list to upload multiple images:

file_list = [  
       ('Key_here', ('file_name1.jpg', open('file_path1.jpg', 'rb'), 'image/png')),
       ('key_here', ('file_name2.jpg', open('file_path2.jpg', 'rb'), 'image/png'))
   ]

r = requests.post(url, files=file_list)

If you want to send files on the same key you need to keep the key same for each element, and for a different key just change the keys.

Source : https://stackabuse.com/the-python-requests-module/

score 2 · Answer 6 · answered Jun 29 '19 at 01:41

I'm a little bit confused, but directly opening file in request (however same is written in official requests guide) is not so "safe".

Just try:

import os
import requests
file_path = "/home/user_folder/somefile.txt"
files = {'somefile': open(file_path, 'rb')}
r = requests.post('http://httpbin.org/post', files=files)

Yes, all will be ok, but:

os.rename(file_path, file_path)

And you will get:

PermissionError:The process cannot access the file because it is being used by another process

Please, correct me if I'm not right, but it seems that file is still opened and I do not know any way to close it.

Instead of this I use:

import os
import requests
#let it be folder with files to upload
folder = "/home/user_folder/"
#dict for files
upload_list = []
for files in os.listdir(folder):
    with open("{folder}{name}".format(folder=folder, name=files), "rb") as data:
        upload_list.append(files, data.read())
r = request.post("https://httpbin.org/post", files=upload_list)
#trying to rename uploaded files now
for files in os.listdir(folder):
    os.rename("{folder}{name}".format(folder=folder, name=files), "{folder}{name}".format(folder=folder, name=files))

Now we do not get errors, so I recommend to use this way to upload multiple files or you could receive some errors. Hope this answer well help somebody and save priceless time.

I don't think what you're trying to do will work, because the file pointers will already be closed, _before_ they're used by `requests.post`. Therefore, we're left with the option to have open pointers. We can, I think, put the pointers in variables, and after they're used, close them OR have parallel opens, post _inside_ the `with` block. The pointers will automatically be closed. — Mooncrater, Jul 06 '20 at 10:34

score 1 · Answer 7 · answered Mar 09 '22 at 12:38

Using These methods File will Automatically be closed.

Method 1

with open("file_1.txt", "rb") as f1, open("file_2.txt", "rb") as f2:
    files = [f1, f2]
    response = requests.post('URL', files=files)

But When You're Opening Multiple Files This Can Get Pretty long

Method 2:

files = [open("forms.py", "rb"), open("data.db", "rb")]
response = requests.post('URL', files=files)

# Closing all Files
for file in files: 
    file.close()

score 0 · Answer 8 · answered Sep 05 '19 at 13:12

If you have multiple files in a python list, you can use eval() in a comprehension to loop over the files in the requests post files parameter.

file_list = ['001.jpg', '002.jpg', '003.jpg']
files=[eval(f'("inline", open("{file}", "rb"))') for file in file_list ]

requests.post(
        url=url,
        files=files
)

score 0 · Answer 9 · answered Dec 15 '20 at 13:59

In my case uploading all the images which are inside the folder just adding key with index

e.g. key = 'images' to e.g. 'images[0]' in the loop

 photosDir = 'allImages'
 def getFilesList(self):
        listOfDir = os.listdir(os.path.join(os.getcwd()+photosDir))
        setOfImg = []
        for key,row in enumerate(listOfDir):
            print(os.getcwd()+photosDir+str(row) , 'Image Path')
            setOfImg.append((
                'images['+str(key)+']',(row,open(os.path.join(os.getcwd()+photosDir+'/'+str(row)),'rb'),'image/jpg')
            ))
        print(setOfImg)
        return  setOfImg

Uploading multiple files in a single request using python requests module

9 Answers9

Linked

Related