How can I use requests_mock to unit test a file upload?

Question

I'm writing a Python (3) wrapper for an API, and I'm trying to unit test a part of it that requires a file to be uploaded. I would like to verify the filename and the content are sent properly by my client.

I'm using Python's unittest library, along with requests and requests_mock for testing this.

The way that I was planning to approach this problem was to have a callback function for validating the file is sent and all the headers are properly set. Here's what I have so far:

import unittest
import requests
import requests_mock

from my_class import my_class

from my_class.API import API

class  TestAPI(unittest.TestCase):

    def setUp(self):
        self.hostname = 'https://www.example.com'

    def validate_file_upload(self, request, context, filename, content):
        # self.assertEqual(something, something_else)
        # better solution goes here

    def test_submit_file(self):
        API_ENDPOINT = self.hostname + '/api/tasks/create/file/'
        DUMMY_FILE = 'file'
        DUMMY_CONTENT = 'here is the\ncontent of our\nfile'

        s = API(self.hostname)

        with open(DUMMY_FILE, 'w+') as f:
            f.write(DUMMY_CONTENT)

        with requests_mock.Mocker() as m:
            def json_callback(request, context):
                self.validate_file_upload(request, context, DUMMY_FILE,
                    DUMMY_CONTENT)
                return {}

            m.post(API_ENDPOINT, json=json_callback)
            s.upload_file(DUMMY_FILE)

I have determined that, upon successful file upload, the request parameter to the validate_file_upload has a couple relevant bits of data, namely request.headers and request.text. Here is the content of both of them after the validate_file_upload function is called:

request.headers

{'User-Agent': 'python-requests/2.19.1', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '171', 'Content-Type': 'multipart/form-data; boundary=e1a0aa05f83735e85ddca089c450a21b'}

request.text

'--e1a0aa05f83735e85ddca089c450a21b\r\nContent-Disposition: form-data; name="file"; filename="file"\r\n\r\nhere is the\ncontent of our\nfile\r\n--e1a0aa05f83735e85ddca089c450a21b--\r\n'

Now, here's the thing. I know that I can just parse the request.text string and get the data I want; it's easy enough to validate.

However, that sort of logic seems like it really doesn't belong in my unit testing. I can't imagine there isn't a better solution to this; either someone has implemented this functionality already in a different module or I'm overlooking something obvious.

I shouldn't have to implement the HTTP spec for file uploads to unit test something as simple as a file upload, right? Is there a better way of doing this?

Here is the output of dir(request):

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_allow_redirects', '_case_sensitive', '_cert', '_create', '_matcher', '_proxies', '_qs', '_request', '_stream', '_timeout', '_url_parts', '_url_parts_', '_verify', 'allow_redirects', 'cert', 'hostname', 'json', 'matcher', 'netloc', 'path', 'port', 'proxies', 'qs', 'query', 'scheme', 'stream', 'text', 'timeout', 'verify']

I have checked all of the non-underscore attributes for any other representation of the file upload data, to no avail. I've also tried searching StackOverflow and Google, and am no closer to finding a better way of doing this. This is the only post that shows up for either of the searches.

score 2 · Accepted Answer · answered Oct 13 '19 at 18:36

I do not have my_class.API to test with your exact code, but I believe that you can use cgi.

import cgi

body_file = io.BytesIO(request.body)
_, pdict = cgi.parse_header(request.headers['Content-Type'])
parsed = cgi.parse_multipart(fp=body_file, pdict=pdict)
# Now inspect `parsed`

score 0 · Answer 2 · answered Aug 08 '18 at 00:16

For the time being, I have decided to take this rather simple approach:

def validate_file_upload(self, request, context, filename, content):
    self.assertTrue(filename in request.text)
    self.assertTrue(content in request.text)

While it is not perfect, it's significantly less logic than parsing the HTTP request and seems to do at least a basic validation that the file is being uploaded properly. As I mentioned before, I'm using the requests library, so I'm not too worried about messing up a file upload, and this should catch it in most cases, anyways.

As a precaution against incorrectly matching the name file against something else in the request.text, I have changed it to rather_unique_filename.

How can I use requests_mock to unit test a file upload?

2 Answers2