Where am I going wrong with patching a function with mock_open?

Question

I have a function that calls a sub-function to open up a file. I am trying to test the parent function, but I want to patch the sub-function and have it return the data I pass in (as if it read from a file).

tests.py

# Read in the sample data
__SAMPLE_LOG = os.path.join(settings.BASE_DIR, "apps/tests/log_viewer/sample_logs/sample_manager_log.log")
sample_data = []
for line in reversed_lines(open(__SAMPLE_LOG)):
    sample_data.append(line)

sample_data = ('').join(sample_data)

class ReadLog(TestCase):
    @patch('apps.log_viewer.utils.reversed_lines', new_callable = mock_open, read_data = sample_data)
    def test_returnsDictionaryContainingListOfDictionaries(self, mock_file):
        activity = read_log()

        # Make sure the sample data was read ==> this fails.
        self.assertEqual(open(settings.ACTIVITY_LOG_FILE).read(), sample_data)

utils.py

def read_log():

   # This is the line I am trying to patch
   for line in reversed_lines(open(settings.ACTIVITY_LOG_FILE)):      
      # process data

# see: https://stackoverflow.com/questions/260273/most-efficient-way-to-search-the-last-x-lines-of-a-file-in-python/260433#260433
def reversed_lines(file):
    "Generate the lines of file in reverse order."
    part = ''
    for block in reversed_blocks(file):
        for c in reversed(block):
            if c == '\n' and part:
                yield part[::-1]
                part = ''
            part += c
    if part: yield part[::-1]

def reversed_blocks(file, blocksize=4096):
    "Generate blocks of file's contents in reverse order."
    file.seek(0, os.SEEK_END)
    here = file.tell()
    while 0 < here:
        delta = min(blocksize, here)
        here -= delta
        file.seek(here, os.SEEK_SET)
        yield file.read(delta)

The error

I am trying to patch reversed_lines() in utils.py within the read_log() method, but read_log() is still reading from the actual log, indicating that I am not patching reversed_lines() correctly.

When I change

@patch('apps.log_viewer.utils.reversed_lines', new_callable = mock_open, read_data = sample_data)

to

@patch('builtins.open', new_callable = mock_open, read_data = sample_data)

I get

======================================================================
ERROR: test_returnsDictionaryContainingListOfDictionaries 
(tests.log_viewer.test_utils.ReadLog)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/mock.py", line 1209, in patched
    return func(*args, **keywargs)
  File "/webapp/apps/tests/log_viewer/test_utils.py", line 32, in test_returnsDictionaryContainingListOfDictionaries
    activity = read_log()
  File "/webapp/apps/log_viewer/utils.py", line 64, in read_log
    for line in reversed_lines(open(settings.ACTIVITY_LOG_FILE)):
  File "/webapp/apps/log_viewer/utils.py", line 173, in reversed_lines
    for block in reversed_blocks(file):
  File "/webapp/apps/log_viewer/utils.py", line 164, in reversed_blocks
    while 0 < here:
TypeError: '<' not supported between instances of 'int' and 'MagicMock'

Where am I going wrong?

Could you provide a self-contained example? I have quite some experience with mocking, but in the current snippets I'm missing imports and directory structure, which makes it too much work to get to the point where I could start helping you — F. Pareto, Aug 20 '19 at 18:16

azundo · Accepted Answer · 2019-08-16T17:27:48.270

3

Following the example from the docs at https://docs.python.org/3.3/library/unittest.mock.html#mock-open I think you want

@patch('builtins.open', mock_open(read_data = sample_data), create=True)

However reading through the source of mock_open: https://github.com/python/cpython/blob/3.7/Lib/unittest/mock.py#L2350

It appears that the tell method for filehandles is not implemented by the mock. The only supported methods are read, readline, readlines, write and iterating over the contents. You'll need to manually set up the mock for the tell method. This is not a general implementation but will work in your specific case:

class ReadLog(TestCase):
    @patch('builtins.open', mock_open(read_data = sample_data), create=True)
    def test_returnsDictionaryContainingListOfDictionaries(self, mock_file):
        mock_file.return_value.tell.return_value = len(sample_data)
        ...

edited Aug 16 '19 at 17:27

answered Aug 15 '19 at 23:45

azundo

5,902
1
14
21

This doesn't work for me. It says that the `mock_file` argument isn't passed in, and then when I remove it I get `TypeError: '<' not supported between instances of 'int' and 'MagicMock'` – Hunter Aug 16 '19 at 12:08
@Hunter see my update - python's `mock_open` implementation doesn't implement the `tell` method so you'll need to manually mock that as is appropriate for your test – azundo Aug 16 '19 at 17:28
Thanks this was really close! I changed your patch line to `@patch('builtins.open', new_callable = mock_open, read_data = sample_data)` and it worked! Using your patch line I was getting the error `Traceback (most recent call last): File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/mock.py", line 1209, in patched return func(*args, **keywargs) TypeError: test_returnsMessagesNewestFirst() missing 1 required positional argument: 'mock_file'` – Hunter Aug 19 '19 at 20:57

Where am I going wrong with patching a function with mock_open?

1 Answers1