Python: Creating a mock or fake directory with files for unittesting

Question

I am trying to create a unit test for the following function:

def my_function(path):
    #Search files at the given path
    for file in os.listdir(path):
        if file.endswith(".json"):
            #Search for file i'm looking for
            if file == "file_im_looking_for.json":
                #Open file
                os.chdir(path)
                json_file=json.load(open(file))
                print json_file["name"]

However I am having trouble successfully creating a fake directory with files in order for the function to work correctly and not through errors.

Below is what I have so far but it is not working for me, and I'm not sure how to incorporate "file_im_looking_for" as the file in the fake directory.

tmpfilepath = os.path.join(tempfile.gettempdir(), "tmp-testfile")
@mock.patch('my_module.os')

def test_my_function(self):

    # make the file 'exist'
    mock_path.endswith.return_value = True

    file_im_looking_for=[{
      "name": "test_json_file",
      "type": "General"
    }]

    my_module.my_function("tmpfilepath")

Any advice where I'm going wrong or other ideas to approach this problem are appreciated!

Mauro Baraldi · Accepted Answer · 2016-05-11T13:10:11.750

First of all, you forgot to pass the mocked object to test function. The right way to use mock in your test should be like this.

@mock.patch('my_module.os')
def test_my_function(self, mock_path):

Anyway, you shouldn't mock the endswith, but the listdir. The snippet below is an example and may help you.

app.py

def check_files(path):
    files = []
    for _file in os.listdir(path):
        if _file.endswith('.json'):
            files.append(_file)
    return files

test_app.py

import unittest
import mock
from app import check_files


class TestCheckFile(unittest.TestCase):

    @mock.patch('app.os.listdir')
    def test_check_file_should_succeed(self, mock_listdir):
        mock_listdir.return_value = ['a.json', 'b.json', 'c.json', 'd.txt']
        files = check_files('.')
        self.assertEqual(3, len(files))

    @mock.patch('app.os.listdir')
    def test_check_file_should_fail(self, mock_listdir):
        mock_listdir.return_value = ['a.json', 'b.json', 'c.json', 'd.txt']
        files = check_files('.')
        self.assertNotEqual(2, len(files))

if __name__ == '__main__':
    unittest.main()

Edit: Answering your question in comment, you need to mock the json.loads and the open from your app.

@mock.patch('converter.open')
@mock.patch('converter.json.loads')
@mock.patch('converter.os.listdir')
def test_check_file_load_json_should_succeed(self, mock_listdir, mock_json_loads, mock_open):
    mock_listdir.return_value = ['a.json', 'file_im_looking_for.json', 'd.txt']
    mock_json_loads.return_value = [{"name": "test_json_file", "type": "General"}]
    files = check_files('.')
    self.assertEqual(1, len(files))

But remember! If your is too broad or hard to maintain, perhaps refactoring your API should be a good idea.

Thank you very much for explaining, but how do I make, for instance 'b.json', in the mock directory equal the structure of "file_im_looking_for" (shown in my code) — Catherine, May 11 '16 at 12:42
Just one last question, For the mock_json_loads and mock_listdir the return values seem to be mixed up? Using a print statement it seems the line "for _file in os.listdir(path):" returns [{"name": "test_json_file", "type": "General"}] when it should return ['a.json', 'file_im_looking_for.json', 'd.txt']. Do you know why this would be? — Catherine, May 11 '16 at 13:33
This is an important behavior from mock, the order of its declaration and patching. — Mauro Baraldi, May 11 '16 at 13:41
Is it possible to mock the contents of the fake files created ? — Kandarp Gandhi, Aug 04 '19 at 13:45
@kandarpgandhi does [this answer](https://stackoverflow.com/questions/38454272/mock-file-open-in-python) helps? — Mauro Baraldi, Aug 05 '19 at 14:08

Adrian · Answer 2 · 2022-03-02T08:20:00.777

I would suggest to use Python's tempfile library, specifically TemporaryDirectory.

The issue with your and Mauro Baraldi's solution is that you have to patch multiple functions. This is a very error prone way, since with mock.patch you have to know exactly what you are doing! Otherwise, this may cause unexpected errors and eventually frustration.

Personally, I prefer pytest, since it has IMO nicer syntax and better fixtures, but since the creator used unittest I will stick with it.

I would rewrite your test code like this:

import json
import pathlib
import tempfile
import unittest

wrong_data = {
      "name": "wrong_json_file",
      "type": "Fake"
    }

correct_data = {
      "name": "test_json_file",
      "type": "General"
    }

class TestMyFunction(unittest.TestCase):
    def setUp(self):
        """ Called before every test. """
        self._temp_dir = tempfile.TemporaryDirectory()
        temp_path = pathlib.Path(self._temp_dir.name)
        self._create_temporary_file_with_json_data(temp_path / 'wrong_json_file.json', wrong_data)
        self._create_temporary_file_with_json_data(temp_path / 'file_im_looking_for.json', correct_data)
        
    def tearDown(self):
        """ Called after every test. """
        self._temp_dir.cleanup()

    def _create_temporary_file_with_json_data(self, file_path, json_data):
        with open(file_path, 'w') as ifile:
            ifile.write(json.dumps(content))

    def test_my_function(self):
        my_module.my_function(str(self._temp_dir))

You see that your actual test is compressed down to a single line! Admittedly, there is no assert, but if your function would return something, the result would behave as expected.

No mocking, because everything exists and will be cleaned up afterwards. And the best thing is that you now can add more tests with a lower barrier of entry.

Python: Creating a mock or fake directory with files for unittesting

2 Answers2

Linked