Python unit testing code which calls OS/Module level python functions

Question

I have a python module/script which does a few of these

1. At various nested levels inside the script I take command line inputs, validate them, apply sensible defaults
2. I also check if a few directories exist

The above are just two examples. I am trying to find out what is the best "strategy" to test this. What I have done is that I have constructed wrapper functions around raw_input and os.path.exists in my module and then in my test I override these two functions to take input from my array list or do some mocked behaviour. This approach has the following disadvantages

1. Wrapper functions just exist for the sake of testing and this pollutes the code
2. I have to remember to use the wrapper function in the code everytime and not just call os.path.exists or raw_input

Any brilliant suggestions?

Answer 1

Solution1: I would do something like this beacuse it works:

def setUp(self):
    self._os_path_exists = os.path.exists
    os.path.exists = self.myTestExists # mock

def tearDown(self):
    os.path.exists = self._os_path_exists

It is not so nice.

Solution2: Restructuring your code was not an option as you said, right? It would make it worse to understand and unintuitive.

Answer 2

The short answer is to monkey patch these system calls.

There are some good examples in the answer to How to display the redirected stdin in Python?

Here is a simple example for raw_input() using a lambda that throws away the prompt and returns what we want.

System Under Test

$ cat ./name_getter.py
#!/usr/bin/env python

class NameGetter(object):

    def get_name(self):
        self.name = raw_input('What is your name? ')

    def greet(self):
        print 'Hello, ', self.name, '!'

    def run(self):
        self.get_name()
        self.greet()

if __name__ == '__main__':
    ng = NameGetter()
    ng.run()

$ echo Derek | ./name_getter.py 
What is your name? Hello,  Derek !

Test case:

$ cat ./t_name_getter.py
#!/usr/bin/env python

import unittest
import name_getter

class TestNameGetter(unittest.TestCase):

    def test_get_alice(self):
        name_getter.raw_input = lambda _: 'Alice'
        ng = name_getter.NameGetter()
        ng.get_name()
        self.assertEquals(ng.name, 'Alice')

    def test_get_bob(self):
        name_getter.raw_input = lambda _: 'Bob'
        ng = name_getter.NameGetter()
        ng.get_name()
        self.assertEquals(ng.name, 'Bob')

if __name__ == '__main__':
    unittest.main()

$ ./t_name_getter.py -v
test_get_alice (__main__.TestNameGetter) ... ok
test_get_bob (__main__.TestNameGetter) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

Answer 3

Johnnysweb is spot on with what you need to do, but instead of rolling your own, you can import and use mock. Mock is specifically designed for unit testing, and makes it extremely simple to do what you're trying to do. It's built-in to Python 3.3.

For example, if want to run a unit test that replaces os.path.isfile and always returns True:

try:
    from unittest.mock import patch
except ImportError:
    from mock import patch

class SomeTest(TestCase):

    def test_blah():
        with patch("os.path.isfile", lambda x: True):
            self.assertTrue(some_function("input"))

This can save you a LOT of boilerplate code, and it's quite readable.

If you need something a bit more complex, for example, replacing supbroccess.check_output, you can create a simple helper function:

def _my_monkeypatch_function(li):
     x,y = li[0], li[1]
     if x == "Reavers":
        return "Gorram"
     if x == "Inora":
        return "Shiny!"
     if x == y:
        return "The Ballad of Jayne"

def test_monkey():
     with patch("subprocess.check_output", _my_monkeypatch_function):
            assertEquals(subprocess.check_output(["Mudder","Mudder"]),
                                                 "The Ballad of Jayne")