1

Background

I have a class which performs fairly slow, complex operations on data. I want to write some tests for that class, but I'm having a hard time finding a suitable testing framework.

The class has an interface like this:

class thing():

    def __init__(self, datafile):
        data = read datafile

    def a(self):
        self.data_a = expensive(self.data)

    def b(self):
        self.data_b = expensive(self.data_a)

    def c(self):
        self.data_c = expensive(self.data_b)

    etc...

The way we use this class is we usually sequentially perform the operations and analyze the results, usually at each step, often in a jupyter notebook.

Question

How should one test the methods in a class which have this structure? Is there a framework / convention for building sequential test suites?

I tried...

  • I tried using pytest, like so: 

    @pytest.mark.incremental
class Test_thing(object):

    def setUp(self):
        generated_data = generate(data)
        generated_data.write(somewhere)
        self.t = thing(somewhere)

    def test_a(self):
        self.t.a()
        assert self.t.data_a hasProperties

    def test_b(self):
        self.t.b()
        assert self.t.data_b hasProperties

    But test_b would fail with data_a is not an attribute of thing which is a feature, not a bug, of pytest: the tests operate in isolation from one another. But I want the opposite.

  • I also tried unittest some time ago, but couldn't find a way to phrase my tests in that framework.

  • At present, I've written the tests without a testing framework, basically the pytest code from above, without pytest references, and at the end: 

    t = Test_thing()
t.setUp()
t.test_a()
t.test_b()
...

    Is that the best I can do?

Notes

I'm also looking for recommendations on a better title for this question

Alex Lenail
  • 12,992
  • 10
  • 47
  • 79
  • Does expensive(x) change x? if not, why not run all of t.a(), t.b(), etc in setUp()? – rcriii Jun 13 '18 at 20:38
  • expensive(x) does not change x. I want to test each of the individual functions -- why would t.b() in setup() if I wanted to _test_ t.b() ? – Alex Lenail Jun 13 '18 at 21:29
  • Good point, plus I missed the part where the computations are slow. Maybe this answer will help: https://stackoverflow.com/questions/3145720/how-to-accumulate-state-across-tests-in-py-test – rcriii Jun 13 '18 at 21:57
  • Why not just test them all in one function because they depend on each other anyway. You can have multiple asserts in one test. Or can you mock the results of the expensive function and feed that into the others. – Zev Jun 14 '18 at 12:50

1 Answers1

1

Perhaps you could consider using the subtest feature in the unitest package:

import unittest

class TestExpensive(unittest.TestCase):

    def setUp(self):
        generated_data = generate(data)
        generated_data.write(somewhere)
        self.t = thing(somewhere)

    def test_abc(self):
        with self.subTest(status_code='a'):
             self.t.a()
             # Verify self.t.data_a has required properties
             self.assertTrue(True) 

        with self.subTest(status_code='b'):
             self.t.b()
             # Verify self.t.data_b has required properties
             self.assertTrue(False)

You can put all your expensive function calls in one test so that you can reuse any intermediate results over the course of the test. Moreover, if an error occurs, you will still be able to trace it back to the subtest that is causing the problem through the status_code variable:

======================================================================
FAIL: test_abc (__main__.TestExpensive) (status_code='b')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 17, in test_abc
    self.assertTrue(False)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1)
alta
  • 353
  • 1
  • 8