How do I run multiple Python test cases in a loop?

Question

I am new to Python and trying to do something I do often in Ruby. Namely, iterating over a set of indices, using them as argument to function and comparing its results with an array of fixture outputs.

So I wrote it up like I normally do in Ruby, but this resulted in just one test case.

  def test_output(self):
    for i in range(1,11):
      ....
      self.assertEqual(fn(i),output[i])

I'm trying to get the test for every item in the range. How can I do that?

Do you want to generate 10 distinct test cases? The code you've shown works, it just does all ten assertions in one test case. — , Sep 28 '13 at 21:05

score 26 · Answer 1 · answered Jun 28 '16 at 07:55

26

Starting from python 3.4, you can do it like this:

def test_output(self):
    for i in range(1,11):
        with self.subTest(i=i):
            ....
            self.assertEqual(fn(i),output[i])

https://docs.python.org/3.4/library/unittest.html?highlight=subtest#distinguishing-test-iterations-using-subtests

answered Jun 28 '16 at 07:55

Antoine Fontaine

792
7
16

@picardo: Please consider accepting this answer rather than the currently accepted one so that other people don't get confused as to what the standard approach should be. – ccpizza Jul 26 '20 at 10:01
1

This seems to be a very elegant answer, but unfortunately (as far as I know), the `setUp` and `tearDown` methods are not being automatically called before and after each sub test. Would be nice to know if there is a way of calling `setUp` and `tearDown` before and after each sub test without doing so explicitly. – Kevin Südmersen Jan 22 '21 at 18:47
@KevinSüdmersen, that's true, the `setUp` and `tearDown` are not executed for each subtest, so it's not the silver bullet. – Antoine Fontaine Jan 25 '21 at 14:08

Trevor Merrifield · Accepted Answer · 2018-01-29T20:30:58.663

Using unittest you can show the difference between two sequences all in one test case.

seq1 = range(1, 11)
seq2 = (fn(j) for j in seq1)
assertSequenceEqual(seq1, seq2)

If that's not flexible enough, using unittest, it is possible to generate multiple tests, but it's a bit tricky.

def fn(i): ...
output = ...

class TestSequence(unittest.TestCase):
    pass

for i in range(1,11):
    testmethodname = 'test_fn_{0}'.format(i)
    testmethod = lambda self: self.assertEqual(fn(i), output[i])
    setattr(TestSequence, testmethodname, testmethod)

Nose makes the above easier through test generators.

import nose.tools

def test_fn():
    for i in range(1, 11):
        yield nose.tools.assert_equals, output[i], fn(i)

Similar questions:

That makes sense, thanks for the explanation. I'll get to updating it. — Trevor Merrifield, Sep 28 '13 at 21:36
Probably shouldn't put the test method generation in the if __name__ == '__main__' clause because it's modifying the module. The test sequence code doesn't make sense without the generation code. — Jappie Kerk, Jan 24 '18 at 22:27

score 14 · Answer 3 · answered Sep 28 '13 at 21:34

In python world two most popular options to write tests are:

In pytest you parametrize tests very easly:

@pytest.mark.parametrize(('param1', 'param2'),[
                         (1, 'go'),
                         (2, 'do not go')])
def test_me(param1, param2):
    # write test

This will produce nice output also while running tests:

go.py:2: test_me[1-go] PASSED
go.py:2: test_me[2-do not go] PASSED

I am using pytest for two years now and it's very nice tool. You have many features there. Besides parametrization there are fixtures also, very very nice assertions (you do not need to write assertEqual, just assert a==b and pytest can generate very nice and helpful output for it.)

https://pypi.org/project/parameterized/ - looks like the function was extracted and can be used outside of pytest in to execute any parametrization. — spinus, Feb 21 '19 at 20:24

score 1 · Answer 4 · answered Aug 24 '18 at 12:14

If Your question is regarding, when you are solving the problem on competitive places like hackerrank or anywhere else. If they have not provided their environment to run test cases in a loop.

And locally if You are running code for python compiler It would be useful.

you can simply use a while loop or range function of python.

e.g:

t = int(input("Enter Number of testcases"))
type(t)

while(t!=0):

    n = int(input("Enter number of data"))
    type(n)

    // Code logic or function Call

    t = t-1

How do I run multiple Python test cases in a loop?

4 Answers4

Linked