1

I have a python unittest like below , I want to run this whole Test N Number of times

class Test(TestCase)

    def test_0(self):
            .........
            .........
            .........

Test.Run(name=__name__)

Any Suggestions?

ulta
  • 11
  • 1
  • 2
  • 2
    The good question is why to you want that? The unittest frameword is intented to allow to test results of methods, so running same test multiple times does not really make sense. And if you want to control execution times, the module to use is `timeit`. – Serge Ballesta Aug 06 '17 at 07:26
  • Why because... the test_0 method contains a random option.. so each time it runs it selects random number of configuration and tests against those configurations. so I am not testing the same thing multiple times.. – ulta Aug 07 '17 at 07:44
  • 1
    Even running multiple tests against random value is weird. Tests are intended to be run again with always same result specially when you want to control that an evolution has brought no regression. Choose a number of *interesting* configurations and test against them. If you want to learn about parameterized test you could have a look at https://stackoverflow.com/q/32899/3545273 – Serge Ballesta Aug 07 '17 at 08:33

3 Answers3

1

You can use parameterized tests. There are different modules to do that. I use nose to run my unittests (more powerful than the default unittest module) and there's a package called nose-parameterized that allows you to write a factory test and run it a number of times with different values for variables you want. If you don't want to use nose, there are several other options for running parameterized tests.

Alternatively you can execute any number of test conditions in a single test (as soon as one fails the test will report error). In your particular case, maybe this makes more sense than parameterized tests, because in reality it's only one test, only that it needs a large number of runs of the function to get to some level of confidence that it's working properly. So you can do:

import random
class Test(TestCase)

    def test_myfunc(self):
        for _ in range(100):
            input = random.random() 
            self.assertEquals(input, input + 2)


Test.Run(name=__name__)
portusato
  • 46
  • 4
1

Why because... the test_0 method contains a random option.. so each time it runs it selects random number of configuration and tests against those configurations. so I am not testing the same thing multiple times..

Randomness in a test makes it non-reproducible. One day you might get 1 failure out of 100, and when you run it again, it’s already gone.

Use a modern testing tool to parametrize your test with a sequential number, then use random.seed to have a random but reproducible test case for each number in a sequence.

portusato suggests nose, but pytest is a more modern and popular tool:

import random, pytest

@pytest.mark.parametrize('i', range(100))
def test_random(i):
    orig_state = random.getstate()
    try:
        random.seed(i)
        data = generate_random_data()
        assert my_algorithm(data) == works
    finally:
        random.setstate(orig_state)

pytest.mark.parametrize “explodes” your single test_random into 100 individual tests — test_random[0] through test_random[99]:

$ pytest -q test.py
....................................................................................................
100 passed in 0.14 seconds

Each of these tests generates different, random, but reproducible input data to your algorithm. If test_random[56] fails, it will fail every time, so you will then be able to debug it.

Vasiliy Faronov
  • 11,840
  • 2
  • 38
  • 49
0

If you don't want your test to stop after the first failure, you can use subTest.

class Test(TestCase):
    def test_0(self):
        for i in [1, 2, 3]:
            with self.subTest(i=i):
                self.assertEqual(squared(i), i**2)

Docs

calvh
  • 336
  • 2
  • 6