1

I was given a class decorator to generate automatically the number of a test case in unittest this is the decorator:

def generate_test_numbers(test_class):
    counter = 1
    for method_name in dir(test_class):
        if not method_name.startswith('test_n_'):
            continue
        method = getattr(test_class, method_name)
        if not callable(method):
            continue
        new_method_name = method_name.replace('_n_', '_{:02d}_'.format(counter))
        counter += 1
        setattr(test_class, new_method_name, method)
        delattr(test_class, method_name)
    return test_class

My problem is when I try to run the class decorator in my unittest file:

import unittest
from generator import generate_test_numbers


@generate_test_numbers
class TestStringMethods(unittest.TestCase):

    def test_n_remove(self):
        print("1")

    def test_n_add(self):
        print("2")

    def test_n_c(self):
        print("3")

    def test_n_d(self):
        print("4")

    def test_n_e(self):
        print("5")

    def test_n_f(self):
        print("6")

    def test_n_g(self):
        print("7")


if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestStringMethods)
    unittest.TextTestRunner(verbosity=2).run(suite)

I expect to get test_n_remove as the first test and not last:

test_01_add (__main__.TestStringMethods) ... 2
ok
test_02_c (__main__.TestStringMethods) ... 3
ok
test_03_d (__main__.TestStringMethods) ... 4
ok
test_04_e (__main__.TestStringMethods) ... 5
ok
test_05_f (__main__.TestStringMethods) ... 6
ok
test_06_g (__main__.TestStringMethods) ... 7
ok
test_07_remove (__main__.TestStringMethods) ... 1
ok
jonrsharpe
  • 115,751
  • 26
  • 228
  • 437
andrew_k
  • 73
  • 5
  • 3
    Dont assume any order when running unit test. It is a wrong design. – balderman Oct 06 '19 at 07:22
  • Looks like they're sorted alphabetically... – Ulrich Eckhardt Oct 06 '19 at 07:29
  • If your tests depend on ordering, then they have side effects that make it much harder to run them in isolation or to correctly interpret their results, making them much less useful. (If your tests don't depend on ordering, then enforcing ordering anyway is going to make it easier for unwanted order-dependence to slip in.) – user2357112 Oct 06 '19 at 07:30
  • This *might* work as you expect from Python 3.7 where dictionaries, including the class dict (see https://stackoverflow.com/q/58235381/3001761) retain insertion order, so `dir` *might* give you the methods in definition order. But as pointed out above, if you need to know or rely on this, they aren't unit tests. This is possibly an XY problem, could you provide some context on *why* you're trying to do this? – jonrsharpe Oct 06 '19 at 07:31
  • 1
    The reason the tests are numbered in alphabetical order is that `dir(test_class)` returns the method names in alphabetical order. Actually, best to my knowledge, python simply doesn't have the information of the method order in your source code, because they are stored in `__dict__`, which is unordered (until python 3.7, as mentioned by @jonrsharpe). To figure out the ordering, the only solution would be to read back the source code, which is so hacky that I won't provide an example for it. – gnvk Oct 06 '19 at 07:39
  • In fact `dir` is [explicitly documented](https://docs.python.org/3/library/functions.html#dir) as returning them alphabetically, so you'd have to look in the `__dict__` yourself even in 3.7+. – jonrsharpe Oct 06 '19 at 07:45
  • @r4ch I found some solution to put letters instead of numbers if it numbered in alphabetical. How can I put this https://pastebin.com/zqD9ELcY solution in your class decorator? – andrew_k Oct 06 '19 at 09:05
  • 1
    Possible duplicate of [Unittest tests order](https://stackoverflow.com/questions/4095319/unittest-tests-order) – Udi Oct 06 '19 at 11:19

0 Answers0