11

I was looking at similar questions and I couldn't find an answer to my problem.

I wrote Tests in a python class that derives from unittest.TestCase

class TestEffortFormula(unittest.TestCase)

I need to give an order to the tests (please, do not tell me that I shouldn't rely on test's order, I just do).

Before I needed to give order to the tests the command I used to run the tests was:

unittest.main(testRunner=TeamcityTestRunner())

Then I wanted to make the order dissappear, so I tried the following:

loader = unittest.TestLoader()
loader.sortTestMethodsUsing(None)
loader.loadTestsFromTestCase(TestEffortFormula)
suite = loader.suiteClass()

but from here I don't know how to run the tests, specially with testRunner=TeamcityTestRunner() as I did before.

Appreciate your help

Thomas Dickey
  • 51,086
  • 7
  • 70
  • 105
slashms
  • 928
  • 9
  • 26
  • 5
    `unittest` makes no guarantee about the order of execution. And if your tests rely on the order, then you're not doing unit testing properly! Initial and final state should be handled by setup and teardown, not be other test methods – wim May 17 '15 at 11:23
  • 2
    I read some places around stackoverflow that there is a way of doing that, I just couldn't understand it under my conditions. Also, I asked not to tell me why I shouldn't... – slashms May 17 '15 at 11:25
  • Do you need unittest, or can you use another framework? AFAIR proboscis allows test dependency. – Filip Malczak May 17 '15 at 11:46
  • Does this answer your question? [Python unittest.TestCase execution order](https://stackoverflow.com/questions/5387299/python-unittest-testcase-execution-order) – ggorlen Jun 18 '20 at 05:22

2 Answers2

6

Option 1.

One solution to this (as a workaround) was given here - which suggests writing the tests in numbered methods step1, step2, etc., then collecting and storing them via dir(self) and yielding them to one test_ method which trys each.

Not ideal but does what you expect. Each test sequence has to be a single TestClass (or adapt the method given there to have more than one sequence generating method).

Option 2.

Another solution, also in the linked question, is you name your tests alphabetically+numerically sorted so that they will execute in that order.

But in both cases, write monolithic tests, each in their own Test Class.

P.S. I agree with all the comments that say unit testing shouldn't be done this way; but there are situations where unit test frameworks (like unittest and pytest) get used to do integration tests which need modular independent steps to be useful. Also, if QA can't influence Dev to write modular code, these kinds of things have to be done.

Community
  • 1
  • 1
aneroid
  • 12,983
  • 3
  • 36
  • 66
2

I've searched a long time to solve this problem myself.
One of the answers in this question does exactly what you need.

Applied to your code:

ln = lambda f: getattr(TestEffortFormula, f).im_func.func_code.co_firstlineno
lncmp = lambda _, a, b: cmp(ln(a), ln(b))
unittest.TestLoader.sortTestMethodsUsing = lncmp

suite = unittest.TestLoader().loadTestsFromTestCase(TestEffortFormula)
unittest.TextTestRunner(failfast=True).run(suite)

Unfortunately, setting unittest.TestLoader.sortTestMethodsUsing=None does not work, although it is documented that this should avoid sorting the tests alphabetically.

Community
  • 1
  • 1
saroele
  • 9,481
  • 10
  • 29
  • 39
  • 1
    Because `unittest` calls `dir()` to get a list of (possible) test methods and they are already sorted... :-( – nodakai Oct 18 '16 at 10:23