Python unittest counting the number of tests

Question

It's my first time playing around with Python's unittest for an assignment in school. I basically have a Circle Object, where I am using pyunit to make sure the data is stored properly.

I noticed that Python only counts the number of methods as test cases as opposed to the number of assert statements.

For instance I want to test that methods are working correctly, Python only counts the following as 2 tests, despite having 4 assert statements. It really caught me off guard, as with Java's JUnit it will count the number of assert statements instead.

def test_xcrd(self): 
    self.assertTrue(self.point.xcrd() == 1) 
    self.assertFalse(self.point.xcrd() == 5)

def test_ycrd(self): 
    self.assertTrue(self.point.ycrd() == 2) 
    self.assertFalse(self.point.ycrd() == 10)

What's the "norm" in python? Should there only be one assert statement per method?

"as with Java's JUnit, it will count the number of assert statements, instead" - what? Since when? — user2357112, Feb 17 '17 at 18:45
In past experience, as long as I have @Test above methods I want to test, all assertion statements in the respective method will show up in the JUnit Testing panel of Eclipse. For instance, a method with 4 assertion statements written in Java - all four asserts will show up in Eclipse. These same assertions written in Python, with pyunit, say "Ran 1 test in 0.001s, OK." — Fandom_Lover, Feb 17 '17 at 18:53
`pyunit` is a 3rd party testing module that is distinct from the `unittest` module that comes with CPython in the standard library. If you are using `unittest`, then you should edit the text to remove `pyunit` and the `pyunit` tag. If you had posted an MCVE https://stackoverflow.com/help/mcve, there would be no question about which module you are using. — Terry Jan Reedy, Feb 17 '17 at 18:57
@TerryJanReedy https://wiki.python.org/moin/PyUnit begs to differ? [The docs](https://docs.python.org/2/library/unittest.html) also say that `unittest` is sometimes referred to as PyUnit. That part is gone in the docs for Python 3 though... — Dartmouth, Feb 17 '17 at 19:05
It appears that unittest began as the 3rd party module called pyunit, after Junit. That page and the separate release, last updated in 2001, is a bit of a fossil. Unittest, in the stdlib, has seen many fixes and updates and has a different primary maintainer. — Terry Jan Reedy, Feb 17 '17 at 23:58

score 5 · Accepted Answer · answered Feb 17 '17 at 19:02

Python's unittest package allows you to structure your unit tests in separate methods like you are noticing. This is useful in cases where you want to tests things which are very closely related and don't require separate unit tests.

unittest tests start by subclassing unittest.Test, and then adding methods to this. So, you can add several layers separation between different unittests which are more less related.

An example from the Python Docs demonstrates what is considered to be best practice for Python unit tests:

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()

There are a number of things you can observe here:

The three methods of TestStringMethods are the separate unittests.
test_isupper and test_split both contain two asserts, since they are very closely related. Adding separate tests for the two asserts present in test_isupper would be adding lots of bloat to the code, and it could lead to very weird problems.

For example, if str.isupper() would break in a weird way, the single unittest covering this single function would break. However, if the two tests for "FOO" and "Foo" were separate, one test might pass, and the other fail. So, testing the functionality of a single function is better kept in a single unittest with several asserts.

The same applies to the test_split method; checking that str.split() works and checking that it raises a TypeError are closely related, and should therefore best be kept close together in code as well.

So, to come back to your question: There can (and sometimes should) be more than one assert per method, since it leads to simpler and clearer code, and less confusion. To quote the "Zen of Python" (found by running import this in a python shell): "Simple is better than complex". So, keep your unittests simple and structured by grouping similar asserts in a single method.

score 0 · Answer 2 · answered Feb 17 '17 at 19:03

The answer to your question '''What's the "norm" in python? Should there only be one assert statement per method?''' is "No". Some people might say 'yes' but CPython core developers, including me, routinely use multiple asserts in test methods. Take a look at the test_xyz files in Lib/test (if your installation includes that directory).

It is true that one method should test one unit or even one behavior of one unit.

Python unittest counting the number of tests

2 Answers2

Linked