47

I'm writing some test cases for my application using Python's unittest. Now I need to compare a list of objects with a list of another objects to check if the objects from the first list are what I'm expecting.

How can I write a custom .assertFoo() method? What should it do? Should it raise an exception on failure? If yes, which exception? And how to pass the error message? Should the error message be a unicode string or a bytestring?

Unfortunately, the official documentation doesn't explain how to write custom assertion methods.

If you need a real-world example for this, continue reading.

The code I'm writing is somewhat like this:

def assert_object_list(self, objs, expected):
    for index, (item, values) in enumerate(zip(objs, expected)):
        self.assertEqual(
            item.foo, values[0],
            'Item {0}: {1} != {2}'.format(index, item.foo, values[0])
        )
        self.assertEqual(
            item.bar, values[1],
            'Item {0}: {1} != {2}'.format(index, item.bar, values[1])
        )

def test_foobar(self):
    objs = [...]  # Some processing here
    expected = [
        # Expected values for ".foo" and ".bar" for each object
        (1, 'something'),
        (2, 'nothing'),
    ]
    self.assert_object_list(objs, expected)

This approach makes it extremely easy to describe the expected values of each object in a very compact way, and without needing to actually create full objects.

However... When one object fails the assertion, no further objects are compared, and this makes debugging a bit more difficult. I would like to write a custom method that would unconditionally compare all objects, and then would display all objects that failed, instead of just the first one.

Denilson Sá Maia
  • 47,466
  • 33
  • 109
  • 111
  • This [answer](https://stackoverflow.com/a/54499865/4934640) for the question `Continuing in Python's unittest when an assertion fails` should help you override the `TestCase.assertEqual()` correctly. And this other question [Comparison of multi-line strings in Python unit test](https://stackoverflow.com/questions/32359402) should help you in creating a custom `myAssertEqual()` and register it to be automatically called. – Evandro Coan Feb 03 '19 at 04:20

3 Answers3

45

I use the multiple inheritance in these cases. For example:

First. I define a class with methods that will incorporate.

import os

class CustomAssertions:
    def assertFileExists(self, path):
        if not os.path.lexists(path):
            raise AssertionError('File not exists in path "' + path + '".')

Now I define a class that inherits from unittest.TestCase and CustomAssertion

import unittest

class MyTest(unittest.TestCase, CustomAssertions):
    def test_file_exists(self):
        self.assertFileExists('any/file/path')

if __name__ == '__main__':
    unittest.main()
Alan Cristhian
  • 614
  • 7
  • 8
  • 3
    Good alternative approach to subclassing - this was helpful. – Josh Werts Jan 21 '14 at 22:04
  • 9
    This approach is also called [mixin](https://en.wikipedia.org/wiki/Mixin#In_Python). – Denilson Sá Maia Sep 11 '14 at 17:37
  • @DenilsonSáMaia - Is it okay to make a custom assert return something ? – Erran Morad Oct 08 '16 at 21:55
  • @DenilsonSáMaia - Actually, how do you develop unittests for a custom assert, or develop a custom assert using test driven development ? – Erran Morad Oct 08 '16 at 22:56
  • @BoratSagdiyev: Those comments look like they should be new questions at StackOverflow or at https://programmers.stackexchange.com/ – Denilson Sá Maia Oct 10 '16 at 17:50
  • @DenilsonSáMaia - Thanks. I created a question for that here-https://programmers.stackexchange.com/questions/333538/is-it-okay-to-allow-a-custom-assertion-to-return-something I wonder if it will get as much attention and answers as stack overflow though. – Erran Morad Oct 13 '16 at 18:19
  • 4
    I did something similar, but I think it is very annoying that, when the test fails, the stack trace always shows the line where you raise the assertion error and not the line that calls the assertion function in your actual test. Is there a way to fix that? – Cristóbal Ganter Feb 12 '18 at 03:03
  • 5
    @CristóbalGanter to fix the line in stack trace see https://stackoverflow.com/questions/49927872/prevent-custom-assert-from-showing-in-traceback-of-python-unittest/49929579#49929579 – shao.lo Apr 19 '18 at 21:37
  • 3
    The recommend way to signal test failures is to call `self.fail()` or equivalently `raise self.failureException(msg)`. While raising and AssertionError will work for now it may not always be treated equivalently by the unittest framework: See https://docs.python.org/2/library/unittest.html#re-using-old-test-code – sbc100 Mar 03 '19 at 18:29
19

You should create your own TestCase class, derived from unittest.TestCase. Then put your custom assert method into that test case class. If your test fails, raise an AssertionError. Your message should be a string. If you want to test all objects in the list rather than stop on a failure, then collect a list of failing indexes, and after the loop over all the objects, build an assert message that summarizes your findings.

Ned Batchelder
  • 364,293
  • 75
  • 561
  • 662
  • 1
    Is there any difference in raising an `AssertionError` and any other `Exception` (with respect to custom asserts and `unittest`)? – hiwaylon Apr 03 '13 at 21:25
3

Just an example to sum up with a numpy comparaison unittest

import numpy as np
class CustomTestCase(unittest.TestCase):
    def npAssertAlmostEqual(self, first, second, rtol=1e-06, atol=1e-08):
        np.testing.assert_allclose(first, second, rtol=rtol, atol=atol)


class TestVector(CustomTestCase):
    def testFunction(self):
        vx = np.random.rand(size)
        vy = np.random.rand(size)
        self.npAssertAlmostEqual(vx, vy)
parisjohn
  • 301
  • 2
  • 12