How does test = False magic attribute work for test discovery

Question

So I'm attempting to implement something similar to how unittesting frameworks do the following thing:

class BaseTest(T.TestCase):
    # Disables this test from being run
    __test__ = False

    def test_foo(self): pass


# However this test is picked up because it doesn't directly have __test__ set
class InheritingTest(BaseTest): pass

A thing I find peculiar:

# >> InheritingTest.__test__
# False

Which would indicate to me that it isn't using a metaclass to set __test__ to True on construction of the type.

I tried grepping through the python library find . -name "*.py" | xargs grep '__test__' but did not seem to find anything related to this.

My "guess" approach at solving this problem is to do the following:

def is_class_tested(cls):
    return cls.__dict__.get('__test__', True)

However this feels fragile to me... Is there a cleaner / nicer way to do this that works in all cases? Is there ever a chance that a class will not have a __dict__ property?

Why do you find it perculiar that InheritingTest.__test__ is False? The name resolution for this variable will be first to check the dict of InheritingTest and then to check the dict of BaseTest, where it finds it. — Owen, Aug 26 '13 at 00:03
I find it peculiar because that test is run, despite it having `__test__ = False` — anthony sottile, Aug 26 '13 at 00:04
Can you post the code that decides whether or not a test should run? — Owen, Aug 26 '13 at 00:06
What framework is this? The Python 2 `unittest` documentation doesn't mention this attribute; I'd never heard of it before. — user2357112, Aug 26 '13 at 00:56
I'm using a noselike test framework called https://github.com/Yelp/Testify — anthony sottile, Aug 26 '13 at 04:14
They do something similar to what I've posted above -- I wonder why my grep didn't find it. — anthony sottile, Aug 26 '13 at 04:15

nneonneo · Accepted Answer · 2013-08-26T13:58:10.007

Testify, the library you are using, does the following in testify/test_discovery.py around line 140:

    # it's not a list, it's not a bare module - let's see if it's an honest-to-god TestCaseBase
    elif isinstance(test_module, MetaTestCase) and (not '__test__' in test_module.__dict__ or bool(test_module.__test__)):
        ...
    # detect unittest test cases
    elif issubclass(test_module, unittest.TestCase) and (not '__test__' in test_module.__dict__ or bool(test_module.__test__)):

So, in other words, it is doing exactly what your "guess" approach does, in a slightly more verbose way: it tests for the presence and value of __test__ in the class __dict__ directly.

score -1 · Answer 2 · edited May 23 '17 at 12:05

Is there ever a chance that a class will not have a __dict__ property? My initial reaction to that is no. That might not be exactly what you were asking because you can have an instance of a class without a __dict__ if it defines __slots__ and does not inherit from a class with a __dict__ already in it. See the __slots__ description in the Python Data Model description

EDIT:

as pointed out by @nneonneo, comments below about the double underscore are not correct becuase there are trailing underscores. The content is left in for historical reasons.

The behavior you claim looks peculiar, I think looks natural. @Owen was right in asking about the code that that expects something different and thankyou for posting the reference to Yelp/Testify. That framework makes extensive use of the double_underscore.

Alex Martelli's SO answer on the double_underscore helps shed light on what might be going on but the very shortest answer is that the double underscore is returning the results above because the dot notation of InheritingTest.__test__ goes through the normal attribute resolution machinery but the Testify framework, by deciding to use the leading double_underscore, has reserved the right to access it at it's class scope even if your subclass overrides it.

Test frameworks are, by their very nature, magical beasts and the use of __test__ hear looks proper on their part. I checked out their docs and the documentation feels sparse so your using __test__ at all feels dangerous (the double underscore kind of signals hands off this "class local" variable anyway).

`__test__` has *two trailing underscores*, exempting it from the usual name mangling. It is *not* a 'class-local attribute'. — nneonneo, Aug 26 '13 at 13:57
Unless I'm very much mistaken, a class with `__slots__` does not have to have a `__dict__`. — Marcin, Aug 26 '13 at 17:55
@Marcin more like usually does not have a `__dict__`. The question appeared to be concerned with the *class* level `__dict__` as opposed the the instance level. `__slots__` means that there is not `__dict__` on the *instnace*. Also, `__slots__` does not guarantee that there is no `__dict__` at the instance level because if one is there from a parent class then there will be both `__slots__` and `__dict__`. — Phil Cooper, Aug 28 '13 at 12:31

How does test = False magic attribute work for test discovery

2 Answers2

Linked

How does __test__ = False magic attribute work for test discovery

2 Answers2

Linked

How does test = False magic attribute work for test discovery