0

I have 2 classes that are identical aside from their inheritance. Each class overrides the same methods. I tried to avoid redundancy by using a common global method, but it wasn't enough to avoid a JSCPD error.

I'm not quite sure how to arrange things so that I only have 1 instance of the over-ridden methods, but each over-riding the methods of different base classes...

Probably if I removed the doc strings (which I've removed below), I'd avoid the linting error, but I'd prefer knowing how to accomplish this correctly.

import time

from django.test import TestCase, TransactionTestCase

LONG_TEST_THRESH_SECS = 20
LONG_TEST_ALERT_STR = f" [ALERT > {LONG_TEST_THRESH_SECS}]"


class TracebaseTestCase(TestCase):

    maxDiff = None
    databases = "__all__"

    def setUp(self):
        self.testStartTime = time.time()

    def tearDown(self):
        _reportRunTime(self.id(), self.testStartTime)

    def setUpClass(self):
        self.classStartTime = time.time()

    def setUpTestData(self):
        _reportRunTime(f"{self.__class__.__name__}.setUpTestData", self.classStartTime)

    class Meta:
        abstract = True


class TracebaseTransactionTestCase(TransactionTestCase):

    maxDiff = None
    databases = "__all__"

    def setUp(self):
        self.testStartTime = time.time()

    def tearDown(self):
        _reportRunTime(self.id(), self.testStartTime)

    def setUpClass(self):
        self.classStartTime = time.time()

    def setUpTestData(self):
        _reportRunTime(f"{self.__class__.__name__}.setUpTestData", self.classStartTime)

    class Meta:
        abstract = True


def _reportRunTime(id, startTime):
    t = time.time() - startTime
    heads_up = ""  # String to include for tests that run too long

    if t > LONG_TEST_THRESH_SECS:
        heads_up = LONG_TEST_ALERT_STR

    print("TEST TIME%s: %s: %.3f" % (heads_up, id, t))

I have a vague recollection back in my C++ school days about creating classes where you could supply a class as input for it's "inheritance" but I don't remember what that was called. I recall it used angle brackets in its declaration. I feel like that's what I need. Something like:

class abstractBaseClass(<base class input>):
    # This would be where the methods would be defined
    # I would also make reportRunTime be a member function here if I knew how to implement this "class template"

class TracebaseTestCase(abstractBaseClass(TestCase))
    pass

class TracebaseTransactionTestCase(abstractBaseClass(TransactionTestCase))
    pass

Am I close?

UPDATE: I fleshed out the rest of my source code, since there seemed to be some question about if it would affect the answer.

The point is that in each case, I am either over-riding the methods of TestCase or TransactionTestCase. And wherever I inherit from (for example) TracebaseTestCase, TestCase is determining when to run setUp, tearDown, setUpClass, and setUpTestData.

The code works as it is. I just want to avoid the jscpd linting error and reduce the redundancy.

hepcat72
  • 890
  • 4
  • 22
  • Oh right. I'm recalling "class templates" from C++. Can you do that in Python? – hepcat72 Sep 12 '22 at 17:22
  • 1
    What's a JSCPD error? Are you looking for a _mix-in_? – jonrsharpe Sep 12 '22 at 17:23
  • Do you mean [metaclasses](https://docs.python.org/3/reference/datamodel.html?highlight=metaclass#metaclasses) or do you just want a Mixin class (for multiple inheritance)? – Daniil Fajnberg Sep 12 '22 at 17:24
  • jscpd is a linting tool that finds redundant code. It errors out on the code duplication in the two classes. `ERROR: jscpd found too many duplicates (38.38%) over threshold (0%)` – hepcat72 Sep 12 '22 at 17:25
  • A mix-in is almost certainly sufficient instead of a metaclass. It's not clear what `_reportRunTime` is, but I suspect it can be wrapped or replaced with an inherited method in some way. – chepner Sep 12 '22 at 17:26
  • metaclass looks promising. I don't know much about mixins. I just need for my methods to override the parent class's methods in each of the 2 classes. – hepcat72 Sep 12 '22 at 17:28

2 Answers2

2

You might consider using a mix-in.

class TestSkeleton:
    maxDiff = None
    databases = "__all__"

    def setUp(self):
        self.testStartTime = time.time()

    def tearDown(self):
        _reportRunTime(self.id(), self.testStartTime)

    def setUpClass(self):
        self.classStartTime = time.time()

    def setUpTestData(self):
        _reportRunTime(f"{self.__class__.__name__}.setUpTestData", self.classStartTime)

    class Meta:
        abstract = True


class TracebaseTestCase(TestSkeleton, TestCase):
    pass


class TracebaseTransactionTestCase(TestSkeleton, TransactionTestCase):
    pass

tearDown and setUpTestData probably need some adjustment depending on exactly what _reportRunTime is and what it's first argument is supposed to represent.

chepner
  • 497,756
  • 71
  • 530
  • 681
  • `_reportRunTime` is very simple. It just prints the number of seconds every test took and the number of seconds `setUpTestData` took. It adds a string to notify when the time is over an arbitrary threshold (so it's easy to search for in test output). – hepcat72 Sep 12 '22 at 17:30
  • I'm not 100% sure that the "TestSkeleton" class would correctly override the methods in `TestCase` and `TransactionTestCase` here. I need the code in TracebaseTestCase.tearDown() for example to run after it's superclass's version has run. If both `TestSkeleton` and `TestCase` have the same methods, which one would get called when `super().tearDown()` is called from the class that inherits `TracebaseTestCase`? Or if they're both called, wouldn't they be called serially? – hepcat72 Sep 12 '22 at 17:35
  • ...I'm confusing myself. I don't think I need TestCase.tearDown() to be called at a specific time. It's automatically called... and I just need my code to be run when it's automatically called by TestCase... and I'm not sure how that would happen here. Say I have a class in one of my test modules like: `class CalculationTests(tracebasetestCase):`. It doesn't override tearDown. tearDown is called upon completion of each individual test by the inherited TestCase machinery. – hepcat72 Sep 12 '22 at 17:42
  • I tried your suggestion, but it doesn't appear to work. Every class that inherits from TraceBaseTestCase exits with an error: `TypeError: setUpClass() missing 1 required positional argument: 'self'`. I tried adding `@classmethod` decorators but then I got errors about other missing decorators. I think that "TestSkeleton" needs to inherit from each of the 2 base classes in order to work. This is the complication I was afraid of with your suggestion. I really would need a "template class" like in C++. I have played with metaclass, but I can't figure out how to make that work either. – hepcat72 Sep 12 '22 at 18:32
  • [`setUpClass`](https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUpClass) is supposed to be defined as a class method. You are defining it as an instance method. – chepner Sep 12 '22 at 18:45
  • Like I said, I tried making it a class method, but I get errors about other missing decorators. The problem is that TestCase and TransactionTestCase are the classes that are calling these methods. With inheritance, all I have to do is override them. This solution, assuming I could make it work if I implement all the same decorators, would have the overhead of reimplementing those decorators anytime the base class is updated. I could try that, but it would be more elegant if the base class was somehow a variable. Is that even possible? – hepcat72 Sep 12 '22 at 19:02
  • Yes, it's possible. However, I wouldn't worry too much about any overhead due to multiple inheritance. It should be minimal compared to whatever work the individual tests actually do. – chepner Sep 12 '22 at 19:04
  • OK, after adding `@classmethod` to setUpClass `TestCase.tearDownClass`, complains `AttributeError: 'function' object has no attribute 'wrapped'`. If I add class method: `def tearDownClass(cls): pass`, I get the error: `django.utils.connection.ConnectionDoesNotExist: The connection '_' doesn't exist.`... where does this end? I also tried creating a factory like here: https://stackoverflow.com/questions/3876921/metaclass-to-parametrize-inheritance and I get the same behavior/errors. It just seems like this isn't cleanly possible - to override base class methods where the base class is variable. – hepcat72 Sep 12 '22 at 19:31
  • That seems related to a use of `functools.wraps` that doesn't appear in your question. If you want help, you'll need to provide a [mcve]. – chepner Sep 12 '22 at 19:37
  • The code I included (after the update) is the entire code of the file, minus comments and doc strings. However, I think I may have been mistaken about the code fully working. It's hard to say because I keep tweaking and rerunning. The last things I'd added was the setUpClass and setUpTestData overrides and it may be that those never worked. It worked before I added those methods bacause I still have that output in a terminal window. Maybe overriding class methods is a bad idea in general. I'll check my code and either update or ask a new question. – hepcat72 Sep 12 '22 at 19:59
0

@chepner's answer is correct, though I didn't understand why or how it worked because I never understood the concept of mixins. I couldn't map it to my naive notion of multiple inheritance. So I did some testing with trial and error and now have more confidence that I have a better conceptual (though perhaps technically inaccurate) understanding of how they work. I had previously been conceptually dissuaded by the notion of "multiple inheritance" to imply multiple independent parent classes/objects. However (to use a geeky analogy) I now see it more like the parent being "Tuvix" from Star Trek Voyager. The parents (Tuvok and Neelix) are not independent individuals. There's only 1 parent: Tuvix, who is a merging of the 2 parents.

And from my testing, I have come to understand that the order of the superclasses establishes whose characteristics dominate. Precedence goes from left to right. Anything you "override" in the left-side class is what gets set/called when a derived class calls/gets it.

I don't need to reiterate @chepner's answer, but I will provide an example to demonstrate why it works... Take these classes as an example:

class mybaseclass():
    """a.k.a. TestCase"""

    classvar = "base"

    def member_override_test(self):
        print(f"member_override_test in mybaseclass, classvar: [{self.classvar}]")

    @classmethod
    def classmethod_override_test(cls):
        print(f"classmethod_override_test in mybaseclass, classvar: [{cls.classvar}]")

    def run_member_super_test(self):
        print(f"classvar: {self.classvar}")
        print("Calling member_override_test from mybaseclass:")
        self.member_override_test()
        print("Calling classmethod_override_test from mybaseclass:")
        self.classmethod_override_test()
        print("Calling run_test from mybaseclass:")
        self.run_test()


class mymixinclass():
    """a.k.a. TestSkeleton"""

    classvar = "mixin"

    def member_override_test(self):
        print(f"member_override_test in mymixinclass, classvar: [{self.classvar}]")

    @classmethod
    def classmethod_override_test(cls):
        print(f"classmethod_override_test in mymixinclass, classvar: [{cls.classvar}]")


class mypseudoderivedclass(mymixinclass, mybaseclass):
    """a.k.a. TracebaseTestCase"""

    def member_override_test(self):
        print(f"member_override_test in mypseudoderivedclass, classvar: [{self.classvar}]")
        print(f"Calling super.member_override_test")
        super().member_override_test()

    def classmethod_override_test(self):
        print(f"classmethod_override_test in mypseudoderivedclass, classvar: [{self.classvar}]")
        print(f"Calling super.classmethod_override_test")
        super().classmethod_override_test()

    def run_test(self):
        print(f"classvar: {self.classvar}")
        print("Calling member_override_test from mypseudoderivedclass object:")
        self.member_override_test()
        print("Calling classmethod_override_test from mypseudoderivedclass object:")
        self.classmethod_override_test()


class reversemypseudoderivedclass(mybaseclass, mymixinclass):
    """a.k.a. TracebaseTestCase - reversing the order of the mixins"""

    def member_override_test(self):
        print(f"member_override_test in reversemypseudoderivedclass, classvar: [{self.classvar}]")
        print(f"Calling super.member_override_test")
        super().member_override_test()

    def classmethod_override_test(self):
        print(f"classmethod_override_test in reversemypseudoderivedclass, classvar: [{self.classvar}]")
        print(f"Calling super.classmethod_override_test")
        super().classmethod_override_test()

    def run_test(self):
        print(f"classvar: {self.classvar}")
        print("Calling member_override_test from mypseudoderivedclass object:")
        self.member_override_test()
        print("Calling classmethod_override_test from mypseudoderivedclass object:")
        self.classmethod_override_test()

And here's what you see when you play with those classes in the python shell:

In [1]: from DataRepo.tests.tracebase_test_case import mypseudoderivedclass, reversemypseudoderivedclass
   ...: mpdc = mypseudoderivedclass()

In [2]: mpdc.run_test()
   ...: 
classvar: mixin
Calling member_override_test from mypseudoderivedclass object:
member_override_test in mypseudoderivedclass, classvar: [mixin]
Calling super.member_override_test
member_override_test in mymixinclass, classvar: [mixin]
Calling classmethod_override_test from mypseudoderivedclass object:
classmethod_override_test in mypseudoderivedclass, classvar: [mixin]
Calling super.classmethod_override_test
classmethod_override_test in mymixinclass, classvar: [mixin]

In [3]: mpdc.member_override_test()
member_override_test in mypseudoderivedclass, classvar: [mixin]
Calling super.member_override_test
member_override_test in mymixinclass, classvar: [mixin]

In [4]: mpdc.classmethod_override_test()
classmethod_override_test in mypseudoderivedclass, classvar: [mixin]
Calling super.classmethod_override_test
classmethod_override_test in mymixinclass, classvar: [mixin]

In [5]: mpdc.run_member_super_test()
classvar: mixin
Calling member_override_test from mybaseclass:
member_override_test in mypseudoderivedclass, classvar: [mixin]
Calling super.member_override_test
member_override_test in mymixinclass, classvar: [mixin]
Calling classmethod_override_test from mybaseclass:
classmethod_override_test in mypseudoderivedclass, classvar: [mixin]
Calling super.classmethod_override_test
classmethod_override_test in mymixinclass, classvar: [mixin]
Calling run_test from mybaseclass:
classvar: mixin
Calling member_override_test from mypseudoderivedclass object:
member_override_test in mypseudoderivedclass, classvar: [mixin]
Calling super.member_override_test
member_override_test in mymixinclass, classvar: [mixin]
Calling classmethod_override_test from mypseudoderivedclass object:
classmethod_override_test in mypseudoderivedclass, classvar: [mixin]
Calling super.classmethod_override_test
classmethod_override_test in mymixinclass, classvar: [mixin]

In [6]: rmpdc = reversemypseudoderivedclass()
   ...: rmpdc.run_test()
classvar: base
Calling member_override_test from mypseudoderivedclass object:
member_override_test in reversemypseudoderivedclass, classvar: [base]
Calling super.member_override_test
member_override_test in mybaseclass, classvar: [base]
Calling classmethod_override_test from mypseudoderivedclass object:
classmethod_override_test in reversemypseudoderivedclass, classvar: [base]
Calling super.classmethod_override_test
classmethod_override_test in mybaseclass, classvar: [base]

In [7]: rmpdc.member_override_test()
member_override_test in reversemypseudoderivedclass, classvar: [base]
Calling super.member_override_test
member_override_test in mybaseclass, classvar: [base]

In [8]: rmpdc.classmethod_override_test()
classmethod_override_test in reversemypseudoderivedclass, classvar: [base]
Calling super.classmethod_override_test
classmethod_override_test in mybaseclass, classvar: [base]

In [9]: rmpdc.run_member_super_test()
classvar: base
Calling member_override_test from mybaseclass:
member_override_test in reversemypseudoderivedclass, classvar: [base]
Calling super.member_override_test
member_override_test in mybaseclass, classvar: [base]
Calling classmethod_override_test from mybaseclass:
classmethod_override_test in reversemypseudoderivedclass, classvar: [base]
Calling super.classmethod_override_test
classmethod_override_test in mybaseclass, classvar: [base]
Calling run_test from mybaseclass:
classvar: base
Calling member_override_test from mypseudoderivedclass object:
member_override_test in reversemypseudoderivedclass, classvar: [base]
Calling super.member_override_test
member_override_test in mybaseclass, classvar: [base]
Calling classmethod_override_test from mypseudoderivedclass object:
classmethod_override_test in reversemypseudoderivedclass, classvar: [base]
Calling super.classmethod_override_test
classmethod_override_test in mybaseclass, classvar: [base]

Note that the class variable classvar's value and the method handling the (super) calls depend on the order of the mixins in the class's multiple inheritances.

Factory Alternative

Before I understood the mixins, I tried another solution that solves the sample problem in a different way. You can indeed parameterize the base class, which I learned from another answer, by creating a factory function, then vivify the derived classes by making factory calls.

Note, in this answer, I realized I could avoid overriding setUpClass by setting the "class start time" in the member data:

import time

from django.test import TestCase, TransactionTestCase

LONG_TEST_THRESH_SECS = 20
LONG_TEST_ALERT_STR = f" [ALERT > {LONG_TEST_THRESH_SECS}]"


def test_case_class_factory(base_class):
    class TracebaseTestCaseTemplate(base_class):
        maxDiff = None
        databases = "__all__"
        classStartTime = time.time()

        def setUp(self):
            self.testStartTime = time.time()

        def tearDown(self):
            reportRunTime(self.id(), self.testStartTime)

        @classmethod
        def setUpTestData(cls):
            super().setUpTestData()
            reportRunTime(f"{cls.__name__}.setUpTestData", cls.classStartTime)

        class Meta:
            abstract = True

    return TracebaseTestCaseTemplate


def reportRunTime(id, startTime):
    t = time.time() - startTime
    heads_up = ""  # String to include for tests that run too long

    if t > LONG_TEST_THRESH_SECS:
        heads_up = LONG_TEST_ALERT_STR

    print("TEST TIME%s: %s: %.3f" % (heads_up, id, t))


# Classes created by the factory with different base classes:
TracebaseTestCase = test_case_class_factory(TestCase)
TracebaseTransactionTestCase = test_case_class_factory(TransactionTestCase)

Both solutions work, and while I like the preserved inheritance of the factory method, once you understand mixins, I feel like @chepner's code is easier to read, so I will select his answer.

hepcat72
  • 890
  • 4
  • 22