6

I'm attempting to create algebras in Python, and I'm finding it difficult to create parameterized classes.

As an example, consider this ProductWeight class. It holds two other Weight objects, with enforced types (at least statically, via mypy).

This will fail because (by design) I cannot access the classes for W1 and W2 to call their classmethods like zero. (They're not specified; ProductWeight is not templated.) ProductWeight doesn't know, when I create an instance, which types to bind to it.

from typing import Generic, TypeVar
W1 = TypeVar("W1")
W2 = TypeVar("W2")

class ProductWeight(Generic[W1, W2]):
    def __init__(self, value1: W1, value2: W2):
        self.value1_ = value1
        self.value2_ = value2

    @classmethod
    def zero(cls):
        return cls(W1.zero(), W2.zero())  # Will fail - no access to W1 and W2.
    

By contrast, this is straightforward in C++: because the type is parameterized, it is able to look up W1::Zero.

template<typename W1, typename W2>
public:
    ProductWeight(W1 w1, W2 w2) : value1_(w1), value2_(w2) {}
    static ProductWeight Zero() {
        return ProductWeight(W1::Zero(), W2::Zero());
    }
private:
    W1 value1_;
    W2 value2_;
};

Is there a workaround for this in Python? Either creating an inner class, or otherwise somehow providing types to the class (rather than to class instances)?

For the sake of a minimal reproducible example, you can use this implementation of another Weight type.

class SimpleWeight:
    def __init__(self, value):
        assert value in {True, False}
        self.value_ = value

    @classmethod
    def zero(cls):
        return cls(False)

weight1 = SimpleWeight(False)
weight2 = SimpleWeight(True)
product = ProductWeight(weight1, weight2)

print(SimpleWeight.zero())  # So far, so good.
print(ProductWeight.zero())  # Oof.
# Predictably, it failed because `ProductWeight` is not specialized.

And here's the predictable error message:

Traceback (most recent call last):
  File "garbage.py", line 32, in <module>
    print(ProductWeight.zero())  # Oof.
  File "garbage.py", line 14, in zero
    return cls(W1.zero(), W2.zero())  # Will fail - no access to W1 and W2.
AttributeError: 'TypeVar' object has no attribute 'zero'

Ideally, it would be possible to create a parametric type like this:

product = ProductWeight[SimpleWeight, SimpleWeight](weight1, weight2)
# And similarly:
print(ProductWeight[SimpleWeight, SimpleWeight].zero())
Arya McCarthy
  • 8,554
  • 4
  • 34
  • 56
  • What exactly are you trying to do with `W1` and `W2`? If you want all methods to have access to it, you need to declare a `self.W1` or `self.W2` variable, if I understand your question correctly. – M-Chen-3 Feb 26 '21 at 19:07
  • But that's the point—I'm working on the class itself, not an instance. There's no `self`. W1 and W2 should be accessible attributes of the _type_, not the _instance_. In other words, I'm trying to create templatic types. – Arya McCarthy Feb 26 '21 at 19:09
  • Couldn't you just have two variables passed to `__init__`, make them `self` variables, then in `ProductWeight` call `var.zero()`? – M-Chen-3 Feb 26 '21 at 19:10
  • I'm also confused why you have `value1 : W1`. – M-Chen-3 Feb 26 '21 at 19:11
  • "I'm also confused why you have `value1 : W1`."—it's a type annotation. – Arya McCarthy Feb 26 '21 at 19:12
  • Shouldn't it be `value1 : SimpleWeight` since the parameters will be of type `SimpleWeight`? – M-Chen-3 Feb 26 '21 at 19:14
  • 1
    No, it's a generic—to indicate that any type would be acceptable there. They won't always be `SimpleWeight`s. For instance, it could be another `ProductWeight`, or a weight type that wasn't necessary for the [mre]. – Arya McCarthy Feb 26 '21 at 19:17
  • So shouldn't this be `self.value1_.zero(), self.value2_.zero()`? Also, sorry if I'm on the wrong track - I'm just making educated guesses to try to figure out where the problem is. – M-Chen-3 Feb 26 '21 at 19:18
  • There is no `self`. See that I'm calling `ProductWeight.zero()` as opposed to `product.zero()`. – Arya McCarthy Feb 26 '21 at 19:20
  • Try having `.zero` take parameters then. – M-Chen-3 Feb 26 '21 at 19:21
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/229269/discussion-between-m-chen-3-and-arya-mccarthy). – M-Chen-3 Feb 26 '21 at 19:21
  • 1
    This isn't really a construct that Python supports to my knowledge. Type hints are there as a guide, whereas here they would be being used for functionality. – Axe319 Feb 26 '21 at 19:29
  • Are you against having 2 more arguments in your child `classmethod`? – Axe319 Feb 26 '21 at 19:29
  • 1
    Yes—see the discussion in chat. It's a gateway to bad design. It's inconceivable that my goal is not possible in Python. – Arya McCarthy Feb 26 '21 at 19:31
  • "Type hints are there as a guide, whereas here they would be being used for functionality."—yes, exactly. It should still be possible to create parametric types in Python. – Arya McCarthy Feb 26 '21 at 19:45
  • In Java, type parameters are instance data, since each instance can use a different generic type. – ncmathsadist Feb 26 '21 at 19:49
  • @AryaMcCarthy I answered your last edit. See edit to my answer. – sanitizedUser Feb 26 '21 at 22:30
  • @AryaMcCarthy I added one more version that supports both explicit and implicit types. See last section of my answer. – sanitizedUser Feb 26 '21 at 23:04
  • @AryaMcCarthy One last thing. I added a section about type checking that will throw an exception if you provide arguments that don't match their required types. – sanitizedUser Feb 26 '21 at 23:44
  • Does this answer your question? [Python Typing : Create a class function with a generic type, and also access that type through the class](https://stackoverflow.com/questions/65265983/python-typing-create-a-class-function-with-a-generic-type-and-also-access-tha) – MisterMiyagi Feb 27 '21 at 06:10
  • Do you actually need static type checking? This pattern is commonly solved by classically parameterizing the class, either via a factory function or metaclass. – MisterMiyagi Feb 27 '21 at 06:25
  • Do you want W1/W2 to be the concrete types of value1/value2, or can they be abstract or base classes as well? – MisterMiyagi Feb 27 '21 at 06:38

1 Answers1

8

To solve this, it's paramount to realize that generics in C++ create multiple classes while in Python you will always have only one class, no matter what types the parameters in its constructor have.

In other words, in C++ vector<int> and vector<string> are two classes. If you bound them to Python interpreter you would have to assign them two different names, for example VectorInt and VectorString.

The code

def ProductWeight(value1, value2):
    def init(self, value1, value2):
        self.value1 = value1
        self.value2 = value2

    def zero(cls):
        return cls(cls.W1.zero(), cls.W2.zero())

    W1 = type(value1)
    W2 = type(value2)
    name = f'ProductWeight{W1}{W2}'

    try:
        return ProductWeight.types[name](value1, value2)
    except KeyError:
        pass

    cls = type(name, (), {'__init__': init})
    cls.W1 = W1
    cls.W2 = W2
    cls.zero = classmethod(zero)
    ProductWeight.types[name] = cls

    return cls(value1, value2)

ProductWeight.types = {}

class SimpleWeight:
    def __init__(self, value):
        assert value in {True, False}
        self.value_ = value

    @classmethod
    def zero(cls):
        return cls(False)

weight1 = SimpleWeight(False)
weight2 = SimpleWeight(True)
product = ProductWeight(weight1, weight2)

print(SimpleWeight.zero())
print(type(product).zero())

The code tries to respect your original API, you may create instances of class ProductWeight with generic parameters. However, you have to access an underlying type of an instance if you want to call its classmethod zero (notice the change from ProductWeight to type(product) at the last line).

You may save this reference to a variable for convenience.

The function ProductWeight serves as a generic factory. Each time you call it, it creates a name for the new class based on the types of the parameters. If such class already exists, it just returns a new instance of it. Otherwise, it creates the new class using the type function and then returns a new instance.

ProductWeight on its own is also a singleton object that has a dictionary of already created types.

Conclusion

You may notice that this solution uses significantly more memory than its C++ counterpart. However, given that you chose to use Python instead of C++, you will probably not worry about that too much.

More importantly, you will have to decide if this is the right path for you to take. Remember, in Python there are no generics, there are only "dynamics". So your way of thinking will bring in more obstacles than it will remove in the long run.

Making Python sound like C++

This part answers this edit to your question:

Ideally, it would be possible to create a parametric type like this:

product = ProductWeight[SimpleWeight, SimpleWeight](weight1, weight2)
# And similarly:
print(ProductWeight[SimpleWeight, SimpleWeight].zero())

Fear not, because this is actually possible. The following code uses __class_getitem__ from Python 3.7, however with a workaround it can be run on older versions as well, see this question about static getitem method.

Code rewritten

# SimpleWeight didn't change
from simple_weight import SimpleWeight

class ProductWeight:
    types = {}

    def __class_getitem__(cls, key):
        try:
            W1, W2 = key
        except ValueError:
            raise Exception('ProductWeight[] takes exactly two arguments.')

        name = f'{ProductWeight.__name__}<{W1.__name__}, {W2.__name__}>'

        try:
            return cls.types[name]
        except KeyError:
            pass

        new_type = type(name, (), {'__init__': cls.init})
        new_type.W1 = W1
        new_type.W2 = W2
        new_type.zero = classmethod(cls.zero)
        cls.types[name] = new_type

        return new_type

    def __init__(self):
        raise Exception('ProductWeight is a static class and cannot be instantiated.')

    def init(self, value1, value2):
        self.value1 = value1
        self.value2 = value2

    def zero(cls):
        return cls(cls.W1.zero(), cls.W2.zero())

weight1 = SimpleWeight(False)
weight2 = SimpleWeight(True)
product = ProductWeight[SimpleWeight, SimpleWeight](weight1, weight2)

print(SimpleWeight.zero())
print(ProductWeight[SimpleWeight, SimpleWeight].zero())

This takes an advantage of the fact that the bracket operator __getitem__ takes arbitrary number of arguments and packs them into a tuple which is its first parameter. You can unpack the tuple and get all the types. This can be extended to account for any number of types and even such a number that was chosen at runtime.

Inferred type

Last version lost the ability to infer the type from arguments passed to the constructor. By creating an abstract factory we can get this functionality back.

# SimpleWeight didn't change
from simple_weight import SimpleWeight

class ProductWeightAbstractFactory:
    def __call__(self, value1, value2):
        return self[type(value1), type(value2)](value1, value2)

    def __getitem__(self, types):
        W1, W2 = types
        name = f'ProductWeight<{W1.__name__}, {W2.__name__}>'

        try:
            return self.types[name]
        except KeyError:
            pass

        cls = type(self)
        new_type = type(name, (), {'__init__': cls.init})
        new_type.W1 = W1
        new_type.W2 = W2
        new_type.zero = classmethod(cls.zero)
        self.types[name] = new_type

        return new_type

    def __init__(self):
        self.types = {}

    def init(self, value1, value2):
        self.value1 = value1
        self.value2 = value2

    def zero(cls):
        return cls(cls.W1.zero(), cls.W2.zero())

ProductWeight = ProductWeightAbstractFactory()

weight1 = SimpleWeight(False)
weight2 = SimpleWeight(True)
product = ProductWeight[SimpleWeight, SimpleWeight](weight1, weight2)
inferred_product = ProductWeight(weight1, weight2)

print(SimpleWeight.zero())
print(ProductWeight[SimpleWeight, SimpleWeight].zero())
print(type(inferred_product).zero())

Note that you have to create an instance of the factory before using it:

ProductWeight = ProductWeightAbstractFactory()

Now you can create an object with an explicit type using the brackets:

product = ProductWeight[SimpleWeight, SimpleWeight](weight1, weight2)

Or you can infer the type to make the code concise:

product = ProductWeight(weight1, weight2)

Now it's closer to C++ like syntax than ever.

Type checking

To provide more safety when developping you can also introduce type checking in the constructor.

def init(self, value1, value2):
    def check_types(objects, required_types):
        for index, (obj, t) in enumerate(zip(objects, required_types)):
            if not issubclass(type(obj), t):
                raise Exception(f'Parameter {index + 1} is not a subclass of its required type {t}.')

    cls = type(self)
    check_types((value1, value2), (cls.W1, cls.W2))

    self.value1 = value1
    self.value2 = value2

This code will then fail:

product = ProductWeight[SimpleWeight, SimpleWeight](1, 2)
sanitizedUser
  • 1,723
  • 3
  • 18
  • 33