Is typed implicit conversion (coercion) in Python 3.x possible?

Question

Is it possible to implement custom automatic/implicit conversion (aka coercion) in Python 3.6+ that won't make mypy and other static analyzers sad? An example would be a def(foo: A), and given def b_to_a(b: B) -> A, is there a way I could potentially write foo(some_b) (where some_b: B ) instead of foo(b_to_a(some_b))?

I think there are definitely some good ways to do this in the dynamics of Python (tacking on members to classes that include converters, for instance), or even tacking on such converters to the function object itself so that it can handle conversion for selected types, but my current understanding of Python types makes me thing it would not satisfy mypy and the like.

For comparison, see Scala's implicit conversions.

What sort of classes are `A` and `B`? How does the conversion work? You might be able to do this by writing a decorator that sends all the inputs to the correct converter, maybe through some [single dispatch generic function](https://docs.python.org/3/library/functools.html#functools.singledispatch) — Patrick Haugh, Jun 19 '18 at 02:02
Any type of class. The idea is the user of a supplied function that takes a value of type `A` could write an implicit converter for any type `T` that would automatically be called as needed, as long as the converter is in scope (at least that's how it works in Scala) — bbarker, Jun 19 '18 at 02:07
No, python doesn't offer anything like that. You could write something to achieve a similar effect, but it would produce way more code than a handful of `b_to_a` calls. Depending on what the operation you want to do with the object is, you might use duck typing to equip certain classes to perform that operation, and then use an [Abstract Base Class](https://docs.python.org/3/library/abc.html#abc.ABCMeta.__subclasshook__) to describe that group of classes. — Patrick Haugh, Jun 19 '18 at 02:17

Patrick Haugh · Answer 1 · 2018-06-19T15:31:36.267

Here's an implementation of this feature I came up with. We keep a dictionary of single-dispatch converters for types we know the "implicit" conversions for. We add converters to this using the @implicit decorator.

We then have a @coerce decorator that can inspect the function annotations at runtime, get the appropriate converters and apply the conversions. Below is the framework:

from functools import wraps, singledispatch
from inspect import signature
from collections import OrderedDict

converters = {}

def implicit(func):
    ret = func.__annotations__.get('return', None)
    if not ret or len(func.__annotations__) != 2:
        raise ValueError("Function not annotated properly or too many params")
    if ret not in converters:    
        @singledispatch
        def default(arg):
            raise ValueError("No such converter {} -> {}".format(type(arg).__name__, ret.__name__))    
        converters[ret] = default
    else:
        default = converters[ret]
    t = next(v for k, v in func.__annotations__.items() if k != 'return')
    default.register(t)(func)
    return wraps(func)(default)

def convert(val, t):
    if isinstance(val, t):
        return t
    else:
        return converters[t](val)


def coerce(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        sig = signature(func)
        bound = sig.bind(*args, **kwargs)
        bound.apply_defaults()
        bound.arguments = OrderedDict(
            (param, convert(val, sig.parameters[param].annotation)) 
            for param, val in bound.arguments.items())
        return func(*bound.args, **bound.kwargs)    
    return wrapper

And an example:

from typing import Tuple, Type


@implicit
def str_to_int(a: str) ->  int:
    return int(a)

@implicit
def float_to_int(a: float) -> int:
    return int(a)

@coerce
def make_ints(a: int, b: int) -> Tuple[Type, Type]:
    return (type(a), type(b))

print(make_ints("20", 5.0))
# (<class 'int'>, <class 'int'>)

This is amazing (reminds me of reflection in Java land though), will have to study this and try it out. (But if Protocols are realized, I may prefer them for my particular use case in the end). — bbarker, Jun 19 '18 at 15:25

Blckknght · Answer 2 · 2018-06-19T17:11:55.080

It sounds like you may be looking for something like the Protocol types proposed in PEP 544. That PEP isn't approved yet (and may not have a complete implementation yet either), so it may be a while before you get the feature you need (Python 3.8 at the earliest).

Anyway, according to the PEP, Protocols would let you describe a kind of abstract type based on what methods and attributes it has, without the concrete types needing to know about the Protocol or do anything in particular (it doesn't need to inherit from an abstract base class, just have the required methods). It's similar to how you can customize how isinstance and issubclass work using metaclasses, but it works with static type checking, not only at runtime.

For instance, iterators in Python are an existing protocol that many unrelated classes implement. If the PEP is approved and implemented, you won't need to declare a custom iterator type as inheriting from typing.Iterator any more, it would figure it out automatically, just because the class has __iter__ and __next__ methods.

In your example, you could make an A_Like protocol that requires a to_A method:

class A_Like(typing.Protocol):
    def to_A(self) -> A:
        ...

Then you'd implement A.to_A with a simple return self, while B.to_A does the appropriate conversion. Both classes will be seen as matching the A_Like protocol type, so def foo(a: A_Like) would satisfy type checkers (with the body of the class needing to do a = a.to_A() before calling any A specific methods).

You can do this now with inheritance from a common abstract base class (which can be a simple mixin), but it's definitely not as elegant as it will be with Protocols. Another option if you don't have many classes you need to convert is to just use Union types declarations: def foo(a: Union[A, B])

"static duck typing" is actually sort of how I think of type classes anyway, though I'm not yet sure if they are an exact match. +1, great to see that on the way (hopefully). In fact, this is even better than what I was hoping for, since I was just looking for implicit coercion as a way to do static ducktyping/typeclasses! — bbarker, Jun 19 '18 at 15:22
It looks like I can go ahead and use the `typing_extensions` package from pypi (or just drop the file in your project from the typings github repo) in Python 3.6.5 without issue (so far). Of course, fair warning the standard may change, or even not be accepted, as you noted. — bbarker, Jun 20 '18 at 01:54

score 1 · Answer 3 · answered Jun 19 '18 at 02:07

I don't think this is a conversion problem. But it looks like a annotation problem.

First, if foo can only handle A, how could it accept B? And if foo can handle B too, why it should only accpet A?

Second, if you want to annotate that foo accept A or B, you can use def(foo: Union[A, B]).

Finally, if you mean B should have some methods that make it can be handled by function which can only handle A. It is still an instance of B. Without right annotation, your static analyzers will still warn you.

Basically, I'd like to use the notion of type classes from Haskell, which are more flexible than inheritance. — bbarker, Jun 19 '18 at 02:09

Is typed implicit conversion (coercion) in Python 3.x possible?

3 Answers3