Assign a default callable value for a callable protocol

Question

I have the following callable Protocol:

class OnStarted(Protocol):
    def __call__(self, kwargs: Dict[str, Any]) -> Optional[Dict[str, Any]]: ...

that I want to assign a default function, which I did like this:

def foo(on_started: OnStarted = lambda _: {}):
    pass

but MyPy isn't happy with it and complains with this error:

foo.py:172: error: Incompatible default for argument "on_started" (default has type "Callable[[Any], Dict[<nothing>, <nothing>]]", argument has type "OnStarted")  [assignment]
foo.py:172: note: "OnStarted.__call__" has type "Callable[[Arg(Dict[str, Any], 'kwargs')], Optional[Dict[str, Any]]]"

How can I fix that so that on_started has a default value and MyPy doesn't error?

It can't infer that `{}` is potentially `Dict[str, Any]`, `lambda _: None` would work, or maybe `lambda _: cast(Dict[str, Any], {})`. — jonrsharpe, Apr 28 '23 at 11:03
@jonrsharpe oh I see. This means mypy doesn't know it's an empty dict that actually would satisfy the requirement (which was my assumption), but it recognizes an untyped dict thus failing the typing hints. — t3chb0t, Apr 28 '23 at 11:09
You can see what it's inferred - `Dict[, ]`, which is apparently not compatible with `Dict[str, Any]`. `{"": 0}` is about the shortest value that would get inferred correctly, but probably isn't what you want to be returning! — jonrsharpe, Apr 28 '23 at 11:14
@jonrsharpe now that you're saying this LOL, face-palm heh. I'll give you the tick if you want to post it as an answer ;-] — t3chb0t, Apr 28 '23 at 11:17

Mathias Réus · Answer 1 · 2023-04-28T12:02:42.537

2

As stated in PEP8 *:

Always use a def statement instead of an assignment statement that binds a lambda expression directly to an identifier.

Namely, you will gain clarity if you write a regular function to set it as a default value. Plus, you can add documentation to explain why you need a default function, what the default does, it's easier to change.

For this you can follow the PEP8 recommendation and simply define your default function as a regular one.

def default_on_started(kwargs: Dict[str, Any]) -> Optional[Dict[str, Any]]: 
    return {}

def foo(on_started: OnStarted = default_on_started):
    print(on_started({}))

*: We might argue that this line is only for regular body code (@jonrshape) in comment in this post gave good points. If you are interested, there is a discussion in the comments.

edited Apr 28 '23 at 12:02

answered Apr 28 '23 at 11:23

Mathias Réus

21
3

There's no point putting an implementation in `OnStarted`, because `TypeError: Protocols cannot be instantiated`. – jonrsharpe Apr 28 '23 at 11:30
I can use a `def` as a default!? That's just cool! Also with the `return` in the same line. – t3chb0t Apr 28 '23 at 11:38
You are right, we cannot instanciate Protocol, big mistake. Thanks. – Mathias Réus Apr 28 '23 at 11:41
(Also I don't think that's the intent of _"an assignment statement that binds a lambda expression directly to an identifier"_, e.g. PyCharm's Pylint doesn't warn using a `lambda` as a default value but does for top-level assignments. See e.g. https://stackoverflow.com/a/25010243/3001761. And if you're following PEP8 _"Compound statements (multiple statements on the same line) are generally discouraged"_!) – jonrsharpe Apr 28 '23 at 11:41
@t3chb0t you can use _any_ value as a default, [`def`](https://docs.python.org/3/reference/compound_stmts.html#function-definitions) is just a statement for defining a function value. – jonrsharpe Apr 28 '23 at 11:43
@jonrsharpe I do think this PEP8 line can hold in this case. Assigning a default value to a parameter is a "binding" to an identifier. But I get what you are saying and I am not 100% of my opinion. Actually So I will add this post to my answer, thanks again. – Mathias Réus Apr 28 '23 at 11:49
@MathiasRéus I see what you're saying, but flake8's E731 doesn't consider it a violation and a Pylint maintainer says it's [completely fine](https://github.com/pylint-dev/pylint/issues/5976#issuecomment-1079076107). It's a very common usage pattern! – jonrsharpe Apr 28 '23 at 11:53
Yes, I totally agree. I should have put my return line after a new line. – Mathias Réus Apr 28 '23 at 11:54
@jonrsharpe Wait, use a lambda expression for **an argument** is totally ok. In this case, we use it as **a parameter** which is different. So it's not the same use case at all. I do think it's thin ice here about using a lambda in the parameter case. From what I understand, when something is statically defined such as a variable in code body, we do not use lambda. Although, when you use a function such as `sort` with its argument `key`, it's ok to use a lambda function because if you have to repeat the usage of sort, you won't type 20 def. While the default value is typed only once. – Mathias Réus Apr 28 '23 at 11:57

score 1 · Accepted Answer · answered Apr 28 '23 at 11:31

The core of the error is the return type, which is required to be Optional[Dict[str, Any]] but being inferred for the lambda expression as Dict[<nothing>, <nothing>], a permanently empty dictionary.

(<nothing> isn't documented, but it's a "bottom type", you get the same behaviour using Never.)

This is hard to predict, because any simpler example is just fine:

d: Dict[str, Any] = {}  # fine

c: Callable[[Dict[str, Any]], Dict[str, Any]] = lambda _: {}  # fine


def f(on_started: Callable[[Dict[str, Any]], Dict[str, Any]] = lambda _: {}):  # fine
    pass

def foo(on_started: OnStarted = lambda _: {}):  # no thanks
    pass

If you want to keep it inline, you have to give the type system a little more information, e.g.:

def foo(on_started: OnStarted = lambda _: cast(Dict[str, Any], {})):
    pass

Unfortunately this also breaks type safety, as cast(Dict[str, Any], "lol") is accepted too.

Given that the response type is optional, lambda _: None would be a valid and inferrable default.

Assign a default callable value for a callable protocol

2 Answers2