Precise specification of __await__

Question

The Python Language Reference specifies object.__await__ as follows:

object.__await__(self)

Must return an iterator. Should be used to implement awaitable objects. For instance, asyncio.Future implements this method to be compatible with the await expression.

That's it. I find this specification very vague and not very specific (ironically). Ok, it should return an iterator, but can it be an arbitrary iterator? Obviously not:

import asyncio


class Spam:
    def __await__(self):
        yield from range(10)


async def main():
    await Spam()


asyncio.run(main())

RuntimeError: Task got bad yield: 0

I'm assuming the asyncio event loop expects a specific kind of object being yielded by the iterator. Then what exactly should it yield? (And why isn't this documented?)

---

Edit: as far as I can see, this isn't documented anywhere. But I've been investigating on my own, and I think that the key to understanding what objects the asyncio expects its coroutines to yield lies in task_step_impl in _asynciomodule.c.

---

Update: I've made a PR to the cpython repository with the aim of clarifying this: "Clarify the vague specification of object.__await__".

The PR has now been merged and should be available in the docs for Python 3.10+.

Answer 1

The language doesn't care which iterator you return. The error comes from a library, asyncio, which has specific ideas about the kind of values that must be produced by the iterator. Asyncio requires __await__ to produce asyncio futures (including their subtypes such as tasks) or None. Other libraries, like curio and trio, will expect different kinds of values. Async libraries by and large don't document their expectations from __await__ because they consider it an implementation detail.

As far as asyncio is concerned, you're supposed to be using higher-level constructs, such as futures and tasks, and await those, in addition to coroutines. There is rarely a need to implement __await__ manually, and even then you should use it to delegate the signals of another awaitable. Writing an __await__ that creates and yields a fresh suspend-value of its own requires it to be coupled with the event loop and have knowledge of its internals.

You can think of __await__ as a tool to write a library similar to asyncio. If you are the author of such a library, the current specification is sufficient because you can yield whatever you like from the iterator, only the code in your event loop will observe the yielded values. If you're not in that position, you probably have no need to implement __await__.

Answer 2

Tasks can only wait on other tasks / futures. From the CPython source code:

    /* Check if `result` is FutureObj or TaskObj (and not a subclass) */
    /* ... */

    /* Check if `result` is None */
    /* ... error */

    /* Check if `result` is a Future-compatible object */
    /* ... */

    /* Check if `result` is a generator */
    /* ... */

    /* The `result` is none of the above */
    o = task_set_error_soon(
        task, PyExc_RuntimeError, "Task got bad yield: %R", result);
    Py_DECREF(result);
    return o;

---

Edit: If I understand correctly this restriction is only imposed on tasks, and normal futures can wait on any iterable returned from __await__, though the point is probably that the iterable returned yields to the event loop, then ultimately ends up returning a result.