asyncio awaitable object - basic example

Question

I'm trying to understand how to make an awaitable object. The definition from the documentation states:

An object with an __await__ method returning an iterator.

Guided by that definition I wrote the sample code:

import asyncio

async def produce_list():
        num = await Customer()
        print(num)

class Customer(object):

    def __await__(self):
        return iter([1, 2, 3, 4])

loop = asyncio.get_event_loop()
loop.run_until_complete(produce_list())

The flow that I expected was:

1. Event loop gives control to produce_list(). produce_list() gives up execution on num = await Customer().
2. Customer() is executed and returns an iterator. Which because returns the first value in the iterator. Q1: am not clear here why num isn't becoming the iterator itself. Also what is doing a send here?
3. Once the last value the iterator has been reached. num = 4 the execution of the coroutine continues to print(num), and prints the value 4.

What I got:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
~/workspace/dashboard/so_question_await.py in <module>()
     16 
     17 loop = asyncio.get_event_loop()
---> 18 loop.run_until_complete(produce_list())

/usr/lib/python3.5/asyncio/base_events.py in run_until_complete(self, future)
    464             raise RuntimeError('Event loop stopped before Future completed.')
    465 
--> 466         return future.result()
    467 
    468     def stop(self):

/usr/lib/python3.5/asyncio/futures.py in result(self)
    291             self._tb_logger = None
    292         if self._exception is not None:
--> 293             raise self._exception
    294         return self._result
    295 

/usr/lib/python3.5/asyncio/tasks.py in _step(***failed resolving arguments***)
    239                 result = coro.send(None)
    240             else:
--> 241                 result = coro.throw(exc)
    242         except StopIteration as exc:
    243             self.set_result(exc.value)

~/workspace/dashboard/so_question_await.py in produce_list()
      5 
      6 async def produce_list():
----> 7         num = await Customer()
      8         print(num)
      9 

RuntimeError: Task got bad yield: 1

What concepts have I gotten wrong here?

In the end I'm looking for an example that uses iteration through a list as an event to return to the control of the coroutine.

Answer 1

__await__ returns an iterator because the underlying mechanism for coroutines is originally based on the yield from syntax. In practice, __await__ returns either iter(some_future) or some_coroutine.__await__(). It can be used to create objects that produce different values every time they are awaited. See this simple example:

import asyncio
import random

class RandomProducer:

    def __await__(self):
        return self.producer().__await__()

    async def producer(self):
        sleep = random.random()
        value = random.randint(0, 9)
        return await asyncio.sleep(sleep, result=value)

async def main():
    producer = RandomProducer()
    while True:
        print(await producer)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

---

To answer your comments:

Does every coroutine eventually ends up calling asyncio.sleep?

No, and asyncio.sleep is actually not the end of the chain. At the very bottom, it's always a future that is being yielded: the coroutine chain asks the event loop "please wake me up when this future has a result". In the case of asyncio.sleep, it uses loop.call_later to set the result of the future after a given amount of time. The loop provides more methods for scheduling callbacks: loop.call_at, loop.add_reader, loop.add_writer, loop.add_signal_handler, etc.

An asyncio library such as aiohttp. I'm assuming there is some code somewhere that doesn't rely on existence of previous coroutines.

All the IO operations have to end up delegating to the event loop in order to achieve single-threaded concurrency. For instance, aiohttp relies on the loop.create_connection coroutine to manage the TCP connection.

Answer 2

@Vincent's answer is probably what you want, since it will let you use your __await__ method to call more async code.

However, a bit of searching through python's implementation of Tasks shows that the iterators returned by async are supposed to either return None (sending control back to the event loop) or else raise an exception whose value is their return.

So, the following works:

class MyIter:
    def __init__(self):
        self.n = 2
    def __next__(self):
        self.n -= 1
        if self.n >= 0:
            return None # yields control to outer loop
        raise StopIteration(12) # "return" value stored in exc.value

class MyTask:
    def __await__(self):
        return MyIter()

async def test():
    ans = await MyTask()
    print(ans)