It is possible to implement a Python coroutine in C++, but takes some work. You need to do what the interpreter (in static languages the compiler) normally does for you and transform your async function into a state machine. Consider a very simple coroutine:
async def coro():
x = foo()
y = await bar()
baz(x, y)
return 42
Invoking coro() doesn't run any of its code, but it produces an awaitable object which can be started and then resumed multiple times. (But you don't normally see these operations because they are transparently performed by the event loop.) The awaitable can respond in two different ways: by 1) suspending, or by 2) indicating that it is done.
Inside a coroutine await implements suspension. If a coroutine were implemented with a generator, y = await bar() would desugar to:
# pseudo-code for y = await bar()
_bar_iter = bar().__await__()
while True:
try:
_suspend_val = next(_bar_iter)
except StopIteration as _stop:
y = _stop.value
break
yield _suspend_val
In other words, await suspends (yields) as long as the awaited object does. The awaited object signals that it's done by raising StopIteration, and by smuggling the return value inside its value attribute. If yield-in-a-loop sounds like yield from, you're exactly right, and that is why await is often described in terms of yield from. However, in C++ we don't have yield (yet), so we have to integrate the above into the state machine.
To implement async def from scratch, we need to have a type that satisfies the following constraints:
- doesn't do much when constructed - typically it will just store the arguments it received
- has an
__await__ method that returns an iterable, which can just be self;
- has an
__iter__ which returns an iterator, which can again be self;
- has a
__next__ method whose invocation implements one step of the state machine, with return meaning suspension and raising StopIteration meaning finishing.
The above coroutine's state machine in __next__ will consist of three states:
- the initial one, when it invokes the
foo() sync function
- the next state when it keeps awaiting the
bar() coroutine for as long as it suspends (propagating the suspends) to the caller. Once bar() returns a value, we can immediately proceed to calling baz() and returning the value via the StopIteration exception.
- the final state which simply raises an exception informing the caller that the coroutine is spent.
So the async def coro() definition shown above can be thought of as syntactic sugar for the following:
class coro:
def __init__(self):
self._state = 0
def __iter__(self):
return self
def __await__(self):
return self
def __next__(self):
if self._state == 0:
self._x = foo()
self._bar_iter = bar().__await__()
self._state = 1
if self._state == 1:
try:
suspend_val = next(self._bar_iter)
# propagate the suspended value to the caller
# don't change _state, we will return here for
# as long as bar() keeps suspending
return suspend_val
except StopIteration as stop:
# we got our value
y = stop.value
# since we got the value, immediately proceed to
# invoking `baz`
baz(self._x, y)
self._state = 2
# tell the caller that we're done and inform
# it of the return value
raise StopIteration(42)
# the final state only serves to disable accidental
# resumption of a finished coroutine
raise RuntimeError("cannot reuse already awaited coroutine")
We can test that our "coroutine" works using real asyncio:
>>> class coro:
... (definition from above)
...
>>> def foo():
... print('foo')
... return 20
...
>>> async def bar():
... print('bar')
... return 10
...
>>> def baz(x, y):
... print(x, y)
...
>>> asyncio.run(coro())
foo
bar
20 10
42
The remaining part is to write the coro class in Python/C or in pybind11.
