Can't pass awaitable to asyncio.run_coroutine_threadsafe

Question

I have observed that the asyncio.run_coroutine_threadsafe function does not accept general awaitable objects, and I do not understand the reason for this restriction. Observe

import asyncio


async def native_coro():
    return


@asyncio.coroutine
def generator_based_coro():
    return


class Awaitable:
    def __await__(self):
        return asyncio.Future()


loop = asyncio.get_event_loop()

asyncio.run_coroutine_threadsafe(native_coro(), loop)
asyncio.run_coroutine_threadsafe(generator_based_coro(), loop)
asyncio.run_coroutine_threadsafe(Awaitable(), loop)

Running this with Python 3.6.6 yields

Traceback (most recent call last):
  File "awaitable.py", line 24, in <module>
    asyncio.run_coroutine_threadsafe(Awaitable(), loop)
  File "~/.local/python3.6/lib/python3.6/asyncio/tasks.py", line 714, in run_coroutine_threadsafe
    raise TypeError('A coroutine object is required')
TypeError: A coroutine object is required

where line 24 is asyncio.run_coroutine_threadsafe(Awaitable(), loop).

I know I can wrap my awaitable object in a coroutine defined like

awaitable = Awaitable()

async def wrapper():
    return await awaitable

asyncio.run_coroutine_threadsafe(wrapper(), loop)

however my expectation was that the awaitable would be a valid argument directly to run_coroutine_threadsafe.

My questions are:

1. What is the reason for this restriction?
2. Is the wrapper function defined above the most conventional way to pass an awaitable to run_coroutine_threadsafe and other APIs that demand an async def or generator-defined coroutine?

Answer 1

What is the reason for this restriction?

Looking at the implementation, the reason is certainly not technical. Since the code already invokes ensure_future (rather than, say, create_task), it would automatically work, and work correctly, on any awaitable object.

The reason for the restriction can be found on the tracker. The function was added in 2015 as a result of a pull request. In the discussion on the related bpo issue the submitter explicitly requests the function be renamed to ensure_future_threadsafe (in parallel to ensure_future) and accept any kind of awaitable, a position seconded by Yury Selivanov. However, Guido was against the idea:

I'm against that idea. I don't really see a great important future for this method either way: It's just a little bit of glue between the threaded and asyncio worlds, and people will learn how to use it by finding an example.

[...]

But honestly I don't want to encourage flipping back and forth between threads and event loops; I see it as a necessary evil. The name we currently have is fine from the POV of someone coding in the threaded world who wants to hand off something to the asyncio world.

Why would someone in the threaded world have an asyncio.future that they need to wait for? That sounds like they're mixing up the two worlds -- or they should be writing asyncio code instead of threaded code.

There are other comments in a similar vein, but the above pretty much sums up the argument.

Is the wrapper function defined above the most conventional way to pass an awaitable to run_coroutine_threadsafe and other APIs that demand an async def or generator-defined coroutine?

If you actually need a coroutine object, something like wrapper is certainly a straightforward and correct way to get one.

If the only reason you're creating the wrapper is to call run_coroutine_threadsafe, but you're not actually interested in the result or the concurrent.futures.Future returned by run_coroutine_threadsafe, you can avoid the wrapping by calling call_soon_threadsafe directly:

loop.call_soon_threadsafe(asyncio.ensure_future, awaitable)