Python: Run async from sync from async

Question

My Python program is mostly async, but I have one critical third-party library that is purely synchronous and has a callback. I need to be able to use an async function as the callback (and just let it block, which is fine). I've tried a few ways and cannot get it to work properly.

from asgiref.sync import async_to_sync
from dataclasses import dataclass
from typing import Callable
import asyncio

# Third party library with synchronous callback.
# Library is not editable and cannot be made async.
@dataclass
class ThirdPartyLibrary:
    callback: Callable[[str], None]

    def run(self):
        self.callback('hi')

# My callback to trigger; must be async
async def async_callback(x: str):
    # E.g., write result to database; must be async
    await asyncio.sleep(1)
    print(x)

# My program that uses the third-party library.
# It's already running async, but uses ThirdPartyLibrary which is purely sync.
# How do I set my async_callback to be the callback for my library?
async def main():
    third_party = ThirdPartyLibrary(callback=lambda x: async_to_sync(async_callback)(x)) # Doesn't work
    third_party.run()

asyncio.run(main())

What happens when you run this? What does "doesn't work" mean? Are you sure that the callback is the problem? Does `ThirdPartyLibrary` work as you expect if you don't supply a callback or supply a fully synchronous callback? — CryptoFool, Mar 11 '22 at 19:07

Mikhail Gerasimov · Answer 1 · 2022-03-12T03:25:01.683

If I understand the task correctly, you may want to utilize asyncio.Future - a "bridge" between callback-based and coroutine-based world. Depending on if third_party.run is blocking you may either call it directly or need to run it in a thread using asyncio.to_thread():

from dataclasses import dataclass
from typing import Callable
import asyncio


@dataclass
class ThirdPartyLibrary:
    callback: Callable[[str], None]

    def run(self):
        self.callback("hi")


async def async_callback(x: str):
    await asyncio.sleep(1)
    print(x)


async def main():
    fut = asyncio.Future()

    third_party = ThirdPartyLibrary(callback=lambda x: fut.set_result(x))
    await asyncio.to_thread(third_party.run)  # you can wrap this to asyncio.create_task if you expect this to run after the callback

    result = await fut
    print(result)


asyncio.run(main())

Python: Run async from sync from async

1 Answers1