1

I'm considering using asyncio in a project. The project uses 3rd party libraries that will create pthreads, and also call back into my code.

Changing the project to use asyncio would require a somewhat large rewrite, so I cannot just "try it out", I'd need some information beforehand.

Is it safe to use asyncio and an event loop with third party libraries that use pthreads and callbacks into your code?

1 Answers1

1

Inside asyncio's coroutine you can use any function if it runs relatively fast (< 0.05 sec.), otherwise you risk to block event loop and through this get significant performance degradation.

If function runs relatively slow it still can be used inside asycnio's coroutine without side-effects, but only inside executor.

If function returns fast and later calls callback - it's a good situation. Such function/callback can be cast to nice asyncio's awaitable using asyncio.Future.

Note also that many asyncio's objects are not thread-safe by default.


Long story short, I don't see reason why asyncio can't be safely used with third-party library that uses threads if everything is implemented right.

But before you rewrite something in large code base you should know for certain why you need asyncio and, IMHO, have some experience with it on lesser code base.

Try to take single callback-based function and cast it to coroutine with asyncio.Future. Try to execute multiple such coroutines simultaneously. See if you achieve what you want and if everything is going smoothly. Keep going this way.

Wrapping existing code to use asyncio instead of rewriting sounds like a good idea: you can do it iteratively and only for parts you use.

Mikhail Gerasimov
  • 36,989
  • 16
  • 116
  • 159
  • 1
    A word of caution: if the library invokes the callback **from a different thread** (as async libraries that use threads typically do), it is forbidden to call `future.set_result()` from the callback, or to do anything else asyncio-related. Instead, one must call `loop.call_soon_threadsafe(future.set_result, )` to ensure that asyncio data structures are accessed only from the thread that runs the asyncio event loop. – user4815162342 May 01 '19 at 10:03