1

Currently, asyncio Event Loop supports non-blocking work with network sockets, files, subprocesses, and signals. For other operations, documentations describes executing code in thread or process pools, but that seems much less efficient (threads vs coroutines).

Is there a way to introduce additional low-level non-blocking operations, e.g. wrapping I/O-heavy library calls? What primitives must be implemented?

Andrey Paramonov
  • 409
  • 4
  • 11
  • Can you give an example of the kind of low-level call you'd like to introduce? Is the underlying call asynchronous itself? If so, does it provide a notification mechanism or is the caller expected to poll it for completion? – user4815162342 Oct 28 '18 at 20:10
  • No, the underlying library call is blocking. I'm thinking of the non-blocking interface extension to introduce, so it's compatible with asyncio. For the sake of definiteness, let's consider [Windows Overlapped Serial port communication](https://msdn.microsoft.com/ru-ru/library/ff802693.aspx#serial_topic4) as example of non-blocking interface. – Andrey Paramonov Oct 29 '18 at 07:36
  • 1
    If the underlying library is blocking and you want to wrap it (rather than reimplement it), then threads might be your only option. – user4815162342 Oct 29 '18 at 08:33
  • I have control over the library code, so I can extend the interface with non-blocking calls. But I totally miss what calls are expected by `asyncio`. – Andrey Paramonov Oct 29 '18 at 08:53

1 Answers1

1

Like most event loops, the asyncio event loop is built around polling IO sources, file descriptors on Unix and file handles on Windows. Poll or select is an IO operation that efficiently monitors multiple file descriptors, suspending the current thread until something interesting happens, e.g. new data arrives. Optionally, polling accepts a timeout, so execution can continue even if there is no new IO event. For a more detailed treatment of the topic, refer to this SO answer and other answers to the same question.

The poll-based design allows asyncio to natively support two kinds of events:

  • IO possible
  • timeout expired

All other types of events must be expressed in terms of those two. For example, call_soon_threadsafe wakes up the event loop by writing into an internal pipe monitored by the event loop. The implementation of run_in_executor uses call_soon_threadsafe to inform asyncio that the synchronous function has completed.

To connect an entirely different polling mechanism to asyncio, one would typically spawn a thread dedicated to that kind of polling, and use call_soon_threadsafe to inform asyncio of new events. This answer, for example, shows how to connect the raw multiprocessing pool (which can be terminated) to asyncio.

user4815162342
  • 141,790
  • 18
  • 296
  • 355