1

I have an asyncio program with two tasks:

  • task 1 does some work
  • task 2 provides a command line interface (CLI), it reads commands from the user and sends them to task 1 for processing

The CLI is basically a loop reading lines from an asyncio stream connected to the stdin.

It works, but is not very comfortable. The problem is that the input line provides no editing commands except the [BACKSPACE ctrl-H] and [ctrl-U] which are processed at the Linux terminal level, not in the program. I want at least [LEFT] and [RIGHT] arrows and [DEL].

I tried the builtin input() with readline imported which gives me a comfortable editing, but it must be run in a separate thread, because it is blocking (loop.run_in_executor).

Now the problem is the [ctrl-C] handling (KeyboardInterrupt). The tasks are cancelled, but the input() in its own thread still waits for the [ENTER] key. Only after an [ENTER] the application exits. This is so confusing, and it is a "blocker bug" for this approach. Unfortunately, it is not possible to kill a thread, so I am not able to make a [ctrl-C] keypress terminate the program normally.

Do you have an idea how solve the problem with input() or do you know an alternative way how to enable basic input line editing in an asyncio task?

VPfB
  • 14,927
  • 6
  • 41
  • 75
  • see [ptpython and prompt-toolkit](https://github.com/prompt-toolkit/ptpython) – furas Dec 01 '21 at 10:08
  • @furas the prompt-toolkit looks like it has tons of features and far more functions than I need, but it has an async interface and so is definitely something I should test. Thank you for the hint. – VPfB Dec 01 '21 at 14:20
  • I just stumbled upon basically your question by pure chance: (https://stackoverflow.com/q/58454190/14412190). This might be interesting to look at. – thisisalsomypassword Dec 01 '21 at 17:24
  • I tried `aioconsole.ainput` quickly without reading the docs, but got no editing functions. – VPfB Dec 01 '21 at 18:31

1 Answers1

3

Have you considered keeping the input part in the main thread and running the event loop in a child thread? This might be kind of hacky and definitely needs refinement but you could work with what you have:

import asyncio
from threading import Thread

streaming_queue = asyncio.Queue()


async def main_async():
    while True:
        chunk = await streaming_queue.get()
        print("got chunk: ", chunk)


def event_loop(loop):
    try:
        loop.run_until_complete(main_async())
    except asyncio.CancelledError:
        loop.close()
        print("loop closed")


def send_chunk(chunk):
    streaming_queue.put_nowait(chunk)


def cancel_all():
    for task in asyncio.all_tasks():
        task.cancel()


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    event_thread = Thread(target=event_loop, args=(loop,))
    event_thread.start()
    try:
        while True:
            chunk = input()
            loop.call_soon_threadsafe(send_chunk, chunk)
    except KeyboardInterrupt:
        print("cancelling all tasks")
        loop.call_soon_threadsafe(cancel_all)
        print("joining thread")
        event_thread.join()
        print("done")

The best would of course be an async implementation of input. Not sure if something like that is out there or if the link by @furas is a possibility...

thisisalsomypassword
  • 1,423
  • 1
  • 6
  • 17
  • I just realized that "prompt-toolkit" is a separate project and implements an async API. On the first click on @furas link I only saw the REPL stuff... I'm letting my answer stand as it is as a workaround without any further dependencies, but prompt-toolkit seems like the better choice. – thisisalsomypassword Dec 01 '21 at 13:01
  • Interesting, and +1 for the proof of concept, but I'm afraid to make such a big change to the project. The chance to break something is high, e.g. signals are delivered to the main thread. – VPfB Dec 01 '21 at 14:16
  • Understandable. Thanks for the upvote, anyway! – thisisalsomypassword Dec 01 '21 at 16:46