2

I'd like to use asyncio to continuously read from a process until I stop it some time later. I've tried tons of things but can't figure out how to do it.

Here's one attempt that just doesn't seem to care about the kill at all

import io
import asyncio

class Foo():

    async def run(self):
        proc = await asyncio.create_subprocess_shell('ping www.google.de', stdout=asyncio.subprocess.PIPE)
        await asyncio.gather(self.run_proc(proc), self.kill_proc(proc))

    async def run_proc(self, proc):
        while True:
            await asyncio.sleep(0)
            line = await proc.stdout.readline()
            print(line)

    async def kill_proc(self, proc):
        await asyncio.sleep(1000)
        print("meh")
        proc.kill()


f = Foo()
loop = asyncio.get_event_loop()
loop.run_until_complete(f.run())

UPDATE: Here's the working code incorporating the feedback from user4815162342

import io
import asyncio

class Foo():

    async def run(self):
        proc = await asyncio.create_subprocess_exec('ping', 'www.google.de', stdout=asyncio.subprocess.PIPE)
        await asyncio.gather(self.run_proc(proc), self.kill_proc(proc))

    async def run_proc(self, proc):
        while True:
            line = await proc.stdout.readline()

            if line == b'':
                break

            print(line)

    async def kill_proc(self, proc):
        await asyncio.sleep(1)
        print("meh")
        proc.kill()


f = Foo()
loop = asyncio.get_event_loop()
loop.run_until_complete(f.run())
Christoph
  • 26,519
  • 28
  • 95
  • 133
  • Is there really no better way to check if you've reached the end of stdout than `if line == b'':`? – Tom May 04 '21 at 17:31

1 Answers1

1

Here's one attempt that just doesn't seem to care about the kill at all

Are you actually patient enough to wait for 16 minutes and 40 seconds for the kill to be called? If I change await asyncio.sleep(1000) to await asyncio.sleep(2), the process does get killed in two seconds.

EDIT I've now tried it on a different system, and reproduced the issue. It turns out that proc.kill() kills the shell process (that waits for ping to finish), but not the ping process itself. A cross-platform workaround (that you've already discovered) is to use create_subprocess_exec, which avoids the shell.

Additionally:

  • when ping is killed, run_proc ends up in an infinite loop because the loop is missing something like if line == b'': break.

  • you don't need to await asyncio.sleep(0); the readline() is already a coroutine which will pass the control to the event loop while waiting.

user4815162342
  • 141,790
  • 18
  • 296
  • 355
  • Oops, seems I confused seconds with milliseconds there. However, it's weird because I do see the `meh` printed so I assume the `proc.kill()` is hit but it just keeps on printing pings (which do arrive in real time for me) – Christoph Jul 18 '18 at 19:31
  • Ah, turns out that's caused by this. https://stackoverflow.com/questions/4789837/how-to-terminate-a-python-subprocess-launched-with-shell-true It's working now. Thanks! – Christoph Jul 18 '18 at 19:36
  • @Christoph Edited the answer to cover the new findings. – user4815162342 Jul 19 '18 at 07:35