Pipe two subprocesses in Python with asyncio and interrupt them

Question

I have two subprocesses launched by my asyncio-based Python code with asyncio.create_subprocess_exec. I want one to output data to stdout and the other one to read it through a pipe as soon as it's available (in my case, the first process outputs video frames and the second process encodes them - it's ffmpeg). The only way to create a pipe usable by asyncio that I found is in this post. But following that method the second process (ffmpeg) crashes with the error "Bad file descriptor". Given that the setup works fine when using a shell pipe, I decided to use asyncio.create_subprocess_shell to handle the piping for me. But both processes are designed to run forever until interruption and so I need a way to terminate them. Calling .terminate() on the process returned by create_subprocess_shell, or killing it with SIGTERM or SIGINT do nothing. How can I kill these processes?

Answer 1

The reason that .terminate() doesn't do anything is that the Bash process intercepts the signals by default and doesn't propagate them to the running command. The workaround to that is getting the pid from the process object, and using psutil to get all children of that process, and terminate them one by one which then kills the shell itself.