Event Driven System Call in Python

Question

I'm trying to implement an event driven process with system call or subprocess. Basically I want to launch a non-blocking system command and upon completion of that system call, I want a function to be called. This is so that I can start a GUI progress bar, launch a system command and have the progress bar continue, and when the system call finishes, have the progress bar stop.

What I want to absolutely NOT DO, is to spawn a process, get its process ID and keep checking for the completion of that process in a while loop.

Below is just an example of how I imagine this should work (All of these are inside a class)

def launchTool(self):

    self.progressbar.config(mode = 'indeterminate')
    self.progressbar.start(20)
    self.launchButton.config(state = 'disabled')
    self.configCombobox.config(state = 'disabled')

    ##  here the "onCompletion" is a pointer to a function
    call("/usr/bin/make psf2_dcf", shell=True, onCompletion = self.toolCompleted)


def onCompletion(self):

    print('DONE running Tool')

    self.progressbar.stop()
    self.launchButton.config(state = 'normal')
    self.configCombobox.config(state = 'normal')

What GUI framework are you using? The right way to do this is somewhat dependent on that. — dano, May 06 '15 at 21:15
@Mario , can you be more specific? I'm researching and it turns out I can spawn a process using "Process" & "Queue". Is that what you mean. something like this: q = Queue() p1 = Process(target=someFunction, args=(q,)) p1.start() — nima_santur, May 06 '15 at 22:21
what is your OS? You don't need threads; you could handle `SIGCHLD` signal (write to an fd to trigger I/O event in the signal handler or use OS interface such as `signalfd` directly (subscribe using `tk.createfilehandler` to listen for I/O events) ). — jfs, May 09 '15 at 15:33
btw, you don't need a `while` loop to poll the subprocess, here's [working code example](https://gist.github.com/zed/4067619) — jfs, May 09 '15 at 16:34
Thanks @J.F.Sebastian . The example works. However, it still relies on a loop to listen to "self.process.poll(), right? I was looking for more like a callback method. Where you specify the system command you want to execute and have it call a function upon completion and pass the return code. Something like this, would be most ideal, but not sure if it's possible. https://docs.python.org/2/library/multiprocessing.html 16.6.2.9. Process Pools .... apply_async — nima_santur, May 11 '15 at 19:40
To avoid polling, you could use a signal handler as I've suggested above. It won't work on Windows. A portable solution is to use threads (or a thread pool: `multiprocessing.pool.ThreadPool`, `concurrent.futures.ThreadPoolExecutor` -- they probably have callback interfaces already). — jfs, May 11 '15 at 20:25

jfs · Accepted Answer · 2015-07-12T12:01:25.903

To avoid polling subprocess' status, you could use SIGCHLD signal on Unix. To combine it with tkinter's event loop, you could use the self-pipe trick. It also workarounds the possible tkinter + signal issue without the need to wake the event loop periodically.

#!/usr/bin/env python3
import logging
import os
import signal
import subprocess
import tkinter

info = logging.getLogger(__name__).info

def on_signal(pipe, mask, count=[0]):
    try:
        signals = os.read(pipe, 512)
    except BlockingIOError:
        return # signals have been already dealt with

    # from asyncio/unix_events.py
    #+start
    # Because of signal coalescing, we must keep calling waitpid() as
    # long as we're able to reap a child.
    while True:
        try:
            pid, status = os.waitpid(-1, os.WNOHANG)
        except ChildProcessError:
            info('No more child processes exist.')
            return
        else:
            if pid == 0:
                info('A child process is still alive. signals=%r%s',
                     signals, ' SIGCHLD'*(any(signum == signal.SIGCHLD
                                              for signum in signals)))
                return
            #+end
            # you could call your callback here
            info('{pid} child exited with status {status}'.format(**vars()))
            count[0] += 1
            if count[0] == 2:
                root.destroy() # exit GUI


logging.basicConfig(format="%(asctime)-15s %(message)s", datefmt='%F %T',
                    level=logging.INFO)
root = tkinter.Tk()
root.withdraw() # hide GUI

r, w = os.pipe2(os.O_NONBLOCK | os.O_CLOEXEC) 
signal.set_wakeup_fd(w) 
root.createfilehandler(r, tkinter.READABLE, on_signal)
signal.signal(signal.SIGCHLD, lambda signum, frame: None) # enable SIGCHLD
signal.siginterrupt(signal.SIGCHLD, False) # restart interrupted syscalls automatically
info('run children')
p = subprocess.Popen('sleep 4', shell=True)
subprocess.Popen('sleep 1', shell=True)
root.after(2000, p.send_signal, signal.SIGSTOP) # show that SIGCHLD may be delivered
root.after(3000, p.send_signal, signal.SIGCONT) # while the child is still alive
root.after(5000, lambda: p.poll() is None and p.kill()) # kill it
root.mainloop()
info('done')

Output

2015-05-20 23:39:50 run children
2015-05-20 23:39:51 16991 child exited with status 0
2015-05-20 23:39:51 A child process is still alive. signals=b'\x11' SIGCHLD
2015-05-20 23:39:52 A child process is still alive. signals=b'\x11' SIGCHLD
2015-05-20 23:39:53 A child process is still alive. signals=b'\x11' SIGCHLD
2015-05-20 23:39:54 16989 child exited with status 0
2015-05-20 23:39:54 No more child processes exist.
2015-05-20 23:39:54 done

Event Driven System Call in Python

1 Answers1

Output

Linked