Python: get start and finish time of subprocess

Question

I am running many subprocesses from python flask server via subprocess.Popen() and am saving reference to each one into dictionary. Server is not waiting until subprocess finishes. After some time (5sec, 5min, etc) I try to verify if subprocess finished - via .poll() function. Everything works as expected but now I need to add to it functionality of saving timestamps when was process started (this is no problem) and when was process finished (this is problem) How can I get exact finish timestamp from Popen instance? Since I am checking process after some time it could be that it finished earlier then I check. I tried searching here but it seems Popen instances do not keep this value? https://docs.python.org/3/library/subprocess.html#subprocess.Popen

Thank you for you time and help

I tried to check log of every subprocess for last modified time, but this seems inefficient to me as it could happen that not every subprocess will create log file.

Veysel Olgun · Answer 1 · 2022-11-22T08:13:04.023

You can use os.wait() to wait for any child process that created by your process

os.wait() described as following in the doc

Wait for completion of a child process, and return a tuple containing its pid and exit status indication: a 16-bit number, whose low byte is the signal number that killed the process, and whose high byte is the exit status (if the signal number is zero); the high bit of the low byte is set if a core file was produced.

Availability: Unix, not Emscripten, not WASI.

When any of child process terminated, os.wait() will return pid and exit status of terminated child process

Here is a simple example :

import subprocess as sp
import os,time
from threading import Thread

def wait_for_child():
    while not cancel:
        
        try:
            info = os.wait()
            procs_finish_time[info[0]] = time.time()

        except ChildProcessError:
            time.sleep(0.01)
            continue


procs = []
procs_finish_time = {}
cancel = False

Thread(target=wait_for_child).start()
time.sleep(.1) # to be sure that thread started before the child processes start

for i in range(3):
    procs.append(sp.Popen(["python3","b.py"]))

time.sleep(2) # we assume that b.py process finished
print(procs_finish_time)
cancel = True

You can modify it for your case, after doing procs[1].poll() != None you can check procs[1] termination timestamp info

For Windows solution, you can check this

Python: get start and finish time of subprocess

1 Answers1