Python - Launch detached process, wait for specific output in stdout, then continue

Question

I currently have the following Python code:

my_process = subprocess.Popen(["cmd.exe", "/c", "unchangable_long_running_script.bat"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in iter(my_process.stdout.readline, b""):
    print(f">>> {line.rstrip()}")
    if b"Setup complete" in line:
        break
print("All done!")

It kicks off a long-running Windows batch script (that I cannot modify) and when a specific phrase is encountered, indicating that all is well, my Python script will continue. So the console output when running the above script is something like:

>>> Doing stuff
>>> Doing more stuff
>>> Setup complete
All done!

The problem that I have, is that I would like for the subprocess to continue beyond the lifetime of my Python script.

In order to achieve this, I can use the flags as described in this answer, but then I run into the problem that DETACHED_PROCESS causes the batch script to be run in a separate console to the parent, as per the documentation. In this case, the stdout is now confined to that new console and is inaccessible to my Python script.

tldr; Is there a way to start a detached process in Python, but still read the stdout from that process in order to determine when the detached process has reached a certain point in its execution?

score 1 · Answer 1 · answered Jun 11 '21 at 16:09

Okay, so I figured out what might be considered a "workaround" for this issue.

The workaround is to launch a subprocess and detach from it, whilst having that subprocess write its output to a file:

import tempfile

creation_flags = subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP | subprocess.CREATE_BREAKAWAY_FROM_JOB

with tempfile.TemporaryFile() as out, tempfile.TemporaryFile() as err:
    out.write(str(f"{datetime.now()}\n").encode("utf-8"))
    subprocess.Popen(["cmd.exe", "/c", "unchangable_long_running_script.bat"], creationflags=creation_flags, stdin=subprocess.DEVNULL,
                     stdout=out, stderr=err)

The file(s) can then be read by the script that launched the subprocess in order to determine when the detached subprocess has reached the required point in its execution.

Python - Launch detached process, wait for specific output in stdout, then continue

1 Answers1