How to start, stop and get output from another python program

Question

I would like to be able to start, stop and get the console output of another python program. I don't want to run it within the current program, I want them to run as two separate instances.

Both of the files are in the same directory.

I have seen guides on how to get the console output of another python program, as well as how to start and stop it, but I haven't been able to find a guide on both.

I should also note that the file I want the output from is a .pyw file.

Thanks.

EDIT: Not a duplicate, it is not that simple...

EDIT2:

Here is my attempt

main.py

import subprocess

p = subprocess.Popen(['python', 'sub.py'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
# continue with your code then terminate the child
with p.stdout:
    for line in iter(p.stdout.readline, b''):
        print(line)
p.wait()

sub.py

import time

for i in range(100):
    print(i)
    time.sleep(1)

It sort of works, but it prints it like

b'0\r\n' b'1\r\n' b'2\r\n' b'3\r\n' b'4\r\n'

EDIT3:

import subprocess

p = subprocess.Popen(['python', 'sub.py'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
with p.stdout:
    for line in iter(p.stdout.readline, b''):
        print(line.decode("utf-8").replace("\r\n", ""))
p.wait()

Is this the best way?

However, I am still having issues. I want to run the program completely separately, and so I should be able to run other code in the main.py program at the same time, but that is not working.

import subprocess
import time


def get_output():
    p = subprocess.Popen(['python', 'sub.py'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    with p.stdout:
        for line in iter(p.stdout.readline, b''):
            print(line.decode("utf-8").replace("\r\n", ""))
    p.wait()


def print_stuff():
    for i in range(100):
        print("." + str(i))
        time.sleep(1)


if __name__ == "__main__":
    get_output()
    print_stuff()

import time


def main():
    for i in range(100):
        print(i)
        time.sleep(1)


if __name__ == "__main__":
    main()

EDIT4: Here is my attempt at running them both at the same time

import subprocess
import asyncio


async def get_output():
    p = subprocess.Popen(['python', 'sub.py', 'watch', 'ls'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    with p.stdout:
        for line in iter(p.stdout.readline, b''):
            print(line.decode("utf-8").replace("\r\n", ""))
    p.wait()


async def print_stuff():
    for i in range(100):
        print("." + str(i))
        await asyncio.sleep(1)


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(asyncio.gather(
        print_stuff(),
        get_output()
    ))
    loop.close()

import asyncio


async def main():
    for i in range(100):
        print(i)
        await asyncio.sleep(1)

if __name__ == "__main__":
    asyncio.run(main())

The expected output is

.0 0 .1 1 .2 2 ...

But the output is

.0 0 1 2 3 4 5 ...

EDIT5: I believe the issue is that subprocess.Popen holds the program, so I think I need to use asyncio.create_subprocess_execm but I can't figure out exactly how to get that to work.

"I have seen guides on how to get the console output of another python program, as well as how to start and stop it, but I haven't been able to find a guide on both." -- Really? You know how to do A, and you know how to do B, but you can't figure out to do A then B? — Blorgbeard, Jun 03 '19 at 17:22
I mean, can you explain more clearly how far you have gotten, and exactly where you are stuck? — Blorgbeard, Jun 03 '19 at 17:22

score 1 · Answer 1 · answered Jun 03 '19 at 17:57

Here's how I did it when I asked a similar question. I also needed the output as soon as it was printed, not waiting for a buffer to fill.

CHILD PROCESS

from time import sleep

# Dummy child process for popen demonstration
# Print "sleeping" at 1-sec intervals, then a "QUIT" message.

for _ in range(5):
    print(_, "sleeping")
    sleep(1)

print("Child process finishes")

PARENT PROCESS

import subprocess
import time

# Turn off output buffering in Python;
#   we need each output as soon as it's printed.
environ["PYTHONUNBUFFERED"] = "1"

# Locate the child's source file -- YOU can use a local path.
branch_root = environ["PYTHONPATH"]
cmd = ["/usr/bin/python3", branch_root + "popen_child.py"]

# Start the child process, opening a pipe from its stdout
# Include stderr under stdout.
test_proc = subprocess.Popen(
    cmd,
    universal_newlines=True,
    stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT,
)

print(time.time(), "START")

# Catch and print the output as it's generated.
for out_data in iter(test_proc.stdout.readline, ""):
    print(time.time(), out_data, end="")

print("TEST completed")

Turning the "listening" on and off is a matter of attention in the parent process.

That is helpful, thank you. However, I realised that it does not by default allow me to run other code at the same time, and that is the only reason I'm running it from another file - I want them to essentially run independently. I edited my post again to add some clarification. — J P, Jun 03 '19 at 18:00
I just posted my own answer, do you think that is unbuffered as well? Or do I need to make some changes? — J P, Jun 03 '19 at 19:01

score 0 · Answer 2 · answered Jun 03 '19 at 17:40

sorry for the initial comment.
I know a method but I'm not sure it works on all operating systems. It involves the subprocess module:

import subprocess
import os
#another python file with possible inputs
command = "python3 test2.py"
#get current file dir as thats where test2.py is located
fileDir = os.path.dirname(os.path.realpath(__file__))
#gets outputs printed on the terminal
process = subprocess.check_output(command.split(), cwd=fileDir)
print(process)

This method also ensures the second scripts is done before continuing the main code.
Edit: Output is in byte form, decode with something like process.decode('utf-8')
More here

Sorry I wasn't very clear, but I need it to give a constant live update of the output of the other program, but I would also like to run other code at the same time. — J P, Jun 03 '19 at 17:51

How to start, stop and get output from another python program

2 Answers2