How to make subprocess in python print the outputs immediately?

Question

I have a simple function for executing terminal commands and it looks something like this:

import subprocess

def func(command):
    subprocess.run(command, shell=True)

func('python my_python_program.py --par my_parameter_file.py')

The function indeed executes the command, but there should be some output/print statements coming from the program I am executing. The program takes about 10 min to execute and creates a lot of print statements. They get displayed when I just execute the command in a terminal, but subprocess doesn't display them.

How can I get the output to be printed out when using subprocess? Can you provide me with 2 different options - 1) if I want the output just printed on the screen, each line immediately, not everything when the command is completely done ; 2) if I want the output to be saved into a file, also immediately line after line? Tnx!

Does this answer your question? [python getoutput() equivalent in subprocess](https://stackoverflow.com/questions/6657690/python-getoutput-equivalent-in-subprocess) — C.Nivs, Aug 18 '23 at 11:32
@C.Nivs Thanks, but no. I corrected my answer to explain why - my command executes a program, that takes about 10 min to execute and the program produces a lot of print statements. So I don't want to have the print statements displayed after the program is done running, I want them to appear immediately on the screen. Or as a second option - to be immediately saved onto a file. Does this male sense? Thanks for any further help — NeStack, Aug 18 '23 at 13:49
That makes sense, sounds like what you really want is a non-blocking read of the stdout from that process so you don't have to wait for it to end. [This question](https://stackoverflow.com/q/375427/7867968) might be a good start — C.Nivs, Aug 18 '23 at 15:11
@C.Nivs Thanks a lot! In your second link I found an amazing package that in a simple way does exactly what I want! — NeStack, Aug 18 '23 at 17:01

score 0 · Accepted Answer · answered Aug 18 '23 at 17:00

Based on the second link by C.Nivs I found an answer in another post that solves my issue perfectly by using a new, free package called comman_runner. Here is how my code would look like and would produce my desired screen output:

# pip install command_runner # install the package
from command_runner import command_runner # import the package

def func(command):
    exit_code, output = command_runner(command, shell=True, live_output=True)

func('python my_python_program.py --par my_parameter_file.py')

How to make subprocess in python print the outputs immediately?

1 Answers1