Subprocess Popen stdout is empty if the subprocess is a Python script

Question

Here is some code that should print numbers 0 through 4 to the terminal: (adapted from https://stackoverflow.com/a/59291466/4391249)

import os
import time
import subprocess

cmd = 'python', '-c', 'import time; [(print(i), time.sleep(1)) for i in range(5)]'
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
os.set_blocking(p.stdout.fileno(), False)
start = time.time()
while True:
    # first iteration always produces empty byte string in non-blocking mode
    line = p.stdout.readline()
    if len(line):
        print(line)
    if time.time() > start + 5:
        break
p.terminate()

But it doesn't for me. For me, nothing is printed.

When I instead set cmd = 'ls', it does produce the expected output (prints the contents of my working directory).

Why doesn't the Python one work?

I'm on Ubuntu20.04 with Python3.10.

score 1 · Accepted Answer · answered Feb 23 '23 at 15:10

Disable output buffering using -u cli flag:

import os
import time
import subprocess

cmd = [
    '/Users/xyz/.pyenv/shims/python', '-u', '-c',
    'import time; [(print(i), time.sleep(1)) for i in range(5)]'
]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
os.set_blocking(p.stdout.fileno(), False)
start = time.time()
while True:
    # first iteration always produces empty byte string in non-blocking mode
    line = p.stdout.readline()
    if len(line):
        print(line)
    if time.time() > start + 5:
        break
p.terminate()

Out:

b'0\n'
b'1\n'
b'2\n'
b'3'
b'\n'
b'4\n'

Solved! Btw is this something new in later versions of Python? No one on the linked question seemed to have my issue. — Alexander Soare, Feb 23 '23 at 15:16

Subprocess Popen stdout is empty if the subprocess is a Python script

1 Answers1