Python 3 subprocess pipe blocks

Question

In python 2.7, this works and returns the expected string it works!

process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
process.stdin.write('echo it works!\n')
print process.stdout.readline()

When I know try this in python 3.4 it gets stuck at the readline command

process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
process.stdin.write(bytes('echo it works!\n','UTF-8'))
print(process.stdout.readline().decode('UTF-8'))

May be this helps, http://stackoverflow.com/questions/7585435/best-way-to-convert-string-to-bytes-in-python-3 — Rahul, Nov 20 '15 at 16:09
I was surprised to find that adding `process.stdin.flush()` fixes the problem. I thought it would be line buffered. — tdelaney, Nov 20 '15 at 16:25
Check [this](http://stackoverflow.com/questions/163542/python-how-do-i-pass-a-string-into-subprocess-popen-using-the-stdin-argument) and [this](http://stackoverflow.com/questions/8652767/python3-subprocess-communicate-example). Also @tdelaney's comment... — Remi Guan, Nov 20 '15 at 16:27
thanks for your help. the `process.stdin.flush()` works. But the answer is a little more elegant than always have to flush it. — Rolf Lussi, Nov 20 '15 at 23:46

score 3 · Accepted Answer · edited Jun 20 '20 at 09:12

The hint about buffering is helpfull. The following information can be found in the Subprocess library module documentation:

bufsize will be supplied as the corresponding argument to the open() function when creating the stdin/stdout/stderr pipe file objects:

0 means unbuffered (read and write are one system call and can return short)

1 means line buffered (only usable if universal_newlines=True i.e., in a text mode)

and

If universal_newlines is False the file objects stdin, stdout and stderr will be opened as binary streams, and no line ending conversion is done.

If universal_newlines is True, these file objects will be opened as text streams in universal newlines mode using the encoding returned by locale.getpreferredencoding(False)

Putting it all together gives the following Python3 code:

import subprocess
process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE, bufsize=1, universal_newlines=True)
process.stdin.write('echo it works!\n')
print(process.stdout.readline())

Python 3 subprocess pipe blocks

1 Answers1