Executing bash with subprocess.Popen

Question

I'm trying to write a wrapper for a bash session using python. The first thing I did was just try to spawn a bash process, and then try to read its output. like this:

from subprocess import Popen, PIPE
bash = Popen("bash", stdin = PIPE, stdout = PIPE, stderr = PIPE)
prompt = bash.stdout.read()
bash.stdin.write("ls\n")
ls_output = bash.stdout.read()

But this does not work. First, reading from bash's stdout after creating the process fails, and when I try to write to stdin, I get a broken pipe error. What am I doing wrong?

Just to clarify again, I'm not interested in running a single command via bash and then retrieving its output, I want to have a bash session running in some process with which I can communicate via pipes.

Sounds like you need an Expect module: http://www.noah.org/wiki/pexpect#Q:_Why_not_just_use_a_pipe_.28popen.28.29.29.3F — glenn jackman, Aug 26 '11 at 15:42
OK, I think I found what I needed. Python has a module named pty, with a fork which spawns a child process in a terminal-like environment and returns a read/write fd to it. I can then exec bash in the child. — Dany Zatuchna, Aug 26 '11 at 19:32

score 3 · Answer 1 · answered Aug 26 '11 at 14:53

3

This works:

import subprocess
command = "ls"
p = subprocess.Popen(command, shell=True, bufsize=0, stdout=subprocess.PIPE, universal_newlines=True)
p.wait()
output = p.stdout.read()
p.stdout.close()

answered Aug 26 '11 at 14:53

Gabriel Ross

5,168
1
28
30

2

Ah, but I'm not trying to run a single command. I want the bash process to stay alive while I'm interacting with it. – Dany Zatuchna Aug 26 '11 at 15:03
You might be able to spawn one process after the other, analysing the output of the previous one. What sequence of steps/interactions did you have in mind? – Gabriel Ross Aug 26 '11 at 16:24
Note that this running in 'sh' not 'bash'. There are annoying differences, like 'echo -n Hello' does not work the same way. – Charles Merriam Jul 12 '14 at 05:04

score 0 · Answer 2 · edited May 23 '17 at 12:00

You can refer to my another answer: https://stackoverflow.com/a/43012138/3555925, which use pseudo-terminal and select in handle stdin/stdout.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import sys
import select
import termios
import tty
import pty
from subprocess import Popen

command = 'bash'
# command = 'docker run -it --rm centos /bin/bash'.split()

# save original tty setting then set it to raw mode
old_tty = termios.tcgetattr(sys.stdin)
tty.setraw(sys.stdin.fileno())

# open pseudo-terminal to interact with subprocess
master_fd, slave_fd = pty.openpty()

# use os.setsid() make it run in a new process group, or bash job control will not be enabled
p = Popen(command,
          preexec_fn=os.setsid,
          stdin=slave_fd,
          stdout=slave_fd,
          stderr=slave_fd,
          universal_newlines=True)

while p.poll() is None:
    r, w, e = select.select([sys.stdin, master_fd], [], [])
    if sys.stdin in r:
        d = os.read(sys.stdin.fileno(), 10240)
        os.write(master_fd, d)
    elif master_fd in r:
        o = os.read(master_fd, 10240)
        if o:
            os.write(sys.stdout.fileno(), o)

# restore tty settings back
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_tty)

score 0 · Answer 3 · edited May 23 '17 at 11:46

0

The read() call is blocking, so it reads the output of the first command and hangs because further data could still arrive (ie. bash is still running). To perform a non-blocking read, see Non-blocking read on a subprocess.PIPE in python.

edited May 23 '17 at 11:46

Community

1
1

answered Aug 26 '11 at 18:58

a3nm

8,717
6
31
39

score 0 · Answer 4 · answered Aug 26 '11 at 19:06

0

Why don't you just use the cmd module/raw_input along with subprocess.Popen(shell = True)?

answered Aug 26 '11 at 19:06

Jonathan Sternberg

6,421
7
39
58

Executing bash with subprocess.Popen

4 Answers4

Linked