Interacting with bash through python without terminating

Question

I am trying to create a process that interacts with the /bin/bash shell and get information or execute commands. I do not want to close the process as I want to preserve the state for future interactions.

I tried the solution on the SO answer but the process is terminated after one communication.

Going through the python subprocess API, I find that subprocess.check_output() is perfect for this. However, similar to Popen.communicate() this kills the process after one execution.

The last approach that seems to work but is unsafe and results in deadlocks is using Popen.stdin.write and Popen.stdout.read. The read operation goes into an infinite wait if an EOF is not found.

Is there a safer and simpler way to achieve this?

EDIT: More stuff I tried:

import subprocess
import select
import os

process = subprocess.Popen(['/bin/bash'], shell=True, text=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
process.stdin.write('export SOMETHING=10000abc\n')
process.stdin.flush()
process.stdin.write('echo $SOMETHING\n')
process.stdin.flush()

r, w, e = select.select([process.stdout], [process.stdin], [], 0)
if process.stdout in r:
  print (os.read(process.stdout.fileno(), 50))
else:
  print ("Nothing to read")

The output for this is always Nothing to read

readline() faces the same issue as read(), it can result in a deadlock. So far I am thinking of adding a timeout on the read operations to break out of the deadlock. — Kyuubi, Oct 10 '19 at 22:32
The problem is probably due to stdio buffering when standard output is a pipe. This is why you should use `pexpect`, it works around this by using a pty. — Barmar, Oct 11 '19 at 00:44

score 1 · Answer 1 · answered Oct 11 '19 at 02:11

I ended up doing the following:

import subprocess
import select
import os

process = subprocess.Popen(['/bin/bash'], shell=True, text=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
process.stdin.write('export SOMETHING=10000abc\n')
process.stdin.flush()
process.stdin.write('echo $SOMETHING\n')
process.stdin.flush()

r, w, e = select.select([process.stdout], [process.stdin], [], 0.1)
result = ""
while process.stdout in r:
    result += os.read(process.stdout.fileno(), 100).decode("utf-8")
    r, w, e = select.select([process.stdout], [], [], 0.1)
process.kill()
print (result)

Interacting with bash through python without terminating

1 Answers1