python subprocess.Popen stdin.write

Question

I'm new to python and would like to open a windows cmd prompt, start a process, leave the process running and then issue commands to the same running process.

The commands will change so i cant just include these commands in the cmdline variable below. Also, the process takes 10-15 seconds to start so i dont want to waste time waiting for the process to start and run commands each time. just want to start process once. and run quick commands as needed in the same process

I was hoping to use subprocess.Popen to make this work, though i am open to better methods. Note that my process to run is not cmd, but im just using this as example

import subprocess
cmdline = ['cmd', '/k']
cmd = subprocess.Popen(cmdline, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

cmd.stdin.write("echo hi")       #would like this to be written to the cmd prompt
print cmd.stdout.readline()      #would like to see 'hi' readback

cmd.stdin.write("echo hi again") #would like this to be written to the cmd prompt
print cmd.stdout.readline()      #would like to see 'hi again' readback

The results arent what i expect. Seems as though the stdin.write commands arent actually getting in and the readline freezes up with nothing to read.

I have tried the popen.communicate() instead of write/readline, but it kills the process. I have tried setting bufsize in the Popen line, but that didn't make too much difference

Answer 1

Your comments suggest that you are confusing command-line arguments with input via stdin. Namely, the fact that system-console.exe program accepts script=filename parameter does not imply that you can send it the same string as a command via stdin e.g., python executable accepts -c "print(1)" command-line arguments but it is a SyntaxError if you pass it as a command to Python shell.

Therefore, the first step is to use the correct syntax. Suppose the system-console.exe accepts a filename by itself:

#!/usr/bin/env python3
import time
from subprocess import Popen, PIPE

with Popen(r'C:\full\path\to\system-console.exe -cli -', 
           stdin=PIPE, bufsize=1, universal_newlines=True) as shell:
    for _ in range(10):
        print('capture.tcl', file=shell.stdin, flush=True)
        time.sleep(5)

Note: if you've redirected more than one stream e.g., stdin, stdout then you should read/write both streams concurrently (e.g., using multiple threads) otherwise it is very easy to deadlock your program.

Related:

• Q: Why not just use a pipe (popen())? -- mandatory reading for Unix environment but it might also be applicable for some programs on Windows
• subprocess readline hangs waiting for EOF -- code example on how to pass multiple inputs, read multiple outputs using subprocess, pexpect modules.

The second and the following steps might have to deal with buffering issues on the side of the child process (out of your hands on Windows), whether system-console allows to redirect its stdin/stdout or whether it works with a console directly, and character encoding issues (how various commands in the pipeline encode text).

Answer 2

Here is some code that I tested and is working on Windows 10, Quartus Prime 15.1 and Python 3.5

import subprocess

class altera_system_console:
    def __init__(self):
        sc_path = r'C:\altera_lite\15.1\quartus\sopc_builder\bin\system-console.exe --cli --disable_readline'
        self.console = subprocess.Popen(sc_path, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

    def read_output(self):
        rtn = ""
        loop = True
        i = 0
        match = '% '
        while loop:
            out = self.console.stdout.read1(1)
            if bytes(match[i],'utf-8') == out:
                i = i+1
                if i==len(match):
                    loop=False
            else:
                rtn = rtn + out.decode('utf-8')
        return rtn

    def cmd(self,cmd_string):
        self.console.stdin.write(bytes(cmd_string+'\n','utf-8'))
        self.console.stdin.flush()

c = altera_system_console()
print(c.read_output())
c.cmd('set jtag_master [lindex [get_service_paths master] 0]')
print(c.read_output())
c.cmd('open_service master $jtag_master')
print(c.read_output())
c.cmd('master_write_8 $jtag_master 0x00 0xFF')
print(c.read_output())

Answer 3

You need to use iter if you want to see the output in real time:

import subprocess
cmdline = ['cmd', '/k']
cmd = subprocess.Popen(cmdline, stdin=subprocess.PIPE, stdout=subprocess.PIPE)


cmd.stdin.write("echo hi\n")#would like this to be written to the cmd prompt
for line in iter(cmd.stdout.readline,""):
    print line
    cmd.stdin.write("echo hi again\n")#would like this to be written to the cmd prompt

Not sure exactly what you are trying to do but if you want to input certain data when you get certain output then I would recommend using pexpect