How can I execute commands in the same command window and allow for user input after each command while printing the output in Python?

Question

I am trying to write a Python script that allows the user to input commands in the same command window and displays the output of the executed command. I wrote some code that almost works, but it stops when there is empty input and doesn't display the output of the current command until the next command is entered.

Here is the code that I'm using:

# Import the subprocess module, which allows us to spawn new processes.
import subprocess

# Start a new instance of the Windows command prompt using subprocess.Popen.
# We use stdin, stdout, and stderr pipes to communicate with the process, and 
# set the shell argument to True so that we can execute commands with shell syntax.
cmd = subprocess.Popen('cmd.exe', stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)

# Continuously ask the user for commands and send them to the command prompt process.
# We use a while True loop to keep the program running until the user decides to terminate it.
while True:
    # Prompt the user to enter a command.
    user_input = input("Enter a command: ")
    
    # Send the user's command to the command prompt process via stdin.
    # We encode the input as UTF-8 and add a newline character to simulate pressing Enter.
    # We then flush the stdin buffer to make sure the command is sent immediately.
    cmd.stdin.write(user_input.encode('utf-8') + b"\n")
    cmd.stdin.flush()
    
    # Read the output of the command prompt process from stdout and print it to the console.
    # We use a while True loop to read one line at a time until there is no more output.
    # We decode the output from bytes to a UTF-8 string and append each line to a list.
    # We then join all the lines into a single string and print it to the console.
    output_lines = []
    while True:
        # Read one line of output from stdout and decode it from bytes to a UTF-8 string.
        line = cmd.stdout.readline().decode('utf-8')
        
        # Check if the line is empty. If it is, there is no more output, so we break out of the loop.
        if not line:
            break
        
        # Otherwise, we append the line to a list of output lines.
        output_lines.append(line)

    # Join all the output lines into a single string and print it to the console.
    output = "\n".join(output_lines)
    if output:
        print(output)

I've tested this code with various commands, including simple commands like dir and more complex commands like ping and ipconfig. The output lags behind by one input for all commands, and the program gets stuck on empty input for all commands as well.

I've searched online for similar issues, but all of the solutions that I've found involve creating a new command window for each input. However, I specifically need to execute commands in the same command window and allow for user input after each command without opening a new window.

Answer 1

Somehow, I wrote something that's now working. I don't fully understand why/how it's working. I tried added threading and queuing. Anyway, here's the code:

import subprocess
import threading
import queue
import time


def read_output():
    while not exit_request.is_set():

        line = cmd.stdout.readline().decode('utf-8')

        if not line:
            break

        output_queue.put(line)
    exit_request.set()


cmd = subprocess.Popen('cmd.exe', stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)

output_queue = queue.Queue()

exit_request = threading.Event()

output_thread = threading.Thread(target=read_output)
output_thread.start()

exit_req = False
while not exit_req:

    user_input = input("Enter a command: ")

    if user_input.lower() == "exit":
        cmd.kill()
        exit_req = True

    if cmd.poll() is not None:
        exit_request.set()

        break

    cmd.stdin.write(user_input.encode('utf-8') + b"\n")
    cmd.stdin.flush()

    if cmd.poll() is not None:
        break

    if user_input:

        time.sleep(0.1)

        while not output_queue.empty():
            line = output_queue.get()
            print(line.rstrip())
    else:

        time.sleep(0.1)

output_thread.join()

output_lines = []
while True:

    line = cmd.stdout.readline().decode('utf-8')

    if not line:
        break

    output_lines.append(line)
    output = "\n".join(output_lines)
    if output:
        print(output)