How to convert command into args to pass it to subprocess in Python?

Question

I have the below command that I was running and which was showing as non compliant. In order to make it compliant, I need to pass cmd as args in the input.

cmd = f"{script_path} 2>&1 | tee -a {log_file}"
subprocess.Popen(cmd, shell=True)

When I converted the above code to below, the log part wasn't working:

args = [script_path,"2>&1 | tee -a", log_file ]
subprocess.Popen(args)

Answer 1

Remember you are using shell features like pipe and those redirections. So you either should delegate those to the shell or implement the equivalent in subprocess module's API. You have three options here:

1. You already showed the first option which is using shell=True.

2. Second option is similar - calling the shell manually and pass that string as the argument. Again shell takes care of that, It's just without shell=True:

import subprocess

script_path = "ls"
log_file = "log_file.txt"

cmd = f"{script_path} 2>&1 | tee -a {log_file}"
print(cmd)
subprocess.Popen(["bash", "-c", cmd])

3. Third option is to do the piping and redirection stuff using subprocess module itself:

import subprocess

script_path = "ls"
log_file = "log_file.txt"


script_process = subprocess.Popen(
    [script_path], stdout=subprocess.PIPE, stderr=subprocess.STDOUT
)

tee_process = subprocess.Popen(["tee", "-a", log_file], stdin=script_process.stdout)

Here I created two processes that communicate with each other. The stdout=subprocess.PIPE, stderr=subprocess.STDOUT part is equivalent to 2>&1 and stdin=script_process.stdout in the tee_process does the piping.