subprocess.run() produces different result than manual input

Question

I am trying to run the following Windows console commands through a Python script:

C:\My\Path\openssl.exe x509 -in C:\My\PEM\mypem.pem -noout -subject > C:\My\Data\data.txt

If put directly into the console, produces the expected 1KB file.

Using subprocess.run() does not. It produces a file, but a 0KB file as if it is not grabbing the stdout response.

What I've tried without success:

# produces b''
args = 'C:/My/Path/openssl.exe x509 -in C:/My/PEM/mypem.pem -noout -subject'
data = subprocess.check_output(args)
print (data)

# produces b''
result = subprocess.Popen('C:/My/Path/openssl.exe x509 -in C:/My/PEM/mypem.pem -noout -subject', stdout = subprocess.PIPE)
print (result.stdout) 

# produces a 0KB data.txt
# probably also producing a b'' thus the 0KB
subprocess.run('C:/My/Path/openssl.exe x509 -in C:/My/PEM/mypem.pem -noout -subject > C:/My/Data/data.txt')

Try capturing the `stderr` and see if it's reporting an error. — Barmar, May 23 '17 at 17:36
@Barmar Just used `subprocess.PIPE` for both `stderr` and `stdout` and both yield `b''`. — , May 23 '17 at 17:39
You are using shell redirection (the `>`) and are not running with `shell=True`. Instead, you use `stdout=subprocess.PIPE` to redirect the output to the Python process. You could as well read the output and write the file yourself. — 9000, May 23 '17 at 17:39
@9000 That is what I have been trying to do. So even if I drop the `> ...` and just run up through `-subject` the returned value is always `b''` so I cant write to the file since there is nothing to write. — , May 23 '17 at 17:41
Of *course* there's nothing to write if you manage to successfully perform a redirection with `>` -- you're sending your output to a file, so nothing goes to the pipeline. (On UNIX, that'll only happen with `shell=True`, but this looks like Windows, and the way command execution works on Windows is far more convoluted). — Charles Duffy, May 23 '17 at 19:25
(..on which point, you should **really** specify the platform in the question -- your code here is not at all portable; on UNIX, none of the commands given would work at all without `shell=True`). — Charles Duffy, May 23 '17 at 19:27

Barmar · Answer 1 · 2017-05-23T17:50:56.147

3

You need to use shell=True if you want the string parsed as a command with arguments.

result = subprocess.Popen('C:/My/Path/openssl.exe x509 -in C:/My/PEM/mypem.pem -noout -subject', stdout = subprocess.PIPE, shell=True)
print(result.stdout)

Or you can specify the command as a list:

result = subprocess.Popen(['C:/My/Path/openssl.exe', 'x509', '-in', 'C:/My/PEM/mypem.pem', '-noout', '-subject'], stdout = subprocess.PIPE)

edited May 23 '17 at 17:50

answered May 23 '17 at 17:46

Barmar

741,623
53
500
612

That should be `stdout=open('data.txt', 'w')` to provide parity with the original `>data.txt`, no? – Charles Duffy May 23 '17 at 17:48
My answer is based on his first code with `stdout=subprocess.PIPE`. – Barmar May 23 '17 at 17:51
@Barmar both methods still produce `b''` as if the `stdout` has no data. If I copy the exact same command directly into a command terminal, it executes without problem. – May 23 '17 at 17:57

score 0 · Answer 2 · answered May 05 '20 at 02:22

I know this is an old question, but here is my solution:

It is unclear to me absolutely why this is an issue, but I am hypothesizing that OpenSSL must be one shell, then x509 is another shell opened inside OpenSSL and this creates an issue on Windows using subprocess, but not Linux.

You can type separately and sequentially on the CMDline OpenSSL and then x509 to get an idea of what I am talking about. To solve this, I opened up the supposed OpenSSL and x509 shell in one subprocess command then fed the rest of the required data into the process's stdin using a subprocess.PIPE. Here is an example of my program, I am sure you could modify in yours to make this work:

p=subprocess.Popen("openssl x509 -in " + VARS["CLIENTCERT"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
aissuer = p.communicate(bytes("-noout -issuer -nameopt multiline,-align,-esc_msb,utf8,-space_eq;", "utf-8"))[0].decode().strip("\n") # gets stdout

Keep in mind, if you want to run on UNIX and Windows, you should be specifying two commands and running them based on what os.name returns (e.g., nt for Windows...)

Also, you should be setting OpenSSL to your PATH variable so you don't have to specify a path for OpenSSL. To do so, check out Ahmed's answer.

subprocess.run() produces different result than manual input

2 Answers2