Python subprocess.check_output stderr usage

Question

Hi I am newbie to Python and am trying to understand how stderr is used with subprocess check_output. I've read through the subprocess documentation and am having difficulty understanding how the stderr is used and what specifically subprocess.STDOUT actually accomplishes.

Can I please get some help with some examples or references that explains how stderr is used here?

I've tried these commands using both stderr and without and not seeing any real difference.

Code:

#!/usr/bin/python3
import subprocess

print(subprocess.check_output("echo Hello World!",
                              stderr=subprocess.STDOUT,
                              shell=True))

Output:

# ./ex.py
b'Hello World!\n'

Code:

#!/usr/bin/python3
import subprocess

print(subprocess.check_output("gecho Hello World!",
                              stderr=subprocess.STDOUT,
                              shell=True))

Output:

# ./ex.py
Traceback (most recent call last):
  File "./ex.py", line 6, in <module>
    shell=True))
  File "/usr/lib64/python3.3/subprocess.py", line 589, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command 'gecho Hello World!' returned non-zero exit status 127

You are getting a *CalledProcessError* because the return code is non-zero, all `stderr=subprocess.STDOUT` is *redirecting* stderr to *stdout*, it has nothing to do with catching errors in your python code. If you did not redirect stderr to stdour anything that came out stderr would be outputted to the screen and not captured by check_output .This is an example when you might want to use stderr=subprocess.STDOUT http://stackoverflow.com/questions/39917569/why-doesnt-this-regex-work/39917623#39917623. — Padraic Cunningham, Oct 24 '16 at 16:05
@PadraicCunningham how can I read what is being directed to stderr or redirected to stdout? What is the need to redirect it to stdout? Thanks — MBasith, Oct 24 '16 at 16:08
what if the process you call pipes out stderr instead of stdout and you want to capture the output? You can see exactly that happening in the link above. — Padraic Cunningham, Oct 24 '16 at 16:09
If it pipes out an error will it not by default print the error to the terminal? — MBasith, Oct 24 '16 at 16:11
Do you know what stderr and stdout are? http://stackoverflow.com/questions/3385201/confused-about-stdin-stdout-and-stderr. Run the check_output code in the link, once with `stderr=subprocess.STDOUT` and once without, you will see pretty clearly the difference — Padraic Cunningham, Oct 24 '16 at 16:11
@PadraicCunningham this is link is exactly what I was looking for. Yes I did not fully understand what they were. Very helpful. Thanks. — MBasith, Oct 24 '16 at 16:15
No worries, it is really pretty straightforward once you understand exactly what stderr and stdout are. — Padraic Cunningham, Oct 24 '16 at 16:18
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/126540/discussion-between-mbasith-and-padraic-cunningham). — MBasith, Oct 24 '16 at 16:52

Billy · Accepted Answer · 2016-10-24T16:30:29.017

subprocess.check_output will raise the exception you see if there's a non-zero exit code. It sound like what you're trying to get is a simple way of reading STDERR, which in that case the easiest thing is to use subprocess.run and pipe STDOUT and STDERR (examples are in Windows):

>>> p = subprocess.run(['cmd','/C','echo','hello','world'], stdout = subprocess.PIPE, stderr=subprocess.PIPE)
>>> p.stdout
b'hello world\r\n'

>>> p.stderr
b''

>>> p = subprocess.run(['cmd','/C','gecho','hello','world'], stdout = subprocess.PIPE, stderr=subprocess.PIPE)
>>> p.stdout
b''

>>> p.stderr
b"'gecho' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n"

If you REALLY need to use check_output, the following will ignore the error code from the echo call and still print the error (example taken almost verbatim from the docs):

>>> print(subprocess.check_output('gecho hello& exit 0', stderr=subprocess.STDOUT, shell=True))
b"'gecho' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n"

Or in Linux:

>>> print(subprocess.check_output('gecho hello; exit 0', stderr=subprocess.STDOUT, shell=True))
b'/bin/sh: 1: gecho: not found\n'

As a side note, it's almost never a good idea to use subprocess functions with the option shell=True. It's mostly due to security concerns; read the docs for full explanation.

Python subprocess.check_output stderr usage

1 Answers1