79

I've been reading the Python documentation about the subprocess module (see here) and it talks about a subprocess.check_output() command which seems to be exactly what I need.

However, when I try and use it I get an error that it doesn't exist, and when I run dir(subprocess) it is not listed.

I am running Python 2.6.5, and the code I have used is below:

import subprocess
subprocess.check_output(["ls", "-l", "/dev/null"])

Does anyone have any idea why this is happening?

robintw
  • 27,571
  • 51
  • 138
  • 205

3 Answers3

123

It was introduced in 2.7 See the docs.

Use subprocess.Popen if you want the output:

>>> import subprocess
>>> output = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE).communicate()[0]
user225312
  • 126,773
  • 69
  • 172
  • 181
  • 14
    unlike check_output, this doesn't raise `CalledProcessError` when the process returns non-zero return code. – Sridhar Ratnakumar Apr 13 '12 at 23:36
  • 1
    @SridharRatnakumar: of course because there is a big difference between them, namely: blocking and non-blocking. They are for different use cases! – László Papp Nov 15 '13 at 14:29
  • I shoved it in a `lambda` like this: `check_output = lambda args: Popen(args, stdout = PIPE).communicate()[0]`. Just because I'm in an interactive interpreter and it's kind of a PITA to write out multiline function defs in those. I used `from subprocess import Popen, PIPE` earlier in the session. – ArtOfWarfare May 31 '17 at 14:13
  • how to make a ping then?? can i still use Popen or? – TheCrazyProfessor Jun 06 '17 at 17:03
58

IF it's used heavily in the code you want to run but that code doesn't have to be maintained long-term (or you need a quick fix irrespective of potential maintenance headaches in the future) then you could duck punch (aka monkey patch) it in wherever subprocess is imported...

Just lift the code from 2.7 and insert it thusly...

import subprocess

if "check_output" not in dir( subprocess ): # duck punch it in!
    def f(*popenargs, **kwargs):
        if 'stdout' in kwargs:
            raise ValueError('stdout argument not allowed, it will be overridden.')
        process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs)
        output, unused_err = process.communicate()
        retcode = process.poll()
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
            raise subprocess.CalledProcessError(retcode, cmd)
        return output
    subprocess.check_output = f

Minor fidgeting may be required.

Do bear in mind though the onus is on you to maintain dirty little backports like this. If bugs are discovered and corrected in the latest python then you a) have to notice that and b) update your version if you want to stay secure. Also, overriding & defining internal functions yourself is the next guy's worst nightmare, especially when the next guy is YOU several years down the line and you've forgot all about the grody hacks you did last time! In summary: it's very rarely a good idea.

Community
  • 1
  • 1
Roger Heathcote
  • 3,091
  • 1
  • 33
  • 39
  • 2
    I concur with this method. I'd probably include the location of the source. You can find it at http://hg.python.org/cpython/file/d37f963394aa/Lib/subprocess.py#l544 – Ehtesh Choudhury Mar 04 '14 at 20:28
  • 1
    Note: CalledProcessError does not accept an output in python 2.6. (I'm immediately bitten after using this hack! :( ) – Andy Hayden Apr 09 '14 at 06:54
  • cpython is now on GitHub - `check_output` for Python 2.7 is currently here: https://github.com/python/cpython/blob/2.7/Lib/subprocess.py#L194 – jamesc Sep 12 '18 at 15:08
6

Thanks to the monkey patch suggestion (and my attempts failing - but we were consuming CalledProcessError output, so needed to monkey patch that)

found a working 2.6 patch here: http://pydoc.net/Python/pep8radius/0.9.0/pep8radius.shell/

"""Note: We also monkey-patch subprocess for python 2.6 to
give feature parity with later versions.
"""
try:
    from subprocess import STDOUT, check_output, CalledProcessError
except ImportError:  # pragma: no cover
    # python 2.6 doesn't include check_output
    # monkey patch it in!
    import subprocess
    STDOUT = subprocess.STDOUT

    def check_output(*popenargs, **kwargs):
        if 'stdout' in kwargs:  # pragma: no cover
            raise ValueError('stdout argument not allowed, '
                             'it will be overridden.')
        process = subprocess.Popen(stdout=subprocess.PIPE,
                                   *popenargs, **kwargs)
        output, _ = process.communicate()
        retcode = process.poll()
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
            raise subprocess.CalledProcessError(retcode, cmd,
                                                output=output)
        return output
    subprocess.check_output = check_output

    # overwrite CalledProcessError due to `output`
    # keyword not being available (in 2.6)
    class CalledProcessError(Exception):

        def __init__(self, returncode, cmd, output=None):
            self.returncode = returncode
            self.cmd = cmd
            self.output = output

        def __str__(self):
            return "Command '%s' returned non-zero exit status %d" % (
                self.cmd, self.returncode)
    subprocess.CalledProcessError = CalledProcessError
keen
  • 816
  • 10
  • 11