How do you use subprocess.check_output() in Python?

Question

I have found documentation about subprocess.check_output() but I cannot find one with arguments and the documentation is not very in depth. I am using Python 3 (but am trying to run a Python 2 file through Python 3)

I am trying to run this command: python py2.py -i test.txt

-i is a positional argument for argparse, test.txt is what the -i is, py2.py is the file to run

I have tried a lot of (non working) variations including: py2output = subprocess.check_output([str('python py2.py '),'-i', 'test.txt'])

py2output = subprocess.check_output([str('python'),'py2.py','-i', test.txt'])

Answer 1

The right answer (using Python 2.7 and later, since check_output() was introduced then) is:

py2output = subprocess.check_output(['python','py2.py','-i', 'test.txt'])

To demonstrate, here are my two programs:

py2.py:

import sys
print sys.argv

py3.py:

import subprocess
py2output = subprocess.check_output(['python', 'py2.py', '-i', 'test.txt'])
print('py2 said:', py2output)

Running it:

$ python3 py3.py
py2 said: b"['py2.py', '-i', 'test.txt']\n"

Here's what's wrong with each of your versions:

py2output = subprocess.check_output([str('python py2.py '),'-i', 'test.txt'])

First, str('python py2.py') is exactly the same thing as 'python py2.py'—you're taking a str, and calling str to convert it to an str. This makes the code harder to read, longer, and even slower, without adding any benefit.

More seriously, python py2.py can't be a single argument, unless you're actually trying to run a program named, say, /usr/bin/python\ py2.py. Which you're not; you're trying to run, say, /usr/bin/python with first argument py2.py. So, you need to make them separate elements in the list.

Your second version fixes that, but you're missing the ' before test.txt'. This should give you a SyntaxError, probably saying EOL while scanning string literal.

Meanwhile, I'm not sure how you found documentation but couldn't find any examples with arguments. The very first example is:

>>> subprocess.check_output(["echo", "Hello World!"])
b'Hello World!\n'

That calls the "echo" command with an additional argument, "Hello World!".

Also:

-i is a positional argument for argparse, test.txt is what the -i is

I'm pretty sure -i is not a positional argument, but an optional argument. Otherwise, the second half of the sentence makes no sense.

Answer 2

Since Python 3.5, subprocess.run is recommended instead of subprocess.check_output:

>>> subprocess.run(['cat','/tmp/text.txt'], check=True, stdout=subprocess.PIPE).stdout
b'First line\nSecond line\n'

Since Python 3.7, instead of the above, you can use capture_output=true parameter to capture stdout and stderr:

>>> subprocess.run(['cat','/tmp/text.txt'], check=True, capture_output=True).stdout
b'First line\nSecond line\n'

Also, you may want to use universal_newlines=True or its equivalent since Python 3.7 text=True to work with text instead of binary:

>>> stdout = subprocess.run(['cat', '/tmp/text.txt'], check=True, capture_output=True, text=True).stdout
>>> print(stdout)
First line
Second line

Answer 3

Adding on to the one mentioned by @abarnert

a better one is to catch the exception

import subprocess
try:
    py2output = subprocess.check_output(['python', 'py2.py', '-i', 'test.txt'],stderr= subprocess.STDOUT)  
    #print('py2 said:', py2output)
    print "here"
except subprocess.CalledProcessError as e:
    print "Calledprocerr"

this stderr= subprocess.STDOUT is for making sure you dont get the filenotfound error in stderr- which cant be usually caught in filenotfoundexception, else you would end up getting

python: can't open file 'py2.py': [Errno 2] No such file or directory

Infact a better solution to this might be to check, whether the file/scripts exist and then to run the file/script

Answer 4

@ravi.zombie's answer adapted to work for python 3 and produce output even when there is an non zero exit code error:

try:
    out = subprocess.check_output(['git','commit','-m', commit_message], stderr= subprocess.STDOUT ).decode()
except subprocess.CalledProcessError as e:
    print ('[ERROR]: Exit code != 0')
    out = e.output.decode()
print(out)