62

How can I call an external program with a python script and retrieve the output and return code?

cfischer
  • 24,452
  • 37
  • 131
  • 214
  • 2
    There are some existing questions and answers on SO that will help you: http://stackoverflow.com/questions/89228/how-to-call-external-command-in-python – Jarret Hardie Apr 01 '09 at 19:27
  • Does this answer your question? [How to execute a program or call a system command?](https://stackoverflow.com/questions/89228/how-to-execute-a-program-or-call-a-system-command) – robertspierre Nov 20 '21 at 10:33
  • Can anyone tell me how to store the output to a log.csv file (I need this for git, so the csv file is git commit history) – vpp Mar 17 '22 at 16:50

5 Answers5

87

Look at the subprocess module: a simple example follows...

from subprocess import Popen, PIPE

process = Popen(["ls", "-la", "."], stdout=PIPE)
(output, err) = process.communicate()
exit_code = process.wait()
Victor Uriarte
  • 479
  • 4
  • 9
jkp
  • 78,960
  • 28
  • 103
  • 104
  • 11
    I've edited the answer above to reflect Ambroz's suggestion in case someone does not read the comments and uses the previously incorrect code. – Ben McCann Oct 24 '13 at 17:30
  • if this does not for some reason work, you might want to add shell=True to the params ( when in windows? ) – ntg Oct 31 '16 at 10:05
  • 2
    It seems that [the above solution](https://stackoverflow.com/a/707001/1321680) can be replaced by simple call [`subprocess.run()`](https://docs.python.org/dev/library/subprocess.html#subprocess.run) (Python >= 3.5 is required). – patryk.beza Jun 24 '17 at 12:56
  • 1
    can I get the line number of error line like a line has error and the function returns me the line number like 1.0? so that I can add a tag and highlight the error. – Kumar Saptam May 15 '19 at 16:19
  • 1
    In order to grab the error outputs "stderr=PIPE" should also be added: `process = Popen(["ls", "-la", "."], stdout=PIPE, stderr=PIPE)` – HmT May 31 '19 at 07:18
18

Following Ambroz Bizjak's previous comment, here is a solution that worked for me:

import shlex
from subprocess import Popen, PIPE

cmd = "..."
process = Popen(shlex.split(cmd), stdout=PIPE)
process.communicate()
exit_code = process.wait()
Jabba
  • 19,598
  • 6
  • 52
  • 45
  • 5
    This is by far the best answer. – dotancohen Aug 22 '13 at 09:43
  • 4
    I have a similar post [here](https://stackoverflow.com/questions/1996518/retrieving-the-output-of-subprocess-call/21000308#21000308) that shows how to get three things from a process: exitcode, stdout, stderr. – Jabba Mar 18 '14 at 20:58
5

After some research, I have the following code which works very well for me. It basically prints both stdout and stderr in real time. Hope it helps someone else who needs it.

stdout_result = 1
stderr_result = 1


def stdout_thread(pipe):
    global stdout_result
    while True:
        out = pipe.stdout.read(1)
        stdout_result = pipe.poll()
        if out == '' and stdout_result is not None:
            break

        if out != '':
            sys.stdout.write(out)
            sys.stdout.flush()


def stderr_thread(pipe):
    global stderr_result
    while True:
        err = pipe.stderr.read(1)
        stderr_result = pipe.poll()
        if err == '' and stderr_result is not None:
            break

        if err != '':
            sys.stdout.write(err)
            sys.stdout.flush()


def exec_command(command, cwd=None):
    if cwd is not None:
        print '[' + ' '.join(command) + '] in ' + cwd
    else:
        print '[' + ' '.join(command) + ']'

    p = subprocess.Popen(
        command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=cwd
    )

    out_thread = threading.Thread(name='stdout_thread', target=stdout_thread, args=(p,))
    err_thread = threading.Thread(name='stderr_thread', target=stderr_thread, args=(p,))

    err_thread.start()
    out_thread.start()

    out_thread.join()
    err_thread.join()

    return stdout_result + stderr_result
Jake W
  • 2,788
  • 34
  • 39
  • I only copy pasted the code to see if it works and I got an error that `out` is of type `bytes` so it cannot be used in the `write` method. Also, it prints the characters, but never stops. – ashrasmun Jan 10 '22 at 22:10
3

I've developed a little library (py-execute) that allows you to execute external programs, retrieve the output and the retcode and, at the same time get output in console in real time:

>>> from py_execute.process_executor import execute
>>> ret = execute('echo "Hello"')
Hello
>>> ret
(0, 'Hello\n')

You can avoid printing to console passing a mock user_io:

>>> from mock import Mock
>>> execute('echo "Hello"', ui=Mock())
(0, 'Hello\n')

I wrote it because with plain Popen (In Python 2.7) I was having trouble executing commands with a long output

hithwen
  • 2,154
  • 28
  • 46
3

Check out the subprocess module here: http://docs.python.org/library/subprocess.html#module-subprocess. It should get what you need done.

David Ackerman
  • 12,649
  • 6
  • 24
  • 19