29

Honestly, I just don't understand the lingo of "non-zero" status to really interpret what's going on or what that means (it wasn't even defined) on help pages. What are some examples of using python to call other scripts in which these processes of

subprocess.call subprocess.check_output subprocess.popen

really differ from each other? When would you use either of those, and what are the disambiguated details of these methods? Should I be using os.system instead if I wanted simple OS calls?

Tom
  • 919
  • 3
  • 9
  • 22
  • The following post partially answers your question. http://stackoverflow.com/questions/7681715/whats-the-difference-between-subprocess-popen-and-call-how-can-i-use-them – bergercookie Sep 22 '16 at 13:32

2 Answers2

36

The main difference is that, while popen is a non-blocking function (meaning you can continue the execution of the program without waiting the call to finish), both call and check_output are blocking.

The other difference is in what they return:

The methods call and check_output are, in fact, blocking wrappers of popen, using a Popen object. For example, you can get the returncode attribute by calling Popen.returncode().

otorrillas
  • 4,357
  • 1
  • 21
  • 34
  • 1
    It's probably worth mentioning that all blocking things should nowadays be done using subprocess.run – Bananach Oct 01 '19 at 14:32
25

To expand on @otorrillas's answer with some examples.

Say you have a short script that you only care whether it succeeds or not*, and you just want to do it now and wait for the result. Say, for example, you want to send a single ping to a remote machine. call is the one for you:

res = call(['ping', '127.0.0.1', '-c', '1', '-W', '1'])
# res is 0 if the ping succeeded, non-zero if it failed.

Now say that you have a command you want to assume is going to succeed, and is going to give you some output you want to use for something. check_output is for you. Perhaps a subversion command:

svn_output = check_output(['svn', 'info', path_to_my_working_copy])
# svn_output now contains the svn info output. If it failed, an 
# exception is thrown which more easily allows you to give 
# responsibility of that failure to whoever passed us the wrong 
# path_to_my_working_copy.

In general, if your use case does not simply fall under one of those categories, you are probably going to wind up using Popen.

One simple use case might be, say you have a daemon process that you want to start, but then run alongside of your python process. Now you use Popen:

my_proc = Popen(['my-daemon'])

# We're going to go and do something else now, but want to make sure
# my_proc dies when we do.
import atexit
atexit.register(my_proc.kill)

NB: If you use Popen raw, you must make sure you terminate the process, possibly using an atexit as I have illustrated.

* "non-zero" exit status just means the process failed. A famous computer science quote attributed to Tolstoy is "Happy processes are all alike; every unhappy process is unhappy in its own way", i.e. there's only one way for a process to be happy: to return 0. Everything else is unhappy, but there are lots of ways to be unhappy.

daphtdazz
  • 7,754
  • 34
  • 54
  • 1
    Did you probably meant call_output in second example ? check_call is not providing an output, only return code I believe. – taiko Feb 22 '19 at 22:07
  • 1
    thanks @taiko I've fixed that! Although I think the correct function is check_output not call_output. Amazing it took two years for someone to spot it :-) – daphtdazz Feb 25 '19 at 12:47
  • 2
    It's probably worth mentioning that all blocking things should nowadays be done using subprocess.run – Bananach Oct 01 '19 at 14:33
  • Hi @daphtdazz I don't think `check_output` is suitable for using with `joblib.Parallel` because of the blocking nature all threads continue to wait until timeout even if they were complete. – Koder101 Jul 20 '21 at 06:50