python check_output fails with exit status 1 but Popen works for same command

Question

Command framed to identify if Xcode is running on Mac: cmd = "ps -ax | grep -v grep | grep Xcode"

If Xcode is not running, then above command works well with Popen method of subprocess module, but raises a CalledProcessError with check_output method. I tried to inspect the stderr through the following code, but failed to get appropriate information to understand the reason.

from subprocess import check_output, STDOUT, CalledProcessError

psCmd = "ps -ax | grep -v grep | grep Xcode"
o = None
try:
    o = check_output(psCmd, stderr=STDOUT, shell=True)
except CalledProcessError as ex:
    print 'Error:', ex, o

Exception message is as follows:

Error: Command 'ps -ax | grep -v grep | grep Xcode' returned non-zero exit status 1 None

Question: Why the above command works with Popen, but fails with check_output ?

Note: Command works well with both approach, if Xcode is running.

You're much better off doing the `grep` processing in Python itself anyway. — tripleee, Feb 23 '15 at 14:58
Even if I process the output in python, I have to use `subprocess` module. So I think it's a good way to get all the work done at `bash` end. — the.slow.one, Feb 23 '15 at 15:18
related: [Kill process by name in Python](http://stackoverflow.com/q/2940858/4279) — jfs, Feb 24 '15 at 07:12

jfs · Accepted Answer · 2015-02-24T07:17:22.343

check_output() works as expected. Here's its simplified implementation in terms of Popen():

def check_output(cmd):
    process = Popen(cmd, stdout=PIPE)
    output = process.communicate()[0]
    if process.returncode != 0:
        raise CalledProcessError(process.returncode, cmd, output=output)
    return output

grep returns 1 if it hasn't found anything i.e., you should expect the exception if Xcode is not running.

Note: as the implementation shows, you can get the output even if the exception occurs:

#!/usr/bin/env python
from subprocess import check_output, STDOUT, CalledProcessError

cmd = "ps -ax | grep -v grep | grep Xcode"
try:
    o = check_output(cmd, stderr=STDOUT, shell=True)
    returncode = 0
except CalledProcessError as ex:
    o = ex.output
    returncode = ex.returncode
    if returncode != 1: # some other error happened
        raise

You could probably use pgrep -a Xcode command instead (note: starts with p) or use psutil module for a portable code:

#!/usr/bin/env python
import psutil # $ pip install psutil

print([p.as_dict() for p in psutil.process_iter() if 'Xcode' in p.name()])

I understood now, why check_output behaves as it behaves. Checking for return code is cool. I'll use your idea with errno. Some **doubts**: _1._ Why do we have to init `returncode` in `try` block with value 0? _2_. Can't we compare `ex.returncode` directly with value 1 in `if` condition? Does it serve some purpose, which I have not understood yet. — the.slow.one, Feb 25 '15 at 07:58
`returncode` is set inside try/except in order to 1. show you that it is always zero if `check_output()` hasn't raised an exception 2. both branches (with and without the exception) set both variables: you can use `o` and `returncode` after the code regardless of whether Xcode is running or not. — jfs, Feb 25 '15 at 08:02

score 2 · Answer 2 · answered Feb 23 '15 at 14:34

2

From the Python docs: "If the return code was non-zero it raises a CalledProcessError.". That's what happens to you when Xcode isn't running; the final grep Xcode exits with a non-zero status because grep couldn't find the string Xcode that you're looking for. Hence, check_output() will raise the exception.

BTW, I found this on the Python subprocess documentation.

answered Feb 23 '15 at 14:34

Karel Kubat

1,635
11
12

I don't see so. If I run this command in terminal without Xcode launched and check the errno variable, it still hold the value 0. To check errno I used echo "$?". – the.slow.one Feb 23 '15 at 14:43
1

Ummm... when I try `ps -aef | grep -v grep | grep Xcode ; echo $?`, I get as output: `1` – Karel Kubat Feb 23 '15 at 14:53
@DeepakrajHR you mean even if Xcode is not launched your grep command is finding it? The Error message which you have displayed `Error: Command 'ps -ax | grep -v grep | grep Xcode' returned non-zero exit status 1 None` is the output of your python script because you have handled the exception in your code – Bad_Coder Feb 23 '15 at 14:55
Terminal output when Xcode isn't running `$ ps -ax | grep -v grep | grep Xcode $ echo "$?" 0 $` Terminal output if Xcode is running `$ ps -ax | grep -v grep | grep Xcode 13784 ?? 0:02.34 /Applications/Xcode.app/Contents/MacOS/Xcode $ echo "$?" 0 $` So my point is, I don't think that error code 1 is coming from here in here. I'm missing something else. – the.slow.one Feb 23 '15 at 15:01
I just made sure of it :-) – the.slow.one Feb 23 '15 at 15:06
Guys my bad. I was using command as `echo "$?"` rather than `echo $?`. I too got error code 1. – the.slow.one Feb 23 '15 at 15:13
1

That should not matter, the quotes are harmless (and in fact good form, especially if you are not sure of quoting). – tripleee Feb 23 '15 at 15:45

Bad_Coder · Answer 3 · 2015-02-23T14:51:18.357

1

If your grep command grep Xcode returns no result then the returncode of the command will be non-zero, that's why check_output is calling CalledProcessError, which is what you are seeing in the output of print command

To get the output of your command be it error or success use following piece of code:-

#!/usr/bin/python
from subprocess import check_output, STDOUT, CalledProcessError

psCmd = "ps -aef | grep -v grep | grep Xcode"
o = None
o = check_output(psCmd+";exit 0", stderr=STDOUT, shell=True)

check_output will only show you the output of the command if it's return code is 0 else it calls an exception.

edited Feb 23 '15 at 14:51

answered Feb 23 '15 at 14:35

Bad_Coder

1,019
1
20
38

Please see my comment on Karel Kubat's answer – the.slow.one Feb 23 '15 at 14:46
That's a nice idea as shown in [link](https://docs.python.org/2/library/subprocess.html#subprocess.check_output). But why I'm getting error code as 1 from python script and 0 from terminal. Anyway now I'm using Popen to solve the problem. But the question still remains :-( – the.slow.one Feb 23 '15 at 15:09
I got my answer. Now I'm using this way. But I'm wondering now, what if the command fails for some other unknown reason. I think that too will bypass because of `exit 0`. Is there a way to catch those exceptions. – the.slow.one Feb 23 '15 at 15:24
This is not a sane answer. Don't use `check_output` if you don't want to check the result code from the command. – tripleee Feb 23 '15 at 15:52
`exit 0` is bad. It ignores errors. You can [get the output even if an exception happens](http://stackoverflow.com/a/28689969/4279). – jfs Feb 24 '15 at 07:16

score 0 · Answer 4 · answered Feb 23 '15 at 15:59

The purpose of check_output is to make sure that the command you ran completed successfully. It's supposed to fail if grep Xcode does not return success.

What you want would be a lot simpler with the searching in Python, anyway.

output = check_output(['ps', '-ax'], shell=False)
if 'Xcode' in output:
    print('Xcode appears to be running')

This has the additional (very minor) benefit over the shell version that it actually fails if ps fails for some reason. The shell would simply ignore the exit code of ps when it's not at the end of a pipeline.

python check_output fails with exit status 1 but Popen works for same command

4 Answers4

Linked