Python subprocess.call works but subprocess.check_call does not - what are the differences?

Question

I'm using Python 2.7

I'm trying to run a StatTransfer program from Python.

When I try:

tempname = os.path.abspath('./text.txt')
TEMPFILE = open(tempname, 'wb')
try:
    subprocess.check_call('ST convert.stc', shell = True, stdout = TEMPFILE, stderr = TEMPFILE)
except:
    raise CritError(messages.crit_error_bad_command)

it fails (the CritError is user-defined).

The traceback doesn't tell me anything useful:

Traceback (most recent call last):
  File "C:\...\py\run_program.py", line 181, in run_stcmd
    run.execute_run(current_directory, posix_command, nt_command)
  File "C:\...\py\private\runprogramdirective.py", line 99, in execute_run
    raise CritError(messages.crit_error_bad_command)
CritError: 'ERROR! Cannot execute command'

However changing the relevant line to:

subprocess.call('ST convert.stc', shell = True, stdout = TEMPFILE, stderr = TEMPFILE)

it runs successfully.

The funny thing is, I see the same thing in my TEMPFILE for both cases:

|/-|/-|/-|/-|/- |/-|/-|/-|/-|/- Stat/Transfer - Command Processor (c) 1986-2011 Circle         Systems, Inc.
www.stattransfer.com 
Version 10.1.1866.0714 (32 Bit) - 64 Bit Windows

Serial: ADR4H-L3A3A-N8RJ
User:   XXXXXXXXXXX
Your license is in its grace period -- Please call Circle Systems
Your program will die at the end of the month
Status: Temporarily OK (Expired May 31, 2012)
Transferring from SPSS Portable File: ..\orig\10908970\ICPSR_03775\DS0001\03775-0001-    Data.por
Input file has 26 variables
Optimizing...
Transferring to Stata: ..\data\ABCFeb.dta

504 cases were transferred(0.02 seconds)

Note that if I run "st convert.stc" from the Windows command line, it runs just fine and gives me the same log message above. It does achieve what's written inside convert.stc.

This suggests that the StatTransfer program is called with subprocess.check_call. However, there's an error at the end of it. What kind of error is this? How do I avoid it? Which of the 2 commands should I use and why?

ETA: Following mgilson below, I return the value from subprocess.call and get -1. What does this mean? Why did the program still run and I didn't seem to notice any real error?

Any possible explanations and suggestions on how I should do this here?

Thanks.

You should post the entire traceback -- e.g. what line is causing the error. Your try/except clause is also probably masking the problem because you catch whatever error is happening and then you raise something different (without arguments) — mgilson, Jun 12 '12 at 02:15
Sorry about the except clause. I originally had a user-defined one. It doesn't tell me anything useful but that the subprocess call fails - that's all. Question updated. — cinny, Jun 12 '12 at 02:36

score 6 · Accepted Answer · edited May 23 '17 at 12:32

6

What's probably happening is that your process is exiting with a non-zero exit status. To check, run with retcode=subprocess.call(...) and then print retcode.

subprocess.check_call will raise an exception if retcode (above) is non-zero.

The Exception you're seeing is from the raise subprocess.CalledProcessError in your try/except clause:

>>> import subprocess 
>>> raise subprocess.CalledProcessError
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() takes exactly 3 arguments (1 given)

EDIT

I would still re-write the try/except clause as you're catching an exception and throwing another one (which means that all the information in the original message is lost).

Try something like:

try:
    subprocess.check_call('ST convert.stc', shell = True, stdout = TEMPFILE, stderr = TEMPFILE)
except Exception as e:
    raise CritError(messages.crit_error_bad_command+' '+str(e))

This will still give you some (not all) of the information from the original message. The problem is probably still that your subprogram is exiting with a non-zero exit code. Maybe that's ok (check to see that it accomplished what you want it to do).

You said that you can run the command from the commandline and it all looks OK. You can check to make sure the behavior is the same by checking the exit status from the windows commandline as well ( How do I get the application exit code from a Windows command line? ). I'm guessing the exit status will still be -1 -- If it's not, your program is interacting with the environment (e.g. environment variables) which are somehow different when you call it using python.

Ultimately, if the program does what you want it to do, and you don't care about the exit status, then you should just be using subprocess.call, but I would suggest consulting the manual for the programs exit codes and see what an exit status of -1 actually means.

edited May 23 '17 at 12:32

Community

1
1

answered Jun 12 '12 at 02:16

mgilson

300,191
65
633
696

Does this mean there is or there isn't an error? I don't think there should be any error. How do I avoid it? – cinny Jun 12 '12 at 02:29
I actually used a user-specific error exception and not subprocess.CalledProcessError in my code. I just changed it quickly when asking the question here as my own exception didn't give me anything useful. – cinny Jun 12 '12 at 02:30
@user18115 : Typically when you use a user-specific error, it gets raised as `raise UserError("Some message")`. As far as whether or not there really is an error, that depends on the program. Your program is returning -1. It is standard for a program to return 0 on success which means that the program you're calling is probably having some problem (or at least it thinks it is). What -1 means is entirely program specific so I can't give you any help there. Consult the manual for that program I guess. – mgilson Jun 12 '12 at 12:16
This is really helpful. In this particular case, the program finished what it was supposed to, so I didn't see any reason for it to return -1. The thing is my function is written so that the command changes, so I may use Python to run other things as well. Should I still stick to subprocess.call, or maybe only with StatTransfer I should allow retcode to be <> 0, but not for others? – cinny Jun 12 '12 at 15:43
I tried to check the StatTransfer manual online but it just doesn't mention anything about exit code. I mean, normally I don't see that kind of detail in manuals. This is quite frustrating. – cinny Jun 12 '12 at 15:44
About the try/except clause: How do I get _ALL_ the information from the original error message? How do I print it out? – cinny Jun 12 '12 at 15:48
@user18115 : It will print automatically if you remove the `try/except` block (unless you catch if someplace else). Alternatively, in `except` just put `raise`. That will re-raise the exception. After you've debugged it, you can replace that `raise` with whatever you want. – mgilson Jun 12 '12 at 16:10

Python subprocess.call works but subprocess.check_call does not - what are the differences?

1 Answers1