Stuck inside a python subprocess call

Question

I'm trying to decrypt a file with a gpg executable on a windows 2012 server. Currently the script accesses the 'try' block but then stays there indefinitely. The file is roughly 500mb in size so anything it should take < 10min to decrypt. The longest I've let it run for is an hour. Here is the code for this:

# decrypt the pgp file
comm = (gpg + ' --batch --passphrase passphrase --homedir='+current_path
           +' -o ' + zip_name +' --decrypt ' + file_name)

try:
    subp = subprocess.check_call(comm, stdout=PIPE, stdin=PIPE, stderr=STDOUT)
    stdout_data, stderr_data = subp.communicate()
    print stdout_data, stderr_data
except subprocess.CalledProcessError as e:
    print e.output
    logger.update('Error', process, runtime=0, error=e)
    raise Exception('Error Decrypting File')

Not included, but the key has been imported.

Is there anything I can add/remove or do differently to 1) get a better idea of what's going on within the host system (the windows server) and 2) to not run indefinitely and report back useful information as to why it was running indefinitely.

Let me know if any clarification is needed.

Maybe unrelated, by why do you have `stdin=PIPE` ? gpg may be waiting on something to be fed and just block. — user2722968, Apr 11 '17 at 21:29
Also you should not have your whole commandline in one string, use the program name and individual arguments for everything. You'll run into argument parsing problems on the gpg side otherwise. — user2722968, Apr 11 '17 at 21:31

score 3 · Accepted Answer · answered Apr 11 '17 at 21:42

From python's documentation for subprocess.check_call():

Note: Do not use stdout=PIPE or stderr=PIPE with this function. The child process will block if it generates enough output to a pipe to fill up the OS pipe buffer as the pipes are not being read from.

Considering you are trying to decrypt 500MB, I think it safe to say you will hit the pipe size limit. I suggest you to replace PIPE with an open file descriptor or an existing file object.

BTW, I am a bit surprised with the way you build your command line, it is my understanding that subprocess.check_call expects a list of strings, rather than a string itself.

command = [gpg, "--batch", "--passphrase passphrase", "--homedir=current_path",
           "-o", "zip_name", "--decrypt", file_name]

Starting from version 3.3, a timeout option was added to many function of the subprocess module. You might find it useful to debug your issue.

You don't *have* to pass in a list, but it's decidedly an improvement here. See also http://stackoverflow.com/questions/3172470/actual-meaning-of-shell-true-in-subprocess — tripleee, Apr 11 '17 at 22:03

Stuck inside a python subprocess call

1 Answers1