6

I want to access the traceback of a python programm running in a subprocess.

The documentation says:

Exceptions raised in the child process, before the new program has started to execute, will be re-raised in the parent. Additionally, the exception object will have one extra attribute called child_traceback, which is a string containing traceback information from the child’s point of view.

Contents of my_sub_program.py:

raise Exception("I am raised!")

Contents of my_main_program.py:

import sys
import subprocess
try:
    subprocess.check_output([sys.executable, "my_sub_program.py"])
except Exception as e:
    print e.child_traceback

If I run my_main_program.py, I get the following error:

Traceback (most recent call last):
  File "my_main_program.py", line 6, in <module>
    print e.child_traceback
AttributeError: 'CalledProcessError' object has no attribute 'child_traceback'

How can I access the traceback of the subprocess without modifying the subprocess program code? This means, I want to avoid adding a large try/except clause around my whole sub-program code, but rather handle error logging from my main program.

Edit: sys.executable should be replaceable with an interpreter differing from the one running the main program.

schreon
  • 1,097
  • 11
  • 25
  • 1
    Doc says "before the new program has started to execute", in your case the exception was raised while the new program was executed, hence no `child_traceback`. Once the new program is running you need to catch the `CalledProcessError` exception and do as the following: http://stackoverflow.com/questions/24849998/how-to-catch-exception-output-from-python-subprocess-check-output using `CalledProcessError.output` – mguijarr Jul 18 '16 at 11:02
  • 1
    In my example, `CalledProcessError.output` did only capture the standard output, but not the traceback of the Exception. – schreon Jul 18 '16 at 12:31
  • 1
    This is probably because output is sent on `stderr`. Look at answers from the question I sent you the link above for more details – mguijarr Jul 18 '16 at 14:19

1 Answers1

0

As you're starting another Python process, you can also try to use the multiprocessing Python module ; by sub-classing the Process class it is quite easy to get exceptions from the target function:

from multiprocessing import Process, Pipe
import traceback
import functools

class MyProcess(Process):
    def __init__(self, *args, **kwargs):
        Process.__init__(self, *args, **kwargs)
        self._pconn, self._cconn = Pipe()
        self._exception = None

    def run(self):
        try:
            Process.run(self)
            self._cconn.send(None)
        except Exception as e:
            tb = traceback.format_exc()
            self._cconn.send((e, tb))
            # raise e  # You can still rise this exception if you need to

    @property
    def exception(self):
        if self._pconn.poll():
            self._exception = self._pconn.recv()
        return self._exception


p = MyProcess(target=functools.partial(execfile, "my_sub_program.py"))
p.start()
p.join() #wait for sub-process to end

if p.exception:
    error, traceback = p.exception
    print 'you got', traceback

The trick is to have the target function executing the Python sub-program, this is done by using functools.partial.

mguijarr
  • 7,641
  • 6
  • 45
  • 72
  • Does this solution allow the subprocess to run in a different interpreter than the main program? As I see your suggestion I notice that it was misleading to use `sys.executable` in my example. However, in the context of my question, I need to start scripts in another virtual environment than the main process. (the main process will be some kind of scheduler) – schreon Jul 18 '16 at 12:06
  • Actually `multiprocessing` does `os.fork` on POSIX platforms, so it's not a new virtual environment ; however you can limit the amount of shared data between processes by starting the `Process` objects early, this is maybe fair enough for your needs – mguijarr Jul 18 '16 at 14:14
  • If you're doing a kind of scheduler why not using something like [Celery](http://www.celeryproject.org/) – mguijarr Jul 18 '16 at 14:17