Python trace module - Trace lines as they are executed, but save to file, rather than stdout

Question

I want to trace the lines of a python script as they are executed. However the programme I use needs to print things to stdout. The trace option to the python trace module prints them to stdout. Is there anwyay to tell it to not print them to stdout, but instead save them to a file? I tried setting the outfile parameter, but it doesn't stop the printing of trace lines.

Answer 1

You could copy the trace module code, and make a few changes to get it to write its output to a file of your choosing. There are five print statements between lines 600 and 650 that are the ones you want to change. Since you don't need to make it too pretty, you can add this to the very end of the file:

my_out_file = open("/home/mytrace.txt", "w")

and change the print statements so this:

print "%s(%d): %s" % (bname, lineno,
                      linecache.getline(filename, lineno)),

becomes this:

print >>my_out_file, "%s(%d): %s" % (bname, lineno,
                      linecache.getline(filename, lineno)),

Answer 2

As per the docs of trace module, the outfile is only used to write updated count information.

• http://docs.python.org/library/trace.html

Trace module will continue to print the trace information to stdout.

For a manual tracing, if there is an entry function, I have used sys.settrace to do the work of tracing call flows. It can be extended to trace line by line execution. You can always file the information to a file rather than print it to stdout.

• http://docs.python.org/library/sys.html

A simple structure is

import sys
import os
import linecache

trace_depth = 0

def trace(f):

    def globaltrace(frame, why, arg):
        global trace_depth
        if why == "call":
            # function call event , extract information
            .....
            pass

            trace_depth = trace_depth + 1
        return None

    def localtrace(frame, why, arg):
        global trace_depth
        if why == "line":
            # line execution event
            pass
        elif why == "return":
            trace_depth = trace_depth - 1
            # function return event
        return localtrace

    def _f(*args, **kwds):
        sys.settrace(globaltrace)
        result = f(*args, **kwds)
        sys.settrace(None)
        return result

    return _f

globaltrace and localtrace are callback functions that are called with the event - "why"

say for event when a function is called, you can extract the information from the frame details.

if why == "call":
    # Parent frame details
    p_func = frame.f_back.f_code.co_name
    p_file = frame.f_back.f_code.co_filename
    p_lineinfo = frame.f_back.f_lineno

I have posted the full details here.

Answer 3

Are you invoking trace at the command line? If so, I recommend using the standard method of rerouting any stdout to any file. Redirect all output to file

foo > allout.txt 2>&1

I'm sure windows has something similar if you're in windows.

Answer 4

Another possibility when you're using stdout/stderr for something else is to start a thread to write and feed messages to a queue

import threading
from queue import Queue

def worker(Q, file_dest):
    with open(file_dest, "r+", buffering=1) as fh:
        while True:
            fh.write(f"{}\n".format(Q.get()))

...
    dest_file = "some-writeable/path.txt"
    Q = Queue()  # can be a global, but please name it better!
    t = threading.Thread(
        worker,
        args=(Q, dest_file),
        daemon=True,  # don't block exit
    )
    # set t.daemon = True here instead in earlier Python
    t.start()

whenever you want to write, use the .put() method

Q.put("some message")

buffering=1 is used for line-buffering

Answer 5

It may happen that you need so-called 'function trace' described for PHP. I created Python tool that produces similar output using mentioned by pyfunc sys.settrace() hook.