How do I print Python errors to a txt file, if I don't know where the error is

Question

I have a very large Python 3.x program running in Windows. It works great 99.9% of the time, but occasionally it crashes. I'm not sure what is causing the crash, it could be numerous things. Due to the fact that I have to run the program "compiled" .exe with an invisible console for security reasons (don't ask), I don't get to see any form of console readout when it dies. So obviously it would be great if I could have it output the crash traceback as a text file instead.

I'm familiar with try/except in Python but the piece of code that's causing the issue could be anywhere and I don't want to have to write an individual try/except around every single line of the literally thousands of lines of code. Is there a way that I can get the program to always output any program-stopping error as a text file, no matter what line of code is causing the problem, or what the error might be?

Answer 1

Somewhere in your code you must have a single entry-point into the code that might be crashing (in the main script, at a minimum). You can wrap that in a try/except pair and then use functions from the traceback module to print the exception to a file when it happens:

Change:

if __name__ == "__main__":
    do_stuff()

To:

import traceback

if __name__ == "__main__":
    try:
        do_stuff()
    except:
        with open("exceptions.log", "a") as logfile:
            traceback.print_exc(file=logfile)
        raise

If you want to, you could add some extra code to write extra output to the file, with a time/date stamp or whatever other information you think might be useful. You may want to add additional try/except blocks, more or less like the one above, if you want to give special scrutiny to certain parts of your code. For instance, you could put a block in a loop, where you can print out the loop value if an exception occurs:

for x in some_iterable:
    try:
        do_something_with(x)
    except:
        with open("exceptions.log", "a") as logfile:
            print("Got an exception while handling {!r} in the loop:".format(x)
            traceback.print_exc(file=logfile)
        raise   # you could omit this line to suppress the exception and keep going in the loop

You could also use the logging module, if you want a more configurable system for the file writing end of the issue. The logging.debug and logging.exception functions both read the same exception information used by the traceback module, but with many more options for formatting things yourself (if you want that). Note that setting up logging is a bit more involved than just opening a file manually.

Answer 2

sometimes you cant use try/except or > in terminal.

you can use sys excepthook. add this to beginning:

import sys
import traceback


def excepthook(exctype, value, tb):
    with open("mylog.txt", "w") as mylog:
        traceback.print_exception(exctype, value, tb, file=mylog)


sys.excepthook = excepthook
##########
# your code

after that, all traceback will be print to mylog.txt.

Answer 3

I ended up writing my own logging function

# Takes two inputs - logfile (path to desired .csv), and data to be written
# Writes "Y-M-D|H:M:S, data\n"
    f = open(logfile, 'a+')
    currentdate = time.strftime('%Y-%m-%d|%H:%M:%S')
    f.write(currentdate + ',' + data +'\n')
    f.close()

it requires time or datetime, I'm not sure which. Also you need to make sure that the log file exists.

Then I would just plop it wherever I needed, eg: logger(ERRLOG, "OCR didn't find poop. Check {}".format(ocr_outfilepath))

Answer 4

I'm not sure what kind of program this is or how you are running it, but you could try running your Python program and redirecting all its output (or all errors) to a file.

For example, if I have a very-contrived sample Python script like this

def do_stuff():
    s = [1, 2, 3, 4, 5]
    print(s[6])

if __name__ == "__main__":
    do_stuff()

which is deliberately going to raise an IndexError exception.

You could run it like this:

$ python test.py &> mylogs.txt

$ cat mylogs.txt 
Traceback (most recent call last):
  File "test.py", line 8, in <module>
    do_stuff()
  File "test.py", line 4, in do_stuff
    print(s[6])
IndexError: list index out of range

which redirects all output and errors to a file.

Or, if you can have it displayed on a console and also redirect it to a file:

$ python test.py 2>&1 | tee mylogs.txt
Traceback (most recent call last):
  File "test.py", line 8, in <module>
    do_stuff()
  File "test.py", line 4, in do_stuff
    print(s[6])
IndexError: list index out of range

$ cat mylogs.txt 
Traceback (most recent call last):
  File "test.py", line 8, in <module>
    do_stuff()
  File "test.py", line 4, in do_stuff
    print(s[6])
IndexError: list index out of range

This way, you don't need to modify anything with the code.

Note that this solution is for Linux or Mac systems.
See other StackOverflow posts for redirecting Python output to a file.