Filtering python logging module to only log last line of an exception?

Question

I'm utilizing the following code to print exceptions from a python module I'm running inside of my script:

        except Exception as e:
          logging.info('yt-dlp initated a break', exc_info=True)

Currently it outputs much more information than I need.

Something along the lines of:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/web_ripper.py", line 58, in subscriptions_loop
    ydl.download([url])
  File "/app/venv/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 3113, in download
    self.__download_wrapper(self.extract_info)(
  File "/app/venv/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 3086, in wrapper
    res = func(*args, **kwargs)
  File "/app/venv/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1344, in extract_info
    return self.__extract_info(url, self.get_info_extractor(ie_key), download, extra_info, process)
  File "/app/venv/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1371, in wrapper
    self.report_error(str(e), e.format_traceback())
  File "/app/venv/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 912, in report_error
    self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
  File "/app/venv/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 853, in trouble
    raise DownloadError(message, exc_info)
yt_dlp.utils.DownloadError: ERROR: [youtube:tab] Videogamedunkey: Unable to download API page: HTTP Error 404: Not Found

What's the most efficient way to simply print the last line of the exception?

Does this answer your question? [Formatting exceptions as Python does](https://stackoverflow.com/questions/35498555/formatting-exceptions-as-python-does) — Lenormju, Jan 14 '22 at 10:37

score 0 · Answer 1 · answered Feb 10 '23 at 10:34

You can use a custom Formatter that will process the exc_text of the record you are trying to log.

The following MyFormatter will check if the record has an exception text. If not, it will create it using self.formatException. Then, since record.exc_text is a string storing the trace as text, with \n as separators between lines, you can keep the last line using .split("\n")[-1].

import logging

class MyFormatter(logging.Formatter):
    def format(self, record):
        # https://github.com/python/cpython/blob/main/Lib/logging/__init__.py#L711
        if not record.exc_text:
            record.exc_text = self.formatException(record.exc_info)
        record.exc_text = record.exc_text.split("\n")[-1]
        return super().format(record)

Then, add the following code to use your new MyFormatter:

def a(x):
    return b(x)

def b(x):
    return c(x)

def c(x):
    return d(x)

def d(x):
    return x / 0

logger = logging.getLogger("foobar")
logger.setLevel(logging.INFO)

stream_handler = logging.StreamHandler()
stream_handler.setFormatter(MyFormatter(fmt="%(name)s - %(levelname)s - %(message)s"))
logger.addHandler(stream_handler)

try:
    a(1)
except Exception as e:
    logger.info('yt-dlp initated a break', exc_info=True)

and you get:

foobar - INFO - yt-dlp initated a break
ZeroDivisionError: division by zero

instead of the previous:

foobar - INFO - yt-dlp initated a break
Traceback (most recent call last):
  File "70704112.py", line 31, in <module>
    a(1)
  File "70704112.py", line 12, in a
    return b(x)
  File "70704112.py", line 15, in b
    return c(x)
  File "70704112.py", line 18, in c
    return d(x)
  File "70704112.py", line 21, in d
    return x / 0
ZeroDivisionError: division by zero

Filtering python logging module to only log last line of an exception?

1 Answers1