0

I created a package like:

Package/
├── setup.py
├── Package/
    ├── __init__.py
    ├── __main__.py
    ├── main_steps.py
    ├── code_1.py
    ├── code_2.py
    ├── ...

I performed the analysis of my data step by step in main_steps.py, which is something like:

# main_steps.py
import logging
from . import code_1
from . import code_2


class main_analysis:
    def __init__(self, **kwargs):
        self.param_1 = kwargs.get("param1")
        self.param_2 = kwargs.get("param2")
        # other params....
    def analyze(self, data_file):
        # logging step 1
        # step 1

        # logging step 2
        # step 2
        
        # logging step ....
        # step ...

I have successfully set up the package and tried to logging for every step of my implementations, as designed above. In __main__.py, I can simply do:

# __main__.py
import argparse
from Package import main_steps


def parse_args():
    # configure input data file and parameters here from command line using python's argparse


def main():
    args = parse_args()
    # configure logging here
    logging.basicConfig(...)

    analysis = main_steps.main_analysis(param1=args.param1, param2=args.param2)
    analysis.analyze(args.data_file)


if __name__=="__main__":
    main()

So the package can be run using python -m Package from command line.

I can expect some of the exceptions from the steps I implemented, that can be captured and logged to my .log file using the code like:

try:
    step 1
except expected_exception  # like FileNotFoundError, ValueError, etc.
    logging.exception(msg)
    raise

However, many other unexpected exceptions may raise from the whole package, what should I do? Should I use the following codes in __main__.py?

try:
    analysis.analyze(args.data_file)
except Exception:
    logging.exception("Unexpected exception")
    raise

So if other exception beyond what I have captured in the analysis, the unknown exception can still be logged. If this is not the right way, what's the best practice?

==================

Update for backup

Just found this and this thread related.

Elkan
  • 546
  • 8
  • 23
  • 2
    This is somewhat opinion-based question. What you proposed, with an outermost try/except block, is one way. Another way I've seen is logging from an [exception hook](https://docs.python.org/3/library/sys.html#sys.excepthook). Yet another way is not to log to a file handler at all, but to log to stdout/stderr and then redirect those output streams to an externally configured logging system such as syslog - in this case you would not need to log _and_ raise because an unhandled exception + traceback context would already print on stderr and get consumed that way, when the process exits nonzero. – wim Jun 22 '22 at 03:37
  • Thanks. `sys.excepthook` seems helpful, will try to dig it deeper. – Elkan Jun 22 '22 at 03:55

0 Answers0