I created a package like:
Package/
├── setup.py
├── Package/
├── __init__.py
├── __main__.py
├── main_steps.py
├── code_1.py
├── code_2.py
├── ...
I performed the analysis of my data step by step in main_steps.py
, which is something like:
# main_steps.py
import logging
from . import code_1
from . import code_2
class main_analysis:
def __init__(self, **kwargs):
self.param_1 = kwargs.get("param1")
self.param_2 = kwargs.get("param2")
# other params....
def analyze(self, data_file):
# logging step 1
# step 1
# logging step 2
# step 2
# logging step ....
# step ...
I have successfully set up the package and tried to logging
for every step of my implementations, as designed above. In __main__.py
, I can simply do:
# __main__.py
import argparse
from Package import main_steps
def parse_args():
# configure input data file and parameters here from command line using python's argparse
def main():
args = parse_args()
# configure logging here
logging.basicConfig(...)
analysis = main_steps.main_analysis(param1=args.param1, param2=args.param2)
analysis.analyze(args.data_file)
if __name__=="__main__":
main()
So the package can be run using python -m Package
from command line.
I can expect some of the exceptions from the steps I implemented, that can be captured and logged to my .log
file using the code like:
try:
step 1
except expected_exception # like FileNotFoundError, ValueError, etc.
logging.exception(msg)
raise
However, many other unexpected exceptions may raise from the whole package, what should I do? Should I use the following codes in __main__.py
?
try:
analysis.analyze(args.data_file)
except Exception:
logging.exception("Unexpected exception")
raise
So if other exception beyond what I have captured in the analysis, the unknown exception can still be logged. If this is not the right way, what's the best practice?
==================
Update for backup