0

I am using the following code portion and config YAML to use logger in python.

I want to force logging.conf module creating a folder stated in yaml config ,

filename: log/sft-upload-400-error.log

lets say if "log" directory does not exist it is created automatically.

 try:
        with open(value, 'r') as f:
            config = yaml.safe_load(f.read())
            logging.config.dictConfig(config)
    except FileNotFoundError:
        print(f'File{value} does not exist, default logging conf is used')

config.yaml

version: 1
formatters:
  simple:
    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: simple
    stream: ext://sys.stdout
  sft-upload-400-handler:
    class: logging.handlers.RotatingFileHandler
    level: DEBUG
    formatter: simple
    filename: log/sft-upload-400-error.log
    maxBytes: 10485760 # 10MB
loggers:
  sampleLogger:
    level: DEBUG
    handlers: [console]
    propagate: no
  sftUpload400Logger:
    level: DEBUG
    handlers: [ sft-upload-400-handler ]
    propagate: no
root:
  level: DEBUG
  handlers: [console]

Here is the error when "log" does not exist:

Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/logging/config.py", line 563, in configure
    handler = self.configure_handler(handlers[name])
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/logging/config.py", line 744, in configure_handler
    result = factory(**kwargs)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/logging/handlers.py", line 148, in __init__
    BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/logging/handlers.py", line 55, in __init__
    logging.FileHandler.__init__(self, filename, mode, encoding, delay)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/logging/__init__.py", line 1147, in __init__
    StreamHandler.__init__(self, self._open())
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/logging/__init__.py", line 1176, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/ahmet/projects/sample/log/sft-upload-400-error.log'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/ahmet/Library/Python/3.8/bin/locust", line 8, in <module>
    sys.exit(main())
  File "/Users/ahmet/Library/Python/3.8/lib/python/site-packages/locust/main.py", line 125, in main
    docstring, user_classes, shape_class = load_locustfile(locustfile)
  File "/Users/ahmet/Library/Python/3.8/lib/python/site-packages/locust/main.py", line 81, in load_locustfile
    imported = source.load_module()
  File "<frozen importlib._bootstrap_external>", line 462, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 962, in load_module
  File "<frozen importlib._bootstrap_external>", line 787, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ValueError: Unable to configure handler 'sft-upload-400-handler'
Ahmet Karakaya
  • 9,899
  • 23
  • 86
  • 141
  • 2
    You can subclass RotatingFileHandler to create directories that don't exist. – AKX Feb 24 '22 at 08:19
  • 1
    `FileHandler` and the built-in derived handlers don' try to create the folder. The `FileHandler` constructor [simply creates a file with that path](https://github.com/python/cpython/blob/f4c03484da59049eb62a9bf7777b963e2267d187/Lib/logging/__init__.py#L1144). You'd have to subclass the handler you want and create the folder you want during initialization – Panagiotis Kanavos Feb 24 '22 at 08:36
  • 1
    There's also good info here: https://stackoverflow.com/questions/15693529/how-to-specify-in-yaml-to-always-create-log-file-in-the-projects-folder-using-d – Cow Feb 24 '22 at 08:42

0 Answers0