0

I'm using Pythons logging library to log some events in a few scripts. Each script is run as a separate process, some are run as root whilst others are run as another user - the default Raspberry Pi user in my case. I've set up the configuration in a single YAML file and passed this into dictConfig() when the script starts up. None of the scripts share a logfile - they only share the config.

Simplified YAML file:

version: 1
disable_existing_loggers: true

formatters:
  standard:
    format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"

handlers:
  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: standard
    stream: ext://sys.stdout

  scanner_handler:
    class: logging.handlers.RotatingFileHandler
    formatter: standard
    filename: /tmp/scanner.log
    maxBytes: 10485760 #MB
    backupCount: 3
    encoding: utf8

  prehook_handler:
    (): helpers.AllWriteRotatingFileHandler
    filename: /tmp/prehook.log
    formatter: standard
    maxBytes: 10485760 #MB
    backupCount: 3 
    encoding: utf8

root:
  level: NOTSET
  handlers: [console]
  propagate: yes

loggers:
  scanner:
    level: DEBUG
    handlers: [scanner_handler]
    propagate: yes

  prehook:
    level: DEBUG
    handlers: [prehook_handler]
    propagate: no

Setup method and a custom file handler that ensures every user can read and write to the file:

import os
import stat
import yaml
import logging.config
import logging.handlers
import logging

def setup_logging(default_path, default_level=logging.INFO):
    path = default_path
    if os.path.exists(path):
        with open(path, 'rt') as f:
            config = yaml.safe_load(f.read())
            logging.config.dictConfig(config)
    else:
        print('Failed to load configuration file. The log file could not be opened.')

class AllWriteRotatingFileHandler(logging.handlers.RotatingFileHandler):
    def _open(self):
        rtv = logging.handlers.RotatingFileHandler._open(self)
        curr_mode = os.stat(self.baseFilename).st_mode
        # Adds read and write permissions to all users
        os.chmod(self.baseFilename, curr_mode | stat.S_IWOTH | stat.S_IROTH)
        return rtv

Then in my respective scripts I setup the logger as follows:

helpers.setup_logging('/full/path/to/config.yaml')
logger = logging.getLogger('logger_defined_in_config')

I run one script as my default user, then when I run the other using sudo - the following exception is thrown:

Traceback (most recent call last):
  File "/usr/lib/python3.9/logging/config.py", line 564, in configure
    handler = self.configure_handler(handlers[name])
  File "/usr/lib/python3.9/logging/config.py", line 745, in configure_handler
    result = factory(**kwargs)
  File "/usr/lib/python3.9/logging/handlers.py", line 153, in __init__
    BaseRotatingHandler.__init__(self, filename, mode, encoding=encoding,
  File "/usr/lib/python3.9/logging/handlers.py", line 58, in __init__
    logging.FileHandler.__init__(self, filename, mode=mode,
  File "/usr/lib/python3.9/logging/__init__.py", line 1142, in __init__
    StreamHandler.__init__(self, self._open())
  File "/home/pi/.development/the-cake-can-wait/helpers.py", line 20, in _open
    rtv = logging.handlers.RotatingFileHandler._open(self)
  File "/usr/lib/python3.9/logging/__init__.py", line 1171, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding,
PermissionError: [Errno 13] Permission denied: '/tmp/prehook.log'
  • Are they running at the same time? – JonSG Jan 31 '22 at 16:01
  • Yep, the scripts are running concurrently. – TheGoatRustler Jan 31 '22 at 16:09
  • I suspect that you are running into locking at the OS level. You might take a look at `fcntl` and one or both of these posts: https://stackoverflow.com/questions/11853551/python-multiple-users-append-to-the-same-file-at-the-same-time or https://stackoverflow.com/questions/3211292/two-processes-reading-writing-to-the-same-file-python/3211355 – JonSG Jan 31 '22 at 16:17
  • Each script writes to a separate log file though, neither appends to the same file. In addition, after the scripts have finished running if I try to modify and save the log file manually i.e. `sudo vim file.log` append and `:w` I am told that I can't write to it. – TheGoatRustler Jan 31 '22 at 16:27
  • 1
    oh well that takes python out of the picture nicely :-) Unfortunately I don't really know anything about PIs – JonSG Jan 31 '22 at 16:32

0 Answers0