10

referring to this question here: LINK How can I set up a config, that will only log my root script and my own sub-scripts? The question of the link asked for disabling all imported modules, but that is not my intention.

My root setup:

import logging
from exchangehandler import send_mail
log_wp = logging.getLogger(__name__)

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s - %(levelname)s [%(filename)s]: %(name)s %(funcName)20s - Message: %(message)s',
                    datefmt='%d.%m.%Y %H:%M:%S',
                    filename='C:/log/myapp.log',
                    filemode='a')

handler = logging.StreamHandler()
log_wp.addHandler(handler)

log_wp.debug('This is from root')

send_mail('address@eg.com', 'Request', 'Hi there')

My sub-module exchangehandler.py:

import logging
log_wp = logging.getLogger(__name__)

def send_mail(mail_to,mail_subject,mail_body, mail_attachment=None):
log_wp.debug('Hey this is from exchangehandler.py!')
    m.send_and_save()

myapp.log:

16.07.2018 10:27:40 - DEBUG [test_script.py]: __main__                   <module> - Message: This is from root
16.07.2018 10:28:02 - DEBUG [exchangehandler.py]: exchangehandler        send_mail - Message: Hey this is from exchangehandler.py!
16.07.2018 10:28:02 - DEBUG [folders.py]: exchangelib.folders            get_default_folder - Message: Testing default <class 'exchangelib.folders.SentItems'> folder with GetFolder
16.07.2018 10:28:02 - DEBUG [services.py]: exchangelib.services          get_payload - Message: Getting folder ArchiveDeletedItems (archivedeleteditems)
16.07.2018 10:28:02 - DEBUG [services.py]: exchangelib.services          get_payload - Message: Getting folder ArchiveInbox (archiveinbox)

My problem is, that the log-file contains also a lot of information of the exchangelib-module, that is imported in exchangehandler.py. Either the imported exchangelib-module is configured incorrectly or I have made a mistake. So how can I reduce the log-output only to my logging messages?

EDIT: An extract of the folder.py of the exchangelib-module. This is not anything that I have written:

import logging
log = logging.getLogger(__name__)

def get_default_folder(self, folder_cls):
        try:
            # Get the default folder
            log.debug('Testing default %s folder with GetFolder', folder_cls)
            # Use cached instance if available
            for f in self._folders_map.values():
                if isinstance(f, folder_cls) and f.has_distinguished_name:
                    return f
            return folder_cls.get_distinguished(account=self.account)
Erik Steiner
  • 581
  • 1
  • 5
  • 18

1 Answers1

19

The imported exchangelib module is not configured at all when it comes to logging. You are configuring it implicitly by calling logging.basicConfig() in your main module.

exchangelib does create loggers and logs to them, but by default these loggers do not have handlers and formatters attached, so they don't do anything visible. What they do, is propagating up to the root logger, which by default also has no handlers and formatters attached.

By calling logging.basicConfig in your main module, you actually attach handlers to the root logger. Your own, desired loggers propagate to the root logger, hence the messages are written to the handlers, but the same is true for the exchangelib loggers from that point onwards.

You have at least two options here. You can explicitly configure "your" named logger(s):

main module

import logging
log_wp = logging.getLogger(__name__) # or pass an explicit name here, e.g. "mylogger"
hdlr = logging.StreamHandler()
fhdlr = logging.FileHandler("myapp.log")
log_wp.addHandler(hdlr)
log_wp.addHandler(fhdlr)
log_wp.setLevel(logging.DEBUG)

The above is very simplified. To explicitly configure multiple named loggers, refer to the logging.config HowTo

If you rather want to stick to just using the root logger (configured via basicConfig()), you can also explicitly disable the undesired loggers after exchangelib has been imported and these loggers have been created:

logging.getLogger("exchangelib.folders").disabled = True
logging.getLogger("exchangelib.services").disabled = True

If you don't know the names of the loggers to disable, logging has a dictionary holding all the known loggers. So you could temporarily do this to see all the loggers your program creates:

# e.g. after the line 'log_wp.addHandler(handler)'
print([k for k in logging.Logger.manager.loggerDict])

Using the dict would also allow you to do sth. like this:

for v in logging.Logger.manager.loggerDict.values():
    if v.name.startswith('exchangelib'):
        v.disabled = True
shmee
  • 4,721
  • 2
  • 18
  • 27
  • Thank you! I used fileConfig in my main.py to set up an output to console, file1 with level debug and file2 with level info. However, I got a ValueError in my main.py. This is not logged to file1 nor file2. I can only see it in the colsole. How can I catch such error without knowing that it could occur? [link_to_config](https://pastebin.com/TgQZHq05) – Erik Steiner Jul 16 '18 at 14:23
  • Is the ValueError related to your new logging setup? Could you please add the Traceback to your question? – shmee Jul 16 '18 at 14:34
  • The setup of logging is fine. There is a problem in my regular code that raise a ValueError: Traceback (most recent call last):
    x_usr_name, x_password, x_mail, x_servername, x_server, x_endpoint, x_contacts = load_setup()
    ValueError: too many values to unpack (expected 7)
    But this is a good example for my logging setup which will not log this error in my files. I just see it in the console of PyCharm. So in general write a function that will raise an error and you will see that it is not logged in a logging file.     – Erik Steiner Jul 16 '18 at 14:41
  • In that case, simply put the part of your code that raises the error inside a `try:`-`except ValueError as e:`-block. Inside the `except` you'd do sth. like `log_wp.exception(e)`. Please note that you'll have to re-raise the caught exception after logging it in the `except` block in oder to terminate your program if the severity of the caught exception warrants this. – shmee Jul 16 '18 at 14:48
  • What would I do if I don't know that there could happen something bad? Isn't it possible to catch an error even if I don't use try. Your solution would mean, that I have to know all possible errors. Moreover, how should I fix those errors, if they will not be printed to my logging files? – Erik Steiner Jul 16 '18 at 14:50
  • 1
    This is not an issue introduced by logging, nor can logging solve that on it's own. Logging only does things if you instruct it to do so. You could wrap your whole main program in a `try-except Exception as e`. That would catch any Exception that raises during execution and is not caught. However, it is the developer's responsibility to identify and handle all potential error cases that might occur during execution, either by validating input before processing or by handling potential excpetions accordingly. – shmee Jul 16 '18 at 14:59
  • Thank you for the clarification. That helped a lot. – Erik Steiner Jul 16 '18 at 15:00
  • If you want to see where your program break in order to fix or handle the errors you might want to look at [redirecting sys.stderr](https://stackoverflow.com/a/1956228) to a file – shmee Jul 16 '18 at 15:16