Dynamic imports and logging: No handlers could be found for logger

Question

I am dynamically loading a python module via the map(__import__, 'scripts.myscript') trick. I am configuring the logger in the parent module, and logging in the parent module works fine. However, any logging in the dynamically loaded module returns the following message to stderr:

No handlers could be found for logger

The only workaround that I have is to set the dynamically loaded module's logger attribute to the parent module's logger attribute; e.g.:

logger = logging.getLogger(__name_)
# ... configure logger
script = map(__import__, 'scripts.myscript')[0]
script.myscript.logger = logger

Is there a superior way to get the dynamically loaded module to accept the settings of the logger in the parent?

Incidentally, why doesn't the normal logger propagation take place when dynamically importing modules like this?

My project structure:

/app
    /app
        config.py
        ...
    /scripts
        myscript.py
        ...
    script.py

In script.py, I dynamically load myscript.py similar to the following:

import logging
import sys

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s %(levelname)-8s %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)


# get 'myscript' from args
script_name = sys.argv[1]
script = map(__import__, ['scripts.{}'.format(script_name)])[0]

# The workaround is to set the dynamic module's logger attribute
# to the parent module's attribute, like this:
# getattr(script, script_name).logger = logger

# call main() function in myscript.py
errno = getattr(script, script_name).main()

# This is logged successfully
logger.info("Errno from {} is {}".format(script_name, errno)

myscript.py has something similar to the following:

from app.config import * # See note below
import logging
logger = logging.getLogger(__name__)

def main():
    # This results in: No handlers could be found for logger
    logger.info("Hello world")
    return 0

In case anyone is wondering why I am going through all of this hassle in the first place, it is because of the project structure above. I want the scripts directory to be on the same level as the app directory, but I want individual scripts to have access to the app/config.py module. I don't want to modify the python/sys path inside of the individual scripts, and I also don't want to install the app into site-packages. I also like that there is one entry point, script.py, to all the scripts.

Dynamic imports and logging: No handlers could be found for logger

0 Answers0