python logging __name__ and packaging

Question

The official python logging documentation says this:

The name is potentially a period-separated hierarchical value, like foo.bar.baz (though it could also be just plain foo, for example). Loggers that are further down in the hierarchical list are children of loggers higher up in the list. For example, given a logger with a name of foo, loggers with names of foo.bar, foo.bar.baz, and foo.bam are all descendants of foo. The logger name hierarchy is analogous to the Python package hierarchy, and identical to it if you organise your loggers on a per-module basis using the recommended construction logging.getLogger(__name__). That’s because in a module, __name__ is the module’s name in the Python package namespace.

But I have found that last sentence not to be strictly true. The value of __name__ seems to depend on how a module is imported. The following demonstrates this:

Given two modules, foo and bar both in package pkg:

This is foo:

def showFoosName():
    print __name__

and this is bar:

from pkg import foo
if __name__ == '__main__':
    foo.showFoosName()

running bar produces

pkg.foo

However, if we introduce baz into package pkg, which imports foo simply as foo, which is legal because both are in the same package:

import foo
if __name__ == '__main__':
    foo.showFoosName() 

running baz produces

foo

Now bringing this discussion back to logging and logging configuration, it seems that the python logging recommendation of naming loggers with __name__ only holds good if imports within a package are coded with from pkg import bar syntax. This is not what I would have expected. I would have thought that __name__'s value would be independent of how it was imported.

Answer 1

This is a symptom of your import path being broken. You have both pkg and its containing directory on your path, causing foo.py to correspond to two separate modules, foo and pkg.foo.

Likely, the directory containing pkg is on your import path, but you are running bar.py and baz.py directly by file name, causing pkg itself to be on the path too. Running

python -m pkg.bar

or

python -m pkg.baz

should prevent pkg from being added to the path.

That's only one possible way your import path could have been messed up. We can't really tell. Reading up on the Python import system should help; here's one good resource.

Answer 2

The thing about Python packages and modules is that, like everything else in Python, they are dynamic. The __name__ of each module depends on where it is relative to the Python root paths (on sys.path). So if you have a folder named pkg, even if it contains an __init__.py, it is not actually a "package" if it is on sys.path directly. If the pkg folder is on sys.path directly (for example, because you launched the Python interpreter there), then it is not a package, and any .py files inside it are top-level modules, not modules inside any package. Thus if you have foo.py inside the pkg folder, when you launch the Python interpreter from inside pkg, foo.py's real, official name is foo, not pkg.foo.

Unless the folder that contains pkg is on sys.path, there is no package pkg -- from pkg import foo would fail with an ImportError. And if both pkg and the folder containing pkg are both on sys.path, then your Python path is malformed, and you have two separate importable modules, pkg.foo and foo (with independent copies of all their globals!).

To resolve this kind of problem, it's a best practice to separate executable scripts and importable modules and packages from each other. This allows you to put all your importable modules and packages into a common place (normally, under site-packages for your Python installation), and your executable scripts can import them using the full name, regardless of where the scripts are located (typically, someplace like /usr/bin or /usr/local/bin).