17

Importing the standard "logging" module pollutes sys.modules with a bunch of dummy entries:

Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32
>>> import sys
>>> import logging
>>> sorted(x for x in sys.modules.keys() if 'log' in x)
['logging', 'logging.atexit', 'logging.cStringIO', 'logging.codecs', 
'logging.os', 'logging.string', 'logging.sys', 'logging.thread', 
'logging.threading', 'logging.time', 'logging.traceback', 'logging.types']

# and perhaps even more surprising:
>>> import traceback
>>> traceback is sys.modules['logging.traceback']
False
>>> sys.modules['logging.traceback'] is None
True

So importing this package puts extra names into sys.modules, except that they are not modules, just references to None. Other modules (e.g. xml.dom and encodings) have this issue as well. Why?

Edit: Building on bobince's answer, there are pages describing the origin (see section "Dummy Entries in sys.modules") and future of the feature.

Air
  • 8,274
  • 2
  • 53
  • 88
Peter Hansen
  • 21,046
  • 5
  • 50
  • 72
  • Good link... I hadn't actually spotted this being documented anywhere before. Evidently for some reason I hadn't searched for the obvious title “The Knights Who Say Neeeow ... Wum ... Ping”. Ah, Python... – bobince Dec 24 '09 at 17:01
  • 1
    I was going to post an exact duplicate - my second today :P – Mr_and_Mrs_D Jan 23 '17 at 17:30

1 Answers1

23

None values in sys.modules are cached failures of relative lookups.

So when you're in package foo and you import sys, Python looks first for a foo.sys module, and if that fails goes to the top-level sys module. To avoid having to check the filesystem for foo/sys.py again on further relative imports, it stores None in the sys.modules to flag that the module didn't exist and a subsequent import shouldn't look there again, but go straight to the loaded sys.

This is a cPython implementation detail you can't usefully rely on, but you will need to know it if you're doing nasty magic import/reload hacking.

It happens to all packages, not just logging. For example, import xml.dom and see xml.dom.xml in the module list as it tries to import xml from inside xml.dom.

As Python moves towards absolute import this ugliness will happen less.

bobince
  • 528,062
  • 107
  • 651
  • 834
  • Thanks! I added links in my question to background in, and referenced by, http://www.python.org/dev/peps/pep-0328 – Peter Hansen Dec 24 '09 at 15:39
  • If it "happens to all packages", why have I never seen this with my own stuff? I'd expect to see "mymodule.sys", "mymodule.os" and so forth with dozens and dozens of such dummy modules around but only see it with a few like xml.dom, logging, and encodings (as D.Shawley mentions). – Peter Hansen Dec 24 '09 at 15:47
  • It happens to my packages. But only packages, not simple modules. – bobince Dec 24 '09 at 15:56
  • Strange. I'm dead certain I've frequently checked sys.modules in the past for various debugging purposes, without noticing dozens of stdlib modules referenced in sys.modules as dummy modules under my subpackages. Yet there they are, with things like 'xxx.xxx.sys' showing up 16 times, for example. – Peter Hansen Dec 24 '09 at 15:57