use doctest and logging in python program

Question


#!/usr/bin/python2.4
import logging
import sys
import doctest
def foo(x):
        """
    >>> foo (0)
    0
        """
        print ("%d" %(x))
        _logger.debug("%d" %(x))
def _test():
        doctest.testmod()
_logger = logging.getLogger()
_logger.setLevel(logging.DEBUG)
_formatter = logging.Formatter('%(message)s')
_handler = logging.StreamHandler(sys.stdout)
_handler.setFormatter(_formatter)
_logger.addHandler(_handler)
_test()

I would like to use logger module for all of my print statements. I have looked at the first 50 top google links for this, and they seem to agree that doctest uses it's own copy of the stdout. If print is used it works if logger is used it logs to the root console. Can someone please demonstrate a working example with a code snippet that will allow me to combine. Note running nose to test doctest will just append the log output at the end of the test, (assuming you set the switches) it does not treat them as a print statement.

You might look at the highest voted answer (it was written after the first answer was accepted) and see if it might be worth changing to be the accepted answer? — 00prometheus, Nov 24 '22 at 17:50

00prometheus · Answer 1 · 2022-09-24T14:31:05.870

Within your doctest, before any logging capture is needed, do a addHandler(logging.Streamhandler(sys.stdout)) on your logger.

For example, assuming loggeris your logging object:

"""
This is a doctest that will capture output from the logging module.

>>> logger.addHandler(logging.StreamHandler(sys.stdout))

The rest of your doctest, now able to use output from the logging
module...
"""

Explanation: As you say, "doctest uses it's own copy of the stdout". It does this by spoofing sys.stdout: It sets sys.stdout to DocTestRunner._fakeout. Once the doctest is running, doctest has its spoof in place, so sys.stdout is now set to DocTestRunner._fakeout. If you create a logging.StreamHandler for sys.stdout at this point, sys.stdout will point to doctest's spoof for sys.stdout rather than the real sys.stdout, so you are now adding a StreamHandler for doctest's spoof instead of the real sys.stdout, making doctest see your logged output.

This solution also has the advantage not depending on internal private variables in doctest, such as _fakeout or _SpoofOut, in case they change in the future.

Gotchas: If you get

error: [Errno 128] Transport endpoint is not connected

you might have forgotten to import sys.

`>>> import sys; logger.addHandler(logging.StreamHandler(sys.stdout))` If you need to import sys only in the doctests, and want an one-liner. — luckydonald, Oct 30 '16 at 14:31
This has the added benefit that logger output no longer show up in the sequence of '.' characters that indicate tests being run. — Jim DeLaHunt, Jul 19 '18 at 02:08

score 1 · Accepted Answer · answered Apr 28 '10 at 14:41

I'm not sure why you want to do this, but if you really need to do it, then you can define your own subclass of DocTestRunner, and override the run method:

#imports left out, code not tested - just to point you in the right direction
class MyDocTestRunner(DocTestRunner):
    def run(self, test, compileflags=None, out=None, clear_globs=True):
        if out is None:
            handler = None
        else:
            handler = StreamHandler(out)
        logger = logging.getLogger() # root logger (say)
        if handler:
            logger.addHandler(handler)
        try:
            DocTestRunner.run(self, test, compileflags, out, clear_globs)
        finally:
            if handler:
                logger.removeHandler(handler)
                handler.close()

Then use this runner in place of DocTestRunner.

ok, thanks you for the response I am attempting to implement it. You are of the opinion that developers do not normally mix doctest/logging and would use a mix of logging (which is excluded by a doctest SKIP's) and only use print for their doctest verification? — Luke, Apr 29 '10 at 18:04
I solved it by configuring nose to run doctest which excludes logging, and just using print statements for anything that needs to do the logging comparision.
#!/usr/bin/python2.4 sys.path.append('include/nose') import nose import nose.ext.dtcompat as doctest — Luke, May 03 '10 at 23:37

score 1 · Answer 3 · edited Aug 17 '22 at 07:05

1

One simple and general-purpose approach looks like this:

import sys
import logging

LOGGER = logging.getLogger(__name__)
if hasattr(sys.modules['__main__'], '_SpoofOut'):
    LOGGER.setLevel(logging.DEBUG)
    LOGGER.addHandler(logging.StreamHandler())

The _SpoofOut attribute is injected by the doctest module. If it's present, you can configure logging specifically for doctest. E.g. in my example, set a verbose debug mode and log to the console.

edited Aug 17 '22 at 07:05

mihow

334
4
13

answered Jul 12 '16 at 20:09

Chris Johnson

20,650
6
81
80

This works fine for me in Python 3.9 and seems simpler than other solutions – mihow Aug 16 '22 at 22:33

use doctest and logging in python program

3 Answers3

Linked