Easier way to enable verbose logging

Question

I want to add a debug print statement test, if I enable --verbose from the command line and if I have the following in the script.

logger.info("test")

I went through the following questions, but couldn't get the answer...

• How to implement the --verbose or -v option into a script?

• Python logging - Is there something below DEBUG?

Answer 1

I find both --verbose (for users) and --debug (for developers) useful. Here's how I do it with logging and argparse:

import argparse
import logging

parser = argparse.ArgumentParser()
parser.add_argument(
    '-d', '--debug',
    help="Print lots of debugging statements",
    action="store_const", dest="loglevel", const=logging.DEBUG,
    default=logging.WARNING,
)
parser.add_argument(
    '-v', '--verbose',
    help="Be verbose",
    action="store_const", dest="loglevel", const=logging.INFO,
)
args = parser.parse_args()    
logging.basicConfig(level=args.loglevel)

So if --debug is set, the logging level is set to DEBUG. If --verbose, logging is set to INFO. If neither, the lack of --debug sets the logging level to the default of WARNING.

Answer 2

You need to combine the wisdom of the Argparse Tutorial with Python's Logging HOWTO. Here's an example...

> cat verbose.py 
#!/usr/bin/env python

import argparse
import logging

parser = argparse.ArgumentParser(
    description='A test script for http://stackoverflow.com/q/14097061/78845'
)
parser.add_argument("-v", "--verbose", help="increase output verbosity",
                    action="store_true")

args = parser.parse_args()
if args.verbose:
    logging.basicConfig(level=logging.DEBUG)

logging.debug('Only shown in debug mode')

Run the help:

> ./verbose.py -h
usage: verbose.py [-h] [-v]

A test script for http://stackoverflow.com/q/14097061/78845

optional arguments:
  -h, --help     show this help message and exit
  -v, --verbose  increase output verbosity

Running in verbose mode:

> ./verbose.py -v
DEBUG:root:Only shown in debug mode

Running silently:

> ./verbose.py   
> 

Answer 3

Here is a more concise method, that does bounds checking, and will list valid values in help:

parser = argparse.ArgumentParser(description='This is a demo.')
parser.add_argument("-l", "--log", dest="logLevel", choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'], help="Set the logging level")

args = parser.parse_args()
if args.logLevel:
    logging.basicConfig(level=getattr(logging, args.logLevel))

Usage:

demo.py --log DEBUG

Answer 4

Another variant would be to count the number of -v and use the count as an index to the a list of logging levels:

import argparse
import logging
    
parser = argparse.ArgumentParser()
parser.add_argument('-v', '--verbose', action='count', default=0)
args = parser.parse_args()

levels = [logging.WARNING, logging.INFO, logging.DEBUG]
level = levels[min(args.verbose, len(levels) - 1)]  # cap to last level index
logging.basicConfig(level=level)
    
logging.debug('debug message')
logging.info('info message')
logging.warning('warning message')
logging.error('error message')
logging.critical('critical message')

• If no option is provided then logging.WARNING is selected.
• If one option -v is provided then logging.INFO is selected.
• If two options -v -v (or -vv) or more are provided then logging.DEBUG is selected.

Answer 5

You can explicity specify a level as an integer after the -v flag:

parser = argparse.ArgumentParser()
parser.add_argument("-v", "--verbose", const=1, default=0, type=int, nargs="?",
                    help="increase verbosity: 0 = only warnings, 1 = info, 2 = debug. No number means info. Default is no verbosity.")
args = parser.parse_args()

logger = logging.getLogger()
if args.verbose == 0:
    logger.setLevel(logging.WARN) 
elif args.verbose == 1:
    logger.setLevel(logging.INFO) 
elif args.verbose == 2:
    logger.setLevel(logging.DEBUG) 

Answer 6

if you want to enable logging.DEBUG level for a script you don't want to (or cannot) edit, you can customize your startup:

jcomeau@aspire:~$ python -c "import site; site._script()"
[snip]...
USER_BASE: '/home/jcomeau/.local' (exists)
USER_SITE: '/home/jcomeau/.local/lib/python2.7/site-packages' (exists)
ENABLE_USER_SITE: True
jcomeau@aspire:~$ mkdir -p ~/.local/lib/python2.7/site-packages
jcomeau@aspire:~$ vi ~/.local/lib/python2.7/site-packages/usercustomize.py

enter the following:

import os, logging
if os.getenv('DEBUGGING'):
    logging.basicConfig(level = logging.DEBUG)

then you can just:

jcomeau@aspire:~$ mkdir -p /tmp/some/random/
jcomeau@aspire:~$ echo 'import logging; logging.debug("test")' >> /tmp/some/random/script.py
jcomeau@aspire:~$ DEBUGGING=1 python /tmp/some/random/script.py 
DEBUG:root:test

from Paul Ollis at http://nedbatchelder.com/blog/201001/running_code_at_python_startup.html

---

2017-07-18: I've since switched to a different method:

logging.basicConfig(level=logging.DEBUG if __debug__ else logging.INFO)

what this does is, if you're running without optimization (as in python script.py) you get the DEBUG-level stuff, whereas if you run with python -OO script.py you don't. no environment variables to set.

Answer 7

Here's another take on having argparse count the -v option to increase verbosity up two levels from the default WARNING to INFO (-v) to DEBUG (-vv). This does not map to the constants defined by logging but rather calculates the value directly, limiting the input:

print( "Verbosity / loglevel:", args.v )
logging.basicConfig( level=10*(3-max(0,min(args.v,3))) )
logging.debug("debug") # 10
logging.info("info") # 20
logging.warning("warning") # 30 - The default level is WARNING, which means that only events of this level and above will be tracked
logging.error("error") # 40
logging.critical("critical") # 50