8

I knew the reason for this a while back but I have since forgotten and 5 mins of googling hasn't revealed the answer.

I have written a python script which has two handlers. One for files and one for streams.

Everything works as I want.

Now, I wanted to quickly grep for something in the output that was being printed to the terminal but piping the script's output through grep doesn't appear to be working in that all of the output still get's printed to the terminal.

I am using unix and python 2.7

This is probably a duplicate question but I can't find the answer.

Here's my setup of the logging module:

def setup_logger(verbosity):
    #logger = logging.getLogger(regress.app_name)
    logger = logging.getLogger('components')
    logger.setLevel(logging.DEBUG)
    # create file handler which logs even debug messages
    fh = logging.FileHandler(Config.logging_file_name, mode='w')
    fh.setLevel(logging.DEBUG)
    # create console handler with a higher log level
    ch = logging.StreamHandler()
    ch.setLevel({True:logging.DEBUG, False:logging.INFO}[verbosity])
    # create formatter and add it to the handlers
    file_formatter = logging.Formatter('%(asctime)s - %(pathname)s - %(funcName)s - %(name)s - %(levelname)s - %(lineno)s - %(message)s')

    console_formatter = logging.Formatter('%(filename)s - %(lineno)s - %(levelname)s - %(message)s')
    fh.setFormatter(file_formatter)
    ch.setFormatter(console_formatter)
    # add the handlers to the logger
    logger.addHandler(fh)
    logger.addHandler(ch)

    #logging.abort = abort
    #logging.abort("testing...")
    #logger.info("does this happen?")

    #logging.func = func
    #logging.func()
    #logger.func()

My invocation of the script looks like this:

<script_name> <script args> | grep -i <search_string>

Gregory Kuhn
  • 1,627
  • 2
  • 22
  • 34
  • 5
    Logging prints messages to stderr, but you're piping stdout. https://stackoverflow.com/questions/2342826/how-to-pipe-stderr-and-not-stdout – Blender Oct 30 '15 at 14:43
  • @Blender wow I didn't know that! If you want to put that as the answer, I will mark this answered – Gregory Kuhn Oct 30 '15 at 14:49

1 Answers1

11

As @Blender mentions in the comments below the original question, you just need to redirect stderr. You can do this by adding a 2>&1 redirect before the pipe. So, for example,

python main.py 2>&1 | grep INFO

will filter for any INFO lines logged by main.py.

Doug Bradshaw
  • 1,452
  • 1
  • 16
  • 20