0

I am building some data integration flows with Python 2.7 on Windows and want to standardize logging. Right now, I have many python scripts running and logging is specified in each. I would like to create a separate class and import it into each .py script so logging can be standardized.

I am trying to set this up but getting an error in my test. Here my separate logging file: my_logger.py.

import logging
import logging.handlers

def setup_logging():
    logging.basicConfig(level = logging.INFO)
    logger = logging.getLogger(__name__)
    handler = logging.handlers.RotatingFileHandler('C:\\Users\\Documents\\log_test.log', maxBytes=1024000, backupCount=5)
    handler.setLevel(logging.INFO)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s -    %(message)s')
    handler.setFormatter(formatter)
    logger.addHandler(handler)

Here is a small test file (test_logger.py) that is in the same directory as my_logger.py.

import numpy as np
import pandas as pd
import my_logger

my_logger.setup_logging()

def generate_random_data():
    logger.info("starting test")
    test_df = pd.DataFrame(np.random.randn(5,4), columns=['a','b','c','d'])
    print test_df

generate_random_data()

How can I set this up so I can reference separate logging class and set logging messages with a single line?

UPDATE : I removed the class and globals per @tdelaney's suggestion. I don't get the errors anymore and the info message prints in console. The log file also gets created. However, the info message does not show up in the log.

What am I missing to get the info message set in the log file?

analyticsPierce
  • 2,979
  • 9
  • 57
  • 81
  • Have you tried `my_logger.logger.info` (see e.g. http://stackoverflow.com/a/15959638/3001761)? But more broadly, why are you using a class just to set up some `global` names? Also, you don't actually call the setup: `python_logger.setup_logging()` (note parentheses). – jonrsharpe Sep 15 '14 at 22:06
  • @jonrsharpe for the first question, when I try it I get a "'module' object has no attribute 'logger'" error. For your second suggestion, when I add the parens I get "setup_logging() takes no arguements (1 given)". I can change how I reference the function but how to handle the different error message? – analyticsPierce Sep 15 '14 at 22:10
  • 1
    It doesn't have that attribute until after the method is called (which is very fragile and a definite code smell), and you should include the `self` parameter in the method definition. But you still haven't explained why you're putting that code in a class and/or using global names at all; it doesn't really make any sense. – jonrsharpe Sep 15 '14 at 22:12
  • @jonrsharpe I want to put the logging setup in a class because I have this same snippet in about 30 files and I want place to control logging. I had the global names because that is how it was setup in the separate files. Do you have a suggestion for how I should set this up? – analyticsPierce Sep 15 '14 at 22:16
  • There are other options besides classes for abstraction; this one has no class or instance attributes and only a single method, so should clearly be replaced with a function. However, you should look at e.g. http://stackoverflow.com/a/15011351/3001761 – jonrsharpe Sep 15 '14 at 22:21
  • (1) Make setup_logging() a function instead of a class member. It doesn't need instance data and putting it in a class just fogs things up. Its directly addressable on your module. (2) don't mix configuration with logging. After test_logger.py calls my_logger.setup_logging(), it should create its own `logger = logging.getLogger()`. (3) the handlers and formatters don't need to be global. They'll stick around after you've added them to the root logger. – tdelaney Sep 15 '14 at 22:48
  • @tdelaney Thanks for your suggestions, they were helpful. While the message does show in my console it does not get written to the log. I added the changes you suggested to my question. If you have any advice I appreciate it. If you want you can add this as an answer and I will accept it. – analyticsPierce Sep 17 '14 at 20:43

0 Answers0