Unable to log console warning/outputs to a log file

Question

I want to see all the output displayed on the console in a log file. I ran through basics of logging module, but cannot figure where I am going wrong.

My code is:

# temp2.py
import logging
import pytz
from parameters import *
import pandas as pd
import recos



def create_tbid_cc(tbid_path):
    """Function to read campaign-TBID mapping and remove duplicate entries by
    TBID-CampaignName"""
    logging.debug('Started')
    tbid_cc = pd.read_csv(tbid_path, sep='|')
    tbid_cc.columns = map(str.lower, tbid_cc.columns)
    tbid_cc_unique = tbid_cc[~(tbid_cc.tbid.duplicated())]
    tbid_cc_unique.set_index('tbid', inplace=True)
    tbid_cc_unique['campaignname_upd'] = tbid_cc_unique['campaignname']
    del tbid_cc_unique['campaignname']
    return tbid_cc, tbid_cc_unique


def main():
    logging.basicConfig(
    filename='app.log', filemode='w',
    format='%(asctime)s : %(levelname)s : %(message)s',
    datefmt='%m/%d/%Y %I:%M:%S %p',
    level=logging.DEBUG)
    tbid_cc, tbid_cc_unique = create_tbid_cc(tbid_path=tbid_campaign_map_path)     

if __name__ == '__main__':
    main()

Output on console:

SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  tbid_cc_unique['campaignname_upd'] = tbid_cc_unique['campaignname']

Output of myapp.log is :

09/27/2015 06:29:56 AM : DEBUG : Started

I want to see the warning displayed on the console in the myapp.log file, but not being able to do that. I have the set the logging level to 'DEBUG' but still the output logfile is the only one line as mentioned above. Any help in regarding this would be appreciated.

You have to do more work and write your own log writer in order to capture stdout and stderr. Check the answers here: http://stackoverflow.com/questions/19425736/how-to-redirect-stdout-and-stderr-to-logger-in-py — dopstar, Sep 27 '15 at 09:12

score 0 · Answer 1 · answered Sep 27 '15 at 07:29

The warning being raised won't automatically be logged -- you're seeing them in the console because pandas is writing them to stderr.

If you want to write these messages to the log, you'll need to catch these warnings, and then log the message manually.

See:

https://docs.python.org/2/library/warnings.html

Unable to log console warning/outputs to a log file

1 Answers1