18

This is my scenario: I want to log my_module's activity. This needs to be done, depending on the method executed (let's say, INPUT and OUTPUT), to two different files.

So I have two Handlers, each one point to a different file (my_in_.log & my_out_.log), with the same log level. I would like to know if I can use the same logger to achieve this or I have to define two loggers. My config is:

[loggers]
keys=root, my_log

[handlers]
keys=my_in_hand, my_out_hand

[formatters]
keys=generic_form


...


[logger_my_log]
level=NOTSET
handlers=my_in_hand, my_out_hand
qualname=ws_log

[handler_my_in_hand]
class=handlers.TimeRotatingFileHandler
level=NOTSET
formatter=generic_form
args=('my_in_.log', 'h', 1, 0, None, False, True)

[handler_my_out_hand]
class=handlers.TimeRotatingFileHandler
level=NOTSET
formatter=generic_form
args=('my_out_.log', 'h', 1, 0, None, False, True)

Do I have to define a logger per handler/destination (because I want to log different information in different files)? Is there a way to indicate to the logger which handler will do this? I mean, I have two handlers for one logger, then choose only one handler to log one method.

halfer
  • 19,824
  • 17
  • 99
  • 186
Alberto Megía
  • 2,225
  • 3
  • 23
  • 33

3 Answers3

9

You should instantiate an Handler for each destination you want to send your log to, then add the 2 handlers to your logger. The following should work (didn't test it though):

logger = logging.getLogger()
handler1 = logging.TimedRotatingFileHandler()
handler2 = logging.TimedRotatingFileHandler()
logger.addHandler(handler1)
logger.addHandler(handler2)

Of course add all your configuration and formatting options you may need. Basically it is just to show you that when you instantiate the logging handler you can add it to the logger. From that moment on, your log records will be emitted to every handler added to the logger.

drekyn
  • 438
  • 2
  • 7
  • 2
    Maybe I am not clear: I do not want to log my records to **every handler** added to the logger. What I want is to select which handler will log the record, that is, one logger -> two handlers but choose which destination the record will be allocated in. Reuse the same logger with different handlers, but some records will be logged by one handler, and some with another... Is this possible or I have to define other logger? THX A LOT! :) – Alberto Megía Mar 04 '13 at 11:30
  • 1
    @AlbertoMegía You should define your own method that takes as argument the log destination and uses internally the right handler for the log message. – mike Mar 04 '13 at 11:34
  • So @mike you mean I have to add and remove logger's handler to select it in runtime? This may not be my best option because this module is my frontal view in a web service... with every request I would have to switch handlers... :S – Alberto Megía Mar 04 '13 at 11:41
  • @AlbertoMegía well since this is a frequent action, adding/removing handlers before logging the message is not the right approach (obviously). – mike Mar 04 '13 at 11:50
  • which are the criteria for which you use one logger or the other? You might overwrite the emit() function of the handler in order to do this – drekyn Mar 04 '13 at 11:52
  • @drekyn Criteria: log input request to a file. Log output to another. Output records format != input records format – Alberto Megía Mar 04 '13 at 11:58
  • 5
    I guess that in this case then you need to define 2 loggers with 2 handlers, one each. in this way you'll keep contexts separated and clear – drekyn Mar 04 '13 at 12:10
  • I think that will be the best approach, @drekyn. But I will wait, maybe somebody knows how to deal with it... THX A LOT GUYS! :) – Alberto Megía Mar 04 '13 at 13:12
9

what you want is to

  1. create 2 NON ROOT loggers.
  2. make handler for each one, point to different file
  3. add handler to appropriate logger

    logger1 = logging.getLogger('general_logger')
    logger2 = logging.getLogger('some_other_logger')
    
    log_handler1 = logging.handlers.RotatingFileHandler(file_1, *args)
    log_handler2 = logging.handlers.RotatingFileHandler(file_2, *args)
    
    logger1.addHandler(log_handler1)
    logger2.addHandler(log_handler2)
    

then

    logger1.info("this will be logged to file_1 ")
    logger2.info("this will be logged to file_2 ")

Please note that if you create a ROOT logger and a different logger, root logger will log everything that this different controller is trying to log.

In other words, if

    root_logger = logging.getLogger()
    logger2 = logging.getLogger('some_other_logger')

    root_log_handler = logging.handlers.RotatingFileHandler(file_1, *args)
    log_handler2 = logging.handlers.RotatingFileHandler(file_2, *args)

    root_logger.addHandler(root_log_handler)
    logger2.addHandler(log_handler2)

then

    root_logger.info("this will be logged to file_1 ")
    logger2.info("this will be logged to file_1 AND file_2 ")
Denis Kanygin
  • 1,125
  • 12
  • 15
6

Finally I decided to define two loggers, because:

  • They are for different purposses. In my case, one logs input request to a web service, and the other one logs the response. And they use different files to it

  • I'm using a logging config file, in a frontal web service. Adding/removing handlers before logging messages is not the right approach, as @mike said. Thx to @drekyn too!

Here is my logging config file, just for reference if anyone is interested in:

[loggers]
keys=root, ws_in_log, ws_out_log

[handlers]
keys=consoleHandler, ws_in_hand, ws_out_hand

[formatters]
keys=generic_form

[logger_root]
handlers=consoleHandler
level=NOTSET

[logger_ws_in_log]
level=NOTSET
handlers=ws_in_hand
qualname=ws_in_log

[logger_ws_out_log]
level=NOTSET
handlers=ws_out_hand
qualname=ws_out_log

[handler_ws_in_hand]
class=logging.handlers.TimedRotatingFileHandler
level=NOTSET
formatter=generic_form
args=('/path/ws_in_.log', 'h', 1, 0, None, False, True)

[handler_ws_out_hand]
class=logging.handlers.TimedRotatingFileHandler
level=NOTSET
formatter=generic_form
args=('/path/em/ws_out_.log', 'h', 1, 0, None, False, True)

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=generic_form
args=(sys.stdout,)

[formatter_generic_form]
format='%(asctime)s - %(levelname)s - %(message)s'
datefmt='%Y-%m-%d %H:%M:%S'
class=

See you!

Alberto Megía
  • 2,225
  • 3
  • 23
  • 33
  • 1
    I feel like other answers didn't understand exactly what you were trying to do. But, *this answer* (and your question) is also *exactly what I'm trying to do!* – Ogre Psalm33 Aug 31 '20 at 17:56