Python Logs displaying multiple times in stdout

Question

I currently have two classes. A subclass and a superclass. All my superclass does is thread the methods in the subclass. The subclass's logs were out of order, and I was told to fix it. The logs are now in order, but the log reprints with a format and without format. I've been working on this all day and can't figure out how to fix it. Any and all constructive help would be appreciated.

This is inside of the subclass...

# -----------
    self.component_logger = logging.getLogger(self.__class__.__name__)
    self.stream_handler = logging.StreamHandler(sys.stdout)
    self.buffered_handler = logging.handlers.MemoryHandler(500000, flushLevel='ERROR', target=self.stream_handler)
    self.component_logger.addHandler(self.stream_handler)
    self.component_logger.addHandler(self.buffered_handler)

    self.buffered_handler.setFormatter(logging.Formatter("[%(threadName)s] %(levelname)s - %(asctime)s [%(lineno)d]: %(message)s"))
 # -----------

I call flush in the super class

for component in components:
        t = threading.Thread(target=threaded_start, name=component.component_id, args=(component, exceptions_queue))
        t.start()
        threads.append(t)
    for t in threads:
        t.join(timeout=60)
    for component in components:
        component.buffered_handler.flush()

    #Reset log formatting
    logging.getLogger().handlers[0].setFormatter(logging.Formatter("%(levelname)s - %(asctime)s [%(lineno)d]: %(message)s"))

    #Gather results and reformat logs
    logging.info("Finished threaded starts.")

Currently still printing like this... (general example)

 Starting l2-ilogin   active
 Starting l2-ilogin   active
 [L2-0] DEBUG - 2015-06-03 14:58:18,129 [57]: Starting l2-ilogin   active
 Starting l2-mgmt     active
 Starting l2-mgmt     active
 [L2-0] DEBUG - 2015-06-03 14:58:18,129 [57]: Starting l2-mgmt     active

Where the first two are repeats and the third is the formatted version (this is the output).

Just a guess: try to comment out one of the lines where you `self.component_logger.addHandler...` and see if it solves the issue. — Nir Alfasi, Jun 03 '15 at 19:10
Another shot: try to `setFormatter` to the `stream_handler` as well — Nir Alfasi, Jun 03 '15 at 19:26
Thanks, it worked for formatting the duplicated input. I ended up setting self.component_logger.propogate to False to stop the duplication of output. Thanks @alfasin . Now the only problem is organizing the output from sub processes and I should be able to do that. Thanks! — Ian, Jun 04 '15 at 13:33

score 1 · Accepted Answer · answered Jun 04 '15 at 14:19

Another shot: try to setFormatter to the stream_handler as well – alfasin 18 hours ago

Thanks, it worked for formatting the duplicated input. I ended up setting self.component_logger.propogate to False to stop the duplication of output. Thanks @alfasin . Now the only problem is organizing the output from sub processes and I should be able to do that. Thanks! – Ian 45 mins ago

self.component_logger = logging.getLogger(self.__class__.__name__)
    self.stream_handler = logging.StreamHandler(sys.stdout)
    self.buffered_handler = logging.handlers.MemoryHandler(500000, flushLevel='ERROR', target=self.stream_handler)
    self.stream_handler.setFormatter((logging.Formatter("[%(threadName)s] %(levelname)s - %(asctime)s [%(lineno)d]: %(message)s")))
    # self.component_logger.addHandler(self.stream_handler)
    self.component_logger.addHandler(self.buffered_handler)
    self.component_logger.propagate = False

Python Logs displaying multiple times in stdout

1 Answers1

Linked