-1

I have an existing program (a python executable) but when I run it, it sometimes stops for no reason. So to try and see where the problem arises I aimed to add the logging concept to the python application so I could exactly see what and when a problem arises.

Below is my code, the file log.txt is being created which is good. Now for testing I created an error on purpose by changing the password of the database. Now from experience I know that it should give me an mysql.connector.errors error, which it does in my terminal. But when I open the logfile.txt I get an empty file...

My aim is to catch all different types of errors thats why I use the Exception method.

What I expected to see in log.txt was for example:

2021-09-16 09:04:29,827 ERROR __main__ 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

This is my code:

logging.basicConfig(filename='log.text', level=logging.DEBUG, 
                    format='%(asctime)s %(levelname)s %(name)s %(message)s')
logger=logging.getLogger(__name__)


def main():
    mp_list = get_mp_list() #<-- Get the list created 
    pool = Pool(min(cpu_count(), len(mp_list))) #<-- create a pool
    results = pool.imap_unordered(process_mp, mp_list) #<-- Create the pool.imap object
    while True:
        try:
            result = next(results) #<-- run the program
        except StopIteration:
            break
        except Exception as err: #<-- write to the logfile in  case of error
            logger.error(err)
            raise


if __name__ == '__main__':
    main()
NorthAfrican
  • 135
  • 2
  • 10
  • 2
    Please, provide [mre]. What is `get_mp_list`? what is `process_mp`? – buran Sep 16 '21 at 07:20
  • Does `basicConfig` automatically add a `FileHandler` to your logger? – NotAName Sep 16 '21 at 07:29
  • @buran `get_mp_list` and `process_mp` are just two functions that are to big to show. `get_mp_list` is using an other function named `get_connection()`. So `get_mp_list` is actually creating a connection and fetching data from a table. `process_mp` reads datafiles from directories and processes them – NorthAfrican Sep 16 '21 at 07:38
  • Why don't you use a tkinter messagebox? https://stackoverflow.com/q/68123254#68126475, https://stackoverflow.com/q/67497238#67534869 – D_00 Sep 16 '21 at 07:38
  • @D_00 cost to much time to create it, while its added value is minimum. – NorthAfrican Sep 16 '21 at 07:39
  • @pavel it does the basic configuration for the logging system by creating a StreamHandler with a default Formatter and adding it to the root logger. – NorthAfrican Sep 16 '21 at 07:42
  • 1
    I have tried your code on a Linux system with simple `get_mp_list` and `process_mp` sample functions (that's meant by mre - not the original code) and can't replicate your problem: The deliberately raised exception in `process_mp` is recorded in `log.text`. What kind of system are you using? – Timus Sep 16 '21 at 13:12
  • @Timus Im using Windows 10. – NorthAfrican Sep 16 '21 at 13:47
  • Tried it also on Windows 10 and macOS: Works fine? – Timus Sep 16 '21 at 18:51
  • Since I'm curious: Have you found a solution? – Timus Sep 20 '21 at 10:34
  • @Timus no, not yet – NorthAfrican Sep 20 '21 at 12:03

2 Answers2

0

You seem to be using multiprocessing, and if multiple processes log to the same file, it might not work - for the reasons described in the logging documentation. Basically, you need to use a different strategy to ensure that only one process writes to a particular file. It's too long to reproduce the details here - refer to the linked info for details.

Vinay Sajip
  • 95,872
  • 14
  • 179
  • 191
0

Add filemode it should work then

logging.basicConfig(filename='log.text', level=logging.DEBUG, filemode="w",
                    format='%(asctime)s %(levelname)s %(name)s %(message)s')