3

I use standard RotatingFileHandler within my Flask application with next parameters: maxBytes=10 * 1024 * 1024, backupCount=50. App is managed by uWSGI behind nginx. uWSGI config file part looks like this:

processes = 16
enable-threads = true
threads = 10

Right after start of an app everything (I mean logging) works well. But after first log file rotation some processes (and maybe threads too) continue writing to rotated file and some - to new one. It is obvious. But for me it is not so obvious how can I rotate log file in the way that all of my processes (and threads) start to write messages to new file.

Ivan Velichko
  • 6,348
  • 6
  • 44
  • 90
  • You can rotate files, but it's best done by having a single dedicated process which listens on a socket and writes to the file using a `RotatingFileHandler` as you do now. IMO It's not a large change, and none of the code which actually calls logging should be affected - just the code that does logging configuration. – Vinay Sajip Jan 19 '15 at 12:30
  • Yes, it'll be easy to do. – Ivan Velichko Jan 19 '15 at 12:35
  • 1
    Posting [this link](https://www.packetmischief.ca/2017/10/25/3-ways-to-fail-at-logging-with-flask/) for future readers to explain **why** this behaviour occurs for Flask / pre-forked applications. It's not a solution, but certainly helps with understanding. – S3DEV Sep 08 '20 at 19:59

1 Answers1

2

Note that writing to a single log file from multiple processes isn't supported, because there is no cross-platform synchronisation mechanism that can be used. See this cookbook entry for a suggested approach which might work for you.

Vinay Sajip
  • 95,872
  • 14
  • 179
  • 191
  • So, if I understood correctly, my logging solution is wrong (because I have multiple processes) and I should not try to rotate files atomically, but change entire logging model instead? – Ivan Velichko Jan 19 '15 at 10:12
  • @VinaySajip - May I get your input on this, please? Given the explanation on [this page](https://www.packetmischief.ca/2017/10/25/3-ways-to-fail-at-logging-with-flask/) and [this SO answer](https://stackoverflow.com/a/3969772/6340496), would you agree that using `syslog` is a reasonable solution for logging a small / locally hosted / handful of users Flask service? Thank you. – S3DEV Sep 08 '20 at 20:33
  • 1
    @S3DEV Yes, it's reasonable, but it's not the only approach you can take. If "Small/locally hosted" means a single process in your deployment, then there's no reason _not_ to use `RotatingFileHandler`. The article on Flask logging refers to a scenario with logging from multiple processes. – Vinay Sajip Sep 10 '20 at 10:40
  • @VinaySajip - Excellent, thank you for your response. The deployment has four workers running, so I was experiencing the issues here. Thanks again! – S3DEV Sep 10 '20 at 12:24