8

I'm having a slight logging problem with Python 3.6 (Flask==0.11.1) and gunicorn==19.9.0. I'm using the structlog==18.1.0 (for JSON logging) and python-json-logger==0.1.9 inside the Flask app to log out various stuff which works fine on its own, i.e. when running the Flask app without gunicorn. But when running the Flask app inside gunicorn some logging problems randomly start to appear, i.e. both access logs from gunicorn and from inside the Flask app go missing.

An example of a logging event would be

import structlog

def some_flask_endpoint():
    logger = structlog.getLogger(__name__)
    logger.info('This is an info event', additional_data='some additional data')

Which should output a JSON message to the logs similar to this

{"message": "This is an info event", "additional_data": "some additional data"}

Currently this is all done via a logging FileConfig (which both gunicorn and the flask app is using) as shown below:

[loggers]
keys = root,gunicorn.error,gunicorn.access

[handlers]
keys=console,logfile

[formatters]
keys=json

[logger_root]
level=INFO
handlers=console,logfile

[logger_gunicorn.error]
level = INFO
handlers = logfile
propagate = 1
qualname = gunicorn.error

[logger_gunicorn.access]
level = INFO
handlers = logfile
propagate = 0
qualname = gunicorn.access

[handler_console]
class=logging.StreamHandler
formatter=json
args=(sys.stdout, )

[handler_logfile]
class=logging.handlers.TimedRotatingFileHandler
formatter=json
args=('logs.log', 'midnight', )

[formatter_json]
format=%(message)s
class=pythonjsonlogger.jsonlogger.JsonFormatter

I.e. gunicorn is using this file with the parameter --logconfig logconfig.ini and flask is using it with logging.config.fileConfig('logconfig.ini') from the logging package.

I realize that making the root logger output to the logfile might cause some events from other loggers being double logged but my main concern is that since gunicorn spins up several workers (4 in this case) there might be a collision within each worker where structlog is used (i.e. multiple processes trying to access a single file) which might be the reason why some logs go missing.

So another option I thought of was to simply change the root logger to only use the console handler or simply just create a new logger which structlog uses, i.e.

import structlog

logger = structlog.getLogger('my_app')
logger.info('some info')

[logger_my_app]
level=INFO
handlers=console

And then make the gunicorn master take all logs from stdout and handle the writing of that to a log file and solving the multiprocess-single file problem. However when providing gunicorn with this logging config file (so that it outputs its logs in json and also so I dont have to provide endless amount of parameters to config gunicorn) it seems to completely ignore the parameter --capture-output and therefore my structlog logs only appear in the console but aren't written out to any config file.

So after this rather long but hopefully explanatory post, does anybody have information on the proper way to store logs from logging/structlog to file inside of gunicorn because I can't find any concrete information online and the gunicorn docs are quite wage on this topic?

Tl;dr: How can I send stdout to a log file in gunicorn while also providing a log config file to gunicorn.

gislibergur
  • 131
  • 2
  • 7
  • This link may help: https://stackoverflow.com/questions/26578733/why-is-flask-application-not-creating-any-logs-when-hosted-by-gunicorn – amanb Apr 24 '19 at 13:12
  • 2
    I don't believe that this solves the problem. As I understand it, the gunicorn master handles the logging from each worker. This solves the multiple processes trying to write to a single file problem. However if I would tell pythons logging module to write to the same file I would have multiple processes trying to write to that same file (since we have multiple workers all running the same app), so the only solution I see is to make the gunicorn master pick up from stdout and write that to a file, which isn't working if I'm passing in a logging config file to gunicorn. – gislibergur Apr 24 '19 at 13:29
  • Is "the gunicorn master handles the logging from each worker" documented anywhere? – kennysong Apr 12 '21 at 07:02

0 Answers0