Cannot get logging work for Flask with gunicorn daemon mode

Question

I am running a Flask web app using gunicorn+Nginx. I run gunicorn in daemon mode. I configured gunicorn and nginx to log their access and error to files. But I just cannot get Flask logs to a file.

I use a shell file to start my app with gunicorn:

   #!/bin/bash

   export VIRTUAL_ENV="/to/virtual/path"
   export PATH="$VIRTUAL_ENV/bin:$PATH"
   source "$VIRTUAL_ENV/bin/activate"

   NAME="hello"
   NUM_WORKERS=1

   exec gunicorn hello:app \
       --name $NAME \
       --workers $NUM_WORKERS \
       --log-level=debug \
       --daemon \
       --pid $VIRTUAL_ENV/logs/pid_gunicorn \
       --access-logfile $VIRTUAL_ENV/logs/access_gunicorn.log \
       --error-logfile $VIRTUAL_ENV/logs/error_gunicorn.log

And in my flask app I add logging as per doc requires:

app.debug = False
...
if __name__ == '__main__':
    if app.debug != True:
        import logging
        from logging.handlers import RotatingFileHandler
        handler = RotatingFileHandler("flask.log", maxBytes=10000, backupCount=1)
        handler.setLevel(logging.DEBUG)
        app.logger.addHandler(handler)
        app.logger.debug("test!!")
    app.run()

I also added app.logger.debug at other places.

When I start gunicorn without --daemon, the logging file works fine. But once I add --daemon then no logs are generated.

I tried to use print but it only works without --daemon too.

I have searched a while and it seems gunicorn does not support application logging. But I thought logging to a file would be fine?

Does anybody know how I could log out to a file under my settings?

For whoever downvoted the question I appreciate your effort to keep the quality of questions here, but some explanation would have been helpful. — Yulong, Jul 27 '14 at 15:00
Unless you plan to stay single-server and stateful, I'd suggest avoiding logging to a file and log directly to a remote syslog or anything else that'll keep your logs if your server needs to be destroyed and recreated. — rbanffy, Jan 08 '15 at 18:58

score 2 · Answer 1 · answered Dec 28 '20 at 15:13

So- you're not actually setting any logging. Let me explain-

The first argument to gunicorn is your app. gunicorn hello:app. When you launch gunicorn, it will use regular python imports, basically from hello import app.

In your file hello.py, you are setting up your logging. But, you have that chunk of code wrapped in a if __name__ == "__main__": block. If you do python hello.py that will work. But that is NOT what gunicorn is doing. None of that code is being executed by your service (and you should notice that- after all, your development server is not also running...)

Setup logging at the top of the file, outside of the if block. You would also have the option to set gunicorn to capture-output, in which case it would handle getting your app output into log files. This is PROBABLY closer to what you want. If you DID use the logging config you have, gunicorn is going to run more than one separate process, and they will all be trying to log to the same file on disk.

I am not sure what you mean by "the logging works fine" - its going to a file? My expectation would be that without --daemon it means gunicorn is running in the foreground and its output shows up in your terminal (but isn't going to disk unless you have redirected the output of your script to disk? or are starting it with systemd maybe?)

CESCO · Answer 2 · 2015-05-16T18:51:35.893

The pythonic way to run a daemon process is using something like Supervisord, forget bash, its python. Have you considered using nginx as a proxy pass? Gunicorn can handle the WSGI. I think it is available since 1.3.13. It is meant for websockets, but will work even if you running a http protocol.

something like

server {
    listen 80;
    server_name localhost;
    access_log /var/log/nginx/example.log;

    location / {
        proxy_pass http://127.0.0.1:5000;
        proxy_redirect off;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

score 0 · Answer 3 · answered Dec 28 '20 at 14:46

For your purpose, please set --capture-output to True. It should forward your application output to the error log file.

Another issues I got is that in the daemon mode output to stdout and stderr goes to nowhere. It is annoying when gunicorn runs inside a Docker container.

Cannot get logging work for Flask with gunicorn daemon mode

3 Answers3