Logging a Flask app on remote sever not working

Question

I am logging to a log file in a Flask application under gunicorn and nginx using the following setup:

def setup_logging():
    stream_handler = logging.StreamHandler()
    formatter = logging.Formatter('[%(asctime)s][PID:%(process)d][%(levelname)s][%(name)s.%(funcName)s()] %(message)s')
    stream_handler.setFormatter(formatter)
    stream_handler.setLevel("DEBUG")
    logging.getLogger().addHandler(stream_handler)

    file_handler = RotatingFileHandler("log.txt", maxBytes=100000, backupCount=10)
    file_handler.setFormatter(logging.Formatter(
                '%(asctime)s %(levelname)s: %(message)s '
                '[in %(pathname)s:%(lineno)d]'
                ))
    file_handler.setLevel("DEBUG")
    logging.getLogger().addHandler(file_handler)
    logging.getLogger().setLevel("DEBUG")

Then inializing logging prior to creating the app:

setup_logging()

def create_app(config_name):
    app = Flask(__name__)

Then in modules I am logging using:

import logging
logger = logging.getLogger(__name__)

x = 2
logger.debug('x: {0}' x)

Logging works OK on my local machine - both to stdout and log.txt

However when I run the application on a remote server nothing gets written to log.txt. I have deployed as a user with read and write permission on log.txt on the remote system.

I have tried initializing the app on the remote server with DEBUG = True, still nothing written to log file. The only way I can get view any logs is by viewing /var/log/supervisor/app-stdout---supervisor-nnn.log files but these dont show all logging output

Using the answer from HolgerShurig here Flask logging - Cannot get it to write to a file on the server log file I get only named logger output (ie no output from module level logging)

2017-10-21 00:32:45,125 - file - DEBUG - Debug FILE 

running same code lo local machine I get

2017-10-21 08:35:39,046 - file - DEBUG - Debug FILE
2017-10-21 08:35:42,340 - werkzeug - INFO -  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)  ie 
2017-10-21 08:38:46,236 [MainThread  ] [INFO ]  127.0.0.1 - - [21/Oct/2017 08:38:46] "[37mGET /blah/blah HTTP/1.1[0m" 200 -

I then changed the logging config to:

def setup_logging(app):
    stream_handler = logging.StreamHandler()
    formatter = logging.Formatter('[%(asctime)s][PID:%(process)d][%(levelname)s][%(lineno)s][%(name)s.%(funcName)s()] %(message)s')
    stream_handler.setFormatter(formatter)
    stream_handler.setLevel(Config.LOG_LEVEL)
    app.logger.addHandler(stream_handler)

    file_handler = RotatingFileHandler(Config.LOGGING_FILE, maxBytes=Config.LOGGING_MAX_BYTES, backupCount=Config.LOGGING_BACKUP_COUNT)
    file_handler.setFormatter(logging.Formatter(
                '%(asctime)s %(levelname)s: %(message)s '
                '[in %(pathname)s:%(lineno)d]'
                ))
    file_handler.setLevel(Config.LOG_LEVEL)
    loggers = [app.logger]
    for logger in loggers:
            logger.addHandler(file_handler)

    app.logger.setLevel(Config.LOG_LEVEL)
    app.logger.debug('this message should be recorded in the log file')

and set to us this just after creating the Flask app:

setup_logging(app)

I each module I am still using

import logging
logger = logging.getLogger(__name__)

#for example
def example():
    logger.debug('debug')
    logger.info('info')
    logger.warn('warn')

When I run the application on ther server with

gunicorn manage:app

The only thing printed in the log.txt file is

2017-10-21 02:48:32,982 DEBUG: this message should be recorded in the log file [in /../../__init__.py:82]

But locally MainThread processeses are shown as well

Any ideas?

Answer 1

If your configuration is working on your local machine and not working on your remote server, then your problem is about permission, regarding the file or the directory where the logfile resides.

Here's something that could help you.

Besides that, here's a Gist which could give another perspective regarding logging configuration for Flask applications.