python json logger with config file

Question

I want to configure a json logger, and use the logging config file.
I can achieve the equivalent programmatically (without a config file)
When I use a config file, I have problems with imported package jsonlogger
Note that this is not a duplicate of this

import logging
import logging.config
from pythonjsonlogger import jsonlogger # <----- where should I put this import ?

logging.config.fileConfig("logging.cfg")

logging.info("where is my stuff ?")

And here is my config file:

[loggers]
keys=root

[handlers]
keys=simple

[formatters]
keys=simple

[logger_root]
level=INFO
handlers=simple

[handler_simple]
class=StreamHandler
formatter=simple

[formatter_simple]
format="[ %(asctime)s %(levelname)s ] %(message)s"
datefmt="%d/%m/%y ( %H:%M:%S )"
class=jsonlogger.JsonFormatter # <----- how to let module know this class ?

When I run my program this is the error I get:

Traceback (most recent call last):
  File "main.py", line 5, in <module>
    logging.config.fileConfig("logging.cfg")
  File "/usr/lib/python3.10/logging/config.py", line 72, in fileConfig
    formatters = _create_formatters(cp)
  File "/usr/lib/python3.10/logging/config.py", line 119, in _create_formatters
    c = _resolve(class_name)
  File "/usr/lib/python3.10/logging/config.py", line 90, in _resolve
    found = __import__(used)
ModuleNotFoundError: No module named 'jsonlogger'

I didn't quite understand your comment on my answer. If you do a Web search, you'll see it's quite common to load a dict from e.g. YAML or JSON and use that in a `dictConfig()` call. — Vinay Sajip, Jul 20 '22 at 15:35

score 1 · Accepted Answer · answered Jul 20 '22 at 14:44

1

If you use dictConfig() rather than the older fileConfig(), you should be able to access jsonlogger if it's in the import path (e.g. installed in a venv for your application). You'd just declare a class of e.g. pythonjsonlogger.jsonlogger.JsonFormatter for your formatter.

answered Jul 20 '22 at 14:44

Vinay Sajip

95,872
14
179
191

Is it common to load a configuration file to a dict and use `dictConfig()` ? should I avoid using configuration files in the logging context ? – OrenIshShalom Jul 20 '22 at 14:59

OrenIshShalom · Answer 2 · 2022-07-24T07:04:20.220

Here is a complete working example that demonstrates the following:

loading configuration from a json file
default (root) logger + customized (analyzer) logger
uses pythonjsonlogger.jsonlogger.JsonFormatter
passes parameters to pythonjsonlogger.jsonlogger.JsonFormatter

This answer is based on:

Here is how I use the logging mechanism:

import json
import logging
import logging.config

with open("logging.json") as fl:
    logging.config.dictConfig(
        json.load(fl)
    )

# default (root) logger
logging.info("where is my stuff ?")

# customized (analyzer) logger
logging.getLogger("analyzer").info(
    "it is right there"
)

And here is the complete configuration file (a bit long)

{
    "version": 1,
    "formatters": {
        "formatter_default": {
            "format": "[ %(asctime)s ] [%(levelname)s] %(message)s",
            "datefmt": "%d/%m/%y ( %H:%M:%S )"
        },
        "formatter_analyzer": {
            "()": "pythonjsonlogger.jsonlogger.JsonFormatter",
            "format": "[ %(asctime)s ] %(levelname)s %(message)s",
            "datefmt": "%d/%m/%y ( %H:%M:%S )",
            "rename_fields": {
                "asctime": "time",
                "levelname": "level"
            },
            "json_indent": 4
        }

    },
    "handlers": { 
        "handler_default": { 
            "formatter": "formatter_default",
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stdout"
        },
        "handler_analyzer": { 
            "formatter": "formatter_analyzer",
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stdout"
        }

    },
    "loggers": { 
        "root": {
            "handlers": ["handler_default"],
            "level": "INFO"
        },
        "analyzer": { 
            "handlers": ["handler_analyzer"],
            "propagate": false, # <----- needed !
            "level": "INFO"
        }
    }
}

python json logger with config file

2 Answers2

Linked