How can I log a dictionary into a log file?

Question

I have a dictionary:

 d = {name : "John", age: 10}. 

And a log file setup as:

logging.basicConfig(level = logging.DEBUG, filename = "sample.log")

Is it possible to log this dictionary into the "sample.log" file? If yes, how can I do it?

Answer 1

Simple solution that works

The logging functions will convert the '%s' to string representation (and if the object happens to be a container, then repr() will be used for the contained objects)

import logging
logging.basicConfig(level=logging.DEBUG, filename='sample.log')
logging.debug('Sample dict log: %s', {'name' : "John", 'age': 10})

How it shows in the log file:

DEBUG:root:Sample dict log: {'age': 10, 'name': 'John'}

If you have custom objects (e.g. instances of your own classes), you should implement a sensible __str__ and/or __repr__ for it, and the above will work without any hacks.

More on this here What is the difference between __str__ and __repr__?

A Note on performance

Notice that

logging.debug('%s', some_dict) is not same as

logging.debug('{0}'.format(some_dict))

In the first one, the function is passed 2 arguments '%s' and the original some_dict.

In the second one, the function is passed one argument which is the already-converted some_dict to a string. Because a function needs the arguments evaluated, the second example will always do the conversion, even if logging configs have turned off the logging of debug messages.

That's an unnecessary performance penalty. In the the first example, the logging.debug function can decide to do the conversion, or not.

JSON is not very good for this

For objects that aren't JSON-like (object instances, datetime, etc), json.dumps() would require a custom serializer e.g. with datetime objects you get:

TypeError: datetime.datetime(...) is not JSON serializable

This goes for any type that isn't supported by json.dumps()

Whereas the logging module would figure out a good representation for the object

Answer 2

Simple you can use

dict_data = {"test":"data"}
logger.info("Loging dict ---> {0}".format(dict_data))

Answer 3

you can convert it to string:

string_dictionary = str(d)

and then log the string dictionary variable to the file

using JSON

import json

d = {"name":"John","age":10}
json_string = json.dumps(d)

and if you want to convert the string back:

d = json.loads(json_string)

Answer 4

The problem with str(dictionary) and json.dumps(dictionary) is that the output can human unfriendly, especially if the dictionary is big and has nested structures.

If that's the case, you can Python's built-in pprint to pretty format the dictionary before logging:

import pprint

my_complex_dict = pprint.pformat(my_complex_dict)
logger.info(f"My complex dict:\n{my_complex_dict}")

Answer 5

I came to this question when I wanted to log JSON lines for Cloudwatch.

I ended up using python-json-logger.

Install it: pip install python-json-logger

Use pythonjsonlogger.jsonlogger.JsonFormatter as the formatter class.