Python 3.7 logging: f-strings vs %

Question

I'm running into a performance problem in a project, and I narrowed it down to some of the log lines. It seems that f-strings are calculated even when my logging facility is above the level of the line that is logging.

Consider this example to demonstrate the issue:

import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger('MyLogger')

class MyClass:
    def __init__(self, name: str) -> None:
        self._name = name
    def __str__(self) -> str:
        print('GENERATING STRING')
        return self._name

c = MyClass('foo')
logger.debug(f'Created: {c}')

When this example is run, I get "GENERATING STRING" printed to screen, indicating that the __str__ method is being ran even though my logging level is set to INFO and the log line is for DEBUG.

From what I can tell today, the solution is to use the following vs an f-string.

logger.debug('Created: %s', c)

There are three things going through my head right now.

Most of the examples and docs I read seem to be pretty old.
This project is Python 3.7+ only (not worried about being backwards compatible).
I have a lot of lines of code to update.

I'm curious to know what others do in this situation. Is the %s the best (most modern) approach? Is there a more modern way that I should be logging as demonstrated above?

I have a lot of code to update (fix), and I'm hoping to align with modern best practices.

IMO, use f-strings, they are more efficient *and* more readable. — jpp, Jan 25 '19 at 15:12
Did you read this? https://docs.python.org/3/howto/logging.html#optimization You should use `%s` for the exact reason you described above (If I understood it correctly actually) — redacted, Jan 25 '19 at 15:13
@jpp Even if the f-string is always evaluated and the `%` string only if needed within the logging method? (Note: OP is not evaluating the `%` string immediately but passing the parameters to the logging method) — tobias_k, Jan 25 '19 at 15:14
IIRC, a main advantage of `%` is that the logger will try its best to log *something*, even if interpolating the arguments into the string should fail somehow. f-strings don't have that property, and the entire logging call may fail. Totally `[citation-needed]` though. — deceze, Jan 25 '19 at 15:20
Here's a better link without the `__` requirement: https://stackoverflow.com/questions/13131400/logging-variable-data-with-new-format-string/24683360#24683360 — AChampion, Jan 25 '19 at 15:34
Thanks for linking the possible duplicate @DanielPryden. I was not able to find those with my Google-foo. Hopefully this post helps the next person! — theG, Jan 25 '19 at 16:02

olinox14 · Answer 1 · 2019-01-25T15:17:48.257

88

The documentation says that the logging lib is optimized to use the %s formatting style. I can't remember where it is mentionned exactly, but I read it a few months ago.

Edit - Found! https://docs.python.org/3/howto/logging-cookbook.html#formatting-styles
Edit2 - (thanks to Robin Nemeth): https://docs.python.org/3/howto/logging.html#optimization

edited Jan 25 '19 at 15:17

answered Jan 25 '19 at 15:16

olinox14

6,177
2
22
39

13

here is the link https://docs.python.org/3/howto/logging.html#optimization – redacted Jan 25 '19 at 15:16
1

Too bad, I think f-strings are much more readable. – stefanbschneider Mar 30 '22 at 14:10

score 40 · Accepted Answer · edited May 12 '20 at 11:52

40

IMO, using %s in your strings is NOT the most modern approach. Definitely, most developers will prefer to use f-strings because it is more convenient and easy to read (and write).

But, you interestingly find a specific case where you may not want to use an f-string. If you need to avoid automatic call of __str__() method because of optimisation issue, then it is probably a good enough reason to use %s instead of f-strings. But, this could also indicate that something may be done in your program to reduce the complexity of __str__(). Most of the time it shouldn't take so much time or resources to calculate a string representation for an object...

edited May 12 '20 at 11:52

pfabri

885
1
9
25

answered Jan 25 '19 at 15:17

Antwane

20,760
7
51
84

4

Agreed, I would defiantly prefer the f-string syntax. In this case, the `__str__` method converts an epoch timestamp into a human readable date. It's a performance issue because it's being called millions of times in debug logging. – theG Jan 25 '19 at 16:01
10

Looking at http://docs.python.org/3/howto/logging.html#optimization: "Formatting of message arguments is deferred until it cannot be avoided.". Using f-string syntax might have an extract cost as it will format the message even if it is not logged. See http://docs.python.org/3/howto/logging.html#optimization for additional informations. – Fabrice Jammes Jul 09 '21 at 07:16

Python 3.7 logging: f-strings vs %

2 Answers2

Linked