17

I saw "Why doesn't join() automatically convert its arguments to strings?" and the accepted answer made me think: since

Explicit is better than implicit.

and

Errors should never pass silently.

why does str.format() ignore additional/unused (sometimes accidentally passed) arguments? To me it looks like an error which is passed silently, and it surely isn't explicit:

>>> 'abc'.format(21, 3, 'abc', object(), x=5, y=[1, 2, 3])
'abc'

This actually lead my friend to an issue with os.makedirs(path, exist_ok=True) still raising an error even though the docs for os.makedirs() said that exist_ok=True won't raise an error even if path already exists. It turned out he just had a long line with nested function calls, and the exist_ok was passed in to a nested .format() call instead of the os.makedirs().

Community
  • 1
  • 1
Markus Meskanen
  • 19,939
  • 18
  • 80
  • 119
  • 3
    Would you prefer that something simple like: `'{6} {3}'.format(*range(100))` fail? Or maybe: `'Hi {name} your favourite colour is {colour}'.format(name='Bob', colour='blue', age=21, planet='Earth')` – Jon Clements Jul 13 '16 at 11:08
  • 2
    @JonClements Actually yes. If the string needs `name` and `colour`, I believe it's an error to pass it `age` and `planet`. As mentioned, "errors should neve pass silently.". An error would also force you to be explicit with your statements. – Markus Meskanen Jul 13 '16 at 11:11
  • 8
    So, if I have a `dict` called `person` that I'm passing around and to simply print a string based on certain elements of it, I should be responsible to **subset** that dict to match my formatting? That's an insane requirement -- or do you think people really fancy doing `.format(name=person['name'], colour=['person'])` ? – Jon Clements Jul 13 '16 at 11:14
  • 2
    According to what I understand from the Zen of Python, you shouldn't just be "passing around" the whole dict. If you want to print the person's name and age, pass in the person's name and age. Have a function take the person and print out his name and age to simplify the main function. – Markus Meskanen Jul 13 '16 at 11:15
  • 7
    @MarkusMeskanen You are quoting `"errors should never pass silently."`. The *real* question here is *is the scenario you describe should even be considered as an error*. I think it is not. – DeepSpace Jul 13 '16 at 11:16
  • If you think this is bad behaviour look at `%`-formatting... in that case not only you don't get an error, but *sometimes* you do, sometimes not, depending on a number of obscure edge cases. A program that uses `%` may or may not crash with an error depending on both the format string and the other argument. At least `str.format` is consistent in this and triggers an error only if there is something actually missing that the formatting string expected. – Bakuriu Jul 13 '16 at 11:44
  • @Bakuriu Yeah I'm aware, luckily I don't have to touch `%` ever again. Just waiting for the f-strings to come in Python 3.6... – Markus Meskanen Jul 13 '16 at 11:49
  • @MarkusMeskanen: f-strings are awesome. And highly addictive. :) However, they tend to be a little slower than calling `format` in the current implementation. See http://stackoverflow.com/questions/37365311/why-are-python-3-6-literal-formatted-strings-so-slow and the linked http://bugs.python.org/issue27078 – PM 2Ring Jul 13 '16 at 12:11

1 Answers1

19

Ignoring un-used arguments makes it possible to create arbitrary format strings for arbitrary-sized dictionaries or objects.

Say you wanted to give your program the feature to let the end-user change the output. You document what fields are available, and tell users to put those fields in {...} slots in a string. The end-user then can create templating strings with any number of those fields being used, including none at all, without error.

In other words, the choice is deliberate, because there are practical reasons for allowing more arguments than are converted. Note that the C# String.Formatter implementation that inspired the Python PEP does the same, for those same reasons.

Not that the discussion on this part of the PEP is that clear cut; Guido van Rossum at some point tries to address this issue:

The PEP appears silent on what happens if there are too few or too many positional arguments, or if there are missing or unused keywords. Missing ones should be errors; I'm not sure about redundant (unused) ones. On the one hand complaining about those gives us more certainty that the format string is correct. On the other hand there are some use cases for passing lots of keyword parameters (e.g. simple web templating could pass a fixed set of variables using **dict). Even in i18n (translation) apps I could see the usefulness of allowing unused parameters

to which the PEP author responded that they were still undecided on this point.

For use-cases where you must raise an exception for unused arguments you are expected to subclass the string.Formatter() class and provide an implementation for Formatter.check_unused_args(); the default implementation does nothing. This of course doesn't help your friend's case where you used str.format(*args, **kwargs) rather than Formatter().format(str, *args, **kwargs). I believe that at some point the idea was that you could replace the formatter used by str.format() with a custom implementation, but that never came to pass.

If you use the flake8 linter, then you can add the flake8-string-format plugin to detect the obvious cases, where you passed in an explicit keyword argument that is not being used by the format string.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • This answer looks very correct to me, downvoters care to explain how it is incorrect? – miradulo Jul 13 '16 at 11:19
  • 2
    Referring to [your answer on the other question which inspired me to ask this question](http://stackoverflow.com/questions/22152668/why-doesnt-join-automatically-convert-its-arguments-to-strings-when-would-yo/22152693#22152693), how's this case any different? The `.format()`'s behaviour can hide bugs (as it did with my friend's case), and `.format()` just ignores everything you pass to it. Imo it's even worse than the `join()` would be. – Markus Meskanen Jul 13 '16 at 11:20
  • 3
    @MarkusMeskanen: Also in the Zen: *Although practicality beats purity.* There are good practical reasons to not raise an exception here. I'm trolling through the original PEP discussion at the moment to find the specific decision on this. – Martijn Pieters Jul 13 '16 at 11:21
  • @MarkusMeskanen because it's not an error that's "silently passing". Think of it as def `f(a, b, **kwargs)`... As long as you provide a and b, it doesn't matter if kwargs is empty or contains other bits... It however means, that you can use other bits if needs be, only a & b are **non-optional** – Jon Clements Jul 13 '16 at 11:22
  • 4
    To me it just feels that the Zen is like the US law: it has everything and you can justify anything just by referring to different parts of it. Imo *"practicality beats purity"* would also apply on the other question asked, `str.join()` should actually convert everything to string cause that's just practical. If you're joining `1` and `2` with a dash, what else should it do than `'1-2'`. Although that's a discussion to have in the other question... – Markus Meskanen Jul 13 '16 at 11:25
  • 3
    @MarkusMeskanen: the Zen is a *guideline*, not a law. Even better, it started as a *joke*. It is a philosophy, not a binding document. – Martijn Pieters Jul 13 '16 at 11:26
  • 1
    @MarkusMeskanen: At any rate, features like these are discussed *at length* in the Python core developer mailinglist and the discussion is far more nuanced than just referring to the Zen. And if there are good practical reasons to do something, then it'll be done. I see that the PEP *at some point* included a strict mode that would give you that error, and that Guido didn't have a strong opinion one way or the other for being strict about unused arguments. Still searching. – Martijn Pieters Jul 13 '16 at 11:30
  • 4
    @SkamahOne: you are free to take that up with the Python core developers. – Martijn Pieters Jul 13 '16 at 11:34