How to get Python to gracefully format None and non-existing fields

Question

If I write in Python:

data = {'n': 3, 'k': 3.141594, 'p': {'a': 7, 'b': 8}}
print('{n}, {k:.2f}, {p[a]}, {p[b]}'.format(**data))
del data['k']
data['p']['b'] = None
print('{n}, {k:.2f}, {p[a]}, {p[b]}'.format(**data))

I get:

3, 3.14, 7, 8
Traceback (most recent call last):
  File "./funky.py", line 186, in <module>
    print('{n}, {k:.2f}, {p[a]}, {p[b]}'.format(**data))
KeyError: 'k'

Instead of an error message, how can I get Python to more gracefully format the None's and non existent fields?

To give an example, I would like to see in the output something more like:

3, 3.14, 7, 8
3, ~, 7, ~

Ideally, of course, I would like to be able to specify the string used instead of those missing values.

Answer 1

The recommendation in PEP 3101 is to subclass Formatter:

import string
class PartialFormatter(string.Formatter):
    def __init__(self, missing='~~', bad_fmt='!!'):
        self.missing, self.bad_fmt=missing, bad_fmt

    def get_field(self, field_name, args, kwargs):
        # Handle a key not found
        try:
            val=super(PartialFormatter, self).get_field(field_name, args, kwargs)
            # Python 3, 'super().get_field(field_name, args, kwargs)' works
        except (KeyError, AttributeError):
            val=None,field_name 
        return val 

    def format_field(self, value, spec):
        # handle an invalid format
        if value==None: return self.missing
        try:
            return super(PartialFormatter, self).format_field(value, spec)
        except ValueError:
            if self.bad_fmt is not None: return self.bad_fmt   
            else: raise

fmt=PartialFormatter()
data = {'n': 3, 'k': 3.141594, 'p': {'a': '7', 'b': 8}}
print(fmt.format('{n}, {k:.2f}, {p[a]}, {p[b]}', **data))
# 3, 3.14, 7, 8
del data['k']
data['p']['b'] = None
print(fmt.format('{n}, {k:.2f}, {p[a]:.2f}, {p[b]}', **data))
# 3, ~~, !!, ~~

As set up, it will print ~~ if a field or attribute is not found and !! if an invalid format is used given the field value. (Just use None for the keyword argument bad_fmt if you want the default of a value error raised.)

To handle missing keys, you need to subclass both get_field to catch the KeyError or AttributeError and format_field to return a default value for the missing key.

Since you are catching format_field errors, you can catch a bad format field as well by catching the ValueError from the superclass.

Answer 2

If you're able to do the formatting separately you could use Template.safe_substitute which gracefully handles missing values:

>>> from string import Template
>>> t = Template("$a $b $c")
>>> t.safe_substitute(a=3)
'3 $b $c'

Answer 3

The str.format() method doesn't give you a direct method to handle missing keys or replace values.

You can add a layer of indirection; pass in a mapping that handles missing and None values, and alter the format to use just that argument:

class PlaceholderFormatValue():
    def __format__(self, spec):
        return '~'
    def __getitem__(self, name):
        # handle further nested item access
        return self

class formatting_dict(dict):
    def __getitem__(self, name):
        value = self.get(name)
        if isinstance(value, dict):
            # rewrap nested dictionaries to handle missing nested keys
            value = type(self)(value)
        return value if value is not None else PlaceholderFormatValue()

print('{0[n]}, {0[k]:.2f}, {0[p][a]}, {0[p][b]}'.format(formatting_dict(data)))

Now all slots refer to positional argument 0, which is treated like a dictionary, but key lookups always succeed and both missing values and None are replaced by a placeholder value.

Here the PlaceholderFormatValue() ensures that regardless of what the format spec gives, the value can be interpolated into the format. This makes {0[k]:.2f} work, for example.

By wrapping any dict values and having PlaceholderFormatValue handle item access, the above can also handle failure to provide nested keys or whole dictionaries:

>>> data = {'n': 3, 'k': 3.141594, 'p': {'a': 7, 'b': 8}}
>>> del data['k']
>>> data['p']['b'] = None
>>> print('{0[n]}, {0[k]:.2f}, {0[p][a]}, {0[p][b]}'.format(formatting_dict(data)))
3, ~, 7, ~
>>> del data['p']['a']
>>> print('{0[n]}, {0[k]:.2f}, {0[p][a]}, {0[p][b]}'.format(formatting_dict(data)))
3, ~, ~, ~
>>> del data['p']
>>> print('{0[n]}, {0[k]:.2f}, {0[p][a]}, {0[p][b]}'.format(formatting_dict(data)))
3, ~, ~, ~