string.Formatter throws KeyError ''

Question

I want to print out key+value pairs like in this question,

key a:         1
key ab:        2
key abc:       3
       ^ this colon is what I want

but I don't like the answer there and I tried to subclass string.Formatter like this:

from __future__ import print_function

from string import Formatter

class KeyFormatter(Formatter):
    def parse(self, fmtstr):
        res = super(KeyFormatter, self).parse(fmtstr)
        #for r in res:
        #    print(r)
        return res

kf = KeyFormatter()
w = 10

x = dict(a=1, ab=2, abc=3)

for k in sorted(x):
    v = x[k]
    print(kf.format('key {::<{}} {}', k, w, v))

I want to debug the parsing to see if I can get at the extra ':' inserted in the format string but this throws a

KeyError: '' in both Python 2.7 and 3.4. If I uncomment the for loop to see what is going in the error goes away, but the final print statement then only shows a newline.

When I make the last line:

print('key {:<{}} {}'.format(k, w, v))

this works (with spaces after the key), and when I do:

print('key {::<{}} {}'.format(k, w, v))

I get multiple ':' instead of spaces. But no KeyError.

Why do I get the KeyError? How can I debug this?

What is the output you're actually hoping for? What's the point of the second colon? — jonrsharpe, Sep 08 '15 at 12:21
@jonrsharpe I thought the other question I linked to would be clear, did you look at that? Anyhow I included the expected output. — , Sep 08 '15 at 12:25
No - questions should be standalone. If all you want is a colon after the key, why not put a colon *after the key*, rather than *in the format template*? I.e. `'key {:<{}}: ...` — jonrsharpe, Sep 08 '15 at 12:25
@jonrsharpe that way you don't get the output that I want (the extra spaces are before the ':' ) — , Sep 08 '15 at 12:32
Right, now I see - why not do it in two steps, i.e. build `"key abc:"`, *then* add it left-aligned into the appropriate spaces. — jonrsharpe, Sep 08 '15 at 12:34
You could just put the `:` to the "right" side of the `format`: `"...".format(k + ":", ...)` — tobias_k, Sep 08 '15 at 12:43
@tobias_k that is what is suggested in the question I refer to and doesn't explain the KeyError issue that I ask about, just works around it. — , Sep 08 '15 at 12:49
Thanks for asking a question which resulted in my question getting answered as well :) — luator, Sep 08 '15 at 14:04

score 0 · Accepted Answer · edited Apr 04 '17 at 15:33

There are two somewhat related problems here, the simple answer to how to debug is: you can't, at least not with print statements, or anything itself using string formatting because that happens during another string format and destroys the state of the formatter.

That it throws an error is caused by the fact that string.Formatter() doesn't support empty fields, this was an addition to the formatting going from 2.6 to 3.1 (and 2.7), which is in the C code, but not reflected in the string module.

You can simulate the new behavior by subclassing the class MyFormatter:

from __future__ import print_function

from string import Formatter
import sys

w = 10
x = dict(a=1, ab=2, abc=3)

if sys.version_info < (3,):
    int_type = (int, long)
else:
    int_type = (int)    

class MyFormatter(Formatter):

    def vformat(self, *args):
        self._automatic = None
        return super(MyFormatter, self).vformat(*args)

    def get_value(self, key, args, kwargs):
        if key == '':
            if self._automatic is None:
                self._automatic = 0
            elif self._automatic == -1:
                raise ValueError("cannot switch from manual field specification "
                                 "to automatic field numbering")
            key = self._automatic
            self._automatic += 1
        elif isinstance(key, int_type):
            if self._automatic is None:
                self._automatic = -1
            elif self._automatic != -1:
                raise ValueError("cannot switch from automatic field numbering "
                                 "to manual field specification")
        return super(MyFormatter, self).get_value(key, args, kwargs)

that should get rid of the KeyError. After that you should override the method format_field instead of parse:

if sys.version_info < (3,):
    string_type = basestring
else:
    string_type = str

class TrailingFormatter(MyFormatter):
    def format_field(self, value, spec):
        if isinstance(value, string_type) and len(spec) > 1 and spec[0] == 't':
            value += spec[1]  # append the extra character
            spec = spec[2:]
        return super(TrailingFormatter, self).format_field(value, spec)

kf = TrailingFormatter()
w = 10

for k in sorted(x):
    v = x[k]
    print(kf.format('key {:t:<{}} {}', k, w, v))

and get:

key a:         1
key ab:        2
key abc:       3

Note the format specifier (t) that introduces the trailing character in the format string.

The Python formatting routines are actually smart enough to let you insert the trailing character in the string just like the width formatting:

    print(kf.format('key {:t{}<{}} {}', k, ':', w, v))

gives the same result and lets you dynamically change the ':'

You can also change format_field to be:

    def format_field(self, value, spec):
        if len(spec) > 1 and spec[0] == 't':
            value = str(value) + spec[1]  # append the extra character
            spec = spec[2:]
        return super(TrailingFormatter, self).format_field(value, spec)

and hand in any type:

print(kf.format('key {:t{}<{}} {}', (1, 2), '@', 10, 3))

to get:

key (1, 2)@    3

but since you convert the value to a string before handing it to Formatter.formatfield() that might get you a different result if str(val) gets you a different value than using {0}.format(val) and/or with options after t: that only apply to non-string types (such as + and -)

This only works for strings isn't it? Not for keys that are integers? I am going to try this. — , Sep 08 '15 at 12:56
@Carl I explicitly test for strings, you can leave that out and do `value = string(value) + spec[1]`, with some side effects, I'll update my answer with that. — Anthon, Sep 08 '15 at 13:03
This works, thank you. I tried to append the spec[1] to the return value of super...formatfield, but that put the ':' after the spaces, oh well. Is this part of some library? — , Sep 08 '15 at 13:28
No it is not in library but I might put it in `ruamel.std.string` on PyPI — Anthon, Sep 08 '15 at 13:44
On closer inspection of `string.py` in Python 3.4 the empty `{}` are allowed there by the Formatter() method. — Anthon, Sep 08 '15 at 14:18

string.Formatter throws KeyError ''

1 Answers1