Python string Template Delimiter Issue

Question

I'm confused about setting a custom delimiter for Python string Templates. I saw a tutorial video that said to use a custom subclass. So I did and it worked:

from string import Template

class MyTemplate(Template):
  delimiter = '#'

d = {'key':'value'}
t = MyTemplate('key is #key')
print(t.substitute(d))

#prints "key is value" as expected

So I was thinking, if all I have to do is change a class variable on Template, then shouldn't the following work as well?

from string import Template

Template.delimiter = '#'
d = {'key':'value'}
t = Template('key is #key')
print(t.substitute(d))

#prints "key is #key", but why?

I thought that maybe I had to create a subclass for some reason. So I figured I would write a subclass, but I would allow for the delimiter to be set:

from string import Template

class MyTemplate(Template):
  @classmethod
  def setDelim(cls, delim):
    cls.delimiter = delim

MyTemplate.setDelim('#')
d = {'key':'value'}
t = MyTemplate('key is #key')
print(t.substitute(d))

#prints "key is #key", because it is still looking for '$' as delimiter

I confirmed this behavior in both Python2 and Python3. In all three examples the class variable delimiter was properly set but only in the first example did it actually work. Can anyone explain this please?

Answer 1

This is an implementation detail of how the Template class is defined in the standard python library. Specifically, if you look at the string.py source, Template is implemented using a metaclass that bakes an extra pattern attribute into the class based on the value of the delimiter attribute at class initialization time.

Once the class is initialized, the substitution pattern is frozen, so you have to use the recommended subclass method to override delimiter at class definition time or do something fancier to override the metaclass behavior.

E.g.

from string import Template

class MyTemplate(Template):
  @classmethod
  def setDelim(cls, delim):
    cls.delimiter = delim

>>> MyTemplate.pattern.pattern
# Outputs: '\n    \\$(?:\n      (?P<escaped>\\$) |   # Escape sequence of two delimiters\n      (?P<named>[_a-z][_a-z0-9]*)      |   # delimiter and a Python identifier\n      {(?P<braced>[_a-z][_a-z0-9]*)}   |   # delimiter and a braced identifier\n      (?P<invalid>)              # Other ill-formed delimiter exprs\n    )\n    '
>>> MyTemplate.setDelim('#')
>>> MyTemplate.pattern.pattern # still frozen to '$' as delimiter
# Outputs: '\n    \\$(?:\n      (?P<escaped>\\$) |   # Escape sequence of two delimiters\n      (?P<named>[_a-z][_a-z0-9]*)      |   # delimiter and a Python identifier\n      {(?P<braced>[_a-z][_a-z0-9]*)}   |   # delimiter and a braced identifier\n      (?P<invalid>)              # Other ill-formed delimiter exprs\n    )\n    '

Answer 2

Template inherits from a metaclass. The metaclass's __init__ method performs some setup using Template's delimiter. (Specifically, it creates the regexes used to match delimited text.)

Changing delimiter after this setup has taken place does not re-initialise the setup, hence you need to use a subclass so that the desired value of delimiter is used.