Use of metaclass, setattr and property with Python and weird behavior

Question

I coded a metaclass that defines in run time properties of the class that use it. It just gets these attributes from the REQUIRED_KEYS attribute to then declare a property, for example:

class Base(object):
    REQUIRED_KEYS = ()

    class __metaclass__(type):
        def __init__(cls, name, bases, nmspc):

            type.__init__(cls, name, bases, nmspc)
            for attr in cls.REQUIRED_KEYS:
                setattr(cls, attr, property(lambda self: self._dict.get(attr, None)))

    def __init__(self, **kwargs):
        self._dict = dict(**kwargs)


class Config(Base):
    REQUIRED_KEYS = ('foo', 'bar')

If one key is not given the attribute defined in run time returns a None value. I would expected a right behaviour for the following execution, returning 1 and None:

config = Config(**{'foo': 1})
config.foo
config.bar

But it returns None for the foo attribute where the right value expected will be 1.

If I modify the lambda used by the property function by one closure as the following snippet shows, it works fine.

class __metaclass__(type):
    def __init__(cls, name, bases, nmspc):

        def get(attr):
            def _get(self):
                return self._dict.get(attr, None)
            return _get

        type.__init__(cls, name, bases, nmspc)
        for attr in cls.REQUIRED_KEYS:
            setattr(cls, attr, property(get(attr)))

Following the execution of the first snippet, I realised that the lambda function is always called with the last attr value. It means that when we are trying to access to the foo attribute the lambda uses the bar value.

does anybody know what is going on here ?

score 4 · Answer 1 · answered Dec 26 '15 at 20:36

4

This is how closures work in python.

The value of attr isn't stored in the lambda, only a reference to the variable.

In your code all lambdas contain the same reference to the attr variable, and when the lambda is called, it will contain its last value.

You can force a copy of the value by using a default argument:

setattr(cls, attr, property(lambda self, attr=attr: self._dict.get(attr, None)))
                                         ^^^^^^^^^

answered Dec 26 '15 at 20:36

Karoly Horvath

94,607
11
117
176

Thanks, I did not realized about this issue with the references here. – pfreixes Dec 26 '15 at 22:17

score 0 · Answer 2 · answered Dec 26 '15 at 20:38

According to PEP 227, If a name is used within a code block, but it is not bound there (and is not declared global), the use is treated as a reference to the nearest enclosing function. (formatting mine). The name attr is not declared with in the lambda getter, so it is a reference to the variable attr in the metaclass __init__ function, which always points to the last attribute. In addition, dict(**kwargs) (in the Base class init function) is not necessary because kwargs is already a dictionary (You can type self._dict = kwargs) and in your example, you cank initialize the class as Config(foo=1) instead of Config(**{'foo':1})

About the second coment, give a implicit sentence to build a dictionari from a kwars or a explicit is just different ways to do the same but not an issue. About the last comment, ```Config(**{'foo':'1'})``` in fact comes from something like ```d = {'foo': 1}; Config(**d)``` just relaxed in my comment to make it readable. — pfreixes, Dec 26 '15 at 22:21

Use of metaclass, setattr and property with Python and weird behavior

2 Answers2