Python 3 Strange Metaclass Behaviour

Question

I was playing with metaclasses in Python 3:

class M(type):
    def __new__(cls, clsname, bases, attrs):
        for name, attr in attrs.items():
            if callable(attr):
                attrs[name] = attr
        return type.__new__(cls, clsname, bases, attrs)

class C(metaclass=M):
    def f(self, x):
        print(x)

if __name__ == '__main__':
    c = C()
    c.f(1)
    c.f(2)

Nothing special so far, I just hook into the creation of a class, and substitute its method with... well, itself, so no wonder everything works. But with:

class M(type):
    def __new__(cls, clsname, bases, attrs):
        for name, func in attrs.items():
            if callable(func):
                attrs[name] = lambda *args, **kwds: func(*args, **kwds)
        return type.__new__(cls, clsname, bases, attrs)

It sometime works, and sometimes doesn't:

user$ python test.py
1
2
user$ python test.py
Traceback (most recent call last):
  File "./meta.py", line 23, in <module>
    main()
  File "./meta.py", line 19, in main
    instance.method(1)
  File "./meta.py", line 9, in <lambda>
    attrs[name] = lambda *args, **kwds: func(*args, **kwds)
TypeError: 'str' object is not callable

But I just substituted its method with a lambda wrapper! What does 'str' have to do with anything? What am I doing wrong?

(Just in case it's some weird platform-dependent implementation issue, I'm using Ubuntu Server 12.04.3...)

UPDATE: fixed the name mismatch in the traceback.

Your traceback doesn't match your code; `attribute` != `func`, `attributes != attrs`. — Martijn Pieters, Feb 05 '14 at 22:53
In this case `attribute` appears to be a string object, so the `callable` test appears to be missing. — Martijn Pieters, Feb 05 '14 at 22:54
Also, your next problem: `func` isn't going to point to what you think it's pointing to inside the lambda. (It will refer to the last attribute you saw in your loop, which may not be callable.) To fix this, write a helper function to create the lambda, passing in the function, rather than creating it in your loop. — kindall, Feb 05 '14 at 23:04
possible duplicate of [Local variables in Python nested functions](http://stackoverflow.com/questions/12423614/local-variables-in-python-nested-functions) — Martijn Pieters, Feb 05 '14 at 23:06
@MartijnPieters, I made the variable names shorter so the lines fit in StackOverflow neatly, and forgot to edit the traceback accordingly when I copied it. Anyway, thanks for noting and for helping out! — Dan Gittik, Feb 06 '14 at 06:47

kindall · Accepted Answer · 2014-02-06T15:43:18.743

To further elaborate on my comment:

def makelambda(func):
    return lambda *args, **kwds: func(*args, **kwds)

class M(type):
    def __new__(cls, clsname, bases, attrs):
        for name, func in attrs.items():
            if callable(func):
                attrs[name] = makelambda(func)
        return type.__new__(cls, clsname, bases, attrs)

This is necessary because, in your original code, inside the lambda, func refers to whatever value func had when your __new__ method returned, not the value it had when the lambda was created. This is counterintuitive, but you can verify it:

lambdas = [lambda: x for x in range(10)]
print(lambdas[0]())    # prints 9, as do lambdas[1] through [9]

To fix that, we use a separate function to create the lambda, and thus "freeze" the value of the func variable at the time the lambda was created. You can also do this with a default argument value on the lambda, but since you're using * and ** here, this is a little problematic.

(The behavior has nothing to do with metaclasses, you'd see the same behavior anywhere you defined lambdas and changed the value of variables used in them after creating them. And lambdas are no different from any other function in this respect.)

Python 3 Strange Metaclass Behaviour

1 Answers1