Why do Python descriptors copy?

Question

(I edited the question, since I think it is still basically the same thing I'm asking, though I gained some understanding from the comments. I don't know if that's permitted, or I should have asked a new one.)

The following code

class A: c = lambda:0
a = A()
print(a.c is a.c)

prints False. I have learned it's because Python thinks A.c is a method, since c is assigned a function at class level. I have two questions:

• (less important) How does Python decide whether something is a function? I thought it would have to be def'd explicitly if it were to become a method. "Arbitrary callable" obviously isn't the criterion: for example, builtin functions aren't accepted.

• (more important) I learned that "Whenever you look up a method via class.name or instance.name, the method object is created a-new". Is there any implementation-independent reason why is it so? That is, is there any language feature that wouldn't work right if copies weren't made? (Of course, I know that a1.c is not a2.c, but for same object a, could a.c always be the same object? Or at least, could A.c always be the same object?)

Answer 1

The default function.__get__ method does "copy" (creates new method instance):

/* Bind a function to an object */
static PyObject *
func_descr_get(PyObject *func, PyObject *obj, PyObject *type)
{
    if (obj == Py_None || obj == NULL) {
        Py_INCREF(func);
        return func;
    }
    return PyMethod_New(func, obj);
}

but you can define a descriptor that doesn't copy:

from functools import partial

class D:
    def __init__(self, function, cached=False):
        self.function = function
        if cached:
            self.cache = {}
        else:
            self.cache = None

    def __get__(self, instance, klass):
        if instance is None: # C.m
            return self.function
        if self.cache is None: # no cache
            m = partial(self.function, instance)
        else:
            m = self.cache.get(instance)
            if m is None:
                m = self.cache[instance] = partial(self.function, instance)
        m.__self__ = instance
        return m # C().m

class C:
    m = D(print)
    cached = D(print, cached=True)

assert C.m is C.m
assert C.cached is C.cached
c = C()
assert c.m is not c.m
assert c.cached is c.cached

It might be simpler/more efficient to recreate the method each time .__get__() is called than to keep (possibly weakrefed) mapping (instance -> method) and breaking cycles (due to __self__ = instance), to avoid wasting memory.

Answer 2

I don't have a computer handy. What happens if you type print(id(a.b), id(a.b), id(a.c), id(a.c))? If the second pair differ then separate objects are being created and we don't have a bug.

Answer 3

Here's what the Python 2.x language reference says about attributes on instances (also applies to 3.x) (scroll down to "Class instances"):

When an attribute is not found there, and the instance’s class has an attribute by that name, the search continues with the class attributes. If a class attribute is found that is a user-defined function object or an unbound user-defined method object whose associated class is the class (call it C) of the instance for which the attribute reference was initiated or one of its bases, it is transformed into a bound user-defined method object whose im_class attribute is C and whose im_self attribute is the instance.

And for attributes on classes (only in 2.x, but not 3.x) (scroll to "Classes"):

When a class attribute reference (for class C, say) would yield a user-defined function object or an unbound user-defined method object whose associated class is either C or one of its base classes, it is transformed into an unbound user-defined method object whose im_class attribute is C.

So to answer your questions:

1. How does it decide? The specification specifically says "user-defined function object" or "unbound user-defined method object". So those are the ones that this rule applies to. This is not all callable types. If you scroll up on the same page to the section on callable types, there are many types of callable types, of which "User-defined functions" is only one type.

2. (First, a correction here: for attribute access on instances, yes, a method object is created; for attribute access on classes, a method object is only created on Python 2.x -- in Python 3.x you simply get whatever you put there back, without any wrapping.) To answer your question, presumably a Python implementation could return the same method object. It would require some kind of caching or interning to do this, which has overhead in terms of storage. The specification doesn't say anything about this. And the current CPython implementation does not return the same object. You should not depend on it either way.