Difference between "wrapper" and "method" descriptors?

Question

I was writing a code that finds "unbound methods" of a class using introspection and was surprised to see two different kinds of descriptors for builtin types:

>>> type(list.append), list.append
(<class 'method_descriptor'>, <method 'append' of 'list' objects>)
>>> type(list.__add__), list.__add__
(<class 'wrapper_descriptor'>, <slot wrapper '__add__' of 'list' objects>)

Searching the docs turned up very limited but interesting results:

1. A note in the inspect module that inspect.getattr_static doesn't resolve descriptors and includes a code that can be used to resolve them.
2. an optimization made in python 2.4 claiming that method_descriptor is more efficient than wrapper_descriptor but not explaining what they are:

   The methods list.__getitem__(), dict.__getitem__(), and dict.__contains__() are now implemented as method_descriptor objects rather than wrapper_descriptor objects. This form of access doubles their performance and makes them more suitable for use as arguments to functionals: map(mydict.__getitem__, keylist).

The difference in performance quite intrigued me, clearly there is a difference so I went looking for additional information.

Neither of these types are in the module types:

>>> import types
>>> type(list.append) in vars(types).values()
False
>>> type(list.__add__) in vars(types).values()
False

using help doesn't provide any useful information:

>>> help(type(list.append))
Help on class method_descriptor in module builtins:

class method_descriptor(object)
 |  Methods defined here:
 |  
    <generic descriptions for>
      __call__, __get__, __getattribute__, __reduce__, and __repr__
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __objclass__
 |  
 |  __text_signature__

>>> help(type(list.__add__))
Help on class wrapper_descriptor in module builtins:

class wrapper_descriptor(object)
 |  Methods defined here:
 |  
    <generic descriptions for>
      __call__, __get__, __getattribute__, __reduce__, and __repr__
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __objclass__
 |  
 |  __text_signature__

Searching the internet only came up with results about "what is a descriptor" or vague references to the specific types involved.

So my question is:

What is the actual difference between <class 'method_descriptor'> and <class 'wrapper_descriptor'>?

Answer 1

It's an implementation detail. At C level, a built-in type like list defines methods like append by name through an array of PyMethodDef structs, while special methods like __add__ are defined more indirectly.

__add__ corresponds to a function pointer in either of the two slots sq_concat in the type's tp_as_sequence or nb_add in the type's tp_as_number. If a type defines one of those slots, Python generates a wrapper_descriptor wrapping that slot for the __add__ method of the Python-level API.

The wrapping necessary for type slots and PyMethodDef structs is a bit different; for example, two slots could correspond to one method, or one slot could correspond to six methods. Slots also don't carry their method names with them, while the method name is one of the fields in a PyMethodDef. Since different code is needed for the two cases, Python uses different wrapper types to wrap them.

If you want to see the code, both method_descriptor and wrapper_descriptor are implemented in Objects/descrobject.c, with struct typedefs in Include/descrobject.h. You can see the code that initializes the wrappers in Objects/typeobject.c, where PyType_Ready delegates to add_operators for wrapper_descriptors and add_methods for method_descriptors.

Answer 2

It seems that method_descriptor and wrapper_descriptor are a kind of callables that are available in CPython. The difference between them seems to be simple method_descriptor is apparently used for the methods of built-in (implemented in C) objects:

set.__dict__['union'].__class__
<class 'wrapper_descriptor'>

wrapper_descriptor is used for example the operators of built-in types:

int.__dict__['__add__'].__class__.
<class 'method-wrapper'>

This is the place where I found this information.