How can one use a class method in a class attribute definition

Question

All is in the title. I'd like to create a class method and a class attribute, both constructed only once, when the class is created, using the first in the second's definition.

With my best try, I just get a TypeError: 'classmethod' object is not callable.

Here is my code :

import numpy as np

class Foo( object ) :

    @classmethod
    def bar( cls, x ) :
        return x+1

    bar_vect = np.vectorize( bar )

Foo.bar_vect( np.array([ 1, 2, 3 ]) )

>> TypeError: 'classmethod' object is not callable

EDIT 1 :

'classmethod' object is not callable is a problem raising the same error, but with a lot of workarounds. My question is meant to go straight to the point and have a clear idea of how to use @classmethod without a scope giving access to cls.

Another try that I made was the following :

import numpy as np

class Foo( object ) :

    @classmethod
    def bar( cls, x ) :
        return x+1

    bar_vect = np.vectorize( bar )

>> NameError: name 'Foo' is not defined

Answer 1

@classmethods are implemented as a special object that gets processed using the descriptor protocol when looked up on the class; inside the definition, as a raw name (unqualified), it's a special classmethod object, not a normal function and it's not bound to the class properly. If you check the pure Python definition of classmethod, you'll note it's just a normal object that implements __init__ (for construction) and __get__ (for descriptor lookup), but not __call__, meaning that if you have the raw classmethod object, it's not actually a callable at all.

The trick is to qualify the reference so the "magic" happens to bind it to the class, and move the qualified reference outside the class definition (so Foo is a defined name and can be referenced for binding) changing:

class Foo(object):
    ... rest of class ...
    bar_vect = np.vectorize(bar)  # Indented and unqualified, BAD

to:

class Foo(object):
    ... rest of class ...
# Must qualify both bar_vect and bar, since no longer in class definition
Foo.bar_vect = np.vectorize(Foo.bar)  # Dedented, so Foo is defined for referencing, GOOD

Note that since you're using a classmethod, I suspect you may eventually be interested in subclassing and overriding bar. As written, you'd need to explicitly redefine bar_vect after defining each subclass, or it would use the inherited bar_vect, based on Foo.bar, even if the subclass defines its own bar classmethod. Explicitly redefining bar_vect each time is an option, but the other approach is to use metaclasses to implicitly define bar_vect when a class redefines bar:

class BarVectorized(type):
    def __new__(cls, name, bases, namespace, **kwargs):
        newcls = type.__new__(cls, name, bases, dict(namespace))
        # Make vectorized wrapper for this class (must use new wrapper
        # even if bar unchanged, so cls in bar is correct for lookup of
        # other class attributes/methods)
        try:
            newcls.bar_vect = np.vectorize(newcls.bar)
        except AttributeError:
            pass  # Allow class w/o bar; remove try/except if class must have bar
        return newcls

class Foo(object):
    __metaclass__ = BarVectorized
    @classmethod
    def bar(cls, x): return x + 1

class Foo2(Foo):
    ADD = 2  # Hardcoded 1 is dumb, use class attribute instead!
    @classmethod
    def bar(cls, x):
        return x + cls.ADD

class Foo3(Foo2):
    ADD = 3  # Provide new class attr to change Foo2.bar behavior when called via Foo3

>>> Foo.bar_vect([1,2,3])
array([2, 3, 4])
>>> Foo2.bar_vect([1,2,3])
array([3, 4, 5])
>>> Foo3.bar_vect([1,2,3])
array([4, 5, 6])

No need to define bar_vect explicitly at all, and bar_vect seamlessly uses the most local classes' definition of bar available at class definition time, so unless bar is redefined after class definition, it always works, and it works as efficiently as possible. To make it use bar live, you'd need to resort to more extreme measures that perform dynamic lookup and (barring a cache) reconstruction of the np.vectorize object on each use, which is suboptimal to say the least.

For completeness, a dynamic caching based solution (hat tip to Tadhg McDonald-Jensen's answer) that uses a dynamically populating cache that adds minimal overhead (and more importantly in my opinion, abstracts out the boilerplate code that's irrelevant to the work) for the case where the cache entry already exists by using a dict subclass defining __missing__:

import operator
import numpy as np

class ClassAttrRegistry(dict):
    '''Dictionary keyed by classes which returns optionally wrapped cached attributes'''
    __slots__ = '_wrapper', '_attrgetter'
    def __init__(self, attr, wrapperfunc=lambda x: x):
        self._wrapper = wrapperfunc
        self._attrgetter = operator.attrgetter(attr)
    def __missing__(self, cls):
        self[cls] = wrapped = self._wrapper(self._attrgetter(cls))
        return wrapped

class Foo(object):
    @classmethod
    def bar(cls, x):
        return x + 1

    # Dunder prefix makes cache private to Foo methods; if subclass overrides bar_vect,
    # assumed it's more complex than "vectorized bar"; cache should not be used
    __bar_vect_registry = ClassAttrRegistry('bar', np.vectorize)
    @classmethod
    def bar_vect(cls, x):
        # Get cached vectorized bar (creating if needed) then call it
        return cls.__bar_vect_registry[cls](x)

Subclasses don't need to (and should not) override bar_vect (and can't accidentally access __bar_vect_registry because it's name mangled such that only methods defined by Foo will see it; change name to _bar_vect_registry, one underscore, if it should be accessible to subclasses), they just override bar and Foo's bar_vect will create/cache vectorized accessors when bar_vect is first accessed on the subclass (or an instance thereof).

Answer 2

Your confusion to why this is not an easy work around is understandable, let me elaborate on to why using classmethod in this way isn't going to work...

The way classmethod works is that it creates a descriptor, an object that implements __get__ when it is retrieved as an attribute on an object.

So when you do Foo.bar it basically loads the bar classmethod and calls:

bar.__get__(None, Foo)

Where the None represents the instance (there is None because it is on the class itself) and the second argument represents the class, a classmethod is not callable because then it would not have a class to bind it too!

Not only this but the class object to bind it too doesn't exist until the class definition block has ended (and the metaclass type actually puts it together) so the bare minimum is to create bar_vect after the class is actually defined:

class Foo( object ):
    a = 1 #lets use an example that actually uses the class
    @classmethod
    def bar( cls, x ):
        return x+cls.a

Foo.bar_vect = np.vectorize( Foo.bar )

This will work sure, but then you break the functionality of subclasses, what if you wanted to change a?

class Subfoo(Foo):
    a = 3 #this will have no effect on 

assert Subfoo.bar_vect(np.array([ 1, 2, 3 ])) == np.array([ 4, 5, 6 ])
#this SHOULD work but doesn't because you bound bar_Vect to just Foo
#subclasses mean nothing to your class method

The only way to make it work in this case is to recreate the np.vectorize at least one for each subclass, the simplest version is to just do it every time you call bar_vect:

class Foo( object ):
    a = 1
    @classmethod
    def bar( cls, x ):
        return x+cls.a
    @classmethod
    def bar_vect(cls,arg):
        return np.vectorize(cls.bar)(arg)

This is obviously undesirable because it calls np.vectorize every time x.bar_vect is used, however you could make a record of all the classes and only make it when a new class is used:

_bar_vect_registry = {}
@classmethod
def bar_vect(cls,arg):
    try:
        return cls._bar_vect_registry[cls](arg)
    except KeyError:
        cls._bar_vect_registry[cls] = np.vectorize(cls.bar)
        return cls._bar_vect_registry[cls](arg)

Answer 3

You real problem is that you try to use bar before the class if fully constructed, so you do not get the expected object.

Here is a simplified example:

class Foo:
    @classmethod
    def bar(cls, x):
        print ('bar called in', cls, 'with', x)
    barv = str(bar)

print(str(Foo.bar))
print(Foo.barv)

gives:

<bound method Foo.bar of <class '__main__.Foo'>>
<classmethod object at 0x00000000035B0320>

That shows that until the class is fully constructed, the methods identifier are only bound to the method definitions and not to the real methods.

If you want achieve what you want, you must define the class variable outside of class definition (after last line), as explained by @ShadowRanger