8

I want this code to "just work":

def main():
    c = Castable()
    print c/3
    print 2-c
    print c%7
    print c**2
    print "%s" % c
    print "%i" % c
    print "%f" % c

Of course, the easy way out is to write int(c)/3, but I'd like to enable a simpler perl-ish syntax for a configuration mini-language.

It's notable that if I use an "old-style" class (don't inherit from object) I can do this quite simply by defining a __coerce__ method, but old-style classes are deprecated and will be removed in python3.

When I do the same thing with a new-style class, I get this error:

TypeError: unsupported operand type(s) for /: 'Castable' and 'int'

I believe this is by design, but then how can I simulate the old-style __coerce__ behavior with a new-style class? You can find my current solution below, but it's quite ugly and long-winded.

This is the relevant documentation: (i think)

Bonus points:

    print pow(c, 2, 100)
bukzor
  • 37,539
  • 11
  • 77
  • 111
  • http://docs.python.org/reference/datamodel.html#coercion-rules – John La Rooy Aug 11 '10 at 23:31
  • oh i didn't notice it there in the question. I meant to direct you to the part about new style classes never calling `__coerce__` etc. but I had to pop away for a few minutes – John La Rooy Aug 12 '10 at 00:06
  • If you want Perl, use Perl. Python is strongly typed (for a certain definition of "strongly typed"); trying to change that is trying to change a fundamental part of the language. Your way below is the right way to do this; it is long-winded by necessity. – Katriel Aug 19 '10 at 09:09
  • @katrielalex: I'm just quite surprised that something that was previously easy is now hard. Usually the python devs are so careful about feature changes. – bukzor Aug 19 '10 at 22:23

5 Answers5

8

You need to define __div__ if you want c/3 to work. Python won't convert your object to a number first for you.

Ned Batchelder
  • 364,293
  • 75
  • 561
  • 662
  • I don't expect it to convert for me, but I do expect to be able to do it myself if needed without defining each and every operation. I guess that's what you're suggesting really. – bukzor Aug 11 '10 at 23:17
  • In particular, I'm surprised that I can't catch an attempted call to `__div__` via `__getattribute__`. – bukzor Aug 11 '10 at 23:41
  • 7
    http://docs.python.org/reference/datamodel.html#new-style-special-lookup says "For new-style classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type" and later "implicit special method lookup generally also bypasses the `__getattribute__()` method even of the object’s metaclass". So, no, you have to define each and every operation. – zwol Aug 11 '10 at 23:49
  • @Zack: that's one of the only helpful things people have said. – bukzor Aug 12 '10 at 20:42
5

This works, and is less gross after several improvements (props to @jchl), but still seems like it should be unecessary, especially considering that you get this for free with "old-style" classes.

I'm still looking for a better answer. If there's no better method, this seems to me like a regression in the Python language.

def ops_list():
    "calculate the list of overloadable operators"
    #<type 'object'> has functions but no operations
    not_ops = dir(object)

    #calculate the list of operation names
    ops = set()
    for mytype in (int, float, str):
        for op in dir(mytype):
            if op.endswith("__") and op not in not_ops:
                ops.add(op)
    return sorted(ops)

class MetaCastable(type):
    __ops = ops_list()

    def __new__(mcs, name, bases, dict):
        #pass any undefined ops to self.__op__
        def add_op(op):
            if op in dict:
                return
            fn = lambda self, *args: self.__op__(op, args)
            fn.__name__ = op
            dict[op] = fn

        for op in mcs.__ops:
            add_op( op )
        return type.__new__(mcs, name, bases, dict)


class Castable(object):
    __metaclass__ = MetaCastable
    def __str__(self):
        print "str!"
        return "<Castable>"
    def __int__(self):
        print "int!"
        return 42
    def __float__(self):
        print "float!"
        return 2.718281828459045

    def __op__(self, op, args):
        try:
            other = args[0]
        except IndexError:
            other = None
        print "%s %s %s" % (self, op, other)
        self, other = coerce(self, other)
        return getattr(self, op)(*args)

    def __coerce__(self, other):
        print "coercing like %r!" % other
        if other is None: other = 0.0
        return (type(other)(self), other)
bukzor
  • 37,539
  • 11
  • 77
  • 111
  • I actually think this is a pretty neat solution. Given that Python doesn't seem to provide this functionality, this is not many lines of code to add a pretty powerful feature to the language. – jchl Aug 18 '10 at 09:09
  • There are a few things you could do to make it neater still. Better to modify `dict` before calling `type.__new__` than to use `setattr` afterwards. Better to give your lambda functions an appropriate `__name__`, and to use a closure rather than the default arguments trick, so that they have the proper prototype. And I'm not sure whether you consider it a feature that it assumes that the type doesn't define `__radd__` if it doesn't define `__add__`, or whether this is a bug. – jchl Aug 18 '10 at 09:16
  • @jchl: thanks. I'll do those after work, or you can feel free and try to grab the bounty. I guess the main thing I hate about this solution is the static lists _unary and _binary. It also doesn't support the 3-argument pow() operator. – bukzor Aug 18 '10 at 22:07
  • @bukzor: Yeah, I was hoping the `operator` module had lists of the unary and binary operators, but no such luck. – jchl Aug 19 '10 at 08:30
  • @jchl: I've refactored the setattr's away, as suggested. I'm not sure how you intended to use a closure, since it still has the same late-binding problems as lambda functions (all ops become `__invert__`). – bukzor Aug 20 '10 at 02:59
  • @bukzor: You need to return the function/lambda from within another function. I'll post what I mean; easier than trying to write code in a comment. – jchl Aug 20 '10 at 08:17
  • I don't think it's ugly at all. You probably can't get any better with Python. – Philipp Aug 20 '10 at 10:06
  • @jchl: Thanks. I was able to improve on that and merge most of the unary/binary code. As a bonus, the 3-argument pow operation now works as well. – bukzor Aug 20 '10 at 20:11
  • Doesn't the inline code in the class definition leave `MetaCastable` with two unwanted attributes named `mytype` and `op`? I think you need to either `del` them or move that code to a separate function to avoid that. – jchl Aug 24 '10 at 09:12
  • @jchl: fixed, thanks. I haven't found someone more picky than myself before :P – bukzor Aug 24 '10 at 18:53
3
class MetaCastable(type):
    __binary_ops = ( 
            'add', 'sub', 'mul', 'floordiv', 'mod', 'divmod', 'pow', 'lshift', 
            'rshift', 'and', 'xor', 'or', 'div', 'truediv',
    )

    __unary_ops = ( 'neg', 'pos', 'abs', 'invert', )

    def __new__(mcs, name, bases, dict):
        def make_binary_op(op):
            fn = lambda self, other: self.__op__(op, other)
            fn.__name__ = op
            return fn

        for opname in mcs.__binary_ops:
            for op in ( '__%s__', '__r%s__' ):
                op %= opname
                if op in dict:
                    continue
                dict[op] = make_binary_op(op)

        def make_unary_op(op):
            fn = lambda self: self.__op__(op, None)
            fn.__name__ = op
            return fn

        for opname in mcs.__unary_ops:
            op = '__%s__' % opname
            if op in dict:
                continue
            dict[op] = make_unary_op(op)

        return type.__new__(mcs, name, bases, dict)

class Castable(object):
    __metaclass__ = MetaCastable
    def __str__(self):
        print "str!"
        return "<Castable>"
    def __int__(self):
        print "int!"
        return 42
    def __float__(self):
        print "float!"
        return 2.718281828459045

    def __op__(self, op, other):
        if other is None:
            print "%s(%s)" % (op, self)
            self, other = coerce(self, 0.0)
            return getattr(self, op)()
        else:
            print "%s %s %s" % (self, op, other)
            self, other = coerce(self, other)
            return getattr(self, op)(other)

    def __coerce__(self, other):
        print "coercing like %r!" % other
        return (type(other)(self), other)
jchl
  • 6,332
  • 4
  • 27
  • 51
0
class Castable(object):
    def __div__(self, other):
        return 42 / other
Matt Williamson
  • 39,165
  • 10
  • 64
  • 72
0

New style classes operate faster and more precise than old style classes. Thus no more expensive __getattr__, __getattribute__ , __coerce__ calls for any cheap reasons and in a questionable order.

The old style __coerce__ also had the problem, that it was called even when you have already overloaded an operator method for some special purpose. And it demands casting to equal common types, and is limited to certain binary ops. Think about all the other methods and properties of an int / float / string - and about pow(). Due to all these limitations coerce is missing in PY3. The question examples aim at rather wide virtualization.

With new style classes its just about a loop to provide many "similar" methods with little code, or route those calls to a virtual handler and then its fast and precisely defined and subclassable in correct and fine grained manner. Thats not a "regression in the Python language".

However I would not employ a meta class as shown in other answers just for such a loop or for providing a simple base class kind of behavior. That would be cracking a nut with a sledgehammer.


Here an example helper for virtualization of a "variant":

def Virtual(*methods):
    """Build a (new style) base or mixin class, which routes method or
    operator calls to one __virtualmeth__ and attribute lookups to
    __virtualget__ and __virtualset__ optionally.

    *methods (strings, classes): Providing method names to be routed
    """
    class VirtualBase(object):  
        def __virtualmeth__(self, methname, *args, **kw):
            raise NotImplementedError
    def _mkmeth(methname, thing):
        if not callable(thing):
            prop = property(lambda self:self.__virtualget__(methname),
                            lambda self, v:self.__virtualset__(methname, v))
            return prop
        def _meth(self, *args, **kw):
            return self.__virtualmeth__(methname, *args, **kw)
        _meth.__name__ = methname
        return _meth
    for m in methods:
        for name, thing in (isinstance(m, str) and
                            {m:lambda:None} or m.__dict__).items():
            if name not in ('__new__', '__init__', '__setattr__', ##'__cmp__',
                            '__getattribute__', '__doc__', ):   ##'__getattr__', 
                setattr(VirtualBase, name, _mkmeth(name, thing))
    return VirtualBase

And here an example use case: An Anaphor! (PY2 and PY3) :

import operator
class Anaphor(Virtual(int, float, str)):   
    """remember a sub-expression comfortably:

    A = Anaphor()      # at least per thread / TLS
    if re.search(...) >> A:
        print(A.groups(), +A)
    if A(x % 7) != 0:
        print(A, 1 + A, A < 3.0, A.real, '%.2f' % A, +A)
    """
    value = 0
    def __virtualmeth__(self, methname, *args, **kw):
        try: r = getattr(self.value, methname)(*args, **kw)
        except AttributeError:
            return getattr(operator, methname)(self.value, *args, **kw)
        if r is NotImplemented: # simple type -> coerce
            try: tcommon = type(self.value + args[0])    # PY2 coerce
            except: return NotImplemented
            return getattr(tcommon(self.value), methname)(*args, **kw)
        return r
    def __call__(self, value):   
        self.value = value
        return value
    __lshift__ = __rrshift__ = __call__     # A << x;  x >> A
    def __pos__(self):                      # real = +A
        return self.value
    def __getattr__(self, name):
        return getattr(self.value, name)
    def __repr__(self):
        return '<Anaphor:%r>' % self.value

Seamlessly it also handles the 3-arg opertor pow() :-) :

>>> A = Anaphor()
>>> x = 1
>>> if x + 11 >> A:
...     print repr(A), A, +A, 'y' * A, 3.0 < A, pow(A, 2, 100)
...     
<Anaphor:12> 12 12 yyyyyyyyyyyy True 44
kxr
  • 4,841
  • 1
  • 49
  • 32