0

I have classes like this:

class Tool(object):
    def do_async(*args):
        pass

for which I want to automatically generate non-async methods that make use of the async methods:

class Tool(object):
    def do_async(*args):
        pass
    def do(*args):
        result = self.do_async(*args)
        return magical_parser(result)

This gets to be particularly tricky because each method needs to be accessible as both an object and class method, which is normally achieved with this magical decorator:

class class_or_instance(object):
    def __init__(self, fn):
        self.fn = fn

    def __get__(self, obj, cls):
        if obj is not None:
            f = lambda *args, **kwds: self.fn(obj, *args, **kwds)
        else:
            f = lambda *args, **kwds: self.fn(cls, *args, **kwds)
        functools.update_wrapper(f, self.fn)
        return f

How can I make these methods, and make sure they're accessible as both class and object methods? This seems like something that could be done with decorators, but I am not sure how.

(Note that I don't know any of the method names in advance, but I know that all of the methods that need new buddies have _async at the end of their names.)

I think I've gotten fairly close, but this approach does not appropriately set the functions as class/object methods:

def process_asyncs(cls):

    methods = cls.__dict__.keys()
    for k in methods:
        methodname = k.replace("_async","")
        if 'async' in k and methodname not in methods:

            @class_or_instance
            def method(self, verbose=False, *args, **kwargs):
                response = self.__dict__[k](*args,**kwargs)
                result = self._parse_result(response, verbose=verbose)
                return result

            method.__docstr__ = ("Returns a table object.\n" +
                    cls.__dict__[k].__docstr__)

            setattr(cls,methodname,MethodType(method, None, cls))
keflavich
  • 18,278
  • 20
  • 86
  • 118
  • What are the semantics for calling these methods as instance or class methods? Do they even use the instance for anything when called as instance methods? This sounds like a job for module-level functions, or perhaps `staticmethod` if you're really, absolutely sure these things need to be methods. – user2357112 Aug 04 '13 at 23:06
  • Yes, normally these would just be module-level functions, but there are some cases in which an instance is needed. Basically, this is for a suite of web querying tools, where normally you can just use the classmethod, but sometimes you need to use the object to log in to the service first. There may be better ways, but this approach has worked well so far. (work in progress: astroquery.readthedocs.org) – keflavich Aug 04 '13 at 23:10

1 Answers1

3

Do not get the other method from the __dict__; use getattr() instead so the descriptor protocol can kick in.

And don't wrap the method function in a MethodType() object as that'd neutralize the descriptor you put on method.

You need to bind k to the function you generate; a closured k would change with the loop:

@class_or_instance
def method(self, verbose=False, _async_method_name=k, *args, **kwargs):
    response = getattr(self, _async_method_name)(*args,**kwargs)
    result = self._parse_result(response, verbose=verbose)
    return result

cls.__dict__[methodname] = method

Don't forget to return cls at the end; I've changed this to use a separate function to create a new scope to provide a new local name _async_method_name instead of a keyword parameter; this avoids difficulties with *args and explicit keyword arguments:

def process_asyncs(cls):

    def create_method(async_method):

        @class_or_instance
        def newmethod(self, *args, **kwargs):
            if 'verbose' in kwargs:
                verbose = kwargs.pop('verbose')
            else:
                verbose = False
            response = async_method(*args,**kwargs)
            result = self._parse_result(response, verbose=verbose)
            return result
        return newmethod

    methods = cls.__dict__.keys()
    for k in methods:
        methodname = k.replace("_async","")
        if 'async' in k and methodname not in methods:
            async_method = getattr(cls, k)
            setattr(cls, methodname, create_method(async_method))

    return cls
keflavich
  • 18,278
  • 20
  • 86
  • 118
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • I had tried that, but got the error: `TypeError: 'dictproxy' object does not support item assignment`. My classes have `__metaclass__=abc.ABCMeta`; I don't really know if that's warranted or related to this. – keflavich Aug 04 '13 at 22:52
  • Excellent. I don't need to return `cls`, though, since it is modifying the class directly. I've added the full version of this solution to the answer above. – keflavich Aug 04 '13 at 23:09
  • If you are using it as a decorator, you *do* need to return `cls` because python replaces the decorated object with whatever the decorator returns. – Martijn Pieters Aug 05 '13 at 00:17
  • Good point! That's a much nicer way to do this. I'm still having trouble with the docstrings, but the decorator works as it should (once `return cls` is added). Awesome! – keflavich Aug 05 '13 at 04:44
  • 1
    Your posted solution in your question will suffer from the fact `async_method` is a closure in a loop; all your generated methods will point to **one** async method. – Martijn Pieters Aug 05 '13 at 07:48
  • 1
    See [Local variables in Python nested functions](http://stackoverflow.com/q/12423614) for a more thorough explanation of what will go wrong with the `async_method` name in your code, and why I bound `k` to a keyword argument in my solution. – Martijn Pieters Aug 05 '13 at 09:06
  • You're quite right, and thanks for pointing to that other article. This is somewhat surprising behavior (probably because I haven't fully internalized the problem) and I doubt I would have figured it out without the explicit note. – keflavich Aug 05 '13 at 16:20
  • Unfortunately, specifying the arguments in this manner means that it is impossible to pass positional arguments to `newmethod`, which is actually required. Is there any workaround? (`def newmethod(self, *args, _async_method=async_method, **kwargs):` is not allowed) – keflavich Aug 05 '13 at 17:07
  • 1
    Yes, by creating a new scope; use a *separate* function that returns `newmethod()` when called. See the linked question from my previous comment. – Martijn Pieters Aug 05 '13 at 17:11
  • Updated to do just that. – Martijn Pieters Aug 05 '13 at 17:15
  • There is one more major issue that I've resolved: if you use `getattr` to get the method, it only gets the *class* method, it will never return an instance's attribute. To get around this, bind the *name* instead of the method. See https://github.com/keflavich/astroquery/blob/30deafc3aa057916bcdca70733cba748f1b36b64/astroquery/utils/process_asyncs.py for the full solution. – keflavich Aug 07 '13 at 04:22
  • 1
    Yup, which is what my original answer did but your edit cached the method instead. :-) A late lookup allows you to find instance attributes too, yes. – Martijn Pieters Aug 07 '13 at 08:59