is not a Python function

Question

I'm trying to build a function which I can use as a handler for an RxPy stream that I'm mapping over. The function I have needs access to a variable outside the scope where that variable is defined which, to me, means that I need to use a closure of some kind. So I reached for functools.partial to close over the one variable and return a partial function that I can pass to as an observer to my stream.

However, doing so results in the following:

Traceback (most recent call last):
  File "retry/example.py", line 46, in <module>
    response_stream = message_stream.flat_map(functools.partial(message_handler, context=context))
  File "/home/justin/virtualenv/retry/local/lib/python2.7/site-packages/rx/linq/observable/selectmany.py", line 67, in select_many
    selector = adapt_call(selector)
  File "/home/justin/virtualenv/retry/local/lib/python2.7/site-packages/rx/internal/utils.py", line 37, in adapt_call_1
    argnames, varargs, kwargs = getargspec(func)[:3]
  File "/usr/lib/python2.7/inspect.py", line 816, in getargspec
    raise TypeError('{!r} is not a Python function'.format(func))
TypeError: <method-wrapper '__call__' of functools.partial object at 0x2ce6cb0> is not a Python function

Here is some sample code which reproduces the problem:

from __future__ import absolute_import
from rx import Observable, Observer
from pykafka import KafkaClient
from pykafka.common import OffsetType
import logging
import requests
import functools


logger = logging.basicConfig()


def puts(thing):
    print thing


def message_stream(consumer):
    def thing(observer):
        for message in consumer:
            observer.on_next(message)

    return Observable.create(thing)


def message_handler(message, context=None):
    def req():
        return requests.get('http://httpbin.org/get')

    return Observable.start(req)


def handle_response(message, response, context=None):
    consumer = context['consumer']
    producer = context['producer']
    t = 'even' if message % 2 == 0 else 'odd'
    return str(message) + ': ' + str(response) + ' - ' + t + ' | ' + str(consumer) + ' | ' + producer


consumer = ['pretend', 'these', 'are', 'kafka', 'messages']
producer = 'some producer'
context = {
    'consumer': consumer,
    'producer': producer
}
message_stream = message_stream(consumer)
response_stream = message_stream.flat_map(functools.partial(message_handler, context=context))
message_response_stream = message_stream.zip(response_stream, functools.partial(handle_response, context=context))
message_stream.subscribe(puts)

The problem seems to be that my partial function returns False when calling inspect.isfunction.

How can I make my partial function pass this check? Is there a way to easily convert a partial function into a "real" function type?

Answer 1

You're asking if it's actually a function, and it's telling you isn't not a function. It's a method-wrapper.

You want to duck-type.

>>> def printargs(*args):
...     print args

>>> import inspect
>>> from functools import partial
>>> inspect.isfunction(printargs)
True
>>> f = partial(printargs, 1)
>>> inspect.isfunction(f)
False
# try duck-typing, see if the variable is callable
# check does it work for a method-wrapper?
>>> callable(f)
True
# check an integer, which should be false
>>> callable(1)
False
# ensure it works on an actual function
>>> callable(printargs)
True

This is why you duck-type. You don't care if it's a function. You care if it acts like a function.

EDIT: You could, if desperate enough, write a class and pass a reference to a function in the class.

class A():
    def __init__(self, frozen, *args, **kwds):
        self.frozen = frozen
        self.args = args
        self.kwds = kwds

    def call(self):
        self.frozen(*self.args, **self.kwds)

Then just use A(f).call as your wrapper.

>>> f_ = A(f)
>>> inspect.ismethod(f_.call)
True
>>> f_.call()
(1,)

This works as long as ismethod works.

If not, you really need a decorator.

Final EDIT: If you are truly desperate enough and don't want to write a custom decorator, you could use a lambda function with a tuple to pass to make a partial-like function.

Ex.:

>>> import inspect
>>> def printargs(*args):
...     print args
>>> a = (1,2,3)
>>> f = lambda x: printargs(*x)
>>> f(a)
(1, 2, 3)
>>> inspect.isfunction(f)
True