partial asynchronous functions are not detected as asynchronous

Question

I have a function which accepts both regular and asynchronous functions (not coroutines, but functions returning coroutines).

Internally it uses asyncio.iscoroutinefunction() test to see which type of function it got.

Recently it broke down when I attempted to create a partial async function.

In this demonstration, ptest is not recognized as a couroutine function, even if it returns a coroutine, i.e. ptest() is a coroutine.

import asyncio
import functools

async def test(arg): pass
print(asyncio.iscoroutinefunction(test))    # True

ptest = functools.partial(test, None)
print(asyncio.iscoroutinefunction(ptest))   # False!!

print(asyncio.iscoroutine(ptest()))         # True

The problem cause is clear, but the solution is not.

How to dynamically create a partial async func which passes the test?

OR

How to test the func wrapped inside a partial object?

Either answer would solve the problem.

Martijn Pieters · Accepted Answer · 2019-10-03T12:43:38.193

19

Using Python versions < 3.8 you can't make a partial() object pass that test, because the test requires there to be a __code__ object attached directly to the object you pass to inspect.iscoroutinefunction().

You should instead test the function object that partial wraps, accessible via the partial.func attribute:

>>> asyncio.iscoroutinefunction(ptest.func)
True

If you also need to test for partial() objects, then test against functools.partial:

def iscoroutinefunction_or_partial(object):
    while isinstance(object, functools.partial):
        object = object.func
    return inspect.iscoroutinefunction(object)

In Python 3.8 (and newer), the relevant code in the inspect module (that asyncio.iscoroutinefunction() delegates to) was updated to handle partial() objects, and you no longer have to unwrap partial() objects yourself. The implementation uses the same while isinstance(..., functools.partial) loop.

edited Oct 03 '19 at 12:43

answered Sep 20 '18 at 10:24

Martijn Pieters

1,048,767
296
4,058
3,343

Thank you. Is the `.func` attr documented somewhere? – VPfB Sep 20 '18 at 10:26
1

@VPfB: yes, the [`functools.partial()` documentation](https://docs.python.org/3/library/functools.html#functools.partial) states it exists explicitly; the *Roughly equivalent to* code shows the attributes that are set. – Martijn Pieters Sep 20 '18 at 10:27
1

The functools docs had some problems with links (in 2018), that's why I did not find the info: https://bugs.python.org/issue34748 – VPfB Sep 07 '19 at 06:03
@VPfB interesting; that bug report was opened about an hour after I wrote this answer; I think it is fair to credit your question as the reason the issue was discovered. :-) – Martijn Pieters Sep 07 '19 at 09:00
No surprise. The bug reporter and I share the same office room. – VPfB Sep 07 '19 at 11:33
What about [`functools.partialmethod()`](https://docs.python.org/3/library/functools.html#functools.partialmethod)? – DurandA Oct 03 '19 at 06:08
1

@DurandA: what is your exact question? How to test a `partialmethod` object wraps a coroutine function? Then it depends on exactly what was wrapped and how you accessed the partialmethod object. If you accessed it through `ClassObj.partialmethod_name` or `instance.partialmethod_name`, and it wrapped a normal `async def` method, then descriptor binding takes place and a `partial()` object is returned. So in that case the same technique applies. – Martijn Pieters Oct 03 '19 at 12:53
1

@DurandA: actually, that's not quite correct. I filed [a new bug report](https://bugs.python.org/issue38364) as I discovered that this is a little more complicated. For `instance.partialmethod_name`, you can simply unwrap *first* (and you have to do this in Python 3.8) too). For `ClassObj.partialmethod_name` you *also* have to look for the `_partialmethod` attribute, follow that, and *then* follow the `.func` attribute. – Martijn Pieters Oct 03 '19 at 13:31

score 5 · Answer 2 · answered Mar 14 '21 at 09:01

I solved this by replacing all instances of partial with async_partial:

def async_partial(f, *args):
   async def f2(*args2):
       result = f(*args, *args2)
       if asyncio.iscoroutinefunction(f):
           result = await result
       return result

   return f2

partial asynchronous functions are not detected as asynchronous

2 Answers2

Linked