How to add a coroutine/task to the loop from a 'blocking function' already running in a loop

Question

Hopefully the following code explains what I want to do better than the question title.

import asyncio
import time

loop = asyncio.get_event_loop()

class Foo(object):
    def __init__(self, num):
        self.num = num
    @property
    def number(self):
        # Somehow need to run get_number() in the loop.
        number = self.get_number()
        return number
    @asyncio.coroutine
    def get_number(self):
        yield from loop.run_in_executor(None, time.sleep, self.num)
        return self.num


@asyncio.coroutine
def runner(num):
    print("getting Foo({})".format(num))
    foo = Foo(num)
    yield from asyncio.sleep(1)
    print("accessing number property of Foo({})".format(num))
    number = foo.number
    print("Foo({}) number is {}".format(num, number))


tasks = [
    asyncio.async(runner(3)),
    asyncio.async(runner(1)),
    ]
go = loop.run_until_complete(asyncio.wait(tasks))

I can't work out what to do in the number function, where the comment is. I've tried all sorts of things, but in reality I've just "been throwing **** at the wall and hoping something sticks".

This is a follow-up to this question. I want to access the property without doing a yield from, as I need to access the property from a template (eg mako), and having yield from written everywhere is not ideal (probably not even possible considering mako is probably blocking). In a perfect world, I would have all of this running with a reify decorator.

If I wanted to use yield from, the code would be quite simple.

class Foo(object):
    def __init__(self, num):
        self.num = num
    @property
    @asyncio.coroutine
    def number(self):
        yield from loop.run_in_executor(None, time.sleep, self.num)
        return self.num


@asyncio.coroutine
def runner(num):
    print("getting Foo({})".format(num))
    foo = Foo(num)
    yield from asyncio.sleep(1)
    print("accessing number property of Foo({})".format(num))
    number = yield from foo.number
    print("Foo({}) number is {}".format(num, number))

#getting Foo(3)
#getting Foo(1)
#accessing number property of Foo(3)
#accessing number property of Foo(1)
#Foo(1) number is 1
#Foo(3) number is 3

I found this answer on the topic, but I can't see how adding done callbacks will work with my workflow.

Sorry, you are asking for impossible thing. There is no way to get a value from coroutine without `yield from`. — Andrew Svetlov, Jul 22 '15 at 06:01
Where I have the comment `# Somehow need to run get_number() in the loop.` is where I am hoping to create a future task and put that into the loop, thus suspending the current function - I just don't know how to do it. So for example change `number = self.get_number()` to `number = loop.create_task(self.get_number())`. Is that possible or not? — neRok, Jul 22 '15 at 07:13
@neRok The only way to suspend a currently running function is to use `yield from`, which means it has to be a coroutine. You can add a task to the event loop use `loop.create_task(self.get_number())`, like you suggested, but that task won't actually execute until the method that's actually calling `create_task` gives control back to the event loop, either by returning or making a call that uses `yield from`. Integrating `asyncio`-based code into synchronous code just isn't going to work the way you're hoping. — dano, Jul 23 '15 at 01:31
@neRok There are ways to do some level of integration between synchronous and `asyncio`-based code, just not quite the way you're trying to do it: See http://stackoverflow.com/q/25299887/2073595 and http://stackoverflow.com/q/30155138/2073595. — dano, Jul 23 '15 at 01:33

score 1 · Accepted Answer · answered Jul 22 '15 at 13:51

What you ask for is impossible because when you are in your main thread (where you want to call foo.number without the yield from, you need to explicitly give back control to the main loop. This is exactly what yield from does.

Without that, you need to run the function that call foo.number in a separate thread that would be able to block (no yield from) and wait for the result of get_number without blocking the main loop

score 0 · Answer 2 · answered Jan 06 '17 at 22:17

From your blocking function instead of calculated value you shall return asyncio.Future:

return loop.create_task(self.get_number())

When you get this in your async runner you can wait for result like this:

number = await foo.number

Full test case:

def test_future():
    loop = asyncio.get_event_loop()

    async def target(x: int) -> int:
        loop.run_in_executor(None, time.sleep, 0.1)
        return x + 1

    def intermediate(x: int) -> asyncio.Future:
        return loop.create_task(target(x))

    async def main():
        future = intermediate(5)
        logger.debug('intermediate future = %r', future)
        value = await future
        assert value == 6

    try:
        loop.create_task(main())
        loop.call_later(0.5, loop.stop)
        loop.run_forever()
    finally:
        loop.close()

How to add a coroutine/task to the loop from a 'blocking function' already running in a loop

2 Answers2

Linked