0

I want to run a coroutine in a different thread and get the result that the coroutine returns.

class Main:
    def __init__(self, result_from_io_task=None):
        self._io_task_result = result_from_io_task        

    async def io_task(self):
        await asyncio.sleep(2)
        return "slept of 2s"
    
    def non_async_func(self):
        #This can't be made async.
        if not self._io_task_result:
            #run io_task and get its result

            #event loop will be running in the main thread so I can fire the task
            task = asyncio.create_task(self.io_task)

            #can't await task since I am in non-async func and I cannot 
            #return from non_async_func until and unless I know what
            #self.io_task has returned. Tried following but my app hangs forever.
        
            while not task.done():
                pass

I also tried, but it doesn't work "

def run_in_thread(coro, loop):
       output = []
       def run():
           fut = asyncio.run_coroutine_threadsafe(coro, loop)
           output.append(fut)
       thr = Thread(target=run)
       thr.start()
       return output

async def main():
    main_obj = Main(result_from_io_task=None)
    v = main_obj.non_async_func()

How can I spawn a new thread and run the given coroutine using event loop running in main thread

user6037143
  • 516
  • 5
  • 20
  • Does this answer your question? [How to call a async function from a synchronized code Python](https://stackoverflow.com/questions/51762227/how-to-call-a-async-function-from-a-synchronized-code-python) – larsks Aug 10 '22 at 03:49
  • See this answer: https://stackoverflow.com/questions/70231451/wrapping-python-async-for-synchronous-execution/70254610#70254610 – Paul Cornelius Aug 10 '22 at 06:20
  • What is the reason to run a coroutine in a thread? Both are blocked from loops, and both are effective if you run I/O tasks. – Artyom Vancyan Aug 10 '22 at 06:25
  • 1
    I don't necessarily need a thread. I have updated my example to show that I start `asyncio.run(main())` and within main I want to call a method which may want to get result from an async func. I CANNOT MAKE non_async_func async – user6037143 Aug 10 '22 at 09:35
  • I see the problem. Please check out my [answer](https://stackoverflow.com/questions/72593019/how-can-i-fire-and-forget-a-task-without-blocking-main-thread/72683539#72683539), especially the section **Situation 2**. It is about "How to make synchronous functions asynchronous". I hope you'll get the answer to your questions there. – Artyom Vancyan Aug 10 '22 at 09:55
  • 1
    Unfortunately, my codebase depends on python <3.8 and `asyncio.to_thread` is not available in python 3.7. – user6037143 Aug 10 '22 at 12:19
  • I think that threads are your only choice here. From a non-async function you absolutely cannot yield back to the event loop, run a task, and resume the function at the point where you yielded. Non-async functions do not have the machinery to do that; async functions do. To run an async function in a second thread you must initialize an event loop in that thread. Then you call asyncio.run_coroutine_threadsafe, and you get back an concurrent.futures.Future object. You call it's result() method, which blocks your main thread until the result is available. See the link I posted yesterday. – Paul Cornelius Aug 10 '22 at 21:48
  • The answers here are kind of confused and I would look elsewhere; however Artyom Vancyan’s link to an old post (not the newer answer here) does cover important concepts. – fuzzyTew Sep 02 '23 at 11:30

1 Answers1

1

Unfortunately, my codebase depends on python < 3.8 and asyncio.to_thread is not available in python 3.7

Based on the example of my answer, I'm introducing another implementation of the asynchronous decorator that does not use asyncio.to_thread() function but uses ThreadPoolExecutor instead.

import asyncio
import requests
import concurrent.futures


def asynchronous(func):
    async def wrapper(*args, **kwargs):
        with concurrent.futures.ThreadPoolExecutor() as executor:
            future = executor.submit(func, *args, **kwargs)
            return future.result()

    return wrapper


@asynchronous
def request(url):
    with requests.Session() as session:
        response = session.get(url)
        try:
            return response.json()
        except requests.JSONDecodeError:
            return response.text


async def main():
    task = asyncio.create_task(request("https://google.com/"))
    print("waiting for response...")
    result = await task
    print(result)


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()
Artyom Vancyan
  • 5,029
  • 3
  • 12
  • 34
  • This looks like wrapper() will block its event loop until request completes. – fuzzyTew Aug 22 '23 at 12:46
  • @fuzzyTew, if you call `asyncio.create_task(request(...))` multiple times, e.g. in a loop, it will push the `request` function to the thread pool with the given parameters and wait for the results simultaneously. – Artyom Vancyan Aug 22 '23 at 14:03
  • It looks to me like it waits serially. Any other asyncio tasks in the same thread would block until future.result() returns because it is a threaded future instead of an asyncio future. It works if nothing else is being done in the asyncio loop, but in this case asyncio is not very useful. [Am I wrong?] – fuzzyTew Sep 02 '23 at 00:33
  • This shows how to run coroutines in multiple threads. As we are dealing with Python threads, this approach will work only for I/O bound tasks to achieve concurrency. If the tasks are CPU bound, then you should probably use processes instead of threads. Ask ChatGPT to explain the difference between concurrency and parallelism, I/O and CPU bound tasks, if you do not know yet. It will help you to solve this kind of problems easily. – Artyom Vancyan Sep 02 '23 at 10:39
  • The approach is okay but it is better to use an asyncio waiting facility on threaded code when the calling function is asyncio, such as asyncio.run_in_executor, simply because it is io-bound and you don’t want to block the other asyncio tasks. Compare with the source code of asyncio.to_thread . – fuzzyTew Sep 02 '23 at 11:27
  • @fuzzyTew, I see what you are saying, but I am still unfamiliar with your code to suggest a solution. Please create a question and share the link here, and I will help you with pleasure. – Artyom Vancyan Sep 02 '23 at 12:02