63

I have a controller action in aiohttp application.

async def handler_message(request):

    try:
        content = await request.json()
        perform_message(x,y,z)
    except (RuntimeError):
        print("error in perform fb message")
    finally:
        return web.Response(text="Done")

perform_message is async function. Now, when I call action I want that my action return as soon as possible and perform_message put in event loop.

In this way, perform_message isn't executed

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Pasalino
  • 992
  • 1
  • 9
  • 15

5 Answers5

79

One way would be to use create_task function:

import asyncio

async def handler_message(request):
    ...
    loop = asyncio.get_event_loop()
    loop.create_task(perform_message(x,y,z))
    ...

As per the loop documentation, starting Python 3.10, asyncio.get_event_loop() is deprecated. If you're trying to get a loop instance from a coroutine/callback, you should use asyncio.get_running_loop() instead. This method will not work if called from the main thread, in which case a new loop must be instantiated:

loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.create_task(perform_message(x, y, z))
loop.run_forever()

Furthermore, if the call is only made once throughout your program's runtime and no other loop needs to be is instantiated (unlikely), you may use:

asyncio.run(perform_message(x, y, z))

This function creates an event loop and terminates it once the coroutine ends, therefore should only be used in the aforementioned scenario.

Elazar
  • 20,415
  • 4
  • 46
  • 67
freakish
  • 54,167
  • 9
  • 132
  • 169
  • @Jeppe `create_task` method returns a Task object: `task = loop.create_task(...)`. You can store it somewhere and await later. – freakish Nov 08 '18 at 21:33
  • 4
    Yes, but it is still not started until awaited, right? What if you want to create it, start it, do something else and then await it (it might possibly be done and if not, it blocks until complete) – Jeppe Nov 08 '18 at 21:39
  • 6
    @Jeppe no, create_task will start it as soon as possible. I encourage you to test it (you don't have to await the result at all). – freakish Nov 08 '18 at 21:40
  • 1
    @freakish No create_task doesn't start it as soon as possible you need to await for it in order to be executed. – aldokkani Nov 25 '18 at 12:36
  • 8
    @HossamAl-Dokkani Dude, how about you check it before commenting? It does start the task. You don't have to await it. Period. – freakish Nov 25 '18 at 12:38
  • @freakish I did bro, here is the code snippet `import aiohttp` `import asyncio` `async def main(url):` ` async with aiohttp.ClientSession() as session:` ` await session.get(url)` `loop = asyncio.get_event_loop()` `loop.create_task(main('http://localhost:4000/test/'))` – aldokkani Nov 25 '18 at 12:41
  • @HossamAl-Dokkani that doesn't even start the loop. You need loop.run_forever() at the end – freakish Nov 25 '18 at 12:42
  • 2
    @freakish but I don't want to run this forever I need my function to terminate and respond with a HttpResponse and I don't want to `await` for the result. – aldokkani Nov 25 '18 at 12:44
  • @freakish Can I do this? – aldokkani Nov 25 '18 at 12:51
  • 4
    @HossamAl-Dokkani Listen, there's no way to run an async function on a not-running loop. You **have to** start the loop. You either do it by `loop.run_until_complete(coro)` or by `loop.run_forever()`. See this: https://docs.python.org/3/library/asyncio-eventloop.html#running-and-stopping-the-loop In your case it seems you are looking for the first option. – freakish Nov 25 '18 at 13:03
  • 1
    I donvoted the answer since this code does not run tasks immediately. Even if loop is running. Checked this on python 3.7 – JaktensTid Jan 03 '21 at 11:16
  • 3
    @ЯктенсТид the OP wants to run the task in the event loop, presumably to schedule it for execution, not immediately. We have almost none control over the event loop and when it executes tasks. Also define "immediately" because in a multi-task environment you hardly ever guarantee such thing. – freakish Jan 03 '21 at 12:16
  • 1
    @freakish What do you mean by "We have almost none control over the event loop and when it executes tasks."? It is software development. You MUST have control over your code and it's executions. Assuming I want to run certain task immediately (which pulls data by url). What should I do in this case? If we really don't have control over loop then I'll use threads in this case. – JaktensTid Jan 03 '21 at 12:24
  • @freakish by 'immediately' I mean that my function should at least start executing - i.e go to the first non-async line in debugger. Yeah, we can't control await order, but in my case function is not running at all – JaktensTid Jan 03 '21 at 12:32
  • 2
    @ЯктенсТид you have no control over threads as well. It is up to OS to decide when will it run your code. Things get even worse due to GIL. The Python's event loop is single threaded by design. It won't run tasks in parallel. However it is concurrency what we aim for, i.e. waiting until current function exits should be a submillisecond delay, which is typically not an issue at all,. And no, we have no control over it. Just like we have no control over 99% of software we use. If you want full control, you have to reimplement everything by yourself. That would be a nightmare. – freakish Jan 03 '21 at 12:33
  • 1
    @ЯктенсТид if your code doesn't run at all then it is likely you have a bug somewhere, perhaps you used my answer incorrectly. Or maybe something else is going on. And thus I encourage you to ask a question about it here on StackOverflow. – freakish Jan 03 '21 at 12:35
  • @freakish actually I've got work around in my code Looks like that putting await asyncio.sleep(0.0) 'fires' tasks created by create_task function – JaktensTid Jan 03 '21 at 12:35
  • 1
    @ЯктенсТид every time you use `await` or when `async` function exits the control goes back to the event loop which decides what task to run next. Did you do a blocking infinite loop without `await` inside? Yes, in that scenario, nothing else will run, you simply blocked everything permamently. – freakish Jan 03 '21 at 12:37
  • @freakish actually I am doing some database operations in between. Not sure, maybe because of them it is blocked – JaktensTid Jan 03 '21 at 12:38
  • @ЯктенсТид well, as I said: feel free to post your code and ask question about it. I'm sure someone will help you with your problem. – freakish Jan 03 '21 at 12:38
  • @freakish well, there is a really big chunk of code, and I am not sure where the problem is exactly) I've tested answer above - it works, if your logic is straightforward and there are just two functions. But my code does not work and it might have plenty of reasons) So I'd better spend my time – JaktensTid Jan 03 '21 at 12:41
  • If you use `loop.create_task` inside of a `for` loop and add multiple tasks and then add `time.sleep(1)` (or some other unavoidable blocking function) to the task function, `loop.create_task` will execute the function calls passed to it in a blocking manner. Is there away around this so that each function call executes separately in an async manner? In my case I have to deal with blocking functions, and I just need the function calls to execute concurrently. – SuperCodeBrah Jul 15 '21 at 04:51
  • @SuperCodeBrah Python's event loop is inherently single threaded. And so any blocking call will block the entire loop. Perhaps you should use threads instead? – freakish Jul 15 '21 at 08:00
  • @freakish, I figured out that if I use `await loop.run_in_executor(None, time.sleep, 1)`, as opposed to just `time.sleep(1)`, it will do the job, and I believe `run_in_executor` is just a wrapper that uses threads from what I've read previously. The function being used with `loop.create_task` is going to be pretty complicated and I haven't fully built it out, but I think as long as I wrap anything blocking in `loop.run_in_executor`, it will do the trick. – SuperCodeBrah Jul 15 '21 at 14:49
  • @SuperCodeBrah yes, `run_in_executor` runs code in a separate thread pool by default. I assume that in reality you have more complicated call than `time.sleep(1)` right? Cause that can be replaced by `await asyncio.sleep(1)`. – freakish Jul 15 '21 at 14:55
  • @freakish, Yes, that's correct. I'm going to have to recursively call a function to check on the status of something until the status gets resolved, so not only do I need async functionality, I probably can't use something like `run_until_complete`. It needs to be a true "fire and forget" solution. I have more of a JS background, where this is a trivial problem and there's definitely a learning curve with Python. – SuperCodeBrah Jul 15 '21 at 15:13
  • 1
    Instead of first getting the event loop and calling its `create_task` method, since Python 3.7 you can just use [`asyncio.create_task`](https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task). – mkrieger1 May 21 '22 at 19:17
7

In simplest form:

import asyncio
from datetime import datetime

def _log(msg : str):
    print(f"{datetime.utcnow()} {msg}")

async def dummy(name, delay_sec):
    _log(f"{name} entering ...")
    await asyncio.sleep(delay_sec)
    _log(f"{name} done for the day!")

async def main():
    asyncio.create_task(dummy('dummy1', 5)) # last to finish
    asyncio.create_task(dummy('dummy2', 3)) # 2nd
    asyncio.create_task(dummy('dummy3', 1)) # First to finish

    _log(f"Yo I didn't wait for ya!")
    await asyncio.sleep(10)

asyncio.get_event_loop().run_until_complete(main())

Output:

2022-09-18 00:53:13.428285 Yo I didn't wait for ya!
2022-09-18 00:53:13.428285 dummy1 entering ...
2022-09-18 00:53:13.428285 dummy2 entering ...
2022-09-18 00:53:13.428285 dummy3 entering ...
2022-09-18 00:53:14.436801 dummy3 done for the day!
2022-09-18 00:53:16.437226 dummy2 done for the day!
2022-09-18 00:53:18.424755 dummy1 done for the day!
user3761555
  • 851
  • 10
  • 21
0

You can call it by multiprocessing pool.

from multiprocessing import Pool
import time

def f(x):
    return x*x

def postprocess(result):
    print("finished: %s" % result)

if __name__ == '__main__':
    pool = Pool(processes=1)              # Start a worker processes.
    result = pool.apply_async(f, [10], callback=postprocess) # Evaluate "f(10)" asynchronously calling callback when finished.
    print("waiting...")
    time.sleep(1)
david euler
  • 714
  • 8
  • 12
0

Problem Statement: I Was Using FastApi on Azure Function App and i was also using Beanie ODM for MongoDB and Beanie is an asynchronous ODM so it need to initialized at the startup event but as azure function are serveless so this cannot be done easily and i cannot also use asyncio.run() because FastAPI is handling it's own async loop to solve this issue it took me 2 days but finally fixed

Here's How I did It , I am not sure about the drawback's of this method but would love to hear if anyone knows that

connection_string = os.environ["DCS"]
logging.info(f"connection string {connection_string}")
async def init():
    client = AsyncIOMotorClient(connection_string)
    await init_beanie(database=client['Tradex'],document_models=[Admin,User])

loop = asyncio.get_running_loop()
asyncio.set_event_loop(loop)
loop.create_task(init())
-2

Other way would be to use ensure_future function:

import asyncio

async def handler_message(request):
...
loop = asyncio.get_event_loop()
loop.ensure_future(perform_message(x,y,z))
...
Neuron
  • 5,141
  • 5
  • 38
  • 59
Pasalino
  • 992
  • 1
  • 9
  • 15
  • 8
    That would raise an `AttributeError: '_UnixSelectorEventLoop' object has no attribute 'ensure_future'`. `ensure_future` is a function the `asyncio` module, not a method of the event loop. It is also essentially the same as `loop.create_task` if you are only dealing with coroutines. – Gustavo Bezerra Sep 03 '18 at 10:11