100

What does asyncio.create_task() do? I have looked at the docs and can't seem to understand it. A bit of code that confuses me is this:

import asyncio

async def counter_loop(x, n):
    for i in range(1, n + 1):
        print(f"Counter {x}: {i}")
        await asyncio.sleep(0.5)
    return f"Finished {x} in {n}"

async def main():
    slow_task = asyncio.create_task(counter_loop("Slow", 4))
    fast_coro = counter_loop("Fast", 2)

    print("Awaiting Fast")
    fast_val = await fast_coro
    print("Finished Fast")

    print("Awaiting Slow")
    slow_val = await slow_task
    print("Finished Slow")

    print(f"{fast_val}, {slow_val}")

asyncio.run(main())

This gives the following output:

001 | Awaiting Fast
002 | Counter Fast: 1
003 | Counter Slow: 1
004 | Counter Fast: 2
005 | Counter Slow: 2
006 | Finished Fast
007 | Awaiting Slow
008 | Counter Slow: 3
009 | Counter Slow: 4
010 | Finished Slow
011 | Finished Fast in 2, Finished Slow in 4

I don't understand quite how this is working.

  1. Shouldn't the slow_task not be able to run until the completion of the fast_coro because it was never used in an asyncio.gather method?
  2. Why do we have to await slow_task?
  3. Why is Awaiting Slow printed after the coroutine appears to have started?
  4. What really is a task? I know that what gather is doing is scheduling a task. And create_task supposedly creates a task.

An in-depth answer would be greatly appreciated. Thanks!

It also might be worth mentioning that I know very little about Futures.

Liam
  • 27,717
  • 28
  • 128
  • 190
BeastCoder
  • 2,391
  • 3
  • 15
  • 26

1 Answers1

118

What does asyncio.create_task() do?

It submits the coroutine to run "in the background", i.e. concurrently with the current task and all other tasks, switching between them at await points. It returns an awaitable handle called a "task" which you can also use to cancel the execution of the coroutine.

It's one of the central primitives of asyncio, the asyncio equivalent of starting a thread. (In the same analogy, awaiting the task with await is the equivalent of joining a thread.)

Shouldn't the slow_task not be able to run until the completion of the fast_coro

No, because you explicitly used create_task to start slow_task in the background. Had you written something like:

    slow_coro = counter_loop("Slow", 4)
    fast_coro = counter_loop("Fast", 2)
    fast_val = await fast_coro

...indeed slow_coro would not run because no one would have yet submitted it to the event loop. But create_task does exactly that: submit it to the event loop for execution concurrently with other tasks, the point of switching being any await.

because it was never used in an asyncio.gather method?

asyncio.gather is not the only way to achieve concurrency in asyncio. It's just a utility function that makes it easier to wait for a number of coroutines to all complete, and submit them to the event loop at the same time. create_task does just the submitting, it should have probably been called start_coroutine or something like that.

Why do we have to await slow_task?

We don't have to, it just serves to wait for both coroutines to finish cleanly. The code could have also awaited asyncio.sleep() or something like that. Returning from main() (and the event loop) immediately with some tasks still pending would have worked as well, but it would have printed a warning message indicating a possible bug. Awaiting (or canceling) the task before stopping the event loop is just cleaner.

What really is a task?

It's an asyncio construct that tracks execution of a coroutine in a concrete event loop. When you call create_task, you submit a coroutine for execution and receive back a handle. You can await this handle when you actually need the result, or you can never await it, if you don't care about the result. This handle is the task, and it inherits from Future, which makes it awaitable and also provides the lower-level callback-based interface, such as add_done_callback.

user4815162342
  • 141,790
  • 18
  • 296
  • 355
  • Thanks! I have just two follow up questions. – BeastCoder Jun 23 '20 at 13:18
  • 1. What would happen if you did `asyncio.gather(list_of_tasks)`? 2. Does `create_task` return the results of whatever is running? Or would the only way to get the return statements be to use await and hope that it hadn't finished in the background? 3. Gather also converts all coroutines into tasks, right? But, it waits for them to finish. – BeastCoder Jun 23 '20 at 13:26
  • 5
    @BeastCoder `asyncio.gather(*list_of_tasks)` would wait for the tasks to finish and return the result of their respective coroutines. `create_task` returns a task, which you can `await` to get the result of whatever is (was) running. If it had already finished, don't worry, it'll keep the result anyway. Yes, `gather` calls `create_task` if needed for your convenience - this is because it supports awaitables other than coroutines. – user4815162342 Jun 23 '20 at 13:29
  • Ah, thanks for clearing that up. And, `gather` behind the scenes is basically creating these tasks and then awaiting them? – BeastCoder Jun 23 '20 at 13:52
  • 3
    @BeastCoder Pretty much, yes. Something like `tasks = [asyncio.create_task(c) for c in list_of_coros]; results = [await t for t in tasks]` would be a good first approximation of what `gather` does. The implementation is much more complex, however, because `gather` is very careful about how it propagates exceptions, how it responds to cancelation, as well as various other concerns. – user4815162342 Jun 23 '20 at 14:09
  • One question: when you said create tasks executes coroutines in parallel or concurrent execution? Can you elaborate on it. Does it execute in separate thread ? – Arnab Mukherjee Dec 22 '20 at 15:56
  • 1
    @ArnabMukherjee It executes in the same thread, so I guess it would be concurrent rather than truly parallel. – user4815162342 Dec 22 '20 at 18:20
  • @user4815162342 so what ever tasks we wrap with create_task it will executed in background. We can await and check the results if we want. What is the role of event loop in the tasks ? – Arnab Mukherjee Dec 23 '20 at 02:24
  • 3
    @ArnabMukherjee There is no real "background", they run when you await something, and they are ready to run. These comments should be posted as a separate question; see also [this one](https://stackoverflow.com/questions/49005651/how-does-asyncio-actually-work). – user4815162342 Dec 23 '20 at 07:41
  • @user4815162342 i will draft a question as you said, thanks for sharing knowledge. – Arnab Mukherjee Dec 23 '20 at 07:51
  • Running this code, "counter fast: 1" always appears to be printed first. Is there a deterministic reason for this? If there's a task already submitted by create_task and then an await coroutine call, is the await coroutine guaranteed to run first and if so why? ..or this is just an implementation-dependent determinism – anon2328 Dec 30 '20 at 13:29
  • 5
    @aleks224 If you are referring to the code exactly as in the question, there is. In general asyncio doesn't guarantee the order in which runnable tasks are executed, but here the code awaits a coroutine, not a task. The code in a directly awaited coroutine is immediately executed (without yielding to the event loop) up to the first suspension. Since there is no suspension between `await fast_coro` and the first print, it will always before anything that comes from another task. – user4815162342 Dec 30 '20 at 13:33
  • 2
    @aleks224 Immediately executing awaited coroutine up to a suspension is guaranteed, although you might find it hard to find chapter and verse in the docs. Basically it follows from `await` being both [specified and implemented](https://www.python.org/dev/peps/pep-0492/#await-expression) in terms of `yield from`, and this is how `yield from` must behave to satisfy the _refactoring principle_ of [PEP 380](https://www.python.org/dev/peps/pep-0380/). – user4815162342 Dec 30 '20 at 13:35
  • @user4815162342 should we always call `.create_task()` before calling `.gether()` this task? – Chiefir Mar 03 '21 at 12:56
  • @Chiefir No, in fact it should never be necessary because `asyncio.gather()` creates the task for you. – user4815162342 Mar 03 '21 at 13:01