Python unittest + asyncio hangs forever

Question

Why does the following test hang forever?

import asyncio
import unittest


class TestCancellation(unittest.IsolatedAsyncioTestCase):

    async def test_works(self):
        task = asyncio.create_task(asyncio.sleep(5))
        await asyncio.sleep(2)
        task.cancel()
        await task


if __name__ == '__main__':
    unittest.main()

if you catch the `CancelledError` exception when awaiting the cancelled task things go smooth. So I guess the test runner doesn't do its job properly — Pynchia, Feb 20 '20 at 22:17
You might get some more insight [here](https://stackoverflow.com/a/37345564/6018688) — fabianegli, Feb 20 '20 at 23:18
I have found the pytest-asyncio to be much more reliable for async testing than the unitest lib. — Matthaus Woolard, Feb 21 '20 at 07:38

Pynchia · Accepted Answer · 2020-02-20T22:39:46.673

Catching the CancelledError exception when awaiting the cancelled task makes things go smooth.

So I guess the test runner gets held up in the act.

import asyncio
import unittest


class TestCancellation(unittest.IsolatedAsyncioTestCase):

    async def test_works(self):
        task = asyncio.create_task(asyncio.sleep(5))
        await asyncio.sleep(2)
        task.cancel()
        try:
            await task
        except asyncio.CancelledError:
            print("Task Cancelled already")

if __name__ == '__main__':
    unittest.main()

produces

unittest-hang $ python3.8 test.py 
Task Cancelled already
.
----------------------------------------------------------------------
Ran 1 test in 2.009s

OK

I ignore whether you must await the cancelled task or not.

If you must, since you seem to be testing its cancellation fully, then catch the exception.

If not, then just avoid it, since creating a task starts it immediately and there is no need to await again

import asyncio
import unittest


class TestCancellation(unittest.IsolatedAsyncioTestCase):

    async def test_works(self):
        task = asyncio.create_task(asyncio.sleep(5))
        await asyncio.sleep(2)
        task.cancel()
        # await task

if __name__ == '__main__':
    unittest.main()

produces

unittest-hang $ python3.8 test.py 
.
----------------------------------------------------------------------
Ran 1 test in 2.009s

OK

That doesn't explain why the OP's example hangs forever. Awaiting the cancelled task should leak a `CancelledError` and this should be reported by unittest, but it isn't. — a_guest, Feb 20 '20 at 21:54
@a_guest Point taken indeed. I have done a few tests and reworked the answer. It is still unclear if the OP needs to await the cancelled task. As a test it doesn't make much sense to me, unless he is testing the test runner itself — Pynchia, Feb 20 '20 at 22:34
@Pynchia just found out that plain `raise asyncio.CancelledError()` hangs the test, too. Probably a bug in the `unittest`. — Andrey Moiseev, Feb 20 '20 at 23:25

score 0 · Answer 2 · answered Feb 20 '20 at 22:32

As by the comment from @Pynchia an example solution:

import asyncio
import unittest


class TestCancellation(unittest.IsolatedAsyncioTestCase):

    async def test_works(self):
        task = asyncio.create_task(asyncio.sleep(5))
        await asyncio.sleep(2)
        task.cancel()
        try:
            await task
        except asyncio.CancelledError:
            print("main(): cancel_me is cancelled now")


if __name__ == '__main__':
    unittest.main()

The solution is taken from the asyncio.Task.cancel documentation. The documentation also explains this behavior:

Request the Task to be cancelled.

This arranges for a CancelledError exception to be thrown into the wrapped coroutine on the next cycle of the event loop.

The coroutine then has a chance to clean up or even deny the request by suppressing the exception with a try … … except CancelledError … finally block. Therefore, unlike Future.cancel(), Task.cancel() does not guarantee that the Task will be cancelled, although suppressing cancellation completely is not common and is actively discouraged.

Python unittest + asyncio hangs forever

2 Answers2