Using alternative event loop without setting global policy

Question

I'm using uvloop with websockets as

import uvloop
coro = websockets.serve(handler, host, port)  # creates new server
loop = uvloop.new_event_loop()
loop.create_task(coro)
loop.run_forever()

It works fine, I'm just wondering whether I could run to some unexpected problems without setting the global asyncio policy to uvloop. As far as I understand, not setting the global policy should work as long as nothing down there doesn't use the global asyncio methods, but works with the passed-down event loop directly. Is that correct?

I don't have any particular reasons to do it like this, I just don't like the global aspect of asyncio, because it makes it feel more like a language platform than an async library. And I like to think of it as the the latter. — jhrmnn, May 16 '17 at 11:23

score 6 · Accepted Answer · edited May 16 '17 at 15:27

6

There are three main global objects in asyncio:

the policy (common to all threads)
the default loop (specific to the current thread)
the running loop (specific to the current thread)

All the attempts to get the current context in asyncio go through a single function, asyncio.get_event_loop.

One thing to remember is that since Python 3.6 (and Python 3.5.3+), get_event_loop has a specific behavior:

If it's called while a loop is running (e.g within a coroutine), the running loop is returned.
Otherwise, the default loop is returned by the policy.

Example 1:

import uvloop
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
loop = asyncio.get_event_loop()
loop.run_forever()

Here the policy is the uvloop policy. The loop returned by get_event_loop is a uvloop, and it is set as the default loop for this thread. When this loop is running, it is registered as the running loop.

In this example, calling get_event_loop() anywhere in this thread returns the right loop.

Example 2:

import uvloop
loop = uvloop.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_forever()

Here the policy is still the default policy. The loop returned by new_event_loop is a uvloop, and it is set as the default loop for this thread explicitly using asyncio.set_event_loop. When this loop is running, it is registered as the running loop.

In this example, calling get_event_loop() anywhere in this thread returns the right loop.

Example 3:

import uvloop
loop = uvloop.new_event_loop()
loop.run_forever()

Here the policy is still the default policy. The loop returned by new_event_loop is a uvloop, but it is not set as the default loop for this thread. When this loop is running, it is registered as the running loop.

In this example, calling get_event_loop() within a coroutine returns the right loop (the running uvloop). But calling get_event_loop() outside a coroutine will result in a new standard asyncio loop, set as the default loop for this thread.

So the first two approaches are fine, but the third one is discouraged.

edited May 16 '17 at 15:27

Quentin Pradet

4,691
2
29
41

answered May 16 '17 at 12:05

Vincent

12,919
1
42
64

Thanks for the mention of 3.6. So #3 explains why my code works even when I'm not passing the uvloop loop down to the coroutine. Is this new behaviour documented somewhere? I didn't see it when browsing the documentation. Also, if it is discouraged, why was it introduced? I understand that my (discouraged) code wouldn't work in 3.5. – jhrmnn May 16 '17 at 12:12
Ok, I see that the discussion about get_event_loop is here https://bugs.python.org/issue28613 – jhrmnn May 16 '17 at 12:14
@azag0 Your code wouldn't work in 3.5, unless you pass the event loop explicitly everywhere (`loop=loop`) as @GerasimovMikhail pointed out. – Vincent May 16 '17 at 12:17
@azag0 Also see [asyncio PR #452](https://github.com/python/asyncio/pull/452). And no, I don't think it's part of the documentation yet. – Vincent May 16 '17 at 12:21
1

So from reading https://groups.google.com/forum/#!msg/python-tulip/yF9C-rFpiKk/tk5oA3GLHAAJ, I wouldn't say that #3 is discouraged. Contrary, the new behaviour was introduced to enable #3. It enables you to forget about the loop once you run it. In this way, it enables you to write code *as if* everything was async/await all the way down, as is for example in curio. The loop is delegated to just run your entry point coroutines and then disappears. Lower-level asyncio code can still pick it up if it needs to. – jhrmnn May 16 '17 at 12:29
@azag0 People have different opinions about this. Mine is that relying on `get_event_loop` within coroutines is much nicer that passing the loop explicitly. However, I still think it's better to set the proper policy and use the loop provided by the policy in order to avoid unexpected problems (try to use `asyncio.create_subprocess_exec` without setting the loop and you'll have a surprise). – Vincent May 16 '17 at 13:08
Yes, I guess it depends on whether one's more comfortable with a global state or a runtime-defined behaviour. – jhrmnn May 16 '17 at 13:20

score 3 · Answer 2 · answered May 16 '17 at 12:05

Custom event loop should be passed as param

If you want to use custom event loop without using asyncio.set_event_loop(loop), you'll have to pass loop as param to every relevant asyncio coroutines or objects, for example:

await asyncio.sleep(1, loop=loop)

or

fut = asyncio.Future(loop=loop)

You may notice that probably any coroutine/object from asyncio module accepts this param.

Same thing is also applied to websockets library as you may see from it's source code. So you'll need to write:

loop = uvloop.new_event_loop()
coro = websockets.serve(handler, host, port, loop=loop)  # pass loop as param

There's no guarantee that your program would work fine if you won't pass your event loop as param like that.

Possible, but uncomfortable

While theoretically you can use some event loop without changing policy I find it's extremely uncomfortable.

You'll have to write loop=loop almost everywhere, it's annoying
There's no guarantee that some third-party would allow you to pass loop as param and won't just use asyncio.get_event_loop()

Base on that I advice you to reconsider your decision and use global event loop.

I understand that it may be felt "unright" to use global event loop, but "right" way is to pass loop as param everywhere is worse on practice (in my opinion).