Coroutine as background job in Jupyter notebook

Question

I have a coroutine which I'd like to run as a "background job" in a Jupyter notebook. I've seen ways to accomplish this using threading, but I'm wondering whether it's also possible to hook into the notebook's event loop.

For example, say I have the following class:

import asyncio
class Counter:
    def __init__(self):
        self.counter = 0

    async def run(self):
        while True:
            self.counter += 1
            await asyncio.sleep(1.0)

t = Counter()

and I'd like to execute the run method (which loops indefinitely), while still being able to check the t.counter variable at any point. Any ideas?

score 5 · Answer 1 · answered Apr 27 '16 at 19:17

The following basically does what I want I think, but it does use a separate thread. However, I can still use the async primitives.

def run_loop():
    loop = asyncio.new_event_loop()
    run_loop.loop = loop
    asyncio.set_event_loop(loop)
    task = loop.create_task(t.run())
    loop.run_until_complete(task)

from IPython.lib import backgroundjobs as bg
jobs = bg.BackgroundJobManager()
jobs.new('run_loop()')
loop = run_loop.loop # to access the loop outside

score 2 · Answer 2 · answered Jan 01 '20 at 23:30

2

I think your code works perfectly, you just need to create a task to wrap the coroutine, i.e. :

import asyncio
class Counter:
    def __init__(self):
        self.counter = 0

    async def run(self):
        while True:
            self.counter += 1
            await asyncio.sleep(1.0)

t = Counter()
asyncio.create_task(t.run())

wait 10 seconds, and check

t.counter

should get

> 10

answered Jan 01 '20 at 23:30

Maverick Meerkat

5,737
3
47
66

This approach is handy, but you have to be cautious of the fact that other cells being run in the notebook will be using the same event loop (unlike in Mark's answer where there's a different loop for the separate thread), so will starve the `Counter.run` method during their blocking operations. If it's important for the counter to increment consistently each second, this won't work unless no other notebook cells are going to be used or they don't perform any blocking operations. – Steven D. Aug 26 '21 at 23:47

score 1 · Answer 3 · answered Feb 08 '18 at 00:08

1

There's a simplified version of what Mark proposed:

from IPython.lib import backgroundjobs as bg
jobs = bg.BackgroundJobManager()
jobs.new(asyncio.get_event_loop().run_forever)

If you need, you can access the loop with asyncio.get_event_loop()

answered Feb 08 '18 at 00:08

Vinicius Fortuna

393
4
11

Coroutine as background job in Jupyter notebook

3 Answers3

Linked