Use asyncio and Tkinter (or another GUI lib) together without freezing the GUI

Question

I want to use asyncio in combination with a tkinter GUI. I am new to asyncio and my understanding of it is not very detailed. The example here starts 10 task when clicking on the first button. The task are just simulating work with a sleep() for some seconds.

The example code is running fine with Python 3.6.4rc1. But the problem is that the GUI is freezed. When I press the first button and start the 10 asyncio-tasks I am not able to press the second button in the GUI until all tasks are done. The GUI should never freeze - that is my goal.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

from tkinter import *
from tkinter import messagebox
import asyncio
import random

def do_freezed():
    """ Button-Event-Handler to see if a button on GUI works. """
    messagebox.showinfo(message='Tkinter is reacting.')

def do_tasks():
    """ Button-Event-Handler starting the asyncio part. """
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(do_urls())
    finally:
        loop.close()

async def one_url(url):
    """ One task. """
    sec = random.randint(1, 15)
    await asyncio.sleep(sec)
    return 'url: {}\tsec: {}'.format(url, sec)

async def do_urls():
    """ Creating and starting 10 tasks. """
    tasks = [
        one_url(url)
        for url in range(10)
    ]
    completed, pending = await asyncio.wait(tasks)
    results = [task.result() for task in completed]
    print('\n'.join(results))


if __name__ == '__main__':
    root = Tk()

    buttonT = Button(master=root, text='Asyncio Tasks', command=do_tasks)
    buttonT.pack()
    buttonX = Button(master=root, text='Freezed???', command=do_freezed)
    buttonX.pack()

    root.mainloop()

A _side problem

...is that I am not able to run the task a second time because of this error.

Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python3.6/tkinter/__init__.py", line 1699, in __call__
    return self.func(*args)
  File "./tk_simple.py", line 17, in do_tasks
    loop.run_until_complete(do_urls())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 443, in run_until_complete
    self._check_closed()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 357, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Multithreading

Whould multithreading be a possible solution? Only two threads - each loop has it's own thread?

EDIT: After reviewing this question and the answers it is related to nearly all GUI libs (e.g. PygObject/Gtk, wxWidgets, Qt, ...).

Answer 1

Trying to run both event loops at the same time is a dubious proposition. However, since root.mainloop simply calls root.update repeatedly, one can simulate mainloop by calling update repeatedly as an asyncio task. Here is a test program that does so. I presume adding asyncio tasks to the tkinter tasks would work. I checked that it still runs with 3.7.0a2.

"""Proof of concept: integrate tkinter, asyncio and async iterator.

Terry Jan Reedy, 2016 July 25
"""

import asyncio
from random import randrange as rr
import tkinter as tk


class App(tk.Tk):
    
    def __init__(self, loop, interval=1/120):
        super().__init__()
        self.loop = loop
        self.protocol("WM_DELETE_WINDOW", self.close)
        self.tasks = []
        self.tasks.append(loop.create_task(self.rotator(1/60, 2)))
        self.tasks.append(loop.create_task(self.updater(interval)))

    async def rotator(self, interval, d_per_tick):
        canvas = tk.Canvas(self, height=600, width=600)
        canvas.pack()
        deg = 0
        color = 'black'
        arc = canvas.create_arc(100, 100, 500, 500, style=tk.CHORD,
                                start=0, extent=deg, fill=color)
        while await asyncio.sleep(interval, True):
            deg, color = deg_color(deg, d_per_tick, color)
            canvas.itemconfigure(arc, extent=deg, fill=color)

    async def updater(self, interval):
        while True:
            self.update()
            await asyncio.sleep(interval)

    def close(self):
        for task in self.tasks:
            task.cancel()
        self.loop.stop()
        self.destroy()


def deg_color(deg, d_per_tick, color):
    deg += d_per_tick
    if 360 <= deg:
        deg %= 360
        color = '#%02x%02x%02x' % (rr(0, 256), rr(0, 256), rr(0, 256))
    return deg, color

loop = asyncio.get_event_loop()
app = App(loop)
loop.run_forever()
loop.close()

Both the tk update overhead and time resolution increase as the interval is decreased. For gui updates, as opposed to animations, 20 per second may be enough.

I recently succeeded in running async def coroutines containing tkinter calls and awaits with mainloop. The prototype uses asyncio Tasks and Futures, but I don't know if adding normal asyncio tasks would work. If one wants to run asyncio and tkinter tasks together, I think running tk update with an asyncio loop is a better idea.

EDIT: A least as used above, exception without async def coroutines kill the coroutine but are somewhere caught and discarded. Silent error are pretty obnoxious.

EDIT2: Additional code and comments at https://bugs.python.org/issue27546

Answer 2

In a slight modification to your code, I created the asyncio event_loop in the main thread and passed it as an argument to the asyncio thread. Now Tkinter won't freeze while the urls are fetched.

from tkinter import *
from tkinter import messagebox
import asyncio
import threading
import random

def _asyncio_thread(async_loop):
    async_loop.run_until_complete(do_urls())


def do_tasks(async_loop):
    """ Button-Event-Handler starting the asyncio part. """
    threading.Thread(target=_asyncio_thread, args=(async_loop,)).start()

    
async def one_url(url):
    """ One task. """
    sec = random.randint(1, 8)
    await asyncio.sleep(sec)
    return 'url: {}\tsec: {}'.format(url, sec)

async def do_urls():
    """ Creating and starting 10 tasks. """
    tasks = [one_url(url) for url in range(10)]
    completed, pending = await asyncio.wait(tasks)
    results = [task.result() for task in completed]
    print('\n'.join(results))


def do_freezed():
    messagebox.showinfo(message='Tkinter is reacting.')

def main(async_loop):
    root = Tk()
    Button(master=root, text='Asyncio Tasks', command= lambda:do_tasks(async_loop)).pack()
    Button(master=root, text='Freezed???', command=do_freezed).pack()
    root.mainloop()

if __name__ == '__main__':
    async_loop = asyncio.get_event_loop()
    main(async_loop)

Answer 3

I'm a bit late to the party but if you are not targeting Windows you can use aiotkinter to achieve what you want. I modified your code to show you how to use this package:

from tkinter import *
from tkinter import messagebox
import asyncio
import random

import aiotkinter

def do_freezed():
    """ Button-Event-Handler to see if a button on GUI works. """
    messagebox.showinfo(message='Tkinter is reacting.')

def do_tasks():
    task = asyncio.ensure_future(do_urls())
    task.add_done_callback(tasks_done)

def tasks_done(task):
    messagebox.showinfo(message='Tasks done.')

async def one_url(url):
    """ One task. """
    sec = random.randint(1, 15)
    await asyncio.sleep(sec)
    return 'url: {}\tsec: {}'.format(url, sec)

async def do_urls():
    """ Creating and starting 10 tasks. """
    tasks = [
        one_url(url)
        for url in range(10)
    ]
    completed, pending = await asyncio.wait(tasks)
    results = [task.result() for task in completed]
    print('\n'.join(results))

if __name__ == '__main__':
    asyncio.set_event_loop_policy(aiotkinter.TkinterEventLoopPolicy())
    loop = asyncio.get_event_loop()
    root = Tk()
    buttonT = Button(master=root, text='Asyncio Tasks', command=do_tasks)
    buttonT.pack()
    buttonX = Button(master=root, text='Freezed???', command=do_freezed)
    buttonX.pack()
    loop.run_forever()

Answer 4

You can keep the GUI alive after pressing the Button by adding a call to root.update_idletasks() in the right spot:

from tkinter import *
from tkinter import messagebox
import asyncio
import random

def do_freezed():
    """ Button-Event-Handler to see if a button on GUI works. """
    messagebox.showinfo(message='Tkinter is reacting.')

def do_tasks():
    """ Button-Event-Handler starting the asyncio part. """
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(do_urls())
    finally:
        loop.close()

async def one_url(url):
    """ One task. """
    sec = random.randint(1, 15)
    root.update_idletasks()  # ADDED: Allow tkinter to update gui.
    await asyncio.sleep(sec)
    return 'url: {}\tsec: {}'.format(url, sec)

async def do_urls():
    """ Creating and starting 10 tasks. """
    tasks = [one_url(url) for url in range(10)]
    completed, pending = await asyncio.wait(tasks)
    results = [task.result() for task in completed]
    print('\n'.join(results))


if __name__ == '__main__':
    root = Tk()

    buttonT = Button(master=root, text='Asyncio Tasks', command=do_tasks)
    buttonT.pack()
    buttonX = Button(master=root, text='Freezed???', command=do_freezed)
    buttonX.pack()

    root.mainloop()

Answer 5

A solution using async_tkinter_loop module (which is written by me).

Internally, the approach is similar to the code from the answer of Terry Jan Reedy, but the usage is much simpler: you just need to wrap your asynchronous handlers into async_handler function calls, and use them as a command or an event handlers, and use async_mainloop(root) in place of root.mainloop().

from tkinter import *
from tkinter import messagebox
import asyncio
import random
from async_tkinter_loop import async_handler, async_mainloop


def do_freezed():
    """ Button-Event-Handler to see if a button on GUI works. """
    messagebox.showinfo(message='Tkinter is reacting.')


async def one_url(url):
    """ One task. """
    sec = random.randint(1, 15)
    await asyncio.sleep(sec)
    return 'url: {}\tsec: {}'.format(url, sec)


async def do_urls():
    """ Creating and starting 10 tasks. """
    tasks = [
        asyncio.create_task(one_url(url))  # added create_task to remove warning "The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11."
        for url in range(10)
    ]
    print("Started")
    completed, pending = await asyncio.wait(tasks)
    results = [task.result() for task in completed]
    print('\n'.join(results))
    print("Finished")


if __name__ == '__main__':
    root = Tk()

    # Wrap async function into async_handler to use it as a button handler or an event handler
    buttonT = Button(master=root, text='Asyncio Tasks', command=async_handler(do_urls))
    buttonT.pack()
    buttonX = Button(master=root, text='Freezed???', command=do_freezed)
    buttonX.pack()

    # Use async_mainloop(root) instead of root.mainloop()
    async_mainloop(root)

Answer 6

I had similar task solved with multiprocessing.

Major parts:

1. Main process is Tk's process with mainloop.
2. daemon=True process with aiohttp service that executes commands.
3. Intercom using duplex Pipe so each process can use it's end.

Additionaly, I'm making Tk's virtual events to simplify massage tracking on app's side. You will need to apply patch manually. You can check python's bug tracker for details.

I'm checking Pipe each 0.25 seconds on both sides.

$ python --version
Python 3.7.3

main.py

import asyncio
import multiprocessing as mp

from ws import main
from app import App


class WebSocketProcess(mp.Process):

    def __init__(self, pipe, *args, **kw):
        super().__init__(*args, **kw)
        self.pipe = pipe

    def run(self):
        loop = asyncio.get_event_loop()
        loop.create_task(main(self.pipe))
        loop.run_forever()


if __name__ == '__main__':
    pipe = mp.Pipe()
    WebSocketProcess(pipe, daemon=True).start()
    App(pipe).mainloop()

app.py

import tkinter as tk


class App(tk.Tk):

    def __init__(self, pipe, *args, **kw):
        super().__init__(*args, **kw)
        self.app_pipe, _ = pipe
        self.ws_check_interval = 250;
        self.after(self.ws_check_interval, self.ws_check)

    def join_channel(self, channel_str):
        self.app_pipe.send({
            'command': 'join',
            'data': {
                'channel': channel_str
            }
        })

    def ws_check(self):
        while self.app_pipe.poll():
            msg = self.app_pipe.recv()
            self.event_generate('<<ws-event>>', data=json.dumps(msg), when='tail')
        self.after(self.ws_check_interval, self.ws_check)

ws.py

import asyncio

import aiohttp


async def read_pipe(session, ws, ws_pipe):
    while True:
        while ws_pipe.poll():
            msg = ws_pipe.recv()

            # web socket send
            if msg['command'] == 'join':
                await ws.send_json(msg['data'])

            # html request
            elif msg['command'] == 'ticker':
                async with session.get('https://example.com/api/ticker/') as response:
                    ws_pipe.send({'event': 'ticker', 'data': await response.json()})

        await asyncio.sleep(.25)


async def main(pipe, loop):
    _, ws_pipe = pipe
    async with aiohttp.ClientSession() as session:
        async with session.ws_connect('wss://example.com/') as ws:
            task = loop.create_task(read_pipe(session, ws, ws_pipe))
            async for msg in ws:
                if msg.type == aiohttp.WSMsgType.TEXT:
                    if msg.data == 'close cmd':
                        await ws.close()
                        break
                    ws_pipe.send(msg.json())
                elif msg.type == aiohttp.WSMsgType.ERROR:
                    break

Answer 7

Using Python3.9, it could be done by making several async functions with one of them responsible to the Tk update(). While in the main loop, ensure_future() can be used to invoke all these async functions before starting the asyncio loop.

#!/usr/bin/env python3.9

import aioredis
import asyncio
import tkinter as tk 
import tkinter.scrolledtext as st 
import json

async def redis_main(logs):
    redisS = await aioredis.create_connection(('localhost', 6379))  
    subCh = aioredis.Channel('pylog', is_pattern=False)
    await redisS.execute_pubsub('subscribe', subCh)
    while await subCh.wait_message():
            msg = await subCh.get()
            jmsg = json.loads(msg.decode('utf-8'))
            logs.insert(tk.INSERT, jmsg['msg'] + '\n')

async def tk_main(root):
    while True:
        root.update()
        await asyncio.sleep(0.05)

def on_closing():
    asyncio.get_running_loop().stop()

if __name__ == '__main__':
    root = tk.Tk()
    root.protocol("WM_DELETE_WINDOW", on_closing)
    logs = st.ScrolledText(root, width=30, height=8)
    logs.grid()
    
    tkmain = asyncio.ensure_future(tk_main(root))
    rdmain = asyncio.ensure_future(redis_main(logs))
    
    loop = asyncio.get_event_loop()
    try:
        loop.run_forever()
    except KeyboardInterrupt:
        pass

    tkmain.cancel()
    rdmain.cancel()

Answer 8

I've had great luck running an I/O loop on another thread, started at the beginning of the app creation, and tossing tasks onto it using asyncio.run_coroutine_threadsafe(..).

I'm kind of surprised that I can make changes to the tkinter widgets on the other asyncio loop/thread, and maybe it's a fluke that it works for me -- but it does work.

Notice that while the asyncio tasks are happening, the other button is still alive and responding. I always like to the disable/enable thing on the other button so you don't fire off multiple tasks accidentally, but that's just a UI thing.

import threading
from functools import partial
from tkinter import *
from tkinter import messagebox
import asyncio
import random


# Please wrap all this code in a nice App class, of course

def _run_aio_loop(loop):
    asyncio.set_event_loop(loop)
    loop.run_forever()
aioloop = asyncio.new_event_loop()
t = threading.Thread(target=partial(_run_aio_loop, aioloop))
t.daemon = True  # Optional depending on how you plan to shutdown the app
t.start()

buttonT = None

def do_freezed():
    """ Button-Event-Handler to see if a button on GUI works. """
    messagebox.showinfo(message='Tkinter is reacting.')

def do_tasks():
    """ Button-Event-Handler starting the asyncio part. """
    buttonT.configure(state=DISABLED)
    asyncio.run_coroutine_threadsafe(do_urls(), aioloop)

async def one_url(url):
    """ One task. """
    sec = random.randint(1, 3)
    # root.update_idletasks()  # We can delete this now
    await asyncio.sleep(sec)
    return 'url: {}\tsec: {}'.format(url, sec)

async def do_urls():
    """ Creating and starting 10 tasks. """
    tasks = [one_url(url) for url in range(3)]
    completed, pending = await asyncio.wait(tasks)
    results = [task.result() for task in completed]
    print('\n'.join(results))
    buttonT.configure(state=NORMAL)  # Tk doesn't seem to care that this is called on another thread


if __name__ == '__main__':
    root = Tk()

    buttonT = Button(master=root, text='Asyncio Tasks', command=do_tasks)
    buttonT.pack()
    buttonX = Button(master=root, text='Freezed???', command=do_freezed)
    buttonX.pack()

    root.mainloop()