How should I get my Tkinter IRC client to continuously read data from the IRC server?

Question

I'm writing a little IRC client in python as an exercise. I have a Tkinter.Tk subclass called Main managing the whole application, which creates a socket in its __init__ method. I've played around with sockets in the interactive mode, so I know how to talk to the IRC server with something like this:

>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.connect(("irc.foonetic.net", 6667))
>>> s.recv(1000)
":anchor.foonetic.net NOTICE AUTH :*** Looking up your hostname...\r\n:anchor.foonetic.net NOTICE AUTH :*** Couldn't resolve your hostname; using your IP address instead\r\n"
>>> s.send("PASS mypassword\r\n")

That is, I carry on the whole conversation using .send and .recv. Thus to get user input in my Tkinter app, I imagine I'll have an event handler mapped to the Enter key which will call .send. But where do I put the calls to .recv? The only thing I know how to do would be to use a timer to call .recv every few seconds, but that's obviously not a good solution for several reasons. How do I deal with the fact that .recv blocks for several seconds (determined by whatever timeout you set) if there's no data to receive? I realize I could just google "multithreading", but I'd like some guidance on what the best approach is for this specific situation.

Answer 1

In my project, I setup a new thread for long term I/O like socket read/write. To write a practical GUI program, you have to face multithread soon or later. That's because GUI framework has an event queue, and an event loop. The event loop is typically a while loop, in which it get events from event queue and dispatch this events to registered functions. Like the following:

while event is not QUIT:
    event = event_queue.get(block=True)
    dispatch(event)

In dispatch, all callback functions registered on that event is called directly.

Such code works in the GUI thread, and if you do long term I/O or blocking action in a GUI callback, the thread is blocked in that callback. In terms of event loop, the program is blocked in the dispatch function which called the blocked callback function. Any new event in the event queue will not be processed. As a result, the program looks like dead because the updating event of GUI is blocked.

When you have setup a worker thread to handle time consuming things, don't try to operate GUI widgets directly from that worker thread. Most GUI frameworks are not thread safe, they keep operation sequence by the event queue. And operating a widget in non-GUI threads will break this sequence.

We can add event to event queue from non-GUI thread, and let GUI thread handle that event, to keep the sequence. This is the normal way for some common language, but not for python. In python, function and method are first class object, so we can put then in the queue. Unfortunately, the event queue for tkinter does not support this feature.

In Programming Python by Mark Lutz there is great cover of tkinter programming. In this book, the author introduced a great method to do multithread in tkinter. Here is my demo:

# python3 source code
from tkinter import *
from tkinter.ttk import *
import threading
import time
import queue


root = Tk()
msg = StringVar()
Label(root, textvariable=msg).pack()

# This is our own event queue, each element should be in this form:
# (function_to_be_called_from_gui_thread, function_arguments)
# In python, functions are objects and can be put in a queue.
my_event_queue = queue.Queue()


def worker():
    """
    This is a time consuming worker, it takes 1 second for each task.
    If you put such a worker in the GUI thread, the GUI will be blocked.
    """
    task_counter = 0
    while True:
        time.sleep(1)  # simulate a time consuming task

        # show how many tasks finished in the Label. We put this action in my_event_queue instead of handle
        # it from this worker thread which is not safe. This action will be handled by my_event_handler which is
        # called from GUI thread.
        my_event_queue.put((msg.set, '{} tasks finished.'.format(task_counter)))
        task_counter += 1


def my_event_handler():
    """
    Query my_event_queue, and handle one event per time.
    """
    try:
        func, *args = my_event_queue.get(block=False)
    except queue.Empty:
        pass
    else:
        func(*args)

    # At last schedule handling for next time.
    # Every 100 ms, my_event_handler will be called
    root.after(100, my_event_handler)


threading.Thread(target=worker, daemon=True).start()  # start worker in new thread

my_event_handler()  # start handler, after root.mainloop(), this method will be called every 100ms. Or you can use root.after(100, my_event_handler)

root.mainloop()

Here is the running picture. You can see I adjust the window size when it is running.(Well I have not enough reputation to post images, so you have to try it yourself)

At last I would suggest you to take a look at Programming Python for tkinter programming. All Python code are in Python3.

Answer 2

I'm pretty new to Python in general and very new to Tk/ttk. But here's an example of what I've been playing with for event triggering/signaling and worker thread stuff in Tk/ttk. I know some people will hate the singleton decorator and I know there are other ways to call code from other classes but the trigger class is very convenient and the worker class works like a charm. Together they make things super easy.

Credits: The worker class is a very slightly modified version of the GObject worker found in Pithos and the singleton decorator is a very slightly modified version of something I found here on stackoverflow somewhere.

import sys
import tkinter
from tkinter import ttk
from tkinter import StringVar
import threading
import queue
import traceback
import time

class TkWorkerThreadDemo:
    def __init__(self):
        self.root = tkinter.Tk()
        self.trigger = Trigger.Singleton()
        self.trigger.connect_event('enter_main_thread', self.enter_main_thread)
        self.worker = Worker()
        self.root.title('Worker Thread Demo')
        self.root.resizable(width='False', height='False')
        self.test_label_text = StringVar()
        self.test_label_text.set('')
        self.slider_label_text = StringVar()
        self.slider_label_text.set('Press either button and try to move the slider around...')
        mainframe = ttk.Frame(self.root)
        test_label = ttk.Label(mainframe, anchor='center', justify='center', textvariable=self.test_label_text)
        test_label.pack(padx=8, pady=8, fill='x')
        slider_label = ttk.Label(mainframe, anchor='center', justify='center', textvariable=self.slider_label_text)
        slider_label.pack(padx=8, pady=8, expand=True, fill='x')
        self.vol_slider = ttk.Scale(mainframe, from_=0, to=100, orient='horizontal', value='100', command=self.change_slider_text)
        self.vol_slider.pack(padx=8, pady=8, expand=True, fill='x')
        test_button = ttk.Button(mainframe, text='Start Test with a Worker Thread', command=self.with_worker_thread)
        test_button.pack(padx=8, pady=8)
        test_button = ttk.Button(mainframe, text='Start Test in the Main Thread', command=self.without_worker_thread)
        test_button.pack(padx=8, pady=8)
        mainframe.pack(padx=8, pady=8, expand=True, fill='both')
        self.root.geometry('{}x{}'.format(512, 256))

    def enter_main_thread(self, callback, result):
        self.root.after_idle(callback, result)

    def in_a_worker_thread(self):
        msg = 'Hello from the worker thread!!!'
        time.sleep(10)
        return msg

    def in_a_worker_thread_2(self, msg):
        self.test_label_text.set(msg)

    def with_worker_thread(self):
        self.test_label_text.set('Waiting on a message from the worker thread...')
        self.worker.send(self.in_a_worker_thread, (), self.in_a_worker_thread_2)

    def in_the_main_thread(self):
        msg = 'Hello from the main thread!!!'
        time.sleep(10)
        self.in_the_main_thread_2(msg)

    def in_the_main_thread_2(self, msg):
        self.test_label_text.set(msg)

    def without_worker_thread(self):
        self.test_label_text.set('Waiting on a message from the main thread...')
        self.root.update_idletasks()#without this the text wil not get set?
        self.in_the_main_thread()

    def change_slider_text(self, slider_value):
        self.slider_label_text.set('Slider value: %s' %round(float(slider_value)))

class Worker:
    def __init__(self):
        self.trigger = Trigger.Singleton()
        self.thread = threading.Thread(target=self._run)
        self.thread.daemon = True
        self.queue = queue.Queue()
        self.thread.start()

    def _run(self):
        while True:
            command, args, callback, errorback = self.queue.get()
            try:
                result = command(*args)
                if callback:
                    self.trigger.event('enter_main_thread', callback, result)
            except Exception as e:
                e.traceback = traceback.format_exc()
                if errorback:
                    self.trigger.event('enter_main_thread', errorback, e)

    def send(self, command, args=(), callback=None, errorback=None):
        if errorback is None: errorback = self._default_errorback
        self.queue.put((command, args, callback, errorback))

    def _default_errorback(self, error):
        print("Unhandled exception in worker thread:\n{}".format(error.traceback))

class singleton:
    def __init__(self, decorated):
        self._decorated = decorated
        self._instance = None

    def Singleton(self):
        if self._instance:
            return self._instance
        else:
            self._instance = self._decorated()
            return self._instance

    def __call__(self):
        raise TypeError('Singletons must be accessed through `Singleton()`.')

@singleton
class Trigger:
    def __init__(self):
        self._events = {}

    def connect_event(self, event_name, func, *args, **kwargs):
        self._events[event_name] = func

    def disconnect_event(self, event_name, *args, **kwargs):
        if event_name in self._events:
            del self._events[event_name]

    def event(self, event_name, *args, **kwargs):
        if event_name in self._events:
            return self._events[event_name](*args, **kwargs)

def main():
    demo = TkWorkerThreadDemo()
    demo.root.mainloop()
    sys.exit(0)

if __name__ == '__main__':
    main()