Why is GUI responsiveness impaired by a Worker thread?

Question

I'm trying to understand why this Worker thread, which deliberately uses a fairly intense amount of processing (sorting of these dictionaries in particular) causes the GUI thread to become unresponsive. Here is an MRE:

from PyQt5 import QtCore, QtWidgets
import sys, time, datetime, random

def time_print(msg):
    ms_now = datetime.datetime.now().isoformat(sep=' ', timespec='milliseconds')
    thread = QtCore.QThread.currentThread()
    print(f'{thread}, {ms_now}: {msg}')
    
def dict_reorder(dictionary):
    return {k: v for k, v in sorted(dictionary.items())}
    
class Sequence(object):
    n_sequence = 0
    simple_sequence_map = {}
    sequence_to_sequence_map = {}
    prev_seq = None
    
    def __init__(self):
        Sequence.n_sequence += 1 
        if Sequence.n_sequence % 1000 == 0:
            print(f'created sequence {Sequence.n_sequence}')
        rand_int = random.randrange(100000)
        self.text = str(rand_int)
        Sequence.simple_sequence_map[self] = rand_int
        if Worker.stop_ordered:
            time_print(f'init() A: stop ordered... stopping now')
            return
        dict_reorder(Sequence.simple_sequence_map)
        if Sequence.prev_seq:
            Sequence.sequence_to_sequence_map[self] = Sequence.prev_seq
            if Worker.stop_ordered:
                time_print(f'init() B: stop ordered... stopping now')
                return
            dict_reorder(Sequence.sequence_to_sequence_map)
        Sequence.prev_seq = self

    def __lt__(self, other):
        return self.text < other.text    
        
class WorkerSignals(QtCore.QObject):
    progress = QtCore.pyqtSignal(int)
    stop_me = QtCore.pyqtSignal()

class Worker(QtCore.QRunnable):
    def __init__(self, *args, **kwargs):
        super().__init__()
        self.signals = WorkerSignals()
        
    def stop_me_slot(self):
        time_print('stop me slot')    
        Worker.stop_ordered = True
    
    @QtCore.pyqtSlot()
    def run(self):
        total_n = 30000
        Worker.stop_ordered = False
        for n in range(total_n):
            progress_pc = int(100 * float(n+1)/total_n)
            self.signals.progress.emit(progress_pc)
            Sequence()
            if Worker.stop_ordered:
                time_print(f'run(): stop ordered... stopping now, n {n}')
                return

class MainWindow(QtWidgets.QMainWindow):
    def __init__(self):
        super().__init__()
        
        layout = QtWidgets.QVBoxLayout()
        self.progress = QtWidgets.QProgressBar()
        layout.addWidget(self.progress)
        
        start_button = QtWidgets.QPushButton('Start')
        start_button.pressed.connect(self.execute)
        layout.addWidget(start_button)
        
        self.stop_button = QtWidgets.QPushButton('Stop')
        layout.addWidget(self.stop_button)
        
        w = QtWidgets.QWidget()
        w.setLayout(layout)
        self.setCentralWidget(w)
        self.show()
        self.threadpool = QtCore.QThreadPool()
        self.resize(800, 600)

    def execute(self):
        self.worker = Worker()
        self.worker.signals.progress.connect(self.update_progress)
        self.stop_button.pressed.connect(self.worker.stop_me_slot)
        self.threadpool.start(self.worker)
        
    def update_progress(self, progress):
        self.progress.setValue(progress)        
              
app = QtWidgets.QApplication(sys.argv)
window = MainWindow()
app.exec_()

On my machine, up until about 12%, the GUI is significantly unresponsive: the buttons do not acquire their "hover-over" colour (light blue) and seem not to be able to be clicked (although clicking "Stop" does cause stop after many seconds). Intermittently the dreaded spinner appears (blue circle in a W10 OS).

After 12% or so it becomes possible to use the buttons normally.

What am I doing wrong?

Answer 1

A very simple solution is to "sleep" the thread by using a basic time.sleep: even with a very small interval, it will give enough time for the main thread to process its event queue avoiding UI locking:

    def run(self):
        total_n = 30000
        Worker.stop_ordered = False
        for n in range(total_n):
            progress_pc = int(100 * float(n+1)/total_n)
            self.signals.progress.emit(progress_pc)
            Sequence()
            if Worker.stop_ordered:
                time_print(f'run(): stop ordered... stopping now, n {n}')
                return
            time.sleep(.0001)

^{Note: that pyqtSlot decorator is useless, because it only works for QObject subclasses (which QRunnable isn't); you can remove it.}

Answer 2

Python cannot run more than one CPU-intensive thread. The cause of it the the GIL. Basically Python threads are not good for anything but waiting for I/O.

If you want a CPU-intensive part, try either rewriting the intensive part using Cython, or use multiprocessing, but then the time to send data back and forth could be significant.

Answer 3

Cannot reproduce, ui stays responsive even with limited resources. Have you tried to run it without debugger?

GIL may be the problem as @9000 suggests.

Or maybe eventloop flooded with progress signals, try to emit it less than one for each sequence.

As a sidenote: program works faster if you dont throw away sorting results every time with dict_reorder. try to replace

dict_reorder(Sequence.simple_sequence_map)

with

Sequence.simple_sequence_map = dict_reorder(Sequence.simple_sequence_map)

and

dict_reorder(Sequence.sequence_to_sequence_map)

with

Sequence.sequence_to_sequence_map = dict_reorder(Sequence.sequence_to_sequence_map)