PyQt5 Best way to plot data without freezing GUI

Question

I'm trying to achieve something similar to this. Specifically, I have a queue which gets filled with data, and I want to plot this data in real-time. I currently having a working version which uses a QTimer (and no threads) in order to call a function every half a second. This function takes items out of the queue (while it's nonempty) and plots them. While this certainly works, I was wondering if there's a better way which I should be using (involving threads and the like). My main concern is that I'm scared my approach, while works, could make things more laggy than they need to be. I tried the approach given above, but (if I'm not messing something up, which I probably am):

1. terminating the thread was problematic (specifically: "QObject::killTimer: Timers cannot be stopped from another thread")
2. The process did not even seem to occur at a different thread, as indicated by the last comment (adding a sleep call in the other thread freezes the entire GUI).

The point is, I was wondering if there's a "better" way to do this than my current approach (such as the one given above, if I can get it to work correctly in my case, or having a thread which checks the queue constantly, and whenever its nonempty, emits a value rather than having a timer).

I'm not giving a minimal reproducible example since this is not specific problem with a particular bit of code necessarily and more so a question about the right approach.

Edit: based on this, would it be best to simply process only one event from the queue at a time and set the timer's timeout to 0ms (rather than looping until the queue isn't empty every half a second)?

Answer 1

This depends on how you want the data plotted. Is every item in the Queue a new plot or do you only need the latest data shown? If you only need to show the latest data then this is a simple process.

I made a library qt_thread_updater to help with this. The qt_thread_updater uses a timer that runs periodically to call functions in the main thread. It is thread safe. However, qt_thread_updater will call the function at a later timeout, so GUI updates will be a fraction of a second delayed.

import multiprocessing as mp
import threading
from qtpy import QtWidgets
from qt_thread_updater import get_updater


app = QtWidgets.QApplication()

fig = Plot()  # Your plotting library
fig.show()

def plot_data(data):
    # Plot the data here
    fig.plot(data)    

def read_queue(que, alive):
    while alive.is_set():
        for _ in range(que.qsize()):
            data = que.get()
            get_updater().call_latest(plot_data, data)
        time.sleep(0.1)  # Some delay to let the QEvent loop process events.

# Processed data queue and Event to close down processes
myque = mp.Queue()
alive = mp.Event()
alive.set()

# Read processed data thread and display
pull_data_th = threading.Thread(target=read_queue, args=(myque, alive))
pull_data_th.start()

# Worker process
worker = mp.Process(target=do_work, args=(myque, alive)
worker.start()

app.exec_()
alive.clear()
worker.join()
pull_data_th.join()

If you need every item in the Queue displayed then you have to be careful of performance. If the QEvent loop does not have time to processes events and gets full the application will crash. If you application is crashing then you need to either slow down your data rate or get a faster plotting library (matplotlib is slow, pyqtgraph is much faster).

You can use the same code as below except change call_latest to call_in_main

def read_queue(que, alive):
    while alive.is_set():
        for _ in range(que.qsize()):
            data = que.get()
            get_updater().call_in_main(plot_data, data)

        # Some delay to let the QEvent loop process events.
        if que.empty():
            time.sleep(0.5)
        else:
            time.sleep(0.001)