pyqtgraph ImageView Freezes when multithreaded

Question

I have multiple cameras that are hooked up wirelessly via wifi and I'm trying to stream the data to a client, which displays the streams on a GUI.

My issue is that the pyqtgraph ImageItems seem to stop repainting after about 30 seconds, or if I click out of the window, or if I adjust the controls on one of the images. After that, I can manage to get the images to repaint by resizing the window, but that's kind of tedious.

I thought maybe pyqtgraph wasn't threadsafe, but I don't even know if I'm using real threads, since I have to have everything run through Qt (QThread) to get things working.

I have found a few forum posts that show similar problems to this. The first one which redirected to here explains that you can use something called NSAppSleepDisabled but that appears to be only on OSX and I'm running Windows 10. The second one explains that their entire GUI freezes, and I'm not having that problem, only the ImageItem freezes, the entire rest of the GUI is responsive.

These are my imports

import time
from multiprocessing import Queue
from threading import Lock
from queue import Empty

import pyqtgraph as pg
from PyQt5.QtCore import QThread

In order to keep my software expandable, I've used worker threads to manage incoming image data, in an attempt to keep the ImageView's threadsafe, I added a lock:

graph_lock = Lock()

class WindowUpdater(QThread):

    stop_q = Queue()

    def __init__(self, cam_q: Queue, img_control: pg.ImageView, **kwargs):
        super().__init__(**kwargs)
        self.q = cam_q
        self.c = img_control

    def run(self) -> None:
        while self.stop_q.empty():
            try:
                timg = self.q.get_nowait()
                graph_lock.acquire(True)
                self.c.setImage(timg.frame)
                self.c.invalidate()
                graph_lock.release()
            except Empty:
                time.sleep(0.1)

Then the main application handles linking these threads to the incoming data

app = QApplication(sys.argv)
window = QMainWindow()

grid = QGridLayout()
grid.setSpacing(10)
widg = QWidget()
widg.setLayout(grid)
window.setCentralWidget(widg)
window.show()

window.setGeometry(300, 300, 500, 400)
window.show()

threads = []

for i, h in enumerate(hosts):
    img = pg.ImageView(window)
    grid.addWidget(img, i, 0)

    img_item = pg.ImageItem()
    img.addItem(img_item)

    threads.append(WindowUpdater(img_queue, img)

for t in threads:
    t.start()

sys.exit(app.exec_())

where hosts is a list of hostnames used to connect to, and img_queue is a multiprocessing Queue specific to that host's camera stream.

Does anybody know why having multiple instances of pyqtgraph ImageView or ImageItem s running simultaneously causes problems here?

score 3 · Accepted Answer · answered Jul 12 '19 at 20:15

3

You should not update the GUI from another thread since Qt forbids it ⁽¹⁾. So you must use signals to transmit the information.

class WindowUpdater(QThread):
    imageChanged = pyqtSignal(np.ndarray)
    stop_q = Queue()

    def __init__(self, cam_q: Queue, **kwargs):
        super().__init__(**kwargs)
        self.q = cam_q

    def run(self) -> None:
        while self.stop_q.empty():
            try:
                timg = self.q.get_nowait()
                graph_lock.acquire(True)
                self.imageChanged.emit(timg.frame)
                graph_lock.release()
            except Empty:
                time.sleep(0.1)

# ...
for i, h in enumerate(hosts):
    img = pg.ImageView(window)
    grid.addWidget(img, i, 0)

    img_item = pg.ImageItem()
    img.addItem(img_item)
    thread = WindowUpdater(img_queue)
    thread.imageChanged.connect(img.setImage)
    threads.append(thread)
# ...

_{(1) GUI Thread and Worker Thread}

answered Jul 12 '19 at 20:15

eyllanesc

235,170
19
170
241

Thank you for answering my question! Is there a faster option? pyqtSignals are too slow, and the Qt event loop backs up which causes memory leaks in the program. – iggy12345 Jul 12 '19 at 22:02
@iggy12345 1) Are the signals too slow? I think not, how have you measured it? What is the ratio between dt / size_of_image ?, a GUI has in itself many tasks, and among them the painting so you must be aware that you will not have an infinite efficiency, each image will be refreshed with a maximum 30ms ratio for screen issues. 2) The QEvents do not cause memory leak since Qt will eliminate them, I do not know where you get that information from. – eyllanesc Jul 12 '19 at 22:07
The camera runs at 90 fps, each frame is collected and processed using cython and placed into a Queue, the images are then flattened and transferred across WiFi to the client, all of this, according my debug lines reads out at around 50-100 fps, the client has a Process which receives the data and converts it back into 2D numpy arrays using cython, this is then placed into an appropriate Queue which is connected to the WindowUpdater class you see above – iggy12345 Jul 12 '19 at 22:12
With the pyqtSignals, if I emit a signal at 90 fps, which is what the WindowUpdater does, but can only handle it at 30 fps, the Qt Framework doesn't drop any of those events, so the events will stack up, and since each event contains an entire image, whithin minutes the program consumes almost a full gigabyte of RAM (~900 MB) – iggy12345 Jul 12 '19 at 22:14
@iggy12345 If you have a data from 50 FPS to 100 FPS, you can not show it on a screen, and although the HW could support it, the human eye could not distinguish it since the view works at 30 FPS, so by a known criterion the screens work at 60 FPS, plus a GUI is only used to show the result so instead of trying to show 50-100 FPS you could do a downsampling and pass it to 40 FPS. I do not understand why you need so fast. – eyllanesc Jul 12 '19 at 22:16
@iggy12345 Exactly, Qt only works at 30 FPS because that is the design of the HW since as our sight signal is limited so that a higher speed is not noticed by us, the cameras have a high FPS to compensate the processing time and for so do not suffer with the processing time, generally in video processing (I do not know what type of processing you are doing) a downsampling is done since the algorithms do not require as much information as that increases the working time.... – eyllanesc Jul 12 '19 at 22:20
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/196400/discussion-between-iggy12345-and-eyllanesc). – iggy12345 Jul 12 '19 at 23:07

pyqtgraph ImageView Freezes when multithreaded

1 Answers1

Linked