Is there any solution for real time video GUI with pyqt5 without freezing?

Question

The code below is the code in question.

class ShowVideo that receives and processes video data from Webcam
class ImageViewer that receives images and displays them in GUI in real time

When the data of Webcam is received as 640x480, this code works without problems. -. Left-click on the title bar of the application window and wait -. Left-click and hold the Minimize, Maximize, Close button In this case, the GUI freezes, but after a while after releasing the left click, the GUI starts working without problems.

However, when the data of Webcam is received in 1920x1080, this code causes a problem. Operations in the above two application windows cause the GUI to freeze, and eventually the process is terminated.

i want to solve this problem I want to know how to solve it to avoid freezing of pyqt5 GUI.

import cv2
import sys
from PyQt5 import QtCore
from PyQt5 import QtWidgets
from PyQt5 import QtGui



class ShowVideo(QtCore.QObject):

    flag = 0
    camera = cv2.VideoCapture(0)
    camera.set(cv2.CAP_PROP_FRAME_WIDTH, 1920 ) # 1920
    camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080) # 1080
    ret, image = camera.read()
    height, width = image.shape[:2]
    print(f"{height} / {width}")
    VideoSignal1 = QtCore.pyqtSignal(QtGui.QImage)
    VideoSignal2 = QtCore.pyqtSignal(QtGui.QImage)

    def __init__(self, parent=None):
        super().__init__()


    @QtCore.pyqtSlot()
    def startVideo(self):
        global image

        run_video = True
        while run_video:
            ret, image = self.camera.read()
            color_swapped_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

            qt_image1 = QtGui.QImage(color_swapped_image.data,
                                    self.width,
                                    self.height,
                                    color_swapped_image.strides[0],
                                    QtGui.QImage.Format_RGB888)

            self.VideoSignal1.emit(qt_image1)


            if self.flag:
                img_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
                img_canny = cv2.Canny(img_gray, 50, 100)

                qt_image2 = QtGui.QImage(img_canny.data,
                                         self.width,
                                         self.height,
                                         img_canny.strides[0],
                                         QtGui.QImage.Format_Grayscale8)

                self.VideoSignal2.emit(qt_image2)


            loop = QtCore.QEventLoop()
            QtCore.QTimer.singleShot(25, loop.quit) #25 ms
            loop.exec_()

    @QtCore.pyqtSlot()
    def canny(self):
        self.flag = 1 - self.flag



class ImageViewer(QtWidgets.QWidget):

    def __init__(self, parent=None):
        #     super(ImageViewer, self).__init__(parent)
        super().__init__()
        self.image = QtGui.QImage()
        self.setAttribute(QtCore.Qt.WA_OpaquePaintEvent)


    def paintEvent(self, event):
        painter = QtGui.QPainter(self)
        painter.drawImage(0, 0, self.image)
        # self.image = QtGui.QImage()

    def initUI(self):
        self.setWindowTitle('Test')

    @QtCore.pyqtSlot(QtGui.QImage)
    def setImage(self, image):

        image = QtGui.QImage(image).scaled(400, 300, QtCore.Qt.KeepAspectRatio, QtCore.Qt.FastTransformation)

        if image.isNull():
            print("Viewer Dropped frame!")

        self.image = image
        if image.size() != self.size():
            self.setFixedSize(image.size())
        self.update()



if __name__ == '__main__':
    app = QtWidgets.QApplication(sys.argv)


    thread = QtCore.QThread()
    thread.start()
    vid = ShowVideo()
    vid.moveToThread(thread)

    image_viewer1 = ImageViewer()
    image_viewer2 = ImageViewer()

    vid.VideoSignal1.connect(image_viewer1.setImage)
    vid.VideoSignal2.connect(image_viewer2.setImage)


    push_button1 = QtWidgets.QPushButton('Start')
    push_button2 = QtWidgets.QPushButton('Canny')
    push_button1.clicked.connect(vid.startVideo)
    push_button2.clicked.connect(vid.canny)

    vertical_layout = QtWidgets.QVBoxLayout()
    horizontal_layout = QtWidgets.QHBoxLayout()
    horizontal_layout.addWidget(image_viewer1)
    horizontal_layout.addWidget(image_viewer2)
    vertical_layout.addLayout(horizontal_layout)
    vertical_layout.addWidget(push_button1)
    vertical_layout.addWidget(push_button2)

    layout_widget = QtWidgets.QWidget()
    layout_widget.setLayout(vertical_layout)

    main_window = QtWidgets.QMainWindow()
    main_window.setCentralWidget(layout_widget)
    main_window.show()
    sys.exit(app.exec_())

Just as the ShowVideo class operates as a thread, ImageViewer also created and operated each thread, but the problem was not resolved.

2. Modified Code I changed code based on the comments. Of course, I am sure that I can not reach your wisdom.

I removed thread.start()
I connect pushbutton and thread.start()
I connect showVidoe.startVideo() slot and thread.started signal

In this case,

I can not control turn on/off the camera.

When I turn on the 'Canny', I experienced same freezing and program shut off

 app = QtWidgets.QApplication(sys.argv)

 thread = QtCore.QThread()
 vid = ShowVideo()                  
 vid.moveToThread(thread) 

 image_viewer1 = ImageViewer()
 image_viewer2 = ImageViewer()

 vid.VideoSignal1.connect(image_viewer1.setImage)
 vid.VideoSignal2.connect(image_viewer2.setImage)

 push_button1 = QtWidgets.QPushButton('Start')    
 push_button1.clicked.connect(thread.start)         # button
 thread.started.connect(vid.startVideo)

 push_button2 = QtWidgets.QPushButton('Canny')
 push_button2.clicked.connect(vid.canny)

2. Modified code ver 2

I tried two types of thread.

QtCore.Thread
treading.Thread

For both try, I can not suffer from freezing and program close. But Same with the above code, I can not control camera turn on/off also.

import cv2
import sys
from PyQt5 import QtCore
from PyQt5 import QtWidgets
from PyQt5 import QtGui
import threading


class ShowVideo(QtCore.QThread):
# class ShowVideo(threading.Thread, QtCore.QObject):

    flag = 0
    camera = cv2.VideoCapture(0)
    camera.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
    camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
    ret, image = camera.read()
    height, width = image.shape[:2]
    print(f"{height} / {width}")
    VideoSignal1 = QtCore.pyqtSignal(QtGui.QImage)
    VideoSignal2 = QtCore.pyqtSignal(QtGui.QImage)

    def __init__(self, parent=None):
        super().__init__()
        # threading.Thread.__init__(self)
        # QtCore.QObject.__init__(self)

    @QtCore.pyqtSlot()
    def startVideo(self):
        global image

        run_video = True
        while run_video:
            ret, image = self.camera.read()
            color_swapped_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

            qt_image1 = QtGui.QImage(color_swapped_image.data,
                                     self.width,
                                     self.height,
                                     color_swapped_image.strides[0],
                                     QtGui.QImage.Format_RGB888)

            self.VideoSignal1.emit(qt_image1)

            if self.flag:
                img_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
                img_canny = cv2.Canny(img_gray, 50, 100)

                qt_image2 = QtGui.QImage(img_canny.data,
                                         self.width,
                                         self.height,
                                         img_canny.strides[0],
                                         QtGui.QImage.Format_Grayscale8)

                self.VideoSignal2.emit(qt_image2)

            loop = QtCore.QEventLoop()
            QtCore.QTimer.singleShot(25, loop.quit)  # 25 ms
            loop.exec_()

    @QtCore.pyqtSlot()
    def canny(self):
        self.flag = 1 - self.flag


class ImageViewer(QtWidgets.QWidget):

    def __init__(self):
        super().__init__()
        self.image = QtGui.QImage()
        self.setAttribute(QtCore.Qt.WA_OpaquePaintEvent)

    def paintEvent(self, event):
        painter = QtGui.QPainter(self)
        painter.drawImage(0, 0, self.image)

    def initUI(self):
        self.setWindowTitle('Test')

    @QtCore.pyqtSlot(QtGui.QImage)
    def setImage(self, image):

        if image.isNull():
            print("Viewer Dropped frame!")

        self.image = image
        if image.size() != self.size():
            self.setFixedSize(image.size())
        self.update()


if __name__ == '__main__':
    app = QtWidgets.QApplication(sys.argv)

    vid = ShowVideo()
    vid.start()

    image_viewer1 = ImageViewer()
    image_viewer2 = ImageViewer()

    vid.VideoSignal1.connect(image_viewer1.setImage)
    vid.VideoSignal2.connect(image_viewer2.setImage)

    push_button1 = QtWidgets.QPushButton('Start')
    push_button2 = QtWidgets.QPushButton('Canny')

    push_button1.clicked.connect(vid.startVideo)
    push_button2.clicked.connect(vid.canny)

    vertical_layout = QtWidgets.QVBoxLayout()
    horizontal_layout = QtWidgets.QHBoxLayout()
    horizontal_layout.addWidget(image_viewer1)
    horizontal_layout.addWidget(image_viewer2)
    vertical_layout.addLayout(horizontal_layout)
    vertical_layout.addWidget(push_button1)
    vertical_layout.addWidget(push_button2)

    layout_widget = QtWidgets.QWidget()
    layout_widget.setLayout(vertical_layout)

    main_window = QtWidgets.QMainWindow()
    main_window.setCentralWidget(layout_widget)
    main_window.show()
    sys.exit(app.exec_())

In the worker thread, you are running a loop calling the ```cv2.VideoCapture.read()```. So, avoid using slots and running a Qt event loop in the worker thread. Either override the ```QThread.run()``` or use a native Python thread using the ```threading``` module. You can emit a signal in any worker thread. — relent95, Mar 23 '23 at 14:13
I'm trying to understand your opinion. *relent95 1) Is it correct to compose the contents of the run() function in Class VideoShow(Qobject) and call the run function using thread.start() so that the while loop runs stably in the New thread? *ekhumoro 1) If the button is connected to the Worker thread slot (showVideo Class), is showVideo.startVideo() executed in the main thread not in the worker thread due to Button in the main thread , even though the thread is running? — Seokhyen kim, Mar 24 '23 at 02:29
I edit the post with codes that I tried. Could you check the codes ? — Seokhyen kim, Mar 24 '23 at 04:53
I said you should not use slots and should not run a Qt event loop in the worker thread. You need to understand GUI event loop and how QThread works. I posted a [pseudo code](https://gist.github.com/relent95/fb3ea015ff2f3511ae17e725e6d04487) on Gists. And @ekhumoro, FYI, what you pointed is not a primary cause of the problem. See my pseudo code. — relent95, Mar 24 '23 at 07:21
@relent95 Your claims about slots and event-loops are just plain wrong. I have now tested the OP's original example and it works perfectly fine for me with no freezing of any kind. Clicking the "Canny" button starts and stops the second video output without any problems. So there's actually nothing whatsoever wrong with the code itself. — ekhumoro, Mar 24 '23 at 17:06
@Seokhyen kim Please ignore my comment above - I wasn't able to test your code yesterday, so I was just guessing what the issue might be. I tested your original code today, and it all works fine for me. However, I cannot reproduce the problems you describe in your question that are related to clicking on the title-bar and the title-bar buttons. What platform are you testing on, and what window-manager are you using? I am testing on arch linux with an openbox window manager, so I'm not familiar with the title-bar behavior you are describing. — ekhumoro, Mar 24 '23 at 17:15
@ekhumoro, the OP want to run a loop calling ```cv2.VideoCapture.read()``` in a worker thread. So he should not use slots and should not run a Qt event loop in that thread. Why do you think I am wrong? — relent95, Mar 25 '23 at 01:07
@relent95 Because I tested the code and I know that it works (see my comments above). Also, the OP says in the question that the code works without problems under normal circumstances. What actual evidence do you have that it doesn't work? Your claims about slots and event-loops make no sense at all. Please re-read the question and test the original code example yourself. — ekhumoro, Mar 25 '23 at 01:32
@ekhumoro, the OP's worker thread repeats calling a blocking IO API ```cv2.VideoCapture.read()```, doing a CPU intensive image processing task, and running a nested event loop. Do you expect it will work fine predictably? See [this question](https://stackoverflow.com/questions/35561182/why-should-nesting-of-qeventloops-be-avoided). Also see [the reference](https://doc.qt.io/qt-6/threads-qobject.html). — relent95, Mar 25 '23 at 03:14
@relent95 Please re-read the question and test the original code example yourself. — ekhumoro, Mar 25 '23 at 03:21

Is there any solution for real time video GUI with pyqt5 without freezing?

0 Answers0