I'm trying to develop a Pyqt application with face recognition. For the face recognition I use the python package face_recognition. To keep the UI responsive, I did move a worker with the face recognition to another QThread. However, the UI is still extremly sluggish and lagging. To check, whether I made a mistake implementing multithreading, I did then replace the face recognition part with a long active waiting loop, like this:
Original face recognition code:
face_recognition.face_locations(frame, model="hog")
Replaced with active waiting loop:
for x in range(59999999):
pass
However in this case, the UI was very responsive. So the relevant part is indeed on another thread. But why is the UI thread lagging when I replace it with the face recognition?
I also did try to set the priority of the UI thread to QThread.TimeCriticalPriority (highest) and the priority of the worker to QThread.IdlePriority (lowest) but this did not really have an effect.
EDIT:
Here is a minimal example where I also can observe this behavior. While, the lagging is not as much as in my real application, it's still noticeable here when resizing the window. However, I had to add specific QWidgets like ScrollArea or LineEdit to the UI. Otherwise (e.g. only showing a label) the resizing was smooth.
#!/usr/bin/env python3
import sys, cv2, face_recognition
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtWidgets import QApplication, QMainWindow, QFileDialog
from PyQt5.QtCore import QThread, QObject, pyqtSlot
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setup_test_ui()
self.open_file()
# QThread.currentThread().setPriority(QThread.TimeCriticalPriority)
def setup_test_ui(self):
self.centralwidget = QtWidgets.QWidget(self)
self.verticalLayout_2 = QtWidgets.QVBoxLayout(self.centralwidget)
self.scrollArea = QtWidgets.QScrollArea(self.centralwidget)
self.scrollArea.setWidgetResizable(True)
self.lineEdit = QtWidgets.QLineEdit(self.scrollArea)
self.scrollArea.setWidget(self.lineEdit)
self.verticalLayout_2.addWidget(self.scrollArea)
self.setCentralWidget(self.centralwidget)
def open_file(self):
file_name, _ = QFileDialog.getOpenFileName(self, "Open video", "", "All Files (*);;Movie Files (*.mp4 *.avi)")
if (file_name):
self.setup_worker(file_name)
def setup_worker(self, *args, **kwargs):
worker = Worker(*args, **kwargs)
thread = QThread()
self.worker = (worker, thread)
worker.moveToThread(thread)
thread.started.connect(worker.work)
thread.start()
# thread.start(QThread.IdlePriority)
class Worker(QObject):
def __init__(self, video_source=0, *args, **kwargs):
super().__init__(*args, **kwargs)
self.vid = cv2.VideoCapture(video_source)
if not self.vid.isOpened():
raise ValueError("Unable to open video source", video_source)
@pyqtSlot()
def work(self):
while (True):
ret, frame = self.vid.read()
if ret:
# for x in range(59999999):
# pass
face_locations = face_recognition.face_locations(frame, model="hog")
app = QApplication(sys.argv)
window = MainWindow()
window.show()
sys.exit(app.exec())
Note: Make sure to install all dependencies: OpenCV, PyQt5, face-recognition. For the last one, you likely have to compile dlib in advance (follow these instructions). Then start the application, use any video file which is used as input for the face_recognition and resize the window (it's lagging a bit for me). Then you can edit the work method and use the active waiting loop instead of the face_recognition line. When executing the application now, resizing is very responsive.
