Understanding threading and PyQT

Question

I'm trying to implement threading in a PyQT GUI but having trouble. Some background, I have a standalone script that uninstalled some software, deleted some folders, and then reinstalled a newer build. I used the threading module to delete the folders, spinning up a new thread for each. A couple of the folders were large and took some time, so I would iterate over the separate threads but skip the larger folders and join the threads:

    thread = threading.Thread(name=portalDirToDelete,target=deleteFolder,args=(portalDirToDelete,))
    thread.start()
    ....
    for thread in threading.enumerate():
      if not "MainThread" in thread.getName() and not "content" in thread.getName() and not "temp" in thread.getName():
        thread.join()

Once I started making a UI using PyQT4, I found that the threads wouldn't start until I tried to join them. I did some reading and learned that using the threading module wouldn't work anymore, (How to keep track of thread progress in Python without freezing the PyQt GUI?) so I started to look into QThreads but haven't had much luck. Below is a simplified version of what my other script is doing:

# -*- coding: utf-8 -*-

# Form implementation generated from reading ui file 'Samples\ThreadUI1.ui'
#
# Created by: PyQt4 UI code generator 4.11.4
#
# WARNING! All changes made in this file will be lost!

from PyQt4 import QtCore, QtGui
import shutil, os, time

try:
    _fromUtf8 = QtCore.QString.fromUtf8
except AttributeError:
    def _fromUtf8(s):
        return s

try:
    _encoding = QtGui.QApplication.UnicodeUTF8
    def _translate(context, text, disambig):
        return QtGui.QApplication.translate(context, text, disambig, _encoding)
except AttributeError:
    def _translate(context, text, disambig):
        return QtGui.QApplication.translate(context, text, disambig)

class workerThread(QtCore.QObject):

    finished = QtCore.pyqtSignal()

    def deleteFolder(self,path):
        if os.path.exists(path):
            shutil.rmtree(path)
        self.finished.emit()

class Ui_Form(object):
    def setupUi(self, Form):
        Form.setObjectName(_fromUtf8("Form"))
        Form.resize(115, 66)
        self.runApp = QtGui.QPushButton(Form)
        self.runApp.setGeometry(QtCore.QRect(20, 20, 75, 23))
        self.runApp.setObjectName(_fromUtf8("runApp1"))

        self.retranslateUi(Form)
        QtCore.QMetaObject.connectSlotsByName(Form)

        self.runApp.clicked.connect(lambda:self.runSetups())

    def retranslateUi(self, Form):
        Form.setWindowTitle(_translate("Form", "Form", None))
        self.runApp.setText(_translate("Form", "Run App", None))

    def usingMoveToThread(self,path):
        self.app = QtCore.QCoreApplication([])
        self.objThread = QtCore.QThread()
        self.obj = workerThread()
        self.obj.moveToThread(self.objThread)
        self.obj.finished.connect(self.objThread.quit)
        self.objThread.started.connect(self.obj.deleteFolder(path))
        self.objThread.finished.connect(app.exit)
        self.objThread.start()

    def runSetups(self):
        self.usingMoveToThread(r"C:\arcgisportal")
        for x in range(1,11):
            print(x)
            time.sleep(1)

if __name__ == "__main__":
    import sys
    app = QtGui.QApplication(sys.argv)
    Form = QtGui.QWidget()
    ui = Ui_Form()
    ui.setupUi(Form)
    Form.show()
    sys.exit(app.exec_())

I may be wrong in thinking this, but my approach was to kick off the process to delete the folders in a new thread, and see if it then continues on to print in the loop. I've been using Threading and PyQT as a main guide. There seems to be differing opinions regarding subclassing QThread or not, so I just went with this blog, (https://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/) in not doing it.

I'm using Python 2.7, PyQt4 4.11.4 and this app was made through UI Designer. I know it's not a good idea to update the generated ui/py file, which I'm not doing elsewhere, just made this sample to provide as context.

Any help is appreciated!

Update

If I open a modal QMessageBox() and add a button that closes the QMessageBox, while it's open, the folders within the larger folder are deleted. Once I close the QMessageBox(), the folders stop getting deleted, so there's something that's not clear to me regarding threads and the UI.

Answer 1

There are several issues with what you've written (hopefully I've caught them all, I'm reluctant to run code on my machine that does anything related to deleting files!)

1. You are constructing a new QCoreApplication in the usingMoveToThread method. You should only have one QApplication per python instance. I am not sure exactly what doing this will have done, but I suspect it explains the behaviour you see when you create a modal dialog. Creating the QApplication (or QCoreApplication) starts the main Qt event loop which handles everything from processing signals/calling slots and handling window redraws. You've created a second event loop in the main thread, but not started it by called exec_(). I suspect when you create the dialog, this is calling exec_() (or creating a third event loop, which somehow fixes the issues created by the second event loop). Regardless there is no reason to manually create a second event loop. The main one will handle the thread stuff just fine. (note: you can have multiple event loops per pyqt application, for instance threads can have an event loop (see below) and dialogs sometimes have their own event loops. However, Qt handles the creation for you!)

2. The loop in runSetups is going to block the main thread, so your GUI is going to be unresponsive. This shouldn't effect the thread, but it begs the question what the point of offloading it to a thread was if you are just going to block the GUI! Note that the Python GIL prevents multiple threads from running concurrently anyway, so if you were hoping to have multiple threads delete separate folders simultaneously, you'll want to look into multiprocessing instead to avoid the Python GIL. However, this is likely premature optimisation. There are probably disk I/O considerations and you may find that one of these approaches provides any meaningful speedup.

3. The main issue is the following line: self.objThread.started.connect(self.obj.deleteFolder(path)). You are actually running deleteFolder(path) in the main thread, and passing the return value of that function to the connect method. This results in nothing being run in the thread, and everything happening in the main thread (and blocking the GUI). This mistake has come about because you want to pass a parameter to the deleteFolder method. But you can't because you are not emitting the started signal (Qt does it) and so you can't supply the parameter. Usually, you would get around this by wrapping up your method in lambda, or partial, but this brings other issues when working with threads (see here). Instead, you need to define a new signal in your Ui_Form object, which takes a str argument, connect that signal to your deleteFolder slot and emit the signal manually after you start the thread.

Something like this:

class Ui_Form(object):
    deleteFolder = QtCore.pyqtSignal(str)

    ...

    def usingMoveToThread(self,path):
        self.objThread = QtCore.QThread()
        self.obj = workerThread()
        self.obj.moveToThread(self.objThread)
        self.deleteFolder.connect(self.obj.deleteFolder)
        self.objThread.start()
        self.deleteFolder.emit(path)

You might want to check out this question for slightly more information on this approach (basically, Qt can handle connecting between signals/slots in different threads, as each thread has it's own event loop).

Answer 2

# your app initialization:
# application = QApplication(sys.argv)
# main_window.show()
# application.exec_()
...
thread = threading.Thread(name=portalDirToDelete,target=deleteFolder,args=(portalDirToDelete,))
    thread.start()
    ....
    for thread in threading.enumerate():
      if not "MainThread" in thread.getName() and not "content" in thread.getName() and not "temp" in thread.getName():
        while thread.is_alive():
            application.processEvents()