Python/PySide: How can i destroy a terminated thread object?

Question

I would like to implement a button to stop a thread with a process, it works but not as expected: i can't delete the thread object. (EDIT: The reference to the thread object seems to be deleted, but the signals are not disconnected automatically by deleting the thread object, i can access it anyway via the signal.)

I have a modul with a class thread_worker and a function for complex processing which is running as process:

from PySide.QtCore import *
from PySide.QtGui import *
import multiprocessing as mp
import time

# this function runs as a process
def complex_processing(queue):
    # do something
    ...

class thread_worker(QThread):
    message_signal = Signal(str)
    stop_thread_signal = Signal()

    def __init__(self, prozessID, sleepTime, parent=None):
        super(ThreadProzessWorker, self).__init__(parent)
        self.queue = mp.Queue()
        self.process = mp.Process(target=complex_processing, args=(self.queue,))
        self.timeStamp = int(time.time())

    def run(self):
        self.process.start()
        self.process.join()

    @Slot()
    def stop_process_and_thread(self):
        if self.isRunning():
            self.message_signal.emit("Thread %d is running!" % self.timeStamp)
            if self.process.is_alive():
                self.process.terminate()
                self.process.join()      
            self.stop_thread_signal.emit()   
            #self.terminate() # does it works at this place?       
        else:
            self.message_signal.emit("Thread %d is not running!" % self.timeStamp)

I have two buttons in my application to create/run and terminate a thread object.

...
...
# Buttons
self.button_start_thread = QPushButton("Start Thread")
self.button_start_thread.clicked.connect(self.start_thread)
self.button_stop_thread = QPushButton("Stop Thread")
...
...
@Slot()
def start_thread(self):
    self.new_thread = thread_worker(self)
    self.button_stop_thread.clicked.connect(self.new_thread.stop_process_and_thread)
    self.new_thread.stop_thread_signal.connect(self.stop_thread)
    self.new_thread.message_signal.connect(self.print_message)
....
....
@Slot()
def stop_thread(self):
    self.new_thread.terminate()
    #self.button_stop_thread.disconnect(self.new_thread)
    del(self.new_thread)

@Slot(str)
def print_message(self, message):
    print(message)
...
...

If i start and stop the first thread - it works fine and terminate, but if i klick on the 'Stop'-Button again the output is:

Thread 1422117088 is not running!

I don't understand it: the object self.new_thread is deleted by del(self.new_thread) or not? How can i access this object if it was deleted? If i start and stop again a new thread, the output is:

Thread 1422117088 is not running!  # not expected, the thread object is deleted!
Thread 1422117211 is running!      # expected

Now i do it again (start and stop), the output is:

Thread 1422117088 is not running!   # not expected, the thread object is deleted!
Thread 1422117211 is not running!   # not expected, the thread object is deleted!
Thread 1422117471 is running!       # expected

and so on...

First question: I don't understand why the old threads are not deleted? Why i can access them? I think it is not good: my application crashes at some point, if there are too many threads (not deleted objects) in the background.

Second question: I dont't understand why the signals are not disconnected if i delete the object self.new_thread? I don't want to disconnect the signals manually: if i have many signals i can forgot to disconnect some signals.

Third question: I choose this way to stop a thread with one process. Is there another way to do this better?

UPDATE:

The thread object appears to be destroyed:

del(self.new_thread)
print(self.new_thread)
Output: AttributeError: 'MainWindow' object has no attribute 'new_thread'

But my signals are not disconnected!? Here is described that: "A signal-slot connection is removed when either of the objects involved are destroyed." It does not work in my code.

Answer 1

You're right, your problem is, that your object is not deleted. You delete only the reference self.new_thread. The problem is in this line:

self.new_thread = thread_worker(self)

The reason for this is that the parent self of the thread is alive. As long as parent stays alive, the object self.new_thread is not destroyed. Try something like this:

self.threadParent = QObject()
self.new_thread = thread_worker(self.threadParent)

And now you also delete the parent self.threadParent:

self.new_thread.terminate()
del(self.new_thread)
del(self.threadParent)

Your signals should be disconnected now.

You don't need the following line, because the object self.new_thread has been already deleted after you emit the signal stop_thread_signal:

#self.terminate() # does it works at this place?

Answer 2

This happens possibly because your process doesn't contain anything and finishes after it starts.Therefore the code in

def stop_process_and_thread(self):
    if self.isRunning():
        ...
        if self.process.is_alive():
            self.process.terminate()
            self.process.join()      
            self.stop_thread_signal.emit()   
            #self.terminate() # does it works at this place?  

after the if self.process.is_alive() is never reached. As a result the stop_thread_signal is never emitted so the thread is neither terminated nor deleted

Change your test complex_processing function to

def complex_processing(queue):
# do something
  while(True):
    print "in process"
    time.sleep(0.2)

Now to achieve the effect you want you should store threads in a list so start_thread should become

def start_thread(self):
    n = len(self.new_thread)
    self.new_thread.append(thread_worker(self))
    self.new_thread[-1].setTerminationEnabled(True)
    self.new_thread[-1].start()
    if n>0:
        self.button_stop_thread.clicked.disconnect()
    self.button_stop_thread.clicked.connect(self.new_thread[-1].stop_process_and_thread)
    print self.new_thread
    self.new_thread[-1].stop_thread_signal.connect(self.stop_thread)
    self.new_thread[-1].message_signal.connect(self.print_message)

Notice here that before setting up the connections to the last thread of the list (your newly added thread) you must first disconnect the button clicked() signal from all slots. That is because if you start more than one threads the clicked() signal will be connected to the slots of all of them. Therefore stop_process_and_thread and stop_thread will be called as many times as the number of running threads with unpredicted sequence. By disconnecting the signal only the last started thread is terminated and removed from the list. Also be sure that setTerminationEnabled is set

However after that you have to reconnect the button clicked signal, so you also have to change your stop_thread method to

def stop_thread(self):
    del(self.new_thread[-1])
    if self.new_thread: # if no threads left, then no need ro reconnect signal
         self.button_stop_thread.clicked.connect(self.new_thread[-1].stop_process_and_thread)
    print self.new_thread 

As for termination this can be done inside the stop_process_and_thread method. (It's the line you have commented out, place it before emitting the signal just in case)

Now your program should behave as you expected

For your first question I'm not really sure if threads are deleted, but sure now they are terminated. Somehow in your previous code you were losing the reference by assigning all to one variable while some of them were still active. This answers your 2nd question too

As for the third, terminating a thread this way is not recommended in the qt documentation. you should search there for better and safer ways and I'm also not very sure about your use of multiprocessing but I can't tell you more because I havent used this module a lot.

I believe that you should change strategy for programming this. Better use threads inside processes and not processes inside threads as you do

Answer 3

I have dealt with this problem before and have solved it like this:

import time
from threading import Thread

class myClass(object):
    def __init__(self):
        self._is_exiting = False

        class dummpyClass(object):
            pass
        newSelf = dummpyClass()
        newSelf.__dict__ = self.__dict__

        self._worker_thread = Thread(target=self._worker, args=(newSelf,), daemon=True)

    @classmethod
    def _worker(cls, self):
        i = 0
        while not self._is_exiting:
            print('Loop #{}'.format(i))
            i += 1
            time.sleep(3)
    def __del__(self):
        self._is_exiting = True
        self._worker_thread.join()

The use of dummpyClass allows the worker to access all of the attributes of our main class, but it doesn't actually point to it. Thus, when we delete our main class, it is possible for it to be fully de-referenced.

test = myClass()
test._worker_thread.start()

Note that our worker starts printing output.

del test

Worker thread has stopped.

A more complex way of solving this problem that would allow you access to methods myClass would be to first define a base class (call it myClass_base) that stores all data and required methods, but no threading support. Then add on the threading support in a new class.

import time
from threading import Thread
class myClass_base(object):
    def __init__(self, specialValue):
        self.value = specialValue
    def myFun(self, x):
        return self.value + x

class myClass(myClass_base):
    def __init__(self, *args, **kwargs):

        super(myClass, self).__init__(*args, **kwargs)

        newSelf = myClass_base.__new__(myClass_base)
        newSelf.__dict__ = self.__dict__

        self._is_exiting = False

        self.work_to_do = []

        self._worker_thread = Thread(target=self._worker, args=(newSelf,), daemon=True)

    @classmethod
    def _worker(cls, self):
        i = 0
        while not self._is_exiting:
            if len(self.work_to_do) > 0:
                x = self.work_to_do.pop()
                print('Processing {}.  Result = {}'.format(x, self.myFun(x)))
            else:
                print('Loop #{}, no work to do!'.format(i))
            i += 1
            time.sleep(2)

    def __del__(self):
        self._is_exiting = True
        self._worker_thread.join()

To start this class execute:

test = myClass(3)
test.work_to_do+=[1,2,3,4]
test._worker_thread.start()

And note that results start printing out. Deleting works the same as before. execute

del test

and the thread properly exits.