How to kill a Threading pool from parent?

Question

I have threading classing that has the following run function. So when this class is set to run it keeps on checking a multiprocessing manager queue, if there is anything inside it, it starts the pool to run the job(track function). Upon completion of the job, pool closes automatically and the whole queue if not empty check starts.

def runQueue(self):
    print("The current thread is", threading.currentThread().getName())
    while True:
        time.sleep(1)
        self.pstate=False
        if self.runStop: #this stops the whole threading by dropping main loop
            break
        while not self.tasks.empty():
            self.pstate=True
            task = self.tasks.get()
            with ThreadPool(processes=1) as p: #<- want to kill this pool
                ans = p.apply(self.track, args=(task,))
                self.queueSend(ans)
                self.tasks.task_done()
    print("finished job")

I used the pool because the function returns a value which I need to map. What I am looking for is a way such that, upon some parent call, the pool closes by dropping the job, while keeping the primary class thread (run function [main loop] running). Any kind of help is appreciated.

I was reading something about raising an exception, but don't know how that works. — Harsh Nagarkar, Jul 18 '19 at 05:58
have you looked at [this](https://stackoverflow.com/questions/38857379/stopping-processes-in-threadpool-in-python) question? it might have an answer for you — Rotem Tal, Jul 18 '19 at 08:36
I am reading the article, but I don't understand it completely. Would you help me out? I need to first change my code to use multiprocessing processing instead of the pool and then I don't understand the exception part. @rotemtal — Harsh Nagarkar, Jul 18 '19 at 16:23
basically, you need to encapsulate your processing (by processes or threads) in a try-except block, upon an exception (in the parent process, at the try-except block) you merely call `terminate()` on the thread pool or each process. If this does answer your requirements modifying your code should be easy, let me know — Rotem Tal, Jul 19 '19 at 07:17

score 0 · Accepted Answer · answered Jul 20 '19 at 14:26

I found that for my case pool.terminate would work only I/O applications, I did find some solutions online which were not related to the pool but I could implement. One solution is to run the thread as a multiprocessing process and then call process.terminate() or using multiprocessing Pool and then call pool.terminate.
Note that multiprocessing is faster for CPU intensive tasks. If the tasks are I/O intensive threads are the best solution.
The only way I found a way to kill the thread is using win32 ctypes module. If you start a thread and get it's tid with

tid thread.ident

then you can put your in kill_thread(tid) function below

w32 = ctypes.windll.kernel32
THREAD_TERMINATE = 1 
def kill_thread(tid):
        handle = w32.OpenThread(THREAD_TERMINATE, False,tid)
        result = w32.TerminateThread(handle, 0)
        w32.CloseHandle(handle)

Hope this helps someone.

How to kill a Threading pool from parent?

1 Answers1