11

I was previously using the threading.Thread module. Now I'm using concurrent.futures -> ThreadPoolExecutor. Previously, I was using the following code to exit/kill/finish a thread:

def terminate_thread(thread):
    """Terminates a python thread from another thread.

    :param thread: a threading.Thread instance
    """
    if not thread.isAlive():
        return

    exc = ctypes.py_object(SystemExit)
    res = ctypes.pythonapi.PyThreadState_SetAsyncExc(
        ctypes.c_long(thread.ident), exc)
    if res == 0:
        raise ValueError("nonexistent thread id")
    elif res > 1:
        # """if it returns a number greater than one, you're in trouble,
        # and you should call it again with exc=NULL to revert the effect"""
        ctypes.pythonapi.PyThreadState_SetAsyncExc(thread.ident, None)
        raise SystemError("PyThreadState_SetAsyncExc failed")

This doesn't appear to be working with futures interface. What's the best practice here? Just return? My threads are controlling Selenium instances. I need to make sure that when I kill a thread, the Selenium instance is torn down.

Edit: I had already seen the post that is referenced as duplicate. It's insufficient because when you venture into something like futures, behaviors can be radically different. In the case of the previous threading module, my terminate_thread function is acceptable and not applicable to the criticism of the other q/a. It's not the same as "killing". Please take a look at the code I posted to see that.

I don't want to kill. I want to check if its still alive and gracefully exit the thread in the most proper way. How to do with futures?

xendi
  • 2,332
  • 5
  • 40
  • 64
  • someone pls upvote against whoever did that. I've searched around and can't find an example. It's not uncommon or unreasonable to want to kill a thread. – xendi Oct 03 '18 at 16:25
  • Possible duplicate of [Is there any way to kill a Thread in Python?](https://stackoverflow.com/questions/323972/is-there-any-way-to-kill-a-thread-in-python) – Athena Oct 03 '18 at 16:28
  • 1
    it looks like that was copied from https://stackoverflow.com/questions/323972/is-there-any-way-to-kill-a-thread-in-python did you also implement the `StoppableThread`? – Athena Oct 03 '18 at 16:30
  • question edited – xendi Oct 03 '18 at 16:44
  • @Ares : yes, I had previously copied it from there. However, it does, as far as I can tell, conform to the graceful exit criticism in the other answers. Now I'm using futures though and there are no examples for that. – xendi Oct 03 '18 at 16:46
  • The accepted answer here is how my futures threading is working: https://stackoverflow.com/questions/52620563/python-threadpoolexecutor-blocking-how-to-unblock – xendi Oct 03 '18 at 16:48
  • 1
    I see one problem with it. Before I was storing my threads in a list. Now I'm not. Maybe I can store the futures too and pass them in. – xendi Oct 03 '18 at 16:58
  • There are better ways for me to ask this. Guess I actually will close it – xendi Oct 03 '18 at 17:22
  • You probably want to edit your question a bit to clarify, but I've taken a stab at an answer – Athena Oct 03 '18 at 17:35
  • Did you ever find a solution? I think i am having a similar issue: https://stackoverflow.com/questions/63208371/python-threadpoolexecutor-close-all-threads-when-i-get-a-result?noredirect=1#comment111776768_63208371 – Je Je Aug 02 '20 at 11:36
  • This was so long ago now that I don't even remember what I was doing. These days I use asyncio more than anything and thread my async scripts only when absolutely necessary. I'll see if I can dig up my original script though and judge the answer by sargeATM. Does it help you @NonoLondon ? – xendi Aug 08 '20 at 01:24
  • i couldn't make it work. someonelse suggested asyncio, but seems like you cant use requests with it, so will multiply the work. Tx for your answer. – Je Je Aug 08 '20 at 17:14

2 Answers2

2

If you want to let the threads finish their current work use:

thread_executor.shutdown(wait=True)

If you want to bash the current futures being run on the head and stop all ...future...(heh) futures use:

thread_executor.shutdown(wait=False)
for t in thread_executor._threads:
    terminate_thread(t)

This uses your terminate_thread function to call an exception in the threads in the thread pool executor. Those futures that were disrupted will return with the exception set.

SargeATM
  • 2,483
  • 14
  • 24
  • 2
    `shutdown()` does not suddenly stop the thread until "[the currently pending futures are done executing](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.shutdown)". In addition, with `wait=False`, still waits until "*when all pending futures are done executing*". – howdoicode Jan 24 '21 at 13:45
  • "If _wait is False_ then this method _will return immediately_ and the resources associated with the executor will be freed when all pending futures are done executing" – SargeATM Aug 17 '22 at 15:49
-2

How about .cancel() on the thread result?

cancel() Attempt to cancel the call. If the call is currently being executed and cannot be cancelled then the method will return False, otherwise the call will be cancelled and the method will return True.

https://docs.python.org/3/library/concurrent.futures.html

Athena
  • 3,200
  • 3
  • 27
  • 35
  • You would need to cancel every future and any future that is currently running isn't guaranteed to be stopped by `.cancel()` – SargeATM Oct 05 '18 at 22:42