2

I am working on a project that has a similar construct to that shown in the code below. My hope is to have an object that opens a thread upon creation and automatically closes it when the object is destroyed. This works as expected when the object is instantiated within a function, but when the object is created in the global scope, __del__ is not being called, causing the program to hang.

import threading

def thread(event):
    event.wait()
    print("Terminating thread")

class ThreadManager:
    def __init__(self):
        self.event = threading.Event()
        self.thread = threading.Thread(target=thread, args=(self.event,))
        self.thread.start()

    def __del__(self):
        self.event.set()
        print("Event set")
        self.thread.join()

if __name__ == '__main__':
    print("Creating thread")
    manager = ThreadManager()
    #del manager

Unless I explicitly delete the manager object, the program hangs. I presume that the interpreter is waiting to delete the global objects until all non-daemon threads have completed, causing a deadlock situation.

My question is either, can someone confirm this and offer a workaround (I have read this page, so I'm not looking for a solution using a close() function or something similar, I am simply curious to hear ideas of an alternative that would perform automatic clean-up), or else refute it and tell me what I'm doing wrong?

juanpa.arrivillaga
  • 88,713
  • 10
  • 131
  • 172
TallChuck
  • 1,725
  • 11
  • 28

3 Answers3

0

Unlike languages like C++, Python doesn't destroy objects as soon as they go out of scope and that's why __del__ is unreliable. You can read more about that here: https://stackoverflow.com/a/35489349/5971137

As for a solution, I think this is the perfect case for a context manager (with):

>>> import threading
>>>
>>> def thread(event):
...     event.wait()
...     print("Terminating thread")
...
>>> class ThreadManager:
...     def __init__(self):
...             print("__init__ executed")
...             self.event = threading.Event()
...             self.thread = threading.Thread(target=thread, args=(self.event,))
...             self.thread.start()
...     def __enter__(self):
...             print("__enter__ executed")
...             return self
...     def __exit__(self, *args):
...             print("__exit__ executed")
...             self.event.set()
...             print("Event set")
...             self.thread.join()
...
>>> with ThreadManager() as thread_manager:
...     print(f"Within context manager, using {thread_manager}")
...
__init__ executed
__enter__ executed
Within context manager, using <__main__.ThreadManager object at 0x1049666d8>
__exit__ executed
Event set
Terminating thread

The print statements show the execution order works the way you want it to, where __exit__ is now your reliable cleanup method.

Cole
  • 1,715
  • 13
  • 23
  • a worthwhile suggestion, but I see context managers as being for one-time use objects, whereas in most cases it's not ideal to be constantly opening and closing threads – TallChuck Jan 28 '19 at 18:58
  • @TallChuck I believe that's why you have a `ThreadManager` though as it is the one-time use object with an `__exit__` method that sets the event for all threads to terminate. – Cole Jan 28 '19 at 19:04
  • 3
    Python DOES destroy objects as immediately when there is no reference for them, and reliability of `__del__` has been asserted for these cases. The problem in the question is that the code does keep a reference to the object - the `manager` variable itself. – jsbueno Jan 28 '19 at 19:09
  • @Cole I suppose I can see how you would infer that. Though in this snippet I used the name `ThreadManager`, my intended usage is basically to use a thread to manage network connections, so I need the thread to remain open until explicitly deleted. Using a `close()` function is perhaps the best way to do this, but I don't think a context manager fits my needs – TallChuck Jan 28 '19 at 19:09
  • otherwise, yes, a context manager is a nice solution for this case. – jsbueno Jan 28 '19 at 19:10
  • 1
    @jsbueno, as far as I can tell, the only reference is in the table of global variables, but it seems that that table is not cleared until all other (non-daemon) threads terminate, causing sort of a reverse chicken-and-egg situation – TallChuck Jan 28 '19 at 19:11
  • 1
    @jsbueno interesting –– everything I've read so far has said that Python will call `__del__` eventually, you just can't guarantee _when_. – Cole Jan 28 '19 at 19:12
  • 2
    When the reference count to the object reaches 0. It is called synchronously. But being in a global variable, so it is only really deleted on process tear-down, and them, there are problems - which is the context for this question. – jsbueno Jan 28 '19 at 19:15
  • 2
    @TallChuck: "until explicitly deleted" - if you want to make termination explicit, why are you putting it in `__del__`, the least explicit cleanup mechanism Python comes with? – user2357112 Jan 28 '19 at 20:54
  • @user2357112 I am trying to hide implementation details so the user of the class need not know that it uses threads underneath. Would you care to suggest something better? For basically all uses, `__del__` seems to work great as a destructor, it's seems to be only in the case of global objects that it fails. – TallChuck Jan 28 '19 at 23:57
0

This behavior is actually specifically mentioned in the official documentation for object.__del__():

It is not guaranteed that __del__() methods are called for objects that still exist when the interpreter exits.

TallChuck
  • 1,725
  • 11
  • 28
-1

Use:

self.thread = threading.Thread(target=thread, args=(self.event,), daemon=True)

This won't block the exit of the process.

Bobax
  • 55
  • 1
  • 5