Use cases for __del__

Question

What are use cases in python 3 of writing a custom __del__ method or relying on one from stdlib¹? That is, in what scenario is it reasonably safe, and can do something that's hard to do without it?

For many good reasons (1 2 3 4 5 6), the usual recommendation is to avoid __del__ and instead use context managers or perform the cleanup manually:

1. __del__ is not guaranteed to be called if objects are alive on intrepreter exit².
2. At the point one expects the object can be destroyed, the ref count may actually be non-zero (e.g., a reference may survive through a traceback frame held onto by a calling function). This makes the destruction time far more uncertain than the mere unpredictability of gc implies.
3. Garbage collector cannot get rid of cycles if they include more than 1 object with __del__
4. The code inside __del__ must be written super carefully:
   • object attributes set in __init__ may not be present since __init__ might have raised an exception;
   • exceptions are ignored (only printed to stderr);
   • globals may no longer be available.

---

Update:

PEP 442 has made significant improvements in the behavior of __del__. It seems though that my points 1-4 are still valid?

---

Update 2:

Some of the top python libraries embrace the use of __del__ in the post-PEP 442 python (i.e., python 3.4+). I guess my point 3 is no longer valid after PEP 442, and the other points are accepted as unavoidable complexity of object finalization.

---

¹I expanded the question from just writing a custom __del__ method to include relying on __del__ from stdlib.

²It seems that __del__ is always called on interpreter exit in the more recent versions of Cpython (does anyone have a counter-example?). However, it doesn't matter for the purpose of __del__'s usablity: the docs explicitly provide no guarantee about this behavior, so one cannot rely on it (it may change in future versions, and it may be different in non-CPython interpreters).

Answer 1

Context managers (and try/finally blocks) are somewhat more restrictive than __del__. In general they require you to structure your code in such a way that the lifetime of the resource you need to free doesn't extend beyond a single function call at some level in the call stack, rather than, say, binding it to the lifetime of a class instance that could be destroyed at unpredictable times and places. It's usually a good thing to restrict the lifetime of resources to one scope, but there sometimes edge cases where this pattern is an awkward fit.

The only case where I've used __del__ (aside from for debugging, c.f. @MSeifert's answer) is for freeing memory allocated outside of Python by an external library. Because of the design of the library I was wrapping, it was difficult to avoid having a large number of objects that held pointers to heap-allocated memory. Using a __del__ method to free the pointers was the easiest way to do cleanup, since it would have been impractical to enclose the lifespan of each instance inside a context manager.

Answer 2

One use-case is debugging. If you want to track the lifetime of a specific object it's convenient to write a temporary __del__ method. It can be used to do some logging or just to print something. I have used it a few times especially when I was interested in when and how instances are deleted. It's sometimes good to know when you create and discard a lot of temporary instances. But as I said I only ever used this to satisfy my curiosity or when debugging.

Another use-case is subclassing a class that defines a __del__ method. Sometimes you find a class that you want to subclass but the internals require you to actually override __del__ to control the order in which the instance is cleaned up. That's very rare too because you need to find a class with __del__, you nee to subclass it and you need to introduced some internals that actually require to call the superclass __del__ at exactly the right time. I actually did that once but I don't remember where and why it was important (maybe I didn't even know about alternatives then, so treat this as possible use-case).

When you wrap an external object (for example a c object that isn't tracked by Python) that really, really needs to be deallocated even if someone "forgets" (I suspect a lot of people just omit them on purpose!) to use the context manager that you provided.

---

However all these cases are (or should be) very, very rare. Actually it's a bit like with metaclasses: They are fun and it's really cool to understand the concepts because you can probe the "fun parts" of python. But in practice:

If you wonder whether you need them [metaclasses], you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why).

Citation (probably) from Tim Peters (I haven't found the original reference).

Answer 3

One case where i always use __del__, is for closing a aiohttp.ClientSession object.

When you don't, aiohttp will print warnings about the unclosed client session.