0

Surfing on the internet (here) I found that there are some problems to collect objects with __del__ method for the garbage collector.
My doubt is simple: why?

According to the documentation:

Objects that have __del__() methods and are part of a reference cycle cause the entire reference cycle to be uncollectable, including objects not necessarily in the cycle but reachable only from it. Python doesn’t collect such cycles automatically because, in general, it isn’t possible for Python to guess a safe order in which to run the __del__() methods.

Why is the __del__ method so problematic? What's the difference between an object that implements it and one which doesn't? It only destroys an instance.

zer0uno
  • 7,521
  • 13
  • 57
  • 86
  • 1
    This case is specifically covered in [Martijn's answer to your previous question](http://stackoverflow.com/a/24587590/3001761); the garbage collector *does* "detect it and reclaim memory". What has made you think it doesn't? – jonrsharpe Jul 06 '14 at 13:44
  • Yes, I know, but according to the link that I posted: 'Since there is no good solution to this problem, cycles that are referenced from objects with finalizers are not freed. Instead these objects are added to a global list of uncollectable garbage.' – zer0uno Jul 06 '14 at 13:46
  • But lists don't have a finaliser... Read [this](http://stackoverflow.com/a/2452895/3001761). – jonrsharpe Jul 06 '14 at 13:56
  • @jonrsharpe This question is not answered by the previous answer you linked. That answer explains Python's reference counting and its cycle breaker, but doesn't cover objects with `__del__` being specifically excluded from cycle-breaking, which is what this question is about. – user4815162342 Jul 06 '14 at 21:26
  • @user4815162342 you should review the edits - the question has changed – jonrsharpe Jul 06 '14 at 21:27
  • @jonrsharpe You're right, and only now do I understand your previously puzzling remark about lists not having a finalizer! – user4815162342 Jul 06 '14 at 21:36

1 Answers1

4

__del__ doesn't destroy an instance, it is automatically destroyed by the python runtime once its reference count reaches zero. __del__ allows you to hook into that process and perform additional actions, such as freeing external resources associated with the object.

The danger is that the additional action may even resurrect the object - for example, by storing it into a global container. In that case, the destruction is effectively cancelled (until the next time the object's reference count drops to zero). It is this scenario that causes the mere presence of __del__ to exclude the object from those governed by the cycle breaker (also known as the garbage collector). If the collector invoked __del__ on all objects in the cycle, and one of them decided to resurrect the object, this would need to resurrect the whole cycle - which is impossible since __del__ method of other cycle members have already been invoked, possibly causing permanent damage to their objects (e.g. by freeing external resources, as mentioned above).

If you only need to be notified of object's destruction, use weakref.ref. If your object is associated with external resources that need freeing, implement a close method and/or a context manager interface. There is almost never a legitimate reason to use __del__.

user4815162342
  • 141,790
  • 18
  • 296
  • 355
  • I understood about objects with cyclic reference but this made me think...let's suppose I have a simple object which doesn't contain cyclic reference when its reference counter drops to zero the `__del__` method is invoked. If in the `__del__` method it is resurrected by storing it into a global container then it cant be deallocated but it will, because its reference counter is zero. So even with no-containing-cyclic-reference object do we have problems? – zer0uno Jul 06 '14 at 15:57
  • 1
    @antox Once it is stored into the global container, its reference count is no longer zero. So, the algorithm is something like (extremely simplified): `if (obj->refcnt == 0) { run___del__(obj); if (obj->refcnt == 0) free(obj); }`. See [the source](http://hg.python.org/cpython/file/3881c12fa3ae/Objects/typeobject.c#l979) for the gory details. – user4815162342 Jul 06 '14 at 18:35
  • 1 question came to my mind. If I have cyclic references between 2 or more objects, the problem described above there is only if 2 or more objects have defined a `__del__` method, but if just 1 object has the `__del__` and others not I think that there aren't problems to collect them. Is that right? – zer0uno Jul 08 '14 at 14:04
  • 1
    @antox No, the problem is exactly the same. Resurrecting the object would need to resurrect all members of the cycle, and that cannot be done if they were already cleared by the cycle-breaker and reclaimed by the call to `free()`. – user4815162342 Jul 08 '14 at 15:04
  • where the `free()` method is contained? – zer0uno Jul 08 '14 at 15:12
  • @antox In the implementation of `tp_dealloc` type struct member, which is called when the object's reference count reaches zero. – user4815162342 Jul 08 '14 at 15:18
  • Is the finalized method called only once? I mean if the finalizer resurects the object then when this object reference count drops again to zero, is the finalizer called again? – zer0uno Jul 14 '14 at 10:13
  • @antox I haven't checked, but I expect the finalizer is called again because the system doesn't keep track of it being called previously. If you have more questions, feel free to post a new top-level question. I will not be responding here, except in case of mistakes found in the answer. – user4815162342 Jul 14 '14 at 13:15
  • A single `__del__` in a cycle would work just fine. The cycle breaker would have to call the `__del__` first before freeing any other object. Then when `__del__` resurrects the object the GC stops and everything is fine. This also shows why 2 `__del__` can't work because they can't both be called first. – Goswin von Brederlow Jun 01 '15 at 13:16
  • @GoswinvonBrederlow are you talking about python 3.3- or 3.4+ (post-PEP 442)? – max Jun 07 '17 at 00:13
  • @max I'm talking about theory. – Goswin von Brederlow Jul 04 '17 at 09:55