How to reference finalized object in weakref.finalize?

Question

I have a class (that I do not control) that doesn't implement its own cleanup. I thought that this is one of the cases that weakref.finalize is for, but I can't get it working.

def cleanup(obj):
    print('Cleanup obj')
    if not obj.is_closed:
        obj.close()
...

def make_obj():
    obj = SomeClass()

    # this creates an extra ref, so cleanup is never run
    weakref.finalize(obj, cleanup, obj)

    # this always results in ReferenceError; obj is already gone when cleanup is called
    weakref.finalize(obj, cleanup, weakref.proxy(obj))

Am I doing something wrong? What have I misunderstood?

I know that this has been a while since you asked, but does passing the `obj.close` method as the callable arg to `finalize()` work? E.g. `weakref.finalize(obj, obj.close)`. You wouldn't be able to do the state check before calling the method, but if that's just to avoid an exception, I believe the finalizer will swallow that up. — SuperShoot, Apr 12 '19 at 07:23
@SuperShoot: That wouldn't work because `obj.close` is a bound method, and bound methods contain a strong reference to the object they were called on (in this case, `obj`); as a result, `obj` would always have a live reference stored within the `finalizer` itself, which would never be released to be finalized. In theory, the docs wouldn't forbid the `weakref.proxy` solution the OP used, but in practice, the `proxy` seems to be finalized first, so by the time the `finalizer` calls the finalization function, you get the `ReferenceError` the OP sees. — ShadowRanger, Apr 06 '22 at 15:37

score 6 · Accepted Answer · answered Oct 04 '19 at 22:01

6

It is impossible to reference finalized object in weakref.finalize. In "Note" section for weakref.finalize it is stated:

It is important to ensure that func, args and kwargs do not own any references to obj, either directly or indirectly, since otherwise obj will never be garbage collected. In particular, func should not be a bound method of obj.

Thus, it is also impossible to orginize cleanup with bound method like weakref.finalize(obj, obj.close). In such case you need to call cleanup function yourself. Another option is to inherit from SomeClass and make proper __del__ method.

answered Oct 04 '19 at 22:01

sanyassh

8,100
13
36
70

1

Yea, I just covered this case because it was proposed in the comment from SuperShoot to your question. – sanyassh Oct 04 '19 at 22:06
But it doesn’t in any way answer the question – OrangeDog Oct 04 '19 at 22:07
Sometimes the answer is "No, it is not possible". I wanted to show that it is not possibble nor with your two methods from the question body, neither from the comment. – sanyassh Oct 04 '19 at 22:10
The what is the point of finalize if what it says it’s for is impossible? – OrangeDog Oct 04 '19 at 22:11
`Finalize` cannot be used for every possible case. I already mentioned `__del__` method. `Finalize` and `__del__` sometimes can be used interchangebly (see this section for examples: https://docs.python.org/3/library/weakref.html#comparing-finalizers-with-del-methods), sometimes only `finalize` (when you don't have a class to define `__del__`) and sometimes only `__del__` is available. Looks like your case is for `__del__` only. – sanyassh Oct 04 '19 at 22:28
Oh, args of `finalize`, not args of `func`. I see now. So the solution is to monkeypatch `SomeClass.__del__`? – OrangeDog Oct 07 '19 at 09:44
1

Yes, inherit, add `__del__` method and use this child class if you can or monkeypatch otherwise. Something like this thread https://stackoverflow.com/questions/30294458/any-elegant-way-to-add-a-method-to-an-existing-object-in-python will help in monkeypatching. – sanyassh Oct 07 '19 at 11:58
Why doesn't it work to use `weakref.proxy`? The object will still be garbage-collected since we're not increasing the number of strong references to it. – Daniel Walker Apr 06 '22 at 14:34
1

@DanielWalker because it gets garbage-collected before the finalizer runs – OrangeDog Apr 07 '22 at 10:24
Is that required behavior or just how CPython does it? I looked at the docs and I didn’t see anything concrete about the order. – Daniel Walker Apr 07 '22 at 11:48
1

Another possibility is to create a proxy class, that holds the obj. Then to add a weakref finalizer to the proxy class, that has `obj.close` as the target. That way it is still available. – Chris Jun 24 '22 at 03:42
I'm afraid the answer is probably right - it's not possible to do this with weakref. As @Chris mentions, you can use a wrapper class around the object to be finalized, but that's not optimal. If the caller isolates the object and the wrapper class is garbage collected, the finalizer threatens to be applied prematurely, which would result in a use after free scenario for a C extensions use case. It would be much nicer if the finalizer could just make the object available to the function, but I guess that opens problems with possible object resurrection. – mara004 Nov 19 '22 at 16:23
Please note that `__del__` is not a drop-in replacement for `finalize()` because it can be invoked in an arbitrary order, while arbitrary code is executed, and from any arbitrary thread. This makes it completely unsuitable for a C extension that is not thread-safe, and whose close functions need to be applied in correct order (reverse to loading). `finalize()` is much better in this regard. – mara004 Nov 19 '22 at 16:26

How to reference finalized object in weakref.finalize?

1 Answers1