3

From what I understand, Python 3 does not make any guarantees about when the memory for an object is released except that it is not released as long as at least one non-weak reference points to an object.

Does the language make any other guarantees about the weak references? In particular, suppose I create a weakref wr pointing to some object. Suppose by some later time, all non-weak references to that object have been destroyed. Is it guaranteed that at that time wr would evaluate to None, or might it still evaluate to the original object for a while?

Apart from the language guarantees, is there any interesting implementation behavior around weakref?

max
  • 49,282
  • 56
  • 208
  • 355
  • 1
    I hope you aren't planning to write tricky code that relies upon the exact nuances of this behaviour. I've personally never needed `weakref` FWIW. – Karl Knechtel Mar 20 '12 at 23:23
  • I use weakkeydictionary and weakvaluedictionary all the time in PyQt to reference Widgets without keeping them from being destroyed by garbage collection. But...I also never ever rely on their validity. You should not expect a a weakref to remain valid beyond the point of all non-weakrefs reaching zero. – jdi Mar 20 '12 at 23:28
  • @KarlKnechtel: I want to avoid depending on anything that's not guaranteed. I don't mind relying on nuances if they are not implementation-specific, but are guaranteed by the language. – max Mar 20 '12 at 23:42

4 Answers4

1

A weakly referenced object will only be destroyed once GC kicks in. As this is non-deterministic, it is not guaranteed that it will be destroyed as soon as all strong references have been dropped.

From the docs:

...when the only remaining references to a referent are weak references, garbage collection is free to destroy the referent and reuse its memory for something else.

The operative phrase is "free to destroy the referent" - it may not happen immediately.

Mike Chamberlain
  • 39,692
  • 27
  • 110
  • 158
  • I thought maybe object destruction happens first, and is deterministic; but memory is freed later by GC, and this is not deterministic. But now that I think about it, it would be weird if they weren't done at the same time - why not free the memory if the interpreter already knows that the object is destroyed. – max Mar 21 '12 at 00:57
1

No, Python doesn't make any guarantees as to when an object is actually collected and the weakref will return None. It could happen right away (and it often is in CPython, which uses reference counting plus a cyclic-reference garbage collector, but much less often in other Python implementations that don't use refcounting.) It can also be delayed for a number of reasons -- when not using CPython (or using a version of CPython that doesn't use refcounting), or when using refcounting but when your objects participate in a reference cycle.

Thomas Wouters
  • 130,178
  • 23
  • 148
  • 122
  • This follows automatically from your answer at http://stackoverflow.com/a/9796689/336527, correct? There you explain that `__del__` may not be called arbitrarily long after last strong reference is deleted. And weakref cannot return `None` until `__del__` completed (if only because `__del__` may still revive the object). – max Mar 21 '12 at 01:04
0

The weakref either gives you a valid object (from Python's POV), or None. At which point an object is actually deallocated (causing the weakref to go to None) is undefined except that there cannot be any non-weak references to the object from any other live objects. The reference counting behavior you observe in CPython is not guaranteed.

Guido van Rossum
  • 16,690
  • 3
  • 46
  • 49
  • 1
    Thanks! Does your comment also imply that weakref won't evaluate to None until the object is actually deallocated? If such a guarantee exists, it would also imply that if one weakref evaluates to None, so do all of them. – max Mar 21 '12 at 19:02
  • The code that flips the weakrefs runs as part of the deallocation. The weakrefs are linked together and to the object. However this may be an implementation detail -- I don't want to guarantee that a weakref will never go to None before the object is deallocated -- especially since deallocation is not a concept in the language standard per se -- all that matters is whether an object is unreachable. The grayest area is probably whether it may sometimes be possible to recover an otherwise unreachable object by following a weakref -- I don't want to say that can't happen. – Guido van Rossum Mar 22 '12 at 04:23
  • Why do you consider this a gray area? Seems like it's not only possible, but is *sure* to happen if you allow weakref to stay valid after the last strong reference is gone. If at that point, I create a strong reference, I have just revived the object, haven't I (i.e., Python may not destroy it now). Am I missing something? I suppose Python could catch the attempt to assign a weakref to a strong reference, and update the weakref to None prior to the assignment. But such behavior would seem to go against the idea that checking for presence of strong references is never urgent. – max Mar 23 '12 at 02:51
  • What I'm trying to say is that there is no rule that says that Python needs to hurry to destroy an object to which no strong references exist. CPython, unless the object is part of a cycle, _will_ in fact hurry. But other Python implementations may use GC that doesn't detect the absence of live references immediately, and then it may be possible to revive an object via a weakref. However such "revival" doesn't do anything since the lack of strong references hadn't been detected yet so the object hadn't been destroyed yet. So an object with 0 strong references _may_ still be live. That's all. – Guido van Rossum Mar 23 '12 at 03:33
  • Thank you, this makes all the language guarantees perfectly clear. – max Mar 23 '12 at 03:44
  • Oh, quite the contrary. This answer fails to distinguish between liveness and reachability, so taken at face value, it implies that eligibility for garbage collection is just as much of an underspecified mess in Python as it is in JavaScript; compare . Though on the other hand, major implementations in practice seem to converge on defining eligibility for GC in terms of reachability after all. But this is nowhere guaranteed. – user3840170 Nov 18 '22 at 10:02
0

Since It seems that all weak references being released coherently is not guaranteed (as per Guido's post). One thing that you could do is to use a single weak reference across every location where you would otherwise generate a new weak reference (since the single reference must look the same to all that are using it).

Here you would lose the nice callback mechanism for all of the different references-to-references, but you could recover them by subclassing weakref.ref and adding some kind of registration scheme. You would also need some way of locating a weakref's single instance. Either package it with the instance to be referenced and make a helper method to get it or create-it (be aware of thread safety issues, use getattr with a default if you don't want locks too). Alternatively you could make a singleton to find them (use the first method...).

Coherent dissapearence in weakrefs seems like it should be guaranteed for the sake of sane memoization schemes, but using a single weakref should work albeit at the cost of reimplementing the weakref scheme of the interpreter with the guarantees you need.

you actually could use the .__weakref__ attribute, where weakrefs show up on instances within the interpreter and use the helper function method with that.

Lee McCuller
  • 86
  • 3
  • Why is the timing of "nullifying" of weakref a concern? As long as we know it's not guaranteed to happen at the same time, so we don't rely on it, it seems fine. What situation do you think about that would lead to insanity in memoization? – max Mar 23 '12 at 06:26
  • Lets say that two different places share a memoization cache and each one carries a weakref to it, each knowing to rebuild/start a new cache whenever the weakref flips. Without a guarantee that they both return None at the same time, they could start using different caches. I'll admit that this is a bit contrived, and indeed would likely never be seen in cPython, but I figured this would be a way to enforce the expectation that if one weakref returns None then so do all of them. – Lee McCuller Mar 25 '12 at 01:30
  • Wouldn't the (unique) cache manager object be linked with a strong reference, and only from that manager would a (single) weakref point to the actual data location? Maybe I'm missing a good reason for multiple weakref design, sorry - I'm just learning this stuff. – max Mar 26 '12 at 17:06