0

I'm having an issue where the Python destructor for a Tensorflow session object is not being called.

Consider the following code, run under Python 3.4+:

import tensorflow as tf

# CODE THAT CHANGES BEHAVIOR
datashape = [1]
y = tf.ones(shape=datashape)

sess = tf.Session()
sess.delete2 = sess.__del__

def override(p, methods):
    oldType = type(p)
    newType = type(oldType.__name__ + "_Override", (oldType,), methods)
    p.__class__ = newType

def p(self):
    print("__del__ called")
    self.delete2()

def monkey(x):
    override(x, {"__del__": p})

monkey(sess)
print("Deleting sess:")

(We got the monkeypatching code from Monkey patch __del__ to new function. You can also reproduce this without it by installing TensorFlow in a virtualenv and directly editing the source https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/client/session.py#L686)

When we comment out the two lines following the # CODE THAT CHANGES BEHAVIOR comment, upon interpreter exit, we see __del__ called printed.

When we leave those two lines out, we don't see __del__ called printed.

Additionally, if you wrap all the code inside a function (e.g. using the __main__ convention) we also see __del__ called.

Note: according to the accepted answer in Why aren't destructors guaranteed to be called on interpreter exit?, Python 3.4+ guarantees that destructors are always called on interpreter exit.

Why does this behavior occur, and how can we fix it?

Grant Wu
  • 47
  • 5
  • The accepted answer on that other question is wrong. Python 3.4+ makes no such guarantee. It tries pretty hard to call destructors, but it makes no guarantee. – user2357112 Jul 24 '18 at 16:18
  • @user2357112 do you have a post PEP 442 source on that? [PEP 442](https://www.python.org/dev/peps/pep-0442/) does say "Following this scheme, an object's finalizer is always called exactly once, even if it was resurrected afterwards." – Grant Wu Jul 24 '18 at 16:26
  • 1
    [Example](https://ideone.com/2UvmrW) of no destructor call, even post PEP 442, based on sebix's answer down the page. (I'm not sure why this example works.) – user2357112 Jul 24 '18 at 16:28
  • 1
    While we're at it, here's an [example](https://ideone.com/KTgmxM) of repeat finalization even post-PEP 442. I *do* understand this example - the object can't participate in a reference cycle, so it doesn't carry GC metadata, including the flag that's needed to prevent repeat finalization. – user2357112 Jul 24 '18 at 16:32
  • 1
    PEP 442 only covers what happens when an object actually goes through the GC system. The things it says may not apply if an object doesn't get GC'ed. – user2357112 Jul 24 '18 at 16:35
  • If you want to summarize this and put it into an answer, I'd be willing to accept it. – Grant Wu Jul 24 '18 at 16:53
  • @user2357112 wait, can you clarify what you mean by "the object can't participate in a reference cycle"? Do you mean by "it keeps on reviving a reference to itself, but not in a way such that it's a reference cycle"? – Grant Wu Jul 24 '18 at 17:39
  • 1
    Let me correct that a bit - Python *thinks* the object can't participate in a reference cycle, because it has no dict and empty `__slots__`, so Python thinks the object just can't hold references to other objects at all. (Either they forgot about reference cycles through the type pointer, or they deliberately ignored such reference cycles for some reason.) – user2357112 Jul 24 '18 at 17:47
  • @user2357112 Sorry, what is a type pointer here? Near I can tell it's something to do with the Python C API... – Grant Wu Jul 24 '18 at 22:12
  • It's the reference from some object `obj` to `type(obj)`. – user2357112 Jul 24 '18 at 22:15

0 Answers0