Is it necessary to close session after tensorflow InteractiveSession()

Question

I have a question about InteractiveSession in Tensorflow

I know tf.InteractiveSession() is just convenient syntactic sugar for keeping a default session open and basically work the same like below:

with tf.Session() as sess:
    # Do something

However, I have seen some examples online, they did't call close() at the end of the code after using InteractiveSession.

Question:
1. Would it caused any problem without closing the session like session leak?
2. How the GC work for the InteractiveSession if we don't close it?

[with](https://docs.python.org/3/reference/datamodel.html#context-managers) uses a context manager which already handles the entry and exit from its block of code. So no, you don't need to call `close()`. — , May 08 '18 at 09:25
He is talking about not closing after `tf.InteractiveSession()` — Patwie, May 08 '18 at 13:58

Patwie · Accepted Answer · 2018-09-13T16:03:33.973

Yes, tf.InteractiveSession is just convenient syntactic sugar for keeping a default session open.

The Session implementation has a comment

Calling this method frees all resources associated with the session.

A quick test

#! /usr/bin/env python
# -*- coding: utf-8 -*-


import argparse
import tensorflow as tf
import numpy as np


def open_interactive_session():
    A = tf.Variable(np.random.randn(16, 255, 255, 3).astype(np.float32))
    sess = tf.InteractiveSession()
    sess.run(tf.global_variables_initializer())


def open_and_close_interactive_session():
    A = tf.Variable(np.random.randn(16, 255, 255, 3).astype(np.float32))
    sess = tf.InteractiveSession()
    sess.run(tf.global_variables_initializer())
    sess.close()


def open_and_close_session():
    A = tf.Variable(np.random.randn(16, 255, 255, 3).astype(np.float32))
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--num', help='repeat', type=int, default=5)
    parser.add_argument('type', choices=['interactive', 'interactive_close', 'normal'])
    args = parser.parse_args()

    sess_func = open_and_close_session

    if args.type == 'interactive':
        sess_func = open_interactive_session
    elif args.type == 'interactive_close':
        sess_func = open_and_close_interactive_session

    for _ in range(args.num):
        sess_func()
    with tf.Session() as sess:
        print("bytes used=", sess.run(tf.contrib.memory_stats.BytesInUse()))

gives

"""
python example_session2.py interactive
('bytes used=', 405776640)
python example_session2.py interactive_close
('bytes used=', 7680)
python example_session2.py
('bytes used=', 7680)
"""

This provokes a session-leak, when not closing the session.~~Note, even when closing the session, there is currently bug in TensorFlow which keep 1280 bytes per session see Tensorflow leaks 1280 bytes with each session opened and closed?.~~ (This has been fixed now).

Further, there is some logic in the __del__ trying to start the GC.

Interestingly, I never saw the warning

An interactive session is already active. This can cause out-of-memory errors in some cases. You must explicitly call InteractiveSession.close() to release resources held by the other session(s)

which seems to be implemented. It guess the only raison d'être of the InteractiveSession is its usage in Jupyter Notebookfiles or inactive shells in combination with .eval(). But I advised against using eval (see Official ZeroOut gradient example error: AttributeError: 'list' object has no attribute 'eval')

However, I have seen some examples online, they did't call close() at the end of the code after using InteractiveSession.

And I am not surprised by that. Guess how many code snippets are the without a free or delete after some malloc. Bless the OS that it frees up the memory.

Your example and explanation above are amazing!! One more question, (a little bit off topic from this question), would it free the memory after stopping the kernel in Jupyter notebook even if without closing the session? — R.yan, May 08 '18 at 14:20
Honestly, I don't know it exactly but I bet yes. According the jupyter documentation "Note that the RAM is not released until the kernel is shut-down." it is very likely. There is no reason to keep the Python process and TensorFlow session alive. So the OS once more will probably pick up the pieces and clear the memory. — Patwie, May 08 '18 at 18:27

Is it necessary to close session after tensorflow InteractiveSession()

1 Answers1

Linked