Tensorflow variable initialization in a loop

Question

I am trying a Tensorflow example:

import tensorflow as tf
x = tf.Variable(0, name='x')
model = tf.global_variables_initializer()
with tf.Session() as session:
    for i in range(5):
        session.run(model)
        x = x + 1
        print(session.run(x))

But the output is unexpected. I expected it to output:

1 1 1 1 1

BUt the actual output is:

1 2 3 4 5

Why is this? session.run(model) will initialize variable each time, is this statement correct?

Good question. Depends on the implementation. It seems valid to me that the initialization is done only once in a session. — RuDevel, Feb 28 '17 at 00:04

score 2 · Answer 1 · edited May 23 '17 at 12:02

session.run(model) will initialize variable each time

That is correct. The issue is that each time x = x + 1 a new addition is created in the graph, which explains the results you obtain.

Graph after the first iteration:

After the second iteration:

After the third iteration:

After the fourth iteration:

After the fifth iteration:

The code I use, mostly taken from Yaroslav Bulatov's answer in How can I list all Tensorflow variables a node depends on?:

import tensorflow as tf

import matplotlib.pyplot as plt
import networkx as nx

def children(op):
  return set(op for out in op.outputs for op in out.consumers())

def get_graph():
  """Creates dictionary {node: {child1, child2, ..},..} for current
  TensorFlow graph. Result is compatible with networkx/toposort"""

  ops = tf.get_default_graph().get_operations()
  return {op: children(op) for op in ops}

def plot_graph(G):
    '''Plot a DAG using NetworkX'''        
    def mapping(node):
        return node.name
    G = nx.DiGraph(G)
    nx.relabel_nodes(G, mapping, copy=False)
    nx.draw(G, cmap = plt.get_cmap('jet'), with_labels = True)
    plt.show()


x = tf.Variable(0, name='x')
model = tf.global_variables_initializer()
with tf.Session() as session:
    for i in range(5):
        session.run(model)
        x = x + 1
        print(session.run(x))

        plot_graph(get_graph())

Would you like to add a few comments after each graph? – derek Feb 28 '17 at 07:10 — derek, Feb 28 '17 at 07:10

score 0 · Answer 2 · answered Mar 28 '17 at 15:28

It seems as you have to initialize it inside the loop as such

import tensorflow as tf

for i in range(5):
    X = tf.Variable(0, name='x')
    model = tf.global_variables_initializer()
    with tf.Session() as sess:
        sess.run(model)
        X += 1
        print(X.eval())

score 0 · Answer 3 · edited Aug 30 '20 at 07:12

session.run(model)

does initialize variables each time (as it calls model = tf.global_variables_initializer()), but the value of x which is to be used for initialisation is incremented by 1 for each entry into the loop. For example,

for i=0, x is initialised with the value it possesses at that instance, i.e. 0. When i=1, x has been incremented to 1, and that is the value which the initialiser will use, and so on.

Tensorflow variable initialization in a loop

3 Answers3