Are the operations in tf.group executed in order?

Question

Are the operations in tf.group() executed in order?

If they are not executed in order, is there a similar operation that makes them run in order? Or is there a clean way to run them in order?

My aim is to run operations A and B many times over and over again, i.e.

sess.run(A)
sess.run(B)
sess.run(A)
sess.run(B)
sess.run(A)
sess.run(B)
sess.run(A)
sess.run(B)
...

Lior · Accepted Answer · 2017-12-29T13:25:22.723

The operations do not necessarily execute in order.

Here is a test that proves this:

import tensorflow as tf
sess = tf.InteractiveSession()
a = tf.Variable(1.0)
b = tf.Variable(10.0)
c = tf.Variable(0.0)
grp = tf.group(tf.assign(c, a), tf.assign(c, b)) # this is the group op
for i in range(100):
    sess.run(tf.global_variables_initializer()) # initialize c each time
    sess.run(grp) # run the group op
    print(sess.run(c)) # observe results

When I run this on a cpu, I get that some iterations produce 1.0 and some 10.0.

tf.group does not require the operations to be on the same device, which means that they could not be expected to follow an order.

If you want the operations to execute in order, make sure to build them with control_dependencies

import tensorflow as tf
sess = tf.InteractiveSession()
a = tf.Variable(1.0)
b = tf.Variable(10.0)
c = tf.Variable(0.0)
op1  = tf.assign(c, a)
with tf.get_default_graph().control_dependencies([op1]):
    op2 = tf.assign(c, b) # op2 will execute only after op1
grp = tf.group(op1,op2)
for i in range(100):
    sess.run(tf.global_variables_initializer())
    sess.run(grp)
    print(sess.run(c))

Thanks for the answer. It works, but I do not fully understand the workings of it. Does does the control dependency only assert itself when the operations are grouped? I can still run sess.run(op2) on its own, without running sess.run(op1). — Ginger, Dec 29 '17 at 07:40
No. The control dependency is forced also when the operations are not grouped. To see this, switch in my example the ops to be: `op1=tf.assign(c,c+a)` and `op2=tf.assign(c,c*b)`, and then run `sess.run(op2)`. You will see that `op2` always assigns into `c` the value `10.0`, which means that when you run `op2`, `op1` also runs automatically prior to `op2` — Lior, Dec 29 '17 at 13:31
That makes sense now. Thanks for the excellent explanation and examples. You're a real asset to the StackOverflow community! — Ginger, Dec 29 '17 at 21:50

Are the operations in tf.group executed in order?

1 Answers1

Linked