I have a graph with GBs of variables, and a random function (e.g. VAE). I'd like to be able to run a function, and always use the same random sequence (e.g. feed a large number of x, and always get the exact same z and y).
I can achieve this using random seeds, such that I always get the same sequence every time I run the script from scratch (i.e. initialise the session). However I'd like to be able to reset the random sequence without destroying the session, so I can call my function over and over (and get the same sequence). Destroying and reinitialising session isn't really ideal as I have GBs of variables which I lose, and reloading each time is a waste. Setting the random seed again (tf.set_random_seed) doesn't seem to have an affect (I presume the seed from tf.set_random_seed is somehow combined with the op seed and baked into the op when it's created?)
Is there any way around this?
I've read the documentation and a ton of posts regarding random seeds in tensorflow (e.g. TensorFlow: Resetting the seed to a constant value does not yield repeating results, Tensorflow `set_random_seed` not working, TensorFlow: Non-repeatable results, how to get reproducible result in Tensorflow, How to get stable results with TensorFlow, setting random seed) however I can't get the behaviour I'm after.
E.g. toy code
import tensorflow as tf
tf.set_random_seed(0)
a = tf.random_uniform([1], seed=1)
def foo(s, a, msg):
with s.as_default(): print msg, a.eval(), a.eval(), a.eval(), a.eval()
s = tf.Session()
foo(s, a, 'run1 (first session):')
# resetting seed does not reset sequence. is there anything else I can do?
tf.set_random_seed(0)
foo(s, a, 'run2 (same session) :')
# reinitialising session works, but how to do this without closing session and reinitializing?
# and losing GBs of variables which I'd rather not reload
s.close()
s = tf.Session()
foo(s, a, 'run3 (new session) :')
gives results:
run1 (first session): [ 0.53973019] [ 0.54001355] [ 0.43089259] [ 0.30245078]
run2 (same session) : [ 0.112077] [ 0.99792898] [ 0.17628896] [ 0.13141966]
run3 (new session) : [ 0.53973019] [ 0.54001355] [ 0.43089259] [ 0.30245078]
