2

I'm trying to get reproducible results with Keras, however every time I run the program I get different results.

I've set the python hash seed, the Numpy random seed, the random seed, the TensorFlow seed, and the kernel_initializer glorot_uniform seed, but I still don't get reproducible results. Are there any other things I can do to get reproducible results?

I expect the predictions to be the same, however they are not. I get different results every single time.

PV32
  • 81
  • 1
  • 6

4 Answers4

3

with TENSORFLOW 2

import tensorflow as tf

tf.random.set_seed(33)
os.environ['PYTHONHASHSEED'] = str(33)
np.random.seed(33)
random.seed(33)

session_conf = tf.compat.v1.ConfigProto(
    intra_op_parallelism_threads=1, 
    inter_op_parallelism_threads=1
)
sess = tf.compat.v1.Session(
    graph=tf.compat.v1.get_default_graph(), 
    config=session_conf
)
tf.compat.v1.keras.backend.set_session(sess)
Marco Cerliani
  • 21,233
  • 3
  • 49
  • 54
2

Because you're using Keras with Tensorflow as backend, you will find it is pretty hard to get reproducible result especially when GPU is enable. However, there is still a method to achieve this.

First, do not use GPU.

import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = ""

Second, as you've did in code, set seed for Numpy, Random, TensorFlow and so on.

import tensorflow as tf
import numpy as np
import random as rn

sd = 1 # Here sd means seed.
np.random.seed(sd)
rn.seed(sd)
os.environ['PYTHONHASHSEED']=str(sd)

from keras import backend as K
config = tf.ConfigProto(intra_op_parallelism_threads=1,inter_op_parallelism_threads=1)
tf.set_random_seed(sd)
sess = tf.Session(graph=tf.get_default_graph(), config=config)
K.set_session(sess)

One final word, both two pieces of code should be placed at the begining of your code.

guorui
  • 871
  • 2
  • 9
  • 21
1

I created a rule to achieve reproducibility:

  • Works for python 3.6, not 3.7
  • First install Keras 2.2.4
  • After install tensorflow 1.9

And finally in the code:

import numpy as np
import random as rn
import tensorflow as tf
import keras
from keras import backend as K

#-----------------------------Keras reproducible------------------#
SEED = 1234

tf.set_random_seed(SEED)
os.environ['PYTHONHASHSEED'] = str(SEED)
np.random.seed(SEED)
rn.seed(SEED)

session_conf = tf.ConfigProto(
    intra_op_parallelism_threads=1, 
    inter_op_parallelism_threads=1
)
sess = tf.Session(
    graph=tf.get_default_graph(), 
    config=session_conf
)
K.set_session(sess)
#-----------------------------------------------------------------#
Victor Villacorta
  • 577
  • 5
  • 6
0

(Only tested for Tensorflow 2)

Besides setting the random seeds, I found that my RTX 3080 GPU would only give deterministic results if I used tf.float64 instead of the default of tf.float32. This appears to be due to rounding errors on the GPU, which leads to differences in the path taken during gradient descent. Note that this does not guarantee reproducibility across different GPUs. Different GPU architectures are not guaranteed to perform operations in exactly the same way. Such differences in implementation may cause differences in rounding, which can in turn affect the convergence of your model.

Matt Raymond
  • 71
  • 3
  • 5