1

How can I get reproducible results with keras? I followed these steps but I am still getting different results every time I run the Jupyter notebook. I also tried setting shuffle=False when calling model.fit().

My configuration: conda 4.3.25

keras 2.0.6 with tensorflow backend

tensorflow-gpu 1.2.1

python 3.5

windows 10

Enrique
  • 9,920
  • 7
  • 47
  • 59
  • Are you training on GPU or CPU? – McLawrence Sep 21 '17 at 12:17
  • You may be facing this issue: https://github.com/fchollet/keras/issues/2280 – Daniel Möller Sep 21 '17 at 13:00
  • @McLawrence I am using gpu – Enrique Sep 21 '17 at 14:12
  • @DanielMöller I already checked that reference and tried setting the number of threads to 1. – Enrique Sep 21 '17 at 14:14
  • I don't know how strong your results vary, but even if you set the random seed for all used libraries, the implementation of convolutions in cuDNN is non-deterministic by default. So if you are using CNNs, that could be the reason. You can disable it, as described [here](https://github.com/fchollet/keras/issues/2479#issuecomment-213987747). – McLawrence Sep 21 '17 at 14:29
  • @McLawrence maybe you could add your comment as an answer as it seems to be factually correct and answers the question OP had. – Shagun Sodhani Dec 03 '17 at 03:58
  • the answer linked is using Theano...the question is for Tensorflow... – Yossi Farjoun Dec 09 '17 at 19:35
  • Please have a look at my response here (https://stackoverflow.com/a/52897216/9024698) for when using the CPU. – Outcast Oct 22 '18 at 09:55

1 Answers1

0

See the answer I posted at another question. The main idea is to: first, disable the GPU. And then, seed the libraries like "numpy, random, etc". To sum up, including the code below at the beginning of your code may help solve your problem.

import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = ""

import numpy as np
import tensorflow as tf
import random as rn
from keras import backend as K

sd = 1
np.random.seed(sd)
rn.seed(sd)
os.environ['PYTHONHASHSEED']=str(sd)

config = tf.ConfigProto(intra_op_parallelism_threads=1,inter_op_parallelism_threads=1)
tf.set_random_seed(sd)
sess = tf.Session(graph=tf.get_default_graph(), config=config)
K.set_session(sess)
guorui
  • 871
  • 2
  • 9
  • 21