1

I am trying to replicate the results of my model. But each time i run it, the results are different (even with restarting the google colab's runtime). Here's the model

# set random seed
tf.random.set_seed(42)

# 1. Create the model
model_1 = tf.keras.models.Sequential([
    tf.keras.layers.Dense(1, input_shape=[1])
    ])

# 2. Compile the model
model_1.compile(loss=tf.keras.losses.mae,
            optimizer=tf.keras.optimizers.SGD(),
            metrics=["mae"])

# 3. Fit the model
model_1.fit(X_train, y_train, epochs=100)

X_train and y_train are two tensors each with shape=(40,), and dtype=int32

# 4. make a prediction
y_pred = model.predict(X_test)

# 5. mean absolute error
mae = tf.metrics.mean_absolute_error(y_true=y_test,
                                     y_pred=tf.squeeze(tf.constant(y_pred)))

X_test and y_test are two tensors each with shape=(10,), and dtype=int32

Now every time i run the above code i get a different value for mae, for example: 8.63, 21.26, 14.96, 14.93, 14,84, ...

I had expected to get identical runs, because i set the random seed before building and training the model.

How can i exactly reproduce my model's performance?

user1479670
  • 1,145
  • 3
  • 10
  • 22
  • Please check if anything in the answers to this question solves the problem for you: https://stackoverflow.com/q/36288235/3005167 – MB-F Aug 17 '23 at 12:19

1 Answers1

3

Since you are using Keras, there may be different seeds in play. This is documented here.

Keras has a function to set those seeds:

# Set the seed using keras.utils.set_random_seed. This will set:
# 1) `numpy` seed
# 2) `tensorflow` random seed
# 3) `python` random seed
keras.utils.set_random_seed(812)

Maybe you'll also need to:

# This will make TensorFlow ops as deterministic as possible, but it will
# affect the overall performance, so it's not enabled by default.
# `enable_op_determinism()` is introduced in TensorFlow 2.9.
tf.config.experimental.enable_op_determinism()
MB-F
  • 22,770
  • 4
  • 61
  • 116