Different validation accuracy in different training sessions in keras

Question

I trained a keras model on Mnist keeping the training and model hyperparameters same. The training and validation data was exactly same. I got five different accuracies- 0.71, 0.62, 0.59, 0.52, 0.46 in different training sessions. The model was trained on 8 epochs from scratch everytime

This is the code:

def train():
    model = Sequential()
    model.add(Dense(32, input_dim=784))
    model.add(Dense(10, activation="softmax"))
    model.compile(loss='categorical_crossentropy',
                  optimizer='adam',
                  metrics=['accuracy'])

    model.fit(x_train, y_train, batch_size=32, epochs=8, verbose=0)
    results = model.evaluate(x_test, y_test, verbose=0)
    print(results[1])


for i in range(5):
    print(i)
    train()

Results:

0
2019-01-23 13:26:54.475110: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
0.7192
1
0.6223
2
0.5976
3
0.5223
4
0.4624

Answer 1

It may be only because the weights of the models are generated randomly everytime. Suppose, I train 2 models with same data and hyperparameters. Since, they have different weights initially, their loss and accuracy would vary. But, after a certain number of epochs, they both would converge at a same point where the accuracies and losses of both the models seem equal. This point could be the minima with respect to the loss, since the data is same. Otherwise, it could be a point from where both the models acquire a same path towards convergence.

In your case, maybe training for a greater number of epochs would bring equal losses and accuracies to all the models.