There should be someone who can really clarify this..
Here is some initial info from the Keras documentation: The fit function in the Keras just trains the model for a given number of epochs. And the evaluate function returns the loss value & metrics values for the model in test mode.
So, both functions return a loss. For the sake of giving an example, if I have 1 single training example, the loss I get from the fit function after each training step should be identical to the loss I get from the evaluate function (after the same training step). (The assumption here is that I run both the fit and evaluate functions on the same train set (which consists of 1 example only).)
I define my network as following:
def identity_loss(y_true, y_pred):
return K.mean(y_pred - 0 * y_true)
model = ResNet50(weights='imagenet')
model.layers.pop()
x = model.get_layer('flatten_1').output # layer 'flatten_1' is the last layer of the model
model_out = Dense(128, activation='relu', name='model_out')(x)
model_out = Lambda(lambda x: K.l2_normalize(x,axis=-1))(model_out)
new_model = Model(inputs=model.input, outputs=model_out)
anchor_input = Input(shape=(224, 224, 3), name='anchor_input')
pos_input = Input(shape=(224, 224, 3), name='pos_input')
neg_input = Input(shape=(224, 224, 3), name='neg_input')
encoding_anchor = new_model(anchor_input)
encoding_pos = new_model(pos_input)
encoding_neg = new_model(neg_input)
loss = Lambda(triplet_loss)([encoding_anchor, encoding_pos, encoding_neg])
siamese_network = Model(inputs = [anchor_input, pos_input, neg_input],
outputs = loss)
siamese_network.compile(loss=identity_loss, optimizer=Adam(lr=.00003))
Later on, I train my train set (consisting of 1 example only) with the fit function for 10 epochs. Just to check differences between fit and evaluate functions, I also run the evaluate function right after the fit function in each epoch and the output looks like the folloing:
nr_epoch: 0
Epoch 1/1
1/1 [==============================] - 4s 4s/step - loss: 2.0035
1/1 [==============================] - 3s 3s/step
eval_score for train set: 2.0027356147766113
nr_epoch: 1
Epoch 1/1
1/1 [==============================] - 1s 1s/step - loss: 1.9816
1/1 [==============================] - 1s 1s/step
eval_score for train set: 2.001833915710449
nr_epoch: 2
Epoch 1/1
1/1 [==============================] - 1s 1s/step - loss: 1.9601
1/1 [==============================] - 1s 1s/step
eval_score for train set: 2.00126576423645
nr_epoch: 3
Epoch 1/1
1/1 [==============================] - 1s 1s/step - loss: 1.9388
1/1 [==============================] - 1s 1s/step
eval_score for train set: 2.0009117126464844
nr_epoch: 4
Epoch 1/1
1/1 [==============================] - 1s 1s/step - loss: 1.9176
1/1 [==============================] - 1s 1s/step
eval_score for train set: 2.000725746154785
nr_epoch: 5
Epoch 1/1
1/1 [==============================] - 1s 1s/step - loss: 1.8964
1/1 [==============================] - 1s 1s/step
eval_score for train set: 2.0006520748138428
nr_epoch: 6
Epoch 1/1
1/1 [==============================] - 1s 1s/step - loss: 1.8759
1/1 [==============================] - 1s 1s/step
eval_score for train set: 2.0006656646728516
nr_epoch: 7
Epoch 1/1
1/1 [==============================] - 1s 1s/step - loss: 1.8555
1/1 [==============================] - 1s 1s/step
eval_score for train set: 2.0007567405700684
nr_epoch: 8
Epoch 1/1
1/1 [==============================] - 1s 1s/step - loss: 1.8355
1/1 [==============================] - 1s 1s/step
eval_score for train set: 2.0009000301361084
nr_epoch: 9
Epoch 1/1
1/1 [==============================] - 2s 2s/step - loss: 1.8159
1/1 [==============================] - 2s 2s/step
eval_score for train set: 2.001085042953491
As seen, the loss reported by the fit function (at the end of eachepoch) is decreasing. And the loss coming from the evaluate function is just not decreasing.
So the dilemma is: If I run my model on 1 single training example, should I not see the same loss (after each epoch) from both fit and evaluate functions from the same epoch? If I keep training, the train loss is decreasing but the loss coming from the evaluate function somehow remains at the same level and does not decrease
And lastly, here is how I call fit and evaluate functions:
z = np.zeros(len(anchor_path))
siamese_network.fit(x=[anchor_imgs, pos_imgs, neg_imgs],
y=z,
batch_size=batch_size,
epochs=1,
verbose=1,
callbacks=None,
validation_split=0.0,
validation_data=None,
shuffle=True,
class_weight=None,
sample_weight=None,
initial_epoch=0,
steps_per_epoch=None,
validation_steps=None)
eval_score = siamese_network.evaluate(x=[anchor_imgs, pos_imgs, neg_imgs],
y=z,
batch_size = batch_size,
verbose = 1)
print('eval_score for train set: ', eval_score)
So, why does the loss decrease during the execution of fit function but not the evaluate function? Where am I making the mistake?
