I've built and trained a classifier using a custom estimator. As I need to use it on a separate dataset afterwards I've got my model_fn in a separate file. Training and evaluating my dataset straight away as follows (with input_fn shortened for readability)
# Instantiate the estimator
birdsclassifier = tf.estimator.Estimator(
model_fn=cnn_model, model_dir=model_path)
tensors_to_log = {"probabilities": "softmax_tensor"}
logging_hook = tf.train.LoggingTensorHook(
tensors=tensors_to_log, every_n_iter=50)
# Train the model
def input_fn():
#Training dataset preparation happens here
return {'x': features}, labels
birdsclassifier.train(
input_fn=input_fn,
steps=num_steps,
hooks=[logging_hook])
# Evaluate the model and print results
print('begin eval')
def eval_input_fn():
#Validation dataset preparation happens here
return {'x': featuresTest}, labelsT
eval_results = birdsclassifier.evaluate(input_fn=eval_input_fn)
print(eval_results)
and gives the following output
{'accuracy': 0.5952381, 'loss': 1.4494723, 'global_step': 2000}
however running a script with the evaluation code only (having double checked for shuffling issues and that both the validation datasets are identical) yields
{'accuracy': 0.083333336, 'loss': 4.551247, 'global_step': 2000}
Since the global step is the same the graph seems to be at least partially correctly reconstructed from checkpoint - however it looks as if the variables were not correctly loaded. Also running the second script repetitively gives me slightly different outputs. I have also noticed that re-running the training script with small number of steps keeps increasing the global_step count, but also results in an accuracy drop which suggests that the training restarts from random
This thread as well as this suggest that my method should work just fine.
Note: in case the code snippets are insufficient the remainder of the code is here: https://github.com/pbkowalski/cnn_classification_birds
