I am using the Estimator API of Tensorflow and I am running into the following problem. I want to check the f1 score other than the accuracy and when I am evaluating after training there are no problems at all, when I am testing it asks for normalized values, which I have already normalized.
This is the model of my of the network (first part is omitted):
#### architecture omitted #####
predictions = {
"classes": tf.argmax(input=logits, axis=1),
"probabilities": tf.nn.sigmoid_cross_entropy_with_logits(labels=tf.cast(labels, tf.float32), logits=tf.cast(logits, tf.float32), name="sigmoid_tensor")
}
if mode == tf.estimator.ModeKeys.PREDICT:
return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)
loss = tf.losses.sigmoid_cross_entropy(multi_class_labels=labels, logits=logits)
if mode == tf.estimator.ModeKeys.TRAIN:
optimizer = tf.train.AdamOptimizer(learning_rate=0.001)
#optimizer = tf.train.MomentumOptimizer(learning_rate=0.01, momentum=0.96)
train_op = optimizer.minimize(
loss=loss,
global_step=tf.train.get_global_step())
logging_hook = tf.train.LoggingTensorHook({"loss" : loss}, every_n_iter=10)
return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op, training_hooks = [logging_hook])
eval_metric_ops = {
"accuracy": tf.metrics.accuracy(
labels=tf.argmax(input=labels, axis=1),
predictions=predictions["classes"]),
"f1 score" : tf.contrib.metrics.f1_score(
labels = tf.argmax(input=labels, axis=1),
predictions = tf.cast(predictions["classes"],tf.float32)/tf.norm(tf.cast(predictions["classes"], tf.float32)))
}
return tf.estimator.EstimatorSpec(mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)
This is the training script I am using, I don't have any problems when evaluating (no errors here).
classifier = tf.estimator.Estimator(model_fn=instrument_recognition_model, model_dir=saved_model_path)
train_input_fn = tf.estimator.inputs.numpy_input_fn(x=X_train, y=y_train, batch_size=16, num_epochs=30, shuffle=True)
classifier.train(input_fn=train_input_fn)
# Evaluate results on the training set
eval_input_fn = tf.estimator.inputs.numpy_input_fn(x=X_eval,y=y_eval,num_epochs=1,shuffle=False)
eval_results = classifier.evaluate(input_fn=eval_input_fn)
print(eval_results)
This is my testing script, here the program fails:
classifier = tf.estimator.Estimator(model_fn=instrument_recognition_model, model_dir=saved_model_path)
# Keep only certain samples
key_indices = [np.where(instruments == x)[0][0] for x in keys]
example_indices = np.array([])
for ind in key_indices:
tmp = np.argwhere(labels[:,ind] == True).flatten()
example_indices = np.union1d(example_indices, tmp).astype(np.int32)
features = features[example_indices].astype(np.float32)
example_indices = [[x for i in key_indices] for x in example_indices]
labels = labels[example_indices, key_indices].astype(np.int)
# Evaluate results on the test set
print(features)
print(labels)
eval_input_fn = tf.estimator.inputs.numpy_input_fn(x=features, y=labels, batch_size=1, num_epochs=1, shuffle=False)
eval_results = classifier.evaluate(input_fn=eval_input_fn)
print(eval_results)
I have really not idea of what's wrong since I've follwed the same process for evaluation and testing. Without the f1 metric (only accuracy) everything works well, when I add the f1 metric, it fails in the test script.
The fragment of the error is the following:
### trace error omitted ###
File "../models.py", line 207, in instrument_recognition_model
predictions = tf.cast(predictions["classes"],tf.float32)/tf.norm(tf.cast(predictions["classes"], tf.float32)))
### trace error ommitted ###
TheInvalidArgumentError (see above for traceback): assertion failed: [predictions must be in [0, 1]] [Condition x <= y did not hold element-wise:x (div:0) = ] [nan] [y (f1/Cast_1/x:0) = ] [1]
[[Node: f1/assert_less_equal/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_FLOAT, DT_STRING, DT_FLOAT], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](f1/assert_less_equal/Assert/AssertGuard/Assert/Switch, f1/assert_less_equal/Assert/AssertGuard/Assert/data_0, f1/assert_less_equal/Assert/AssertGuard/Assert/data_1, f1/assert_less_equal/Assert/AssertGuard/Assert/Switch_1, f1/assert_less_equal/Assert/AssertGuard/Assert/data_3, f1/assert_less_equal/Assert/AssertGuard/Assert/Switch_2)]]
Thank you in advance
