6

Keras 2.0 removed F1 score, but I would like to monitor its value. I am using a sequential model to train a Neural Net.

I defined a function, as suggested here How to calculate F1 Macro in Keras?.

This function works fine only if used it inside model.compile. In this way I see its value at each step. The problem is that I don't want just to see its value but I would like my training to behave differently according to its value, using the callbacks of Keras.

If I try to insert my custom metric in the callbacks then I get this error:

'function object is not iterable'

Do you know how to define a function such that it can be used as an argument in the callbacks?

spy95
  • 131
  • 8

1 Answers1

0

Callback of Keras will enable us to retrieve the model at different period, based on the metric which we keep track of. This will not affect the training procedure of the model.

You can train your model only with respect to some loss function. For example, cross entropy for classification problem. The readily available loss function in keras are given here

Precision, recall or f1-score are not differentialable functions. Hence, we cannot use that as a loss function for model training.

May be, if you want to tune your hyperparameter (such as learning rate, class weights) for improving f1 score, then you can be do that.

For tuning hyper parameters you can use hyperopt, tutorials

Venkatachalam
  • 16,288
  • 9
  • 49
  • 77
  • 1
    I know that the algorithm that is used try to minimize the loss. But it would be nice to change the learning rate and stop the iterations based on the the F1_Score, instead of the loss. This is what I'd like to do and to do so I wanted to insert the F1_score in the callbacks – spy95 Dec 13 '18 at 08:57