13

I'm currently doing a research for multi class classification. I used categorical crossentropy and i've got a really good result using accuracy as the metrics of the experiment. When i try to use categorical_accuracy, it gives a slightly worse accuracy (1% below). My question will be, is it ok to use accuracy metrics for categorical crossentropy loss instead of the categorical_accuracy?

Aryo Pradipta Gema
  • 1,190
  • 3
  • 13
  • 29

1 Answers1

31

Keras detects the output_shape and automatically determines which accuracy to use when accuracy is specified. For multi-class classification, categorical_accuracy will be used internally. From the source:

            if metric == 'accuracy' or metric == 'acc':
                # custom handling of accuracy
                # (because of class mode duality)
                output_shape = self.internal_output_shapes[i]
                acc_fn = None
                if output_shape[-1] == 1 or self.loss_functions[i] == losses.binary_crossentropy:
                    # case: binary accuracy
                    acc_fn = metrics_module.binary_accuracy
                elif self.loss_functions[i] == losses.sparse_categorical_crossentropy:
                    # case: categorical accuracy with sparse targets
                    acc_fn = metrics_module.sparse_categorical_accuracy
                else:
                    acc_fn = metrics_module.categorical_accuracy

The 1% difference you are seeing can likely be attributed to run-to-run variation, as stochastic gradient descent will encounter different minima, unless the same random seed is used.

dhinckley
  • 2,115
  • 17
  • 16
  • 6
    Would you mind explaining the differences between sparse categorical cross entropy and categorical cross entropy? – natsuapo May 11 '17 at 05:24
  • 5
    2 years later but sparse_categorical_cross_entropy works if you have the labels in the form [1,2,3,4...30] and categorical_cross_entropy works if you have the labels one-hot-encoded [1,0,0....0], [0,1,0,....] – Timbus Calin Aug 29 '19 at 12:21
  • If you look at keras.io/api/metrics/accuracy_metrics/#accuracy-class, I think categorical_accuracy require label to be one hot encoded, while for accuracy the label can't be one hot encoded. My own tests confirms this – user4918159 Aug 04 '20 at 16:09