1

I want to run one of the tensorflow object detection evaluation protocols [1]. I am new with it, and from the webpage I cannot understand where I would have to add the metrics_set configuration. Ex: 

EvalConfig.metrics_set='pascal_voc_detection_metrics'

I tried changing the value in the eval.proto file, where metrics_set is set to the value 8. Does anyone know if this is the right place to change it? I saw no effect on changing this value. And what does the "8" mean? In addition, what is the output I am to expect?

Update: I answered one of my questions: the place where I should change the setting is not the eval.proto, but in the configuration file:

eval_config: {
  metrics_set: 'weighted_pascal_voc_detection_metrics'
}

However, I still do not understand where I am to see the effect of this - I still have the other questions unanswered.

[1] https://github.com/tensorflow/models/blob/fd7b6887fb294e90356d5664724083d1f61671ef/research/object_detection/g3doc/evaluation_protocols.md

jotNewie
  • 426
  • 4
  • 17

1 Answers1

3

I think "8" is just a placeholder - it's the 8th entry in the eval.proto file.

When you run an evaluation job (eval.py), this metrics_set you specify is used as the protocol by which to compute the metrics on the data set specified in eval_input_reader. The results are output to an events summary file prefixed with events.out.tfevents, which you can visualize using TensorBoard or event_accumulator from tensorboard.backend.event_processing. Different metrics set would vary slightly but I haven't tried them all to comment - you'll have to look into the details of each protocol.

yangsiyu
  • 342
  • 3
  • 10