I trained my Mask R-CNN Network with my own data, which i transformed into COCO Style for my Thesis and now i want to evaluate my results. I found two methods to do that. One method is the evaluation from COCO itself. Mask R-CNN itself shows how to evaluate with COCO Metric in their coco.py file: https://github.com/matterport/Mask_RCNN/blob/master/samples/coco/coco.py
I basically use the evaluate_coco(...) function in line 342. As a result i get the result from COCO metric with Average Precisions and Average Recall for different metrics, see the images below. For the parameter eval_type i use eval_type="segm".
For me the mAP is interesting. I know that the mAP50 uses 0.5 for the IuO (Intersection over Union) and their standard mAP is from IuO = 0.5 until 0.95 in 0.05 steps.
The second method is from Mask R-CNN itself in their utils.py: https://github.com/matterport/Mask_RCNN/blob/master/mrcnn/utils.py
The name of the function is compute_ap(...) in line 715 and there is written that the IoU is 0.5. The function returns a mAP value.
This raises the question of what type compute_ap() evaluates for. With COCO you can choose between "bbox" and "segm".
Also i want to know the difference between the mAP value from the compute_ap(...) function and the mAP50 from COCO, because with the same data i get different results.
Unfortunately i can't upload a better image now, because i can only go on Monday-Friday to my university and friday i was in a hurry and took pictures without checking them, but the mean Value over all AP from the compute_ap() is 0.91 and i am pretty sure that the AP50 from COCO is 0.81.
Do someone know the difference or is there no differnce?
Is it because of the maxDets=100 Parameter?: Understanding COCO evaluation "maximum detections" My pictures have only 4 categories and a maximum of 4 instances per picture.
Or is the way how the mAP is calculated different?
EDIT: Now i have a better image with the whole COCO Metric for Type "Segm" and Type "BBox" and compere it with the result from compute_ap(), which is "mAP: 0,41416..." By the way, if there is an "-1" in the COCO Result, does this mean there were no from this typ?
