Merging Overlapping Rectangle in OpenCV

Question

I'm using OpenCV 3.0. I've made a car detection program and I keep running into the problem of overlapping bounding boxes:

Is there a way to merge overlapping bounding boxes as described on the images below? I've used rectangle(frame, Point(x1, y1), Point(x2, y2), Scalar(255,255,255)); to draw those bounding boxes. I've searched for answer from similiar threads but I can't find them helpful. I'd like to form a single outer bounding rectangle after merging those bounding boxes.

a single outer bounding rectangle please. – Boby Pratama Mar 12 '16 at 14:02 — Boby Pratama, Mar 12 '16 at 14:02
take a look at http://stackoverflow.com/a/21770696/5294258 – sturkmen Mar 13 '16 at 17:35 — sturkmen, Mar 13 '16 at 17:35

score 0 · Accepted Answer · answered Mar 12 '16 at 14:50

Problem

Seems as if you are displaying each contour you are getting. You don't have to do that. Follow the algorithm and code given below.

Algorithm

In this case what you can do is iterate through each contour that you detect and select the biggest boundingRect. You don't have to display each contour you detect.

Here is a code that you can use.

Code

for( int i = 0; i< contours.size(); i++ ) // iterate through each contour. 
      {
       double a=contourArea( contours[i],false);  //  Find the area of contour
       if(a>largest_area){
       largest_area=a;
       largest_contour_index=i;                //Store the index of largest contour
       bounding_rect=boundingRect(contours[i]); // Find the bounding rectangle for biggest contour
       }

      }

Regards

score 0 · Answer 2 · answered Feb 11 '19 at 16:57

As I've mentioned in a similar post here, this is a problem best solved by Non Maximum Suppression.

Although your code is in C++, have a look at this pyimagesearch article (python) to get an idea on how this works.

I've translated this code from python to C++,.

struct detection_box
{
    cv::Rect box;               /*!< Bounding box */
    double svm_val;             /*!< SVM response at that detection*/
    cv::Size res_of_detection;  /*!< Image resolution at which the detection occurred */
};

/*!
\brief Applies the Non Maximum Suppression algorithm on the detections to find the detections that do not overlap

The svm response is used to sort the detections. Translated from http://www.pyimagesearch.com/2014/11/17/non-maximum-suppression-object-detection-python/

\param boxes list of detections that are the input for the NMS algorithm
\param overlap_threshold the area threshold for the overlap between detections boxes. boxes that have overlapping area above threshold are discarded


\returns list of final detections that are no longer overlapping
*/
std::vector<detection_box> nonMaximumSuppression(std::vector<detection_box> boxes, float overlap_threshold)
{
    std::vector<detection_box> res;
    std::vector<float> areas;

    //if there are no boxes, return empty 
    if (boxes.size() == 0)
        return res;

    for (int i = 0; i < boxes.size(); i++)
        areas.push_back(boxes[i].box.area());

    std::vector<int> idxs = argsort(boxes);     

    std::vector<int> pick;          //indices of final detection boxes

    while (idxs.size() > 0)         //while indices still left to analyze
    {
        int last = idxs.size() - 1; //last element in the list. that is, detection with highest SVM response
        int i = idxs[last];
        pick.push_back(i);          //add highest SVM response to the list of final detections

        std::vector<int> suppress;
        suppress.push_back(last);

        for (int pos = 0; pos < last; pos++)        //for every other element in the list
        {
            int j = idxs[pos];

            //find overlapping area between boxes
            int xx1 = max(boxes[i].box.x, boxes[j].box.x);          //get max top-left corners
            int yy1 = max(boxes[i].box.y, boxes[j].box.y);          //get max top-left corners
            int xx2 = min(boxes[i].box.br().x, boxes[j].box.br().x);    //get min bottom-right corners
            int yy2 = min(boxes[i].box.br().y, boxes[j].box.br().y);    //get min bottom-right corners

            int w = max(0, xx2 - xx1 + 1);      //width
            int h = max(0, yy2 - yy1 + 1);      //height

            float overlap = float(w * h) / areas[j];    

            if (overlap > overlap_threshold)        //if the boxes overlap too much, add it to the discard pile
                suppress.push_back(pos);
        }

        for (int p = 0; p < suppress.size(); p++)   //for graceful deletion
        {
            idxs[suppress[p]] = -1;
        }

        for (int p = 0; p < idxs.size();)
        {
            if (idxs[p] == -1)
                idxs.erase(idxs.begin() + p);
            else
                p++;
        }

    }

    for (int i = 0; i < pick.size(); i++)       //extract final detections frm input array
        res.push_back(boxes[pick[i]]);

    return res;

}

Merging Overlapping Rectangle in OpenCV

2 Answers2

Problem

Algorithm

Code