OpenCV extract area of an image from a vector of squares

Question

I have an image that contains a square, and I need to extract the area contained in that square. After applying the squares.c script (available in the samples of every OpenCV distribution) I obtain a vector of squares, then I need to save an image for each of them.

The user karlphillip suggested this:

for (size_t x = 0; x < squares.size(); x++) 
{
    Rect roi(squares[x][0].x, squares[x][0].y, 
             squares[x][1].x - squares[x][0].x, 
             squares[x][3].y - squares[x][0].y);
    Mat subimage(image, roi);
}

in order to generate a new Mat called subimage for all the squares detected in the original image

As karl remembered me, the points detected in the image may not represent a perfect square (as you can see in the image above) but the code I just suggested to you assumes they do.

In fact I get this error:

OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width &&
      roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height &&
      roi.y + roi.height <= m.rows) in Mat, file /usr/include/opencv/cxmat.hpp, 
      line 187

terminate called after throwing an instance of 'cv::Exception'
what():  /usr/include/opencv/cxmat.hpp:187: error: (-215) 0 <= roi.x && 
       0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y &&
       0 <= roi.height && roi.y + roi.height <= m.rows in function Mat

Aborted

Suggestion for make the script accept also non perfect squares?

Answer 1

I feel like I need to clarify a few things about that code.

First, it assumes that the region detected is a perfect square because it ignores some of the points inside squares[x] to create a new Mat.

Second, it also assumes that the points that make the region were detected in the clockwise direction, starting with p0 in the top-left corner of the image:

(p0)  1st----2nd  (p1)
       |      |
       |      |
(p3)  4th----3rd  (p2)

which might not be true for all the regions detected. That means that this code:

Rect roi(squares[x][0].x, squares[x][0].y, 
         squares[x][1].x - squares[x][0].x, 
         squares[x][3].y - squares[x][0].y);

probably will generate a ROI with invalid dimensions, such as negative width and height values, and that's why OpenCV throws a cv::Exception at you on Mat subimage(image, roi);.

What you should do, is write a code that will identify the top-left point of the region and call it p0, then it's nearest neightbor on the right side, p1, then find the bottom-right point of the region and call it p2, and then what's left is p3. After this, assembling the ROI is easy:

Rect roi(p0.x, p0.y, 
         p1.x - p0.x, 
         p3.y - p0.y);

EDIT:

I found an excellent solution while reading the documentation of the v2.3 of OpenCV. It automates the process I described earlier and it make things so much easier and clean. You can use this trick to order the 4 Points in the vector to a meaningful Rect structure:

// Data returned and filled by findSquares(). Check the example squares.cpp for more info on this function.
vector<vector<Point> > squares;

for (size_t i = 0; i < squares.size(); i++)
{
    Rect rectangle = boundingRect(Mat(squares[i]));
    cout << "#" << i << " rectangle x:" << rectangle.x << " y:" << rectangle.y << " " << rectangle.width << "x" << rectangle.height << endl;
}