OpenCV: Get the vertices of card

Question

I am trying to detect a card and do warpperspective on the image. I have successfully get the contour of the card. But I am facing difficulties on how to get the vertices point (show as blue point in image) as the input of getPerspectiveTransform function. Actually I have done it with simple algorithm on a static image. However, it does not work well for live frame especially when the card rotate. Is there any simple yet robust method that can get the four vertices point of card regardless of the orientation of card?

MatOfPoint2f curve = new MatOfPoint2f(matched.toArray());
double perimeter = Imgproc.arcLength(curve, true);
MatOfPoint2f approxCurve2f = new MatOfPoint2f();
Imgproc.approxPolyDP(curve, approxCurve2f, 0.003 * perimeter, true);

Mat temp = new Mat(src.rows(),src.cols(),CvType.CV_8UC1);
Imgproc.drawContours(temp, cnt, -1, new Scalar(255,255,255));
Moments moment = Imgproc.moments(matched);
Point centroid = new Point(moment.m10/moment.m00, moment.m01/moment.m00);

MatOfPoint approxCurve = new MatOfPoint();
approxCurve2f.convertTo(approxCurve, CvType.CV_32S);
List<Point> corners = Util.toPointList(approxCurve);
//group points that near each other
List<List<Point>> grouping = Util.cornerPointGrouping(corners, 50);
if(grouping.size()!=4)
{
    return temp;
}

Mat before = new Mat(1,4, CvType.CV_32FC2);
Mat after = new Mat(1,4, CvType.CV_32FC2);
Util.generateTransformationParams(grouping, centroid, before, after);
Mat transformMat = Imgproc.getPerspectiveTransform(before, after);
Mat out = new Mat();
Size dsize = new Size(src.width(), src.height());
Imgproc.warpPerspective(src, out, transformMat, dsize);


public static void generateTransformationParams(List<List<Point>> grouping, 
        Point centroid, Mat before, Mat after)
{

    Point topLeft=new Point(), topRight=new Point(), 
            btmLeft=new Point(), btmRight=new Point();
    for(List<Point> list: grouping)
    {
        List<Point> ySorted = new ArrayList<>(list);
        Collections.sort(list, new xComparator());
        Collections.sort(ySorted, new yComparator());
        if(list.get(0).y<centroid.y) //top
        {
            if(list.get(0).x<centroid.x) //left
            {
                topLeft = new Point(list.get(0).x,ySorted.get(0).y);
            }
            else //right
            {
                topRight = new Point(list.get(list.size()-1).x,ySorted.get(0).y);
            }
        }
        else //bottom
        {
            if(list.get(0).x<centroid.x) //left
            {
                btmLeft = new Point(list.get(0).x,ySorted.get(list.size()-1).y);
            }
            else //right
            {
                btmRight = new Point(list.get(list.size()-1).x,ySorted.get(list.size()-1).y);
            }
        }

    }
    //btmLeft as reference
    before.put(0, 0, topLeft.x, topLeft.y, topRight.x, topRight.y,
            btmRight.x, btmRight.y, btmLeft.x, btmLeft.y);
    double width = Util.getLength(btmLeft, btmRight);
    double height = width/creditCardRatio;
    after.put(0, 0, btmLeft.x, btmLeft.y-height, btmLeft.x+width, btmLeft.y-height,
            btmLeft.x+width, btmLeft.y, btmLeft.x, btmLeft.y);
}

I tried minAreaRect and it works well when the card is rectangle. When the perspective change, it fails to give me the accurate vertices.

this might be helpful https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/ — Krishna, Mar 26 '18 at 07:22
if you proceed as in [this](https://stackoverflow.com/a/49402345/5725763) answer, you'll end up with the vertices of the card [like this](https://imgur.com/a/0KABt) — Gopiraj, Mar 26 '18 at 09:15
After approximating the contour, you'll end with 4 or more points on the contour, form a line using the points. for example line1 = (points[0], points[1]), ... lineN = (points[N], points[0]). Find the length of the line (norm(points[0]-points[1])). Sort the lines based on the length. Get the biggest 4 lines and [find intersection points](https://en.wikipedia.org/wiki/Line%E2%80%93line_intersection#Using_homogeneous_coordinates) of these lines. The intersection of these lines will be the vertices — Gopiraj, Mar 26 '18 at 11:35

Ya boy · Answer 1 · 2018-03-25T21:45:30.593

Once you have the contours of the card defined (either from a contour method or as an intensity image), OpenCV provides these additional functions to detect a rotated rect. The rotated rect can be defined by its vertices.

rect = cv2.minAreaRect(cnt)
box = cv2.boxPoints(rect)

https://docs.opencv.org/3.1.0/dd/d49/tutorial_py_contour_features.html

If the transformation is highly perspective or you need a high accuracy for the points, simply calling cv2.findContours() and sending the flag cv2.CHAIN_APPROXIMATION_SIMPLE should suffice (assuming your card is detected as a quadrilateral contour) as the contour points would be vertices.

EDIT: The syntax above is in Python and I realize the question is in Android java

The language does not matter. Actually, I tried minAreaRect before and I have updated my question. — Sam, Mar 26 '18 at 02:08

OpenCV: Get the vertices of card

1 Answers1