Accurate corners detection?

Question

I have this trivial reference image (others have objects in the white part) and I would like to detect the four corners to fix the perspective. I have tried several strategies:

1. Contours + Houghlines + intersections of lines

This strategy does not work because the hough lines are not correctly found. The white rectangle has a slight barrel deformation and the lines are not straight so I find multiple lines for each edges.

2. Contour + goodFeaturesToTrack

This looked promising, but with cv.goodFeaturesToTrack(u, 4, 0.5, 50) the corners are only found on the two bottom corners

Is there any other method to accurately detect the four corners of this rectangle ?

Answer 1

In your image, rectangle seems a proper rectangle to detect its contours. So fitted rectangle to this contour can fix the problem. My approach is using minAreaRect function:

Here is my code:

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>

using namespace cv;
using namespace std;


int main()
{

    Mat src; Mat src_gray;
    int thresh = 100;
    src = imread( "/ur/image/directory/image.png", 1 );
    Mat source = src.clone();
    cvtColor( src, src_gray, CV_BGR2GRAY );

    Mat threshold_output;
    vector<vector<Point> > contours;
    vector<Vec4i> hierarchy;

    threshold( src_gray, threshold_output, thresh, 255, THRESH_BINARY );
    findContours( threshold_output, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );

    vector<RotatedRect> minRect( contours.size() );

    for( int i = 0; i < contours.size(); i++ )
        minRect[i] = minAreaRect( Mat(contours[i]) );

    for( int i = 0; i< contours.size(); i++ )
    {

        Point2f rect_points[4]; minRect[i].points( rect_points );
        for( int j = 0; j < 4; j++ )
        {
            circle(source,rect_points[j],10,Scalar(255,255,0),2);
            line( source, rect_points[j], rect_points[(j+1)%4], Scalar(0,255,0), 2, 8 );
        }
    }

    imshow("output",source);

    waitKey(0);
    return(0);
}

Result:

Answer 2

Python based solution similar to Yunus Temurlenk's answer. First Otsu thresholding, canny edge detection and dilation applied to get a good outline. Next, the largest contour is extracted by area and four points are extracted based on minimum area rectangle.

This code will also work without dilation step in this image. In that case the outline will be a tighter fit of the rectangle.

Code:

## Press ESC button to get next image

import cv2
import cv2 as cv
import numpy as np



#frame = cv2.imread('resources/pstr1.png')
frame = cv2.imread('resources/pstr2.png')


## keeping a copy of original
print(frame.shape)
original_frame = frame.copy()




## Show the original image
winName = 'Original'
cv.namedWindow(winName, cv.WINDOW_NORMAL)
cv.resizeWindow(winName, 800, 800)
cv.imshow(winName, original_frame)
cv.waitKey(0)



# Otsu's thresholding
grayimg = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
ret2,thresh_n = cv.threshold(grayimg,0,255,cv.THRESH_BINARY+cv.THRESH_OTSU)
thresh_n = cv2.cvtColor(thresh_n, cv2.COLOR_GRAY2BGR)
frame = thresh_n


## edge detection
frame = cv2.Canny(frame,100,200)


## dilate the edges
kernel = np.ones((5,5),np.uint8)
frame = cv2.dilate(frame,kernel,iterations = 1)




## Get largest contour from contours
contours, hierarchy = cv2.findContours(frame, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)



## Get minimum area rectangle and corner points
rect = cv2.minAreaRect(max(contours, key = cv2.contourArea))
box = cv2.boxPoints(rect)
print(box)




## draw anchor points on corner
z = 6
for b in box:
    cv2.circle(original_frame, tuple(b), z, 255, -1)



## show original image with corners
box2 = np.int0(box)
cv2.drawContours(original_frame,[box2],0,(0,0,255), 2)
cv2.imshow('Detected Corners',original_frame)
cv2.waitKey(0)
cv2.destroyAllWindows()

Input Image:

Output Image: