OpenCV: detect flawed rectangle

Question

currently I'm working on a project where I try to find the corners of the rectangle's surface in a photo using OpenCV (Python, Java or C++)

I've selected desired surface by filtering color, then I've got mask and passed it to the cv2.findContours:

cnts, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnt = sorted(cnts, key = cv2.contourArea, reverse = True)[0]
peri = cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, 0.02*peri, True)
if len(approx) == 4:
    cv2.drawContours(mask, [approx], -1, (255, 0, 0), 2)

This gives me an inaccurate result:

Using cv2.HoughLines I've managed to get 4 straight lines that accurately describe the surface. Their intersections are exactly what I need:

edged = cv2.Canny(mask, 10, 200)
hLines = cv2.HoughLines(edged, 2, np.pi/180, 200)
lines = []
for rho,theta in hLines[0]:
    a = np.cos(theta)
    b = np.sin(theta)
    x0 = a*rho
    y0 = b*rho
    x1 = int(x0 + 1000*(-b))
    y1 = int(y0 + 1000*(a))
    x2 = int(x0 - 1000*(-b))
    y2 = int(y0 - 1000*(a))
    cv2.line(mask, (x1,y1), (x2,y2), (255, 0, 0), 2)
    lines.append([[x1,y1],[x2,y2]])

The question is: is it possible to somehow tweak findContours?

Another solution would be to find coordinates of intersections. Any clues for this approach are welcome :)

Can anybody give me a hint how to solve this problem?

"Their intersections is exactly what I need" -- In that case calculate their intersections. — Dan Mašek, Mar 07 '17 at 23:27
There are already many clues avaliable, for example http://stackoverflow.com/questions/563198/how-do-you-detect-where-two-line-segments-intersect or https://en.wikipedia.org/wiki/Line%E2%80%93line_intersection or http://stackoverflow.com/a/7448287/3962537 — Dan Mašek, Mar 07 '17 at 23:38
This is not the case. I detect lines using HoughLines. Since this function returns rho and theta, the intersections could be found as follows: `cos(theta1)*X+sin(theta1)*Y-rho1=0 cos(theta2)*X+sin(theta2)*Y-rho2=0` I've added this solution to my question — Aray Karjauv, Mar 14 '17 at 03:15
When a question is marked as duplicate, aren't they supposed to the refer to the "original" question? — Totoro, Mar 14 '17 at 06:13
It's not duplicate in my opinion. I need 3 more votes to reopen it :) — Aray Karjauv, Mar 14 '17 at 14:41

Aray Karjauv · Accepted Answer · 2023-02-16T09:52:58.883

Finding intersection is not so trivial problem as it seems to be, but before the intersection points will be found, following problems should be considered:

The most important thing is to choose the right parameters for the HoughLines function, since it can return from 0 to an infinite numbers of lines (we need 4 parallel)
Since we do not know in what order these lines go, they need to be compared with each other
Because of the perspective, parallel lines are no longer parallel, so each line will have a point of intersection with the others. A simple solution would be to filter the coordinates located outside the photo. But it may happen that an undesirable intersection will be within the photo.
The coordinates should be sorted. Depending on the task, it could be done in different ways.

cv2.HoughLines will return an array with the values of rho and theta for each line. Now the problem becomes a system of equations for all lines in pairs:

def intersections(edged):
  # Height and width of a photo with a contour obtained by Canny
  h, w = edged.shape

  hl = cv2.HoughLines(edged,2,np.pi/180,190)[0]
  # Number of lines. If n!=4, the parameters should be tuned
  n = hl.shape[0]    
        
  # Matrix with the values of cos(theta) and sin(theta) for each line
  T = np.zeros((n,2),dtype=np.float32)
  # Vector with values of rho
  R = np.zeros((n),dtype=np.float32)

  T[:,0] = np.cos(hl[:,1])
  T[:,1] = np.sin(hl[:,1])
  R = hl[:,0]

  # Number of combinations of all lines
  c = n*(n-1)/2
  # Matrix with the obtained intersections (x, y)
  XY = np.zeros((c,2))
  # Finding intersections between all lines
  for i in range(n):
    for j in range(i+1, n):
      XY[i+j-1, :] = np.linalg.inv(T[[i,j],:]).dot(R[[i,j]])

  # filtering out the coordinates outside the photo
  XY = XY[(XY[:,0] > 0) & (XY[:,0] <= w) & (XY[:,1] > 0) & (XY[:,1] <= h)]
  # XY = order_points(XY) # obtained points should be sorted
  return XY

here is the result:

score -1 · Answer 2 · answered Mar 08 '17 at 02:21

-1

It is possible to:

select the longest contour
break it into segments and group them by gradient
Fit lines to the largest four groups
Find intersection points

But then, Hough transform does nearly the same thing. Is there any particular reason for not using it?

Intersection points of lines are very easy to calculate. A high-school coordinate geometry lesson can provide you with the algorithm.

answered Mar 08 '17 at 02:21

Totoro

3,398
1
24
39

Actually I use Hough transform to detect lines (second example) – Aray Karjauv Mar 14 '17 at 03:20
In that case, the best solution for this case is to find the intersection points as you have already done in the updated question. You will need a way to select the best four candidates for the lines, in a practical situation. – Totoro Mar 14 '17 at 06:11
1

I just use a while loop till i get only 4 lines. not the best solution, but works – Aray Karjauv Mar 14 '17 at 14:37

OpenCV: detect flawed rectangle

2 Answers2