how to fill the hollow lines opencv

Question

I have an image like this:

after I applied some processings e.g. cv2.Canny(), it looks like this now:

As you can see that the black lines become hollow. I have tried erosion and dilation, but if I do them many times, the 2 entrances will be closed(meaning become connected line or closed contour).

How could I make those lines solid like the below image while keep the 2 entrances not affected?

---

Update 1

I have tested the following answers with a few of photos, but the code seems customized to only be able to handle this one particular picture. Due to the restriction of SOF, I cannot upload photos larger than 2MB, so I uploaded them into my Microsoft OneDrive folder for your convenience to test.

https://1drv.ms/u/s!Asflam6BEzhjgbIhgkL4rt1NLSjsZg?e=OXXKBK

Update 2

I picked up @fmw42's post as answer as his answer is the most detailed one. It doesn't answer my question but points out the correct way to process maze which is my ultimate goal. I like his approach of answering questions, firstly tells you what each step should do so that you have a clear idea about how to do the task, then provide the full code example from beginning to end. Very helpful.

Due to the limitation of SOF, I can only pick up one answer. If multiple answers are allowed, I would also pick up Shamshirsaz.Navid's answer. His answer not only points to the correct direction to solve the issue, but also the explanation with visualization really works well for me~! I guess it works equally well for all people who are trying to understand why each line of code is needed. Also he follows up my questions in comments, this makes the SOF a bit interactive :)

The Threshold track bar in Ann Zen's answer is also a very useful tip for people to quickly find out a optimal value.

Answer 1

Here is one way to process the maze and rectify it in Python/OpenCV.

• Read the input
• Convert to gray
• Threshold
• Use morphology close to remove the thinnest (extraneous) black lines
• Invert the threshold
• Get the external contours
• Keep on those contours that are larger than 1/4 of both the width and height of the input
• Draw those contours as white lines on black background
• Get the convex hull from the white contour lines image
• Draw the convex hull as white lines on black background
• Use GoodFeaturesToTrack to get the 4 corners from the white hull lines image
• Sort the 4 corners by angle relative to the centroid so that they are ordered clockwise: top-left, top-right, bottom-right, bottom-left
• Set these points as the array of conjugate control points for the input
• Use 1/2 the dimensions of the input to define the array of conjugate control points for the output
• Compute the perspective transformation matrix
• Warp the input image using the perspective matrix
• Save the results

Input:

import cv2
import numpy as np
import math

# load image
img = cv2.imread('maze.jpg')
hh, ww = img.shape[:2]

# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# threshold
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]

# use morphology to remove the thin lines
kernel = cv2.getStructuringElement(cv2.MORPH_RECT , (5,1))
thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# invert so that lines are white so that we can get contours for them
thresh_inv = 255 - thresh

# get external contours
contours = cv2.findContours(thresh_inv, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]

# keep contours whose bounding boxes are greater than 1/4 in each dimension
# draw them as white on black background
contour = np.zeros((hh,ww), dtype=np.uint8)
for cntr in contours:
    x,y,w,h = cv2.boundingRect(cntr)
    if w > ww/4 and h > hh/4:
        cv2.drawContours(contour, [cntr], 0, 255, 1)
        
# get convex hull from  contour image white pixels
points = np.column_stack(np.where(contour.transpose() > 0))
hull_pts = cv2.convexHull(points)

# draw hull on copy of input and on black background
hull = img.copy()
cv2.drawContours(hull, [hull_pts], 0, (0,255,0), 2)
hull2 = np.zeros((hh,ww), dtype=np.uint8)
cv2.drawContours(hull2, [hull_pts], 0, 255, 2)

# get 4 corners from white hull points on black background
num = 4
quality = 0.001
mindist = max(ww,hh) // 4
corners = cv2.goodFeaturesToTrack(hull2, num, quality, mindist)
corners = np.int0(corners)
for corner in corners:
    px,py = corner.ravel()
    cv2.circle(hull, (px,py), 5, (0,0,255), -1)

# get angles to each corner relative to centroid and store with x,y values in list
# angles are clockwise between -180 and +180 with zero along positive X axis (to right)
corner_info = []
center = np.mean(corners, axis=0)
centx = center.ravel()[0]
centy = center.ravel()[1]
for corner in corners:
    px,py = corner.ravel()
    dx = px - centx
    dy = py - centy
    angle = (180/math.pi) * math.atan2(dy,dx)
    corner_info.append([px,py,angle])

# function to define sort key as element 2 (i.e. angle)
def takeThird(elem):
    return elem[2]

# sort corner_info on angle so result will be TL, TR, BR, BL order
corner_info.sort(key=takeThird)

# make conjugate control points
# get input points from corners
corner_list = []
for x, y, angle in corner_info:
    corner_list.append([x,y])
print(corner_list)

# define input points from (sorted) corner_list
input = np.float32(corner_list)

# define output points from dimensions of image, say half of input image
width = ww // 2
height = hh // 2
output = np.float32([[0,0], [width-1,0], [width-1,height-1], [0,height-1]])

# compute perspective matrix
matrix = cv2.getPerspectiveTransform(input,output)

# do perspective transformation setting area outside input to black
result = cv2.warpPerspective(img, matrix, (width,height), cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT, borderValue=(0,0,0))

# save output
cv2.imwrite('maze_thresh.jpg', thresh)
cv2.imwrite('maze_contour.jpg', contour)
cv2.imwrite('maze_hull.jpg', hull)
cv2.imwrite('maze_rectified.jpg', result)

# Display various images to see the steps
cv2.imshow('thresh', thresh)
cv2.imshow('contour', contour)
cv2.imshow('hull', hull)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Thresholded Image after morphology:

Filtered Contours on black background:

Convex hull and 4 corners on input image:

Result from perspective warp:

Answer 2

You can try a simple threshold to detect the lines of the maze, as they are conveniently black:

import cv2

img = cv2.imread("maze.jpg")
gray = cv2.cvtColor(img, cv2.BGR2GRAY)
_, thresh = cv2.threshold(gray, 60, 255, cv2.THRESH_BINARY)
cv2.imshow("Image", thresh)
cv2.waitKey(0)

Output:

You can adjust the threshold yourself with trackbars:

import cv2

cv2.namedWindow("threshold")
cv2.createTrackbar("", "threshold", 0, 255, id)

img = cv2.imread("maze.jpg")

while True:
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    t = cv2.getTrackbarPos("", "threshold")
    _, thresh = cv2.threshold(gray, t, 255, cv2.THRESH_BINARY)
    cv2.imshow("Image", thresh)
    if cv2.waitKey(1) & 0xFF == ord("q"): # If you press the q key
        break

Answer 3

Canny is an edge detector. It detects the lines along which color changes. A line in your input image has two such transitions, one on each side. Therefore you see two parallel lines on each side of a line in the image. This answer of mine explains the difference between edges and lines.

So, you shouldn’t be using an edge detector to detect lines in an image.

If a simple threshold doesn't properly binarize this image, try using a local threshold ("adaptive threshold" in OpenCV). Another thing that works well for images like these is applying a top hat filter (for this image, it would be a closing(img) - img), where the structuring element is adjusted to the width of the lines you want to find. This will result in an image that is easy to threshold and will preserve all lines thinner than the structuring element.

Answer 4

Check this:

import cv2
import numpy as np

im=cv2.imread("test2.jpg",1)

#convert 2 gray
mask=cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)

#convert 2 black and white
mask=cv2.threshold(mask,127,255,cv2.THRESH_BINARY)[1]

#remove thin lines and texts and then remake main lines
mask=cv2.dilate(mask,np.ones((5, 5), 'uint8'))
mask=cv2.erode(mask,np.ones((4, 4), 'uint8'))

#smooth lines
mask=cv2.medianBlur(mask,3)

#write output mask
cv2.imwrite("mask2.jpg",mask)

From now on, everything can be done. You can delete extra blobs, you can extract lines from the original image according to the mask, and things like that.

---

Median:

Median changes are not much for this project. And it can be safely removed. But I prefer it because it rounds the ends of the lines a bit. You have to zoom in a lot to see the pixels. But this technique is usually used to remove salt/pepper noise.

---

Erode Kernel:

In the case of the kernel, the larger the number, the thicker the lines. Well, this is not always good. Because it causes the path lines to stick to the arrow and later it becomes difficult to separate the paths from the arrow.

---

Update:

It does not matter if part of the Maze is cleared. The important thing is that from this mask you can draw a rectangle around this shape and create a new mask for this image.

Make a white rectangle around these paths in a new mask. Completely whiten the inside of the mask with FloodFill or any other technique. Now you have a new mask that can take the whole shape out of the original image. Now in the next step you can correct Perspective.