How to extract board from sudoku image?

Question

This is my input sudoku image : 0000.png and this is my desired output image : output.

I am trying to find the contours of the sudoku board so I can extract the board separately. This is the code I've tried so far.

import cv2
import operator
import numpy as np
import matplotlib.pyplot as plt

image  = cv2.imread("0000.png")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5,5), 0)
thresh = cv2.adaptiveThreshold(blur, 255, 1, 1, 11, 2)
contours,_ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

max_area = 0
c = 0
for i in contours:
    area = cv2.contourArea(cv2.UMat(i))
    if area > 1000:
        if area > max_area:
            max_area = area
            best_cnt = i
            image = cv2.drawContours(image, contours, c, (0, 255, 0), 3)
    c+=1

mask = np.zeros((gray.shape),np.uint8)
cv2.drawContours(mask,[best_cnt],0,255,-1)
cv2.drawContours(mask,[best_cnt],0,0,2)

out = np.zeros_like(gray)
out[mask == 255] = gray[mask == 255]
blur = cv2.GaussianBlur(out, (11,11), 0)
thresh = cv2.adaptiveThreshold(blur, 255, 1, 1, 11, 2)

c = 0
for i in contours:
    area = cv2.contourArea(i)
    if area > 1000/2:
        cv2.drawContours(image, contours, c, (0, 255, 0), 3)
    c+=1

These are the images I'm able to extract after finding adaptive thresholds and using contours to draw masks.

plt.imshow(out, cmap='gray')

plt.imshow(thresh, cmap='gray')

plt.imshow(image)

How can I modify my code to generate the output image, where the sudoku board is cropped as in output ?

EDIT :

I tried extracting the coordinates of the biggest blog and cropped the largest rectangle, but it's still not accurate as the output image I desire.

contours, h = cv2.findContours(out.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)  
contours = sorted(contours, key=cv2.contourArea, reverse=True)  
polygon = contours[0]  

bottom_right, _ = max(enumerate([pt[0][0] + pt[0][1] for pt in polygon]), key=operator.itemgetter(1))
top_left, _ = min(enumerate([pt[0][0] + pt[0][1] for pt in polygon]), key=operator.itemgetter(1))
bottom_left, _ = min(enumerate([pt[0][0] - pt[0][1] for pt in polygon]), key=operator.itemgetter(1))
top_right, _ = max(enumerate([pt[0][0] - pt[0][1] for pt in polygon]), key=operator.itemgetter(1))
box = [polygon[top_left][0], polygon[top_right][0], polygon[bottom_right][0], polygon[bottom_left][0]]

def distance_between(p1, p2):
    """Returns the scalar distance between two points"""
    a = p2[0] - p1[0]
    b = p2[1] - p1[1]
    return np.sqrt((a ** 2) + (b ** 2))

def crop_and_warp(img, crop_rect):
    """Crops and warps a rectangular section from an image into a square of similar size."""
    top_left, top_right, bottom_right, bottom_left = crop_rect[0], crop_rect[1], crop_rect[2], crop_rect[3]
    src = np.array([top_left, top_right, bottom_right, bottom_left], dtype='float32')
    side = max([
        distance_between(bottom_right, top_right),
        distance_between(top_left, bottom_left),
        distance_between(bottom_right, bottom_left),
        distance_between(top_left, top_right)
    ])
    dst = np.array([[0, 0], [side - 1, 0], [side - 1, side - 1], [0, side - 1]], dtype='float32')
    m = cv2.getPerspectiveTransform(src, dst)
    return cv2.warpPerspective(img, m, (int(side), int(side)))

cropped = crop_and_warp(out, box)

plt.imshow(cropped, cmap='gray')

The border is not neatly extracted unlike my desired output image. The reason I want it in this format is because then then the image can be split into 81 images with the help of a 9x9 grid and I could detect all the numbers in each of the image. The black trace above the image is causing troubles there.

You should first [threshold the image](https://docs.opencv.org/master/d7/d4d/tutorial_py_thresholding.html) so that it's only black and white. Then use `drawContours` to overwrite the black grid with white pixels (use color (255,255,255) and a wide pen width). Then you should have a picture that is only text. — bfris, Sep 16 '20 at 03:23
This may be useful for you: https://stackoverflow.com/questions/10196198/how-to-remove-convexity-defects-in-a-sudoku-square — Andrey Smorodov, Sep 16 '20 at 06:53
Does this answer your question? [How to remove convexity defects in a Sudoku square?](https://stackoverflow.com/questions/10196198/how-to-remove-convexity-defects-in-a-sudoku-square) — T A, Sep 18 '20 at 13:49
I splitted the board into 81 grids eventhough the cropping was not perfect. Later I converted the images to black text on white background and used pytesseract to read these numbers. You can check out the file sudoku_solver.py [here](https://github.com/SurajSubramanian/SudokuSolver/) — Suraj, Sep 30 '20 at 18:58

How to extract board from sudoku image?

0 Answers0