How to extract a 2D array from an image of the same array?

Question

Is it possible to extract a 2D matrix from an image of the same? Having no related experience in this area, I am having trouble proceeding further.

For example, if the image of the array is this, the corresponding 2D array(with blanks denoted by 0) would be as follows:

5 3 0 0 7 0 0 0 0
6 0 0 1 9 5 0 0 0
0 9 8 0 0 0 0 6 0
8 0 0 0 6 0 0 0 3
4 0 0 8 0 3 0 0 1
7 0 0 0 2 0 0 0 6
0 6 0 0 0 0 2 8 0
0 0 0 4 1 9 0 0 5
0 0 0 0 8 0 0 7 9

I would like to know some algorithm or software that would help me extract the array. As I have no prior experience, even a push in the right direction would be appreciated.

Background: I was working on a basic Sudoku solver using Java, and I have implemented the same with a basic backtracking algorithm. Now, instead of giving the input manually by typing out the 2D array, I want to obtain the same from an image of the array.

http://www.aishack.in/tutorials/sudoku-grabber-opencv-detection/ — MBo, Jul 14 '18 at 12:55

score 1 · Accepted Answer · answered Jul 14 '18 at 13:12

Image recognition is not as simple as it appear to be. A low quality picture of low resolution with shadows or with shiny parts brighter than the rest is hard to deal with. Let alone that 3D angle of the camera and 2D rotations in the image plane. All those factors should be diminished or eliminated before image recognition even start.

Assuming you have a clean input image with known width and height what you need is to chop the input image in several squares corresponding to matrix entries. Then for each small subimage run a number recognition algorithm.

For the first part, many times it is better to transform the image from 24bits rgb colors to 8-bits grayscale. In that way image pixels with almost the same color in rgb space will be clustered to have the same intensity in the 8bit grayscale space. Even binary image with only two intensities would be useful in this case. There are image treatment packages that can do it for you. Then chopping the image is not harder than doing 2D array manipulation.

For the second part you can discard all squares with same intensity as empty. For the nonempty squares you have to call a number recognition algorithm.

You can use many packages for pattern recognition out there such as OpenCV or specific OCR (optical character recognition) packages.

It is not hard to write your own feedforward neural network for that:

https://en.m.wikipedia.org/wiki/Feedforward_neural_network

How to extract a 2D array from an image of the same array?

1 Answers1