0

This is a continuation of this question : Finding Squares in Image

I followed the steps in my answer there : https://dsp.stackexchange.com/a/7526/818, And I got the answer as given below :

enter image description here

But at the end of that answer, I have explained a problem, and that is my question.

Explanation :

I already have the centroids of detected squares from step 1 in the previous link (and those detected squares are marked in mask_image below):

enter image description here

I created a grid image as below ( I know their centroid values also) :

enter image description here

I also found which point in grid image to be mapped to corresponding point in mask_image.

With that information, I applied scipy.interpolate.griddata(), and then OpenCV's cv2.remap() function.

And its result is given below :

enter image description here

As you can see, all the squares except the two at center are clipped. It is like, output contain only the region inside a boundary drawn connecting all the centroids of the mask_image.

Scene becomes more worse below :

The case is more worse when the last square (yellow) or any other square in four corners is not detected in first step. Consider last one is not detected. Then below is the result I get and you can see a slant cut at the bottom (marked with yellow color):

enter image description here

Question :

Why remap function not working beyond the points I have given ? And what should I do to remap it without clipping ?

I thought it would work for the full image even if I give some points, which are not at edges.

Expected Output :

Below is the output I expected at the end of my operation. (Region inside red boundary is what I got actually got now )

enter image description here

Looking for some good suggestions ...

UPDATE :

I also add the code here. Only remapping part code is added. Full code is too big to be added here :

# ideal - the grid image - https://i.stack.imgur.com/3QudG.png
# centroids - list of centroids of the squares in mask_image - https://i.stack.imgur.com/jh6bQ.png
# match_pts - list of centroids of the squares in grid image corresponding to squares in mask_image
# warped - the final image obtained after remap - https://i.stack.imgur.com/O26ZA.png

grid_x,grid_y = np.meshgrid(np.arange(ideal.shape[1]),np.arange(ideal.shape[0]))
dst = np.array(centroids) 
src = np.array(match_pts) 
grid_z = griddata(dst,src,(grid_x,grid_y),method='cubic')
map_x_32 = grid_z[:,:,0].astype('float32')
map_y_32 = grid_z[:,:,1].astype('float32')
warped = cv2.remap(ideal, map_x_32, map_y_32, cv2.INTER_CUBIC)

Also added the datas like centroids, match_pts etc, so that if someone want to try it out, they can directly use the data instead of finding it from image : gist.github.com/4540887

Community
  • 1
  • 1
Abid Rahman K
  • 51,886
  • 31
  • 146
  • 157
  • Care to share the code? my guess is that you have made a small mistakes int he code / a small correction fixes that... – fireant Jan 15 '13 at 18:26
  • This comment is only barely-relevant as I didn't read this question at all because after reading the starting considerations of the linked answer I'm not sure why is it considered that this top image of your question was obtained using good methods. For instance, if you had this image: http://i.imgur.com/fBFAM.png, would it make things easier ? – mmgp Jan 15 '13 at 19:21
  • The linked question was actually asked here in SOF, finally migrated to dsp. And since there was not much better answers I started bounty. So if you have any good methods, feel free to post your answer there. And from your image, I think it can be a good solution, if you please could explain how to implement it. – Abid Rahman K Jan 15 '13 at 19:23
  • Please, those who downvote the question put a comment what is the problem with the question. `If I don't know the problem, how can I improve it?` – Abid Rahman K Jan 16 '13 at 01:58

1 Answers1

2

Step 1: Whatever final binary image you are getting from analyzing in B,G,R,H,S,V plane, in that image do a blob counting algorithm.

Step 2: Find the largest blob on basis of area or contour length. Since your blobs will be mostly parallelogram types so area or contour, any one will do.

Step 3: With the largest blob (since largest blob is the best blob resembling your real world squares) try to find the orientation of the blob...this you can get by a fitting a best fit rectangle OR you can get the corner points...get the slope of the lines joining them (in both horizon and vertical direction).

Step 4: Once you get the two slopes draw two lines running through the axis of the blob. for axis you can average the corner points or you can use the centroid (center of mass)...I would go with average of corner points...

Step 5: Since in each horizontal and vertical direction, spacing is equal (ideally horizontal and vertical spacing are also equal as it comes from your ideal square picture but we will not assume it..) just need to locate the possible centroids of the other parallelograms

BOTTOM LINE: If any one square gets detected perfectly you can make the whole grid. Just keep marking centers at an interval of 2H (H = horizontal width of biggest blob) along the horizontal axis of the biggest blob and at an interval of 2V (V = vertical height of biggest blob) vertically along the vertical axis of the blob.

Some pics to support

enter image description here

enter image description here

rotating_image
  • 3,046
  • 4
  • 28
  • 46
  • This method I knew, But I felt the method I explained in my answer is more better (just an intuition, didn't try it), because, in some cases, there are chances that input image may have some defects as in http://stackoverflow.com/questions/10196198/how-to-remove-convexity-defects-in-sudoku-square . So it would be difficult to fit them on a line. ie why I went for remap approach. – Abid Rahman K Jan 15 '13 at 19:02
  • I presume the problem image is the image given by you...not sudoku square detection image...in your image the squares are sparsely spaced and with distinct colors(some of them)...so I am confident at least one square will be prominently detected, of course the orange or the yellow one...(some where you also say chances of orange square detection is high..)...so one square in your image and the whole grid is ready..!! – rotating_image Jan 15 '13 at 19:12
  • Hi, do you mind posting this answer to here : http://dsp.stackexchange.com/questions/3595/finding-squares-in-image . I have called for a bounty there and your answer more suits there. Here, I am much more interested in solving it with remap method. It would be great if you could explain your answer with some code in dsp.stackexchange. – Abid Rahman K Jan 15 '13 at 19:26