How to tell if the most obvious object in a picture is a shape?

Question

Okay guys, I think I have done everything I can except one important thing: Shape Extraction. I already do this fairly in a fairly simple way, but there are some cases where it really messes up. The way I do this is:

• Pick a point in the outline list by generating a bounding rectangle and selecting the closest point to the top left (Point p).
• Create a new Shape object and add p to the shape's outline.
• If the main picture has no more points available for us to test, we simply return the shape object.
• Get a closest point from p and store it in closest.
• while the distance from the closest
  point to p point is less than or
  equal to ten
• ----Add it to the outline
• ----Remove it from the master point list
• ----Set p to closest
• ----Get another closest point from closest
• ----Repeat
• If the shape has ten points or less in its outline, return a null object (ignore all small shapes)
• Otherwise, return the shape object.

I repeat this process until the main point list becomes empty. This means we extracted all the shapes.

Now I do a combine shapes several times. This makes it so if I have |, ---, and | next to each other, it will combine to create a rectangle. Do you understand what I mean? Lets say I have a circle, the above extracting code will sometimes say half of the circle is one shape, and the other half is another shape. So when I combine the shapes, it becomes one circle.

Sigh, I cannot post pictures too and I cannot upload this anywhere but an uploading site. This has some issues. Take a look at the following:

The picture on the left is the starting picture, and the one on the right is the outline points. Now I click the determine shape button. It determines the shape of the most dominant shape in the picture (the shape object that contains the most points in its outline).

Now, it correctly says this is a quadrilateral but for the wrong reasons. Due to my combine shapes (which needed to be done in order to even get a rectangle shape otherwise it will be a small line for one shape and another small line as another shape), it added the outline of the fingers to the testing.

So the entire single shape (the black outline to the right in the second picture) is tested. Not just the "rectangle" part, but everything in that picture is tested. Can you guys think of any way to clean this thing up and extract JUST the rectangle portion instead of having the fingers included? I was thinking of some variation of A* for this, but in a case like this picture, it does not create a loop. So what do I do once A* visits every single point (since it cannot return to the starting point). What do I do then?

Can anybody help me try to get this figured out please?

Answer 1

I think the following part of your code is not so wise:

    int iSelected = selected.ToArgb();
    int iNextRight = nextRight.ToArgb();
    if (Math.Abs(iSelected - iNextRight) > alpha)

because the result is you compare almost only against red color (the others are stored in less significant bites) assuming you don't use alpha in the images.

If you want to achieve better results, you can use Canny Edge Detector or at least take a look at its processing steps.

Other option is to use feature similar to magic wand to separate main object from the background and later extract its edges. For that purposes for example potts model can be used.

If you want something simpler what about detect edges the way yo do it, but use sum of difference of all color channels. Then assume that the point in the middle is part of searched object and fill the shape between nearest edges like paintbrush. If you choose right treshold you will eliminate those fingers.

I wish you good luck.