Obtaining binary mask of skin from bounding box

Question

I have trained an object detection model to detect fingertips of hand. I have extracted the fingertips from the bounding box coordinates. Here is what I got:

Now I want to get the binary mask from these images, which include only the region of fingertip. I have tried these approaches and did not get satisfactory results.

Used canny edge detector approach.
Used Grabcut approach.
Normalized image, then extracted h channel from HSV to find the largest contour.

I want to extract the fingertip region from these bounding boxes, I am currently lost, will appreciate any leads in the right direction.

EDIT Images with red background:

if you already have the neural network and data to train it, you could also just train one to give you this **segmentation** — Christoph Rackwitz, May 25 '22 at 10:48

score 3 · Accepted Answer · answered May 25 '22 at 09:08

3

You can opt for the LAB color space.

One extreme of the A-channel identifies red color. Hence applying Otsu threshold on this extreme can isolate it pretty well.

Code:

img = cv2.imread('finger_1.png')

# convert to LAB color space
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)

# apply Otsu threshold on A-channel
th = cv2.threshold(lab[:,:,1], 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]

Results:

As a further option, you can try normalizing the same A-channel and proceed further.

Also have a look at this post on how to segment green color.

answered May 25 '22 at 09:08

Jeru Luke

20,118
13
80
87

1

This worked perfectly on most of the images that I am testing on. – Muhammad Hassan May 25 '22 at 09:34
there is one case where this method fails and that is if we have red background, I think that's understandable as "A" channel corresponds to Red/Green value. Is there anyway we can make it work for such images as well? – Muhammad Hassan May 30 '22 at 05:29
I have added images with red background in question edit. – Muhammad Hassan May 30 '22 at 05:35
1

@MuhammadHassan You can try enhancing this method by also including edge detection – Jeru Luke May 30 '22 at 06:11

Obtaining binary mask of skin from bounding box

1 Answers1