OpenCV - Draw contours on fingers using convex-hulls & adaptive thresholding

Question

I am pretty new to OpenCV and am trying to achieve drawing simple contours along the outline of my hand using a webcam. I decided on using cv2.adaptiveThreshold() to deal with the different light intensities when the camera adjusts to the hand moving. Everything seems to work fine except that it is struggling with finding the fingers and then also drawing closed contours. See here:

I thought about trying to detect a convex hull and detect anything deviating from it somehow.

How do I go about this best? Firstly I need to manage to maybe not find weird closed contours and then go from there?

Here's the code, I fixed the trackbar values for you :)

import cv2
import numpy as np

#####################################
winWidth = 640
winHeight = 840
brightness = 100

cap = cv2.VideoCapture(0)
cap.set(3, winWidth)
cap.set(4, winHeight)
cap.set(10, brightness)

kernel = (7, 7)


#######################################################################
def empty(a):
    pass


cv2.namedWindow("TrackBars")
cv2.resizeWindow("TrackBars", 640, 240)
cv2.createTrackbar("cVal", "TrackBars", 10, 40, empty)
cv2.createTrackbar("bSize", "TrackBars", 77, 154, empty)


def preprocessing(frame, value_BSize, cVal):
    imgGray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    # mask = cv2.inRange(imgHsv, lower, upper)
    imgBlurred = cv2.GaussianBlur(imgGray, kernel, 4)
    gaussC = cv2.adaptiveThreshold(imgBlurred, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, value_BSize,
                                   cVal)
    imgDial = cv2.dilate(gaussC, kernel, iterations=3)
    imgErode = cv2.erode(imgDial, kernel, iterations=1)

    return imgDial


def getContours(imPrePro):
    contours, hierarchy = cv2.findContours(imPrePro, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
    for cnt in contours:
        area = cv2.contourArea(cnt)
        if area > 60:
            cv2.drawContours(imgCon, cnt, -1, (0, 255, 0), 2, cv2.FONT_HERSHEY_SIMPLEX)
            peri = cv2.arcLength(cnt, True)
            approx = cv2.approxPolyDP(cnt, 0.02 * peri, True)


#######################################################################################################

while cap.isOpened():
    success, frame = cap.read()
    cVal = cv2.getTrackbarPos("cVal", "TrackBars")
    bVal = cv2.getTrackbarPos("bVal", "TrackBars")
    value_BSize = cv2.getTrackbarPos("bSize", "TrackBars")
    value_BSize = max(3, value_BSize)
    if (value_BSize % 2 == 0):
        value_BSize += 1

    if success == True:
        frame = cv2.flip(frame, 1)
        imgCon = frame.copy()
        imPrePro = preprocessing(frame, value_BSize, cVal)
        getContours(imPrePro)
        cv2.imshow("Preprocessed", imPrePro)
        cv2.imshow("Original", imgCon)

        if cv2.waitKey(1) & 0xFF == ord("q"):
            cv2.destroyAllWindows()
            break

Can you post a (raw) image of your hand without any processing? — stateMachine, Feb 01 '21 at 21:56
I can, but it is camera input and no static image :) One moment — itsolidude, Feb 01 '21 at 22:01

score 2 · Accepted Answer · answered Feb 08 '21 at 22:00

L*a*b color space can help find objects brighter than the background. One advantage is that color space is hardware independent, so it should yield relatively similar results from any camera. Using the OTSU option to threshold the image can help it work in different lightning conditions, as it calculates the optimal threshold intensity to separate bright and dark areas in the image. Obviously it is not a silver bullet and will NOT work perfectly in every situation, especially in extreme cases, but as long your hand's brightness is relatively different from the background, it should work.

lab = cv2.cvtColor(frame, cv2.COLOR_BGR2LAB)
tv, thresh = cv2.threshold(lab[:,:,0], 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
plt.imshow(thresh)

Once the hand is properly thresholded, you can proceed to find the contours and do your analysis as needed.

Note: the artifacts in the threholded image are caused by removing the green contour lines from the original posted image.

score 0 · Answer 2 · answered Feb 01 '21 at 22:35

0

I'm using a light threshold so this might work differently depending on the image, but here's what works for this one.

import cv2
import numpy as np

# load image
img = cv2.imread("hand.jpg");

# lab
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB);
l,a,b = cv2.split(lab);

# threshold
thresh = cv2.inRange(l, 90, 255);

# contour
_, contours, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE);

# filter contours by size
marked = img.copy();
cv2.drawContours(marked, contours, -1, (0, 255, 0), 3);

# show
cv2.imshow("marked", marked);
cv2.imshow("Thresh", thresh);
cv2.waitKey(0);

# save
cv2.imwrite("marked_hand.png", marked);

answered Feb 01 '21 at 22:35

Ian Chu

2,924
9
14

The problem is that my code is for a Webcam:(. A global threshold wont work – itsolidude Feb 01 '21 at 22:49
what situations do you want to be able to contour out your hand? Can you give a series of images that are representative of what you might see so that we can try to find a technique that'll work for all of them? – Ian Chu Feb 01 '21 at 22:51
I am trying to work on a program that detects hand gestures. (Thinking about training it using tensor flow) So the person will most likely sit in their office in front of the camera and not somewhere outside but the light intensities will most likely change since a webcame constantly changes these values. I made it work with background subtraction but it is very suboptimal if I move the camera a bit. – itsolidude Feb 01 '21 at 22:57

score 0 · Answer 3 · answered Feb 11 '21 at 23:04

0

Have you looked into google's mediapipe, as an alternate to opencv?

Also, I wonder if adding a thin bottom black border to the frame would help the contour to know to wrap around the wrist.

answered Feb 11 '21 at 23:04

Jen H

147
8

OpenCV - Draw contours on fingers using convex-hulls & adaptive thresholding

3 Answers3