Carving Out exactly humans From Image

Question

I want to carve out the exact human out of the image currently im able to create a box around the humans the code that I'm using is as follows:

import cv2
import numpy as np
import imutils

protopath = r"C:\Users\Admin\MobileNetSSD_deploy.prototxt"
modelpath = r"C:\Users\Admin\MobileNetSSD_deploy.caffemodel"
detector = cv2.dnn.readNetFromCaffe(prototxt=protopath, caffeModel=modelpath)

CLASSES = ["background", "aeroplane", "bicycle", "bird", "boat",
           "bottle", "bus", "car", "cat", "chair", "cow", "diningtable",
           "dog", "horse", "motorbike", "person", "pottedplant", "sheep",
           "sofa", "train", "tvmonitor"]


def main():
    image = cv2.imread(r'E:\trial2.jpg')
    image = imutils.resize(image,300)

    (H, W) = image.shape[:2]

    blob = cv2.dnn.blobFromImage(image, 0.007843, (W, H), 127.5)

    detector.setInput(blob)
    person_detections = detector.forward()

    for i in np.arange(0, person_detections.shape[2]):
        confidence = person_detections[0, 0, i, 2]
        if confidence > 0.5:
            idx = int(person_detections[0, 0, i, 1])

            if CLASSES[idx] != "person":
                continue
                

            person_box = person_detections[0, 0, i, 3:7] * np.array([W, H, W, H])
            (startX, startY, endX, endY) = person_box.astype("int")
            cv2.rectangle(image, (startX, startY), (endX, endY), (0, 0, 255), 2)

    cv2.imshow("Results", image)
    cv2.waitKey(0)
    cv2.destroyAllWindows()
main()

What i want I want to outline the humans exactly

Any help or tips will be greatly appreciated

Edit 1: Suggested by Sunil Kumar

import numpy as np
import cv2
from matplotlib import pyplot as plt

img = cv2.imread(r'E:\trial3.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
# noise removal
kernel = np.ones((3,3),np.uint8)
opening = cv2.morphologyEx(thresh,cv2.MORPH_OPEN,kernel, iterations = 2)

# sure background area
sure_bg = cv2.dilate(opening,kernel,iterations=3)

# Finding sure foreground area
dist_transform = cv2.distanceTransform(opening,cv2.DIST_L2,5)
ret, sure_fg = cv2.threshold(dist_transform,0.7*dist_transform.max(),255,0)

# Finding unknown region
sure_fg = np.uint8(sure_fg)
unknown = cv2.subtract(sure_bg,sure_fg)
# Marker labelling
ret, markers = cv2.connectedComponents(sure_fg)

# Add one to all labels so that sure background is not 0, but 1
markers = markers+1

# Now, mark the region of unknown with zero
markers[unknown==255] = 0
markers = cv2.watershed(img,markers)
img[markers == -1] = [255,0,0]
cv2.imshow("Results", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

and the current output that I'm getting

The SSD network is an object detector model; "object detections" are generally defined as bounding boxes. What you're asking for is a "mask", generally this problem is called "instance segmentation" and you'll need to run a different model, like Mask R-CNN, not YOLO or SSD, etc. See here for a basic overview: https://towardsdatascience.com/a-hitchhikers-guide-to-object-detection-and-instance-segmentation-ac0146fe8e11 — alkasm, Jan 19 '21 at 07:29
@isAif that is my current result......expected result is to create the outline on humans — Sachin Rajput, Jan 19 '21 at 07:30

score 0 · Answer 1 · answered Jan 19 '21 at 10:50

What you are looking for is instance segmentation, which is much more accurate than object detection:
While object detection only outputs a bounding box around each person (as you already noticed). Instance segmentation attempts to output an exact mask for each instance - which is what you are looking for.

I would try out the following instance segmentation frameworks:

Carving Out exactly humans From Image

1 Answers1