Perform object recognition based on colour

Question

I have captured each frame of a video .Then I have used background subtraction to eliminate the background .Now I have the people with their bounding boxes.I have to compare the colour features of this person with the features of the same person in another video. The person will be wearing the same dress in every video. I am developing this in opencv 2.4 and python 2.7

Here it the background subtraction code I have used:

import numpy as np
import cv2
cap = cv2.VideoCapture('test.mp4')
fgbg = cv2.BackgroundSubtractorMOG()
j=0
count = int(cap.get(cv2.cv.CV_CAP_PROP_FRAME_COUNT))
while j<count:
    ret, frame = cap.read()
    cmask = fgbg.apply(frame)
    fgmask = cmask.copy()
    floodfill =cmask.copy()

    (cnts, _) = cv2.findContours(cmask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    cnts = sorted(cnts, key = cv2.contourArea, reverse = True)
    if(len(cnts)!=0):
        h, w = floodfill.shape[:2]
        mask = np.zeros((h+2, w+2), np.uint8)
        cv2.floodFill(floodfill, mask, (0,0), 255)
        floodfill_inv = cv2.bitwise_not(floodfill)
        fgmask=fgmask|floodfill_inv

    # screenCnt = None
    print "K="+str(j)
    j+=1
    for cnt in cnts:
        x,y,w,h = cv2.boundingRect(cnt)
        cv2.rectangle(fgmask,(x,y),(x+w,y+h),255,4)
    if(len(cnts)!=0):
        cv2.imshow('frame',fgmask)

    k = cv2.waitKey(30) & 0xff

    if k == 27:
        break
cap.release()
cv2.destroyAllWindows()

Is there any method for comparing objects based on the colour?

I have not used face recognition because I don't have pictures of the people in the video

Any help would be appreciated

Answer 1

You could try to extract the 'average' color of an image, since the dress will have a large impact on the color shade of the entire image (since you extracted it, it makes up most of the image).

However, depending on lighting conditions and the way you extract the persons from the image, the values can change from time to time. It's therefore most likely not the most secure method, especially not when these persons have similar colors of clothing. But you could give it a try.

You can use the following code to extract the average color of an image (you can apply this to the extracted image you have):

# Get the average color values of each row in an array
average_row_color = numpy.average(input_img, axis=0)
# Now get the average of that array to get the overall average image color
average_image_color = numpy.average(average_row_color, axis=0)
# Convert it to uint8 to get your 3 channel color values, like [100, 150, 200]
average_color_uint8 = numpy.uint8(average_image_color)

You can then compare your results and see how close they are to colors you found in other videos and match the colors that are closest.

Answer 2

I'd consider to match using template matching on color images.

Another option, if applicable, is to extract and match features using feature based methods (SIFT, ect). best option.

Another option (sort of a shortcut) would be to calculate the histogram between the blobs, find the most common color, than match the colors on HSV color space.

Good luck

Answer 3

What you want is Swain and Ballard's Color Indexing technique.

Basically, you create a color histogram to represent the object/person. Using the information in the histogram, you can create a probability mass function of another image that represents the probability of an object being at a certain point. Here's a python/opencv tutorial that explains how to do the back projection. The peak of the probability mass function is the most likely location of your object/person.

You probably may not need to do the back projection if you are just trying to compare two known objects. Here's a previous question about ways to compare color histograms that will give you some options.

If your video has significant lighting changes, you probably won't get the results you need with this technique alone, but someone else came up with a way to introduce color constancy to the method. Instead of using a histogram of colors, you use a histogram of color ratios.