how to notify user if there are common faces detected between two directories with python and opencv

Question

Firstly, I am sorry if the title is long. I am working on face detection using python. I am trying to write a script where it will notify user when there is same picture or almost same picture/faces detected between two directories/folder. Below is the script that I wrote so far.

import cv2
import glob, requests

def detect1():
    face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')

    for img in glob.glob('/Users/Ling/Pythonfiles/Faces/*.jpg'):
        cv_img = cv2.imread(img)
        gray = cv2.cvtColor(cv_img, cv2.COLOR_BGR2GRAY)
        faces1 = face_cascade.detectMultiScale(gray, 1.3, 5)

        for (x,y,w,h) in faces1:
            cv2.rectangle(cv_img,(x,y),(x+w,y+h),(255,0,0),2)


def detect2():
    face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')

    for image in glob.glob('/Users/Ling/Pythonfiles/testfolder/*.jpg'):
        cv_image = cv2.imread(image)
        gray = cv2.cvtColor(cv_image, cv2.COLOR_BGR2GRAY)
        faces2 = face_cascade.detectMultiScale(gray, 1.3, 5)

        for (x,y,w,h) in faces2:
            cv2.rectangle(cv_image,(x,y),(x+w,y+h),(255,0,0),2)

def notify():
    if detect2 == detect1:
        key = "<yourkey>"
        sandbox = "<yoursandbox>.mailgun.org"
        recipient = "<recipient's email>"

        request_url = 'https://api.mailgun.net/v2/{0}/messages'.format(sandbox)
        request = requests.post(request_url, auth=('api', key),
            data={
            'from': '<sender's email',
            'to': recipient,
            'subject': 'face detect',
            'text': 'common face detected'
        })
        print 'Status: {0}'.format(request.status_code)
        print 'Body:   {0}'.format(request.text)

There is no error but there is no notification either. I have a folder with 10 pictures of random faces I downloaded it from Google Image(just for learning purpose)and another folder with 2 picture of people that their face is same as the some of the picture in the previous folder. The picture with the same face is in different angle.

I wrote the script by referring to tutorial from https://pythonprogramming.net/haar-cascade-face-eye-detection-python-opencv-tutorial/ and add some line to send the notification if the program detect the same face from both folder.

My question is how do I exactly notify the user if there are same faces detected. I believe this code is incomplete and hoping that someone can give me suggestion on what to add/edit or what I should not write in this script.

Thank you in advance.

Answer 1

I don't know if I understand you correctly, but I think your looking for face recognition not only a face detection.

The Haar Feature-based Cascade Classifier learned very generell "How a face should look like". It detects the positions of a learned object/shape in a given input image and returns the bounding boxes.

So if you want to know if the detected face matches with a known face you need to train a recognizer. OpenCV has 3 build-in face recognizer: EigenFaceRecognizer, FisherfaceRecognizer, LBPHFaceRecognizer (Local Binary Patterns Histograms Face Recognizer).

use them with e.g. recognizer = cv2.createLBPHFaceRecognizer()

You need a training set for your users. Maybe your trainings folder could look like:

1_001.jpg, 1_002.jpg, 1_003.jpg, 2_001.jpg 2_002.jpg, ..., n_xyz.jpg

where n is the label (user id -> unique for each user) and xyz is maybe a description or a sequence number.

Update:

I used the Faces94 benchmark dataset for testing. Therefore I packed them into the folder trainingSamples and two of them (same person but different face) into the folder testFaces relative to my python script.

To rename all images in a folder matching with the pattern above I used a bash command rename

eg. asamma.[1-20].jpg to 001_[1-20].jpg

rename 's/^asamma./001_/' *

import cv2
import numpy as np
import os

class FaceRecognizer:
    def __init__(self):
        self.cascadeClassifier = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')
        self.faceRecognizer = cv2.face.createLBPHFaceRecognizer()

        if os.path.isfile('faceRecognizer.xml'):
            self.faceRecognizer.load('faceRecognizer.xml')
        else:
            images = []
            labels = []
            for file in os.listdir('trainingSamples/'):
                image = cv2.imread('trainingSamples/'+file, 0)
                images.append(image)
                labels.append(int(file.split('_')[0]))
                ## if you don't have pre-cropped profile pictures you need to detect the face first
                # faces = self.cascadeClassifier.detectMultiScale(image)
                # for (x, y, w, h) in faces
                #     images.append(image[y:y+h, x:x+w])
                #     labels.append(int(file.split('_')[0]))

            self.faceRecognizer.train(images, np.array(labels))
            self.faceRecognizer.save('faceRecognizer.xml')

    def predict(self, image, filename):
        user, confidence = self.faceRecognizer.predict(image)
        if confidence < 100.0:
            print('found user with id {} in picture {} with a confidence of {}'.format(user, filename, confidence))

        ## if you don't have pre-cropped profile pictures you need to detect the face first
        # faces = self.cascadeClassifier.detectMultiScale(image)
        # for (x, y, w, h) in faces
        #     user, confidence = self.faceRecognizer.predict(image[y:y+h, x:x+w]) 
        #     # confidence of 0.0 means perfect recognition (same images)
        #     if confidence < 100.0:
        #         print('found user with id {} in picture {} with a confidence of {}'.format(user, filename, confidence))

faceRecognizer = FaceRecognizer()
for file in os.listdir('testFaces/'):
    image = cv2.imread('testFaces/'+file, 0)
    faceRecognizer.predict(image, file)

The code produces the output:

found user with id 4 in picture 004_20.jpg with a confidence of 27.836526552656732
found user with id 1 in picture 001_6.jpg with a confidence of 22.473253497606876`

So it correctly recognize user 4 and user 1.

The code is tested with OpenCV 3.1-dev on Ubuntu 15.10 using Python 3.4.3 and Python 2.7.9.