1

I have used cv2 to detect my eyes from an uploaded image, but I'd like to paste an image on top of where cv2 detected my face.

View cv2 eye detection photo output here:

View cv2 eye detection photo output here

But instead of outputting the cv2 rectangles around my eyes, I'd like to replace it with an image. Is there a way to paste an image on top of where cv2 detected my eyes?

My python script

import os
import numpy as np
import cv2
from os.path import join, dirname, realpath
from flask import Flask, render_template, request, redirect, url_for, abort
from werkzeug.utils import secure_filename

def upload_files():
    uploaded_file = request.files['file']
    filename = secure_filename(uploaded_file.filename)
    if filename != '':
        file_ext = os.path.splitext(filename)[1]
        if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \
                file_ext != validate_image(uploaded_file.stream):
            abort(400)
        uploaded_file.save('new.png')
   
    face_cascade = cv2.CascadeClassifier('/haarcascade_eye.xml')
    eye_cascade = cv2.CascadeClassifier('/haarcascade_eye.xml')

    img = cv2.imread('new.png')
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

    faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    for (x,y,w,h) in faces:
         cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
         roi_gray = gray[y:y+h, x:x+w]
         roi_color = img[y:y+h, x:x+w]
         eyes = eye_cascade.detectMultiScale(roi_gray)
         for (ex,ey,ew,eh) in eyes:
             cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)
    cv2.imwrite("out.png", img)

Keep in mind that the script above produces this output:

photo output.

I would like to remove the rectangles and use cv2 to paste an image where cv2 detected my eyes.

Jeru Luke
  • 20,118
  • 13
  • 80
  • 87
mattwelter
  • 39
  • 5
  • Does this answer your question? [overlay a smaller image on a larger image python OpenCv](https://stackoverflow.com/questions/14063070/overlay-a-smaller-image-on-a-larger-image-python-opencv) – Ken Y-N Feb 22 '21 at 01:58
  • A little bit but not quite, how do I tell cv2 to paste the image where my eyes were detected? – mattwelter Feb 22 '21 at 02:01

1 Answers1

0

I think that this is what you're looking for:

import os
import numpy as np
import cv2
from os.path import join, dirname, realpath

def upload_files():
   
    face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
    eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')

    img = cv2.imread('face.jpg')
    img_to_place = cv2.imread('img.png')

    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    gray_to_place = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    
    img_h, img_w = gray.shape
    img_to_place_h, img_to_place_w = gray_to_place.shape

    faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    for (x,y,w,h) in faces:
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = img[y:y+h, x:x+w]
        eyes = eye_cascade.detectMultiScale(roi_gray)
        for (ex,ey,ew,eh) in eyes:
            resized_img = cv2.resize(img_to_place, (eh, ew), interpolation = cv2.INTER_AREA)
            resized_img_h, resized_img_w, _ = resized_img.shape

            roi_color[ey:ey+resized_img_h, ex:ex+resized_img_w, :] = resized_img

    cv2.imwrite('out.png', img)

upload_files()

Basically, you load the image you want to put in your eyes, resize it to ROI size, and put it in the coordinates of where the rectangle would be.

Example:

enter image description here

Lucius
  • 1,246
  • 1
  • 8
  • 21
  • That image is exactly what I'm looking for, but the code produces this error: `cv2.error: OpenCV(4.4.0) /private/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/pip-req-build-gi6lxw0x/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor` It looks likes a problem with reading the image, do you know how I can fix this? – mattwelter Feb 23 '21 at 17:52
  • I'm dumb, just had to replace the file name... THANK YOU! Quick question, this it what it produced: https://i.imgur.com/vTRux2s.jpg How can I only output the ones on my eyes? Would something like `for x in range(2)` work? – mattwelter Feb 23 '21 at 17:58
  • I think this is a false positive, there are many ways to solve it, like training a better cascade: https://learnopencv.com/training-better-haar-lbp-cascade-eye-detector-opencv/ (I haven't tried this tutorial, but seems good). This is a complex solution, you could, instead, compare the ratio of the detected eye with the size of the face, to then try to eliminate false positives. Detect only two eyes per face can work, but you could detect the wrong two eyes also. – Lucius Feb 23 '21 at 18:15
  • Thank you! Would you think doing something like: `if ew == top 2 largest numbers in list then paste those image only` in a range(0,2) would work? How can I only return the image with only the largest widths? Does that make sense? As in, putting this code in: `pointsOnFace = []` `integersToAppend = ew` `pointsOnFace.append(integersToAppend)` `print(pointsOnFace)` Returns `[22]` `[196]` `[74]` `[225]` `[52]` `[44]` – mattwelter Feb 23 '21 at 19:50
  • Yeah, selecting the largest two eyes seems like a simple but good approach. You can do it using the variables `eh` and `ew` put them on a list and make another loop to insert the image on the two larger only. – Lucius Feb 24 '21 at 06:17