Opencv - Haar cascade - Face tracking is very slow

Question

I have developed a project to tracking face through camera using OpenCV library. I used haar cascade with haarcascade_frontalface_alt.xml to detect face.

My problem is if image capture from webcame doesn't contain any faces, process to detect faces is very slow so images from camera, which are showed continuosly to user, are delayed.

My source code:

void camera() 
{
    String face_cascade_name = "haarcascade_frontalface_alt.xml";
    String eye_cascade_name = "haarcascade_eye_tree_eyeglasses.xml";
    CascadeClassifier face_cascade;
    CascadeClassifier eyes_cascade;
    String window_name = "Capture - Face detection";
    VideoCapture cap(0);

    if (!face_cascade.load(face_cascade_name))
        printf("--(!)Error loading\n");

    if (!eyes_cascade.load(eye_cascade_name))
        printf("--(!)Error loading\n");

    if (!cap.isOpened()) 
    {
        cerr << "Capture Device ID " << 0 << "cannot be opened." << endl;
    } 
    else 
    {
        Mat frame;
        vector<Rect> faces;
        vector<Rect> eyes;
        Mat original;
        Mat frame_gray;
        Mat face;
        Mat processedFace;

        for (;;) 
        {
            cap.read(frame);
            original = frame.clone();    
            cvtColor(original, frame_gray, CV_BGR2GRAY);
            equalizeHist(frame_gray, frame_gray);
            face_cascade.detectMultiScale(frame_gray, faces, 2, 0,
                    0 | CASCADE_SCALE_IMAGE, Size(200, 200));

            if (faces.size() > 0)
                rectangle(original, faces[0], Scalar(0, 0, 255), 2, 8, 0);

            namedWindow(window_name, CV_WINDOW_AUTOSIZE);
            imshow(window_name, original);
        }

        if (waitKey(30) == 27)
            break;
    }
}

what does "very slow" mean? what's your image solution? `face_cascade.detectMultiScale` needs some time. if you did use release mode opencv libraries already there might be one more chance: maybe openCV has a gpu haarcascade detector too, but I dont know. — Micka, Nov 11 '14 at 16:42
my image solution is 640 x 480. I means function detectMultiscale processed to detect face very slow if image doesn't contain face :( — Nick Viatick, Nov 11 '14 at 16:56
So it is only slow if there is no face detected and it is fast if there are faces?!? — Micka, Nov 12 '14 at 07:55

kazarey · Answer 1 · 2014-11-12T20:39:48.993

Haar classifier is relatively slow by nature. Furthermore, there is not much of optimization you can do to the algorithm itself because detectMultiScale is parallelized in OpenCV.

The only note about your code: do you really get some faces ever detected with minSize which equals to Size(200, 200)? Though surely, the bigger the minSize - the better the performance is.

Try scaling the image before detecting anything:

const int scale = 3;
cv::Mat resized_frame_gray( cvRound( frame_gray.rows / scale ), cvRound( frame_gray.cols / scale ), CV_8UC1 );
cv::resize( frame_gray, resized_frame_gray, resized_frame_gray.size() );
face_cascade.detectMultiScale(resized_frame_gray, faces, 1.1, 3, 0 | CASCADE_SCALE_IMAGE, Size(20, 20));

(don't forget to change minSize to more reasonable value and to convert detected face locations to real scale)

Image size reducing for 2, 3, 5 times is a great performance relief for any image processing algorithm, especially when it comes to some costly stuff like detection.

As it was mentioned before, if resizing won't do the trick, try fetching some other bottlenecks using a profiler.

And you can also switch to LBP classifier which is comparably faster though less accurate.

Hope it will help.

thanks for your help but when no face in image the process detectMultiScale very slow. — Nick Viatick, Nov 15 '14 at 12:32
I am going to try put this into action I was wondering if this method would work before I saw this. I wonder how much this will deteriorate detection performance. I am rather confused how the iPhone can do it so easily compared to a raspberry pi for example. — maxisme, Oct 15 '17 at 13:13
Increasing minSize did the trick changed delay from seconds to milliseconds. Thank you sir! — eric, Apr 28 '19 at 16:05

score 1 · Answer 2 · edited Dec 23 '16 at 07:49

1

I use Haar cascade classifiers regularly, and easily get 15 frames/second for face detection on 640x480 images, on an Intel PC/Mac (Windows/Ubuntu/OS X) with 4GB Ram and 2GHz CPU. What is your configuration?

Here are a few things that you can try.

You don't have to create the window (namedWindow(window_name, CV_WINDOW_AUTOSIZE);) within each frame. Just create it first and update the image.
You can try how fast it runs without histogram equalization. Not always required with a webcam.
As suggested by Micka above, you should check whether your program runs in Debug mode or release mode.
Use a profiler to see whether the bottleneck is.
In case you haven't done it yet, have you measured the frame rate you get if you comment out face detection and drawing rectangles?

edited Dec 23 '16 at 07:49

frogatto

28,539
11
83
129

answered Nov 12 '14 at 05:34

Totoro

3,398
1
24
39

I check and i see the process detectMultislace very slow if no face in image :( – Nick Viatick Nov 15 '14 at 12:33
The waitkey(30) statement can slow down things quite a bit. See http://stackoverflow.com/questions/18701236/cvwaitkey-slowing-capture-process-significantly-workaround for a discussion on that. – Totoro Nov 17 '14 at 00:51
Are you saying that the function runs slower when there is no face?! Surely that is not true! – maxisme Oct 15 '17 at 13:15
1

I am getting 2 FPS on a 900MHZ RPI – maxisme Oct 15 '17 at 13:16
I get 12FPS when there is no detection function which seems super shit – maxisme Oct 15 '17 at 13:16
What are your compiler options? And thanks Frogatto for the great edit! – Totoro Oct 15 '17 at 21:28

ErmIg · Answer 3 · 2016-11-22T05:58:13.210

1

May be it will useful for you:

There is a Simd Library, which has an implementation of HAAR and LBP cascade classifiers. It can use standard HAAR and LBP casscades from OpenCV. This implementation has SIMD optimizations with using of SSE4.1, AVX2 and NEON(ARM), so it works in 2-3 times faster then original OpenCV implementation.

edited Nov 22 '16 at 05:58

answered May 05 '16 at 11:51

ErmIg

3,980
1
27
40

Your library looks cool. How does it compare to Agner Fog's [Vector Class Library](http://www.agner.org/optimize/#vectorclass) or Yeppp!? – Z boson May 06 '16 at 16:13
@Zboson As I know Agner Fog's Vector Class Library and Yeppp are the vector libraries of general purposes. The Simd Library more relates to image processing. – ErmIg May 11 '16 at 05:50

score 0 · Answer 4 · answered Jun 09 '16 at 22:14

0

You can use LBP Cascade to detect faces. It is much more lightweight. You can find lbpcascade_frontalface.xml in OpenCV source directory.

answered Jun 09 '16 at 22:14

fivetech

361
2
13

Opencv - Haar cascade - Face tracking is very slow

4 Answers4