-2

I am using flask-python framework , opencv and face_recognition library to do face recognition. I'm trying to run the webcam using python -opencv and if a face is detected in the frame, it is sent to the backend where the frame is processed and recognition of the person is performed but this api call with response is taking around 0.3 seconds , this delay is causing a lag in the smooth flow of webcam.

what I would like to have is the webcam run continuously and when the face is detected in the frame it should not block the free flow of webcam streaming, instead the processing of this frame should be handled separately in the background while the main thread is running webcam.

Any help in this regard is much appreciated

Hamid Sayeed
  • 163
  • 1
  • 10
  • Move your api call to asynchronous thread, which would be non-blocking call from main thread, hence giving you full FPS required, This [thread](https://stackoverflow.com/questions/1239035/asynchronous-method-call-in-python) would get you started – ZdaR Oct 15 '18 at 10:24
  • Hi @ZdaR Thank you for taking ur time. I'm little confused with moving api to asynchronous thread as i have seen threads working on a continous forever process but this api call is like one time process triggered only when the face is detected .Any example in this case will be of much help. – Hamid Sayeed Oct 15 '18 at 10:39
  • Just wrap your api call in a method and spawn a thread calling that method, when you spawn a thread, then all the instruction in that function would be called as a background task and you won't have to worry. Just read some introductory examples of threads in Python. It would be hard to explain the mechanism in comments. – ZdaR Oct 15 '18 at 11:08

1 Answers1

0

Theres a number of things that you can do to resolve this issue,

  1. Dont pass every frame into detection, chances are that you are using a good webcam, which gives you 30FPS or higher, you dont need such a high FPS for real time detection, what you could do is that, only send every 3rd or 4th frame for detection, and for the other frames, just draw the old boxes to the stream, this will give you accurate enough predictions

  2. Shift your model to GPU, face_recognition gives you the choice between using a CPU and GPU(cnn only), shifting it onto any available GPU's will speed up the process

  3. Resize your input frame, you really don't need a 1080p HD frame to detect frames, anything smaller like 480p will also do (though this is entirely dependent on your problem), resize your frame and make it smaller, the smaller the input matrix the quicker it is

  4. Vectorize your operations, instead of using the default face_recognition API for identity comparisons, i suggest you code a vectorized version of comparisons in numpy that calculates the L2 distance in large matrices, instead of having to do it in a loop, this will save alot of computation cost.

  5. Limit the number of comparisons per person. You could have 10 embeddings of person A and 10 of person B, finding the distance of the unknown embeddings with each of these 20 embeddings, can be computationally expensive, resort to doing something like, taking average embeddings, or the centroids of these embeddings, by clustering them

Imtinan Azhar
  • 1,725
  • 10
  • 26
  • I think the problem is with api call processing it .rest of it is fine .i want to run that api call in a separate thread and i'm little confused in doing it .Any help would be much appreciated. – Hamid Sayeed Oct 16 '18 at 05:48
  • i have used the api you are talking about, im too working on FR, pulling comparisons from different threads might help, but will not solve the problem, you need to apply some or all of the above methods to get your stream to around 15 fps or so, which can be considered real time – Imtinan Azhar Oct 16 '18 at 06:13
  • Thank you @Imtinan Azhar ,I'm working on it ,i'll update my results here.Thank you again for taking ur time – Hamid Sayeed Oct 16 '18 at 06:54
  • no problem :) just accept the answer if it was helpful – Imtinan Azhar Oct 16 '18 at 07:57