How to integrate Huawei HMS Face Liveness Detection with CameraView?

Question

First, I tried Huawei Face Liveness Detection. With the sample code, it works.

Next, I tried CameraView. Also, by just following the sample code, I am able to perform frame processing, achieving face detection and face recognition.

<com.otaliastudios.cameraview.CameraView
        app:cameraFacing="front"
        android:id="@+id/cameraView"
        app:cameraEngine="camera2"
        app:cameraPreview="glSurface"
        android:keepScreenOn="true"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:visibility="visible"
        app:cameraAudio="off"
        app:cameraExperimental="true">
</com.otaliastudios.cameraview.CameraView>

Question: How to integrate Huawei Face Liveness Detection into CameraView?

Provided the Face Liveness Detection code below, I tried changing the view container (mPreviewContainer as below), but it just throws error and app exits.

protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_liveness_custom_detection);
        mPreviewContainer = findViewById(R.id.surface_layout); //<------ Change this to CameraView
        mlLivenessDetectView = new MLLivenessDetectView.Builder()
                .setContext(this)
                .setFaceFrameRect(new Rect(0, 0, 0, 200))
                .setDetectCallback(new OnMLLivenessDetectCallback() {
                ...

    }

I am curious how to integrate the Huawei Face Liveness Detection into CameraView (or even normal Camera2 or CameraX)? Can the HMS take the input frames from CameraView, instead of opening another camera?

P.S.: The first error appeared (out of the lengthy message):

I/BufferQueue: [unnamed-11129-0](this:0x70859fb800,id:0,api:0,p:-1,c:-1) BufferQueue core=(11129:com.example.cv1)
E/AndroidRuntime: FATAL EXCEPTION: CameraViewEngine
    Process: com.example.cv1, PID: 11129
    com.otaliastudios.cameraview.CameraException
        at com.otaliastudios.cameraview.engine.Camera2Engine$2.onDisconnected(Camera2Engine.java:435)
        at android.hardware.camera2.impl.CameraDeviceImpl$7.run(CameraDeviceImpl.java:252)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:226)
        at android.os.HandlerThread.run(HandlerThread.java:65)
E/CameraEngine: EXCEPTION: Handler thread is gone. Replacing.
E/CameraEngine: EXCEPTION: Scheduling on the crash handler...

Thanks for your reply. I will edit my post to include this later. currently away from workstation. — zeroflaw, Dec 08 '20 at 09:14
You may see my answer below. The liveness detection could not be integrated into CameraView for its logic of frame sending is encapsulated. :( — zhangxaochen, Dec 08 '20 at 09:21
i see. it is a great feature. hopefully it could be enhanced one day in future. — zeroflaw, Dec 08 '20 at 09:32
@shirley as the entire error is too long, i only show the first exception here. Based on the wording, i feel it may be caused by camera competition by 2 processes... — zeroflaw, Dec 09 '20 at 02:42

zhangxaochen · Accepted Answer · 2021-01-21T07:34:42.810

0

Update:

Please kindly refer to ML Kit Face Verification. It recognizes and extracts key features of the face in the template, compares the features with those of the face in the input image, and then determines whether the two faces belong to the same person based on their similarity.

To achieve liveness detection and face detection/face recognition, there are two services we need: liveness detection service and face detection (Actually face comparison service, which will be supported in 2021.) Currently, HMS Liveness Detection does not support the method of CameraView (by taking the input frames) to achieve face recognition. You may try this two services: Facial recognition (LocalAuthentication Engine) or Facial comparison (HiAI Engine).

Q: Can the HMS take the input frames from CameraView, instead of opening another camera?

No, it cannot take input frames from CameraView. Because the liveness detection is a multi-frame detection solution. Currently, the logic of frame sending is encapsulated. You app only needs to apply for the camera permission and use the camera on the device for identification or detection.

edited Jan 21 '21 at 07:34

answered Dec 08 '20 at 08:52

zhangxaochen

32,744
15
77
108

i see. So based on this understanding, are 'face detection + recognition' and 'face liveness detection' treated as separate processes (open camera twice)? – zeroflaw Dec 08 '20 at 09:24
Sorry. let me rephrase. So the "Face Liveness detection" is a separate process. If i am using the face detection feature (with landmarks, emotion, etc), i can't have 'face liveness detection' simultaneously, but probably through some workaround (doing one at a time) – zeroflaw Dec 08 '20 at 09:28
Yes, the use cases are different. The liveness detection service is widely used in scenarios such as identity verification and mobile phone unlocking. The face detection service can be used to beautify a detected face or add decorating elements to an image in an image processing app. – zhangxaochen Dec 08 '20 at 10:36
hm... even in identity verification, if face liveness detection is separately handled, then, i kind of feel the lack of the 'seamless integration'. for example, one may use a fake photo of person A to pass the recognition, then quickly switch to a real person B just for 'liveness detection', or the other way round. correct me if i am wrong. – zeroflaw Dec 09 '20 at 02:48
“ one may use a fake photo of person A to pass the recognition” The liveness detection service checks whether a face is alive in the first step. One cannot use a fake phone to pass the liveness detection. The detection result indicates whether the face is of a real person.@zeroflaw – zhangxaochen Dec 09 '20 at 02:58
ya. understand. what if "one uses the real face of person B to pass the face liveness, but when come to face identification, it uses a fake photo of A"? – zeroflaw Dec 09 '20 at 03:11
1

@zeroflaw Then we need the capability of Facial Comparison, which currently is not supported :(. It will be supported in 2021. You may try [LocalAuthentication Engine](https://developer.huawei.com/consumer/en/doc/development/Security-Guides/introduction-0000001051219741) or [Facial comparison](https://developer.huawei.com/consumer/en/doc/development/hiai-Guides/facial-comparison-introduction-0000001053859748?ha_source=hms1) – zhangxaochen Dec 09 '20 at 03:23
Noted. I am looking forward to your promise of 2021 :) – zeroflaw Dec 09 '20 at 04:02
1

by the way, just some thought, if Face Liveness Detection can expose some frames, that would probably resolve the issue too. The HMS has wrapped the camera operation so tidily, if they can be made configurable, that may serve as an alternative to the CameraView. – zeroflaw Dec 10 '20 at 00:53
1

guess i shall close this first. – zeroflaw Dec 28 '20 at 01:37
Hi @zeroflaw Please kindly refer to [Face Vertification Service](https://developer.huawei.com/consumer/en/doc/development/HMSCore-Guides/faceverification-0000001071919926). It can determine whether the two faces belong to the same person based on their similarity. While there are some more details in docs. – zhangxaochen Jan 21 '21 at 07:36
Hi @shirley, thanks for the link, but i am afraid this is a separate issue right? before entering verification, we would like to determine whether it is a real person or not. – zeroflaw Jan 21 '21 at 08:11
@zeroflaw Yes. I think [Liveness Detection](https://developer.huawei.com/consumer/en/doc/development/HMSCore-Guides/liveness-detection-0000001051386243) can support to determine whether a face is of a real person or is a face attack. – zhangxaochen Jan 21 '21 at 08:40

How to integrate Huawei HMS Face Liveness Detection with CameraView?

1 Answers1