Android CameraX - face detection while recording video

Question

I'm using the new library CameraX with Firebase ML Kit in Android and detecting faces every frame the device can.

So I set CameraX like that:

CameraX.bindToLifecycle(this, preview, imageCapture, faceDetectAnalyzer)

All working flowless, now, while I'm doing that, I want to record a video.

So basically I want to to detect faces while recording a video.

I tried:

CameraX.bindToLifecycle(this, preview, imageCapture, faceDetectAnalyzer, videoCapture)

But I'm getting an error saying that there are too many parameters so I guess that's not the right way.

I know that this library still in alpha but I guess there is a way to do that.

Even if there is not jet, what's another way to implement face detection while recording a video with Firebase ML?

Possible duplicate of [an error occurred by CameraX.bindToLifecycle()](https://stackoverflow.com/questions/57126429/an-error-occurred-by-camerax-bindtolifecycle) — Link182, Aug 02 '19 at 13:30
No is not, that's a different error, that come in different circumstances, I'll answer his question. Also my question is about the implementation not just about the error. — GMX, Aug 04 '19 at 11:14
try using the Media Projection API and record the screen instead if you want to try it out... just a suggestion personally I have not used the ML kit — Pemba Tamang, Aug 12 '19 at 10:08
see this https://stackoverflow.com/questions/56054647/can-i-record-video-with-camerax-android-jetpack — Pemba Tamang, Aug 12 '19 at 10:09

score 0 · Answer 1 · edited Sep 09 '19 at 19:41

0

I didn't use CameraX a lot, but I'm usually working with Camera 2 API and Firebase ML Kit.

For use both API together, you should get the Image callbacks from your Preview Size ImageReader. On that callback you can use that Images to create a FirebaseVisionFace through the API and do whatever you want with it. Using Kotlin and Coroutines it should look like these:

 private val options: FirebaseVisionFaceDetectorOptions = FirebaseVisionFaceDetectorOptions.Builder()
    .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS)
    .build()

val detector = FirebaseVision.getInstance().getVisionFaceDetector(options)

suspend fun processImage(image: Image): FirebaseVisionFace {
    val metadata = FirebaseVisionImageMetadata.Builder()
        .setWidth(image.width) // 480x360 is typically sufficient for image recognition
        .setHeight(image.height)
        .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
        .build()

    val visionImage = FirebaseVisionImage.fromMediaImage(image)
    val firebaseVisionFace = detector.detectInImage(visionImage).await()

    return firebaseVisionFace
}

If you want to use the await method for Coroutine support you can give a loot to https://github.com/FrangSierra/Firebase-Coroutines-Android

edited Sep 09 '19 at 19:41

halfer

19,824
17
99
186

answered Aug 13 '19 at 09:47

Francisco Durdin Garcia

12,540
9
53
95

1

I'm already doing this and using both API together (image capture and Face Detection) and is working fine. The question is, on top of that can I record a video? – GMX Aug 13 '19 at 16:02
I dont know about cameraX but API 2 doesn't allow you to do FaceRecognition with Firebase because the preview is not throwing 'Image' objects, you could do it with the preview image reader or maybe a low quality one running together with the videoSession(like you usually do for take snapshots) – Francisco Durdin Garcia Aug 15 '19 at 13:02
1

I don't know too but I'm sure is possible to record video while detecting faces, I saw apps doing it and they are using Firebase. – GMX Aug 15 '19 at 13:05
1

My guess is that they are doing it not with camera x but with camera2 somehow – GMX Aug 15 '19 at 13:06
any clue on this? – qwerty421 Aug 06 '20 at 06:27

Android CameraX - face detection while recording video

1 Answers1