AndroidCamera2 for face detection and distance measurement

Question

I'm building a camera that needs to detect the user's face/eyes and measure distance through the eyes. I found that on this project https://github.com/IvanLudvig/Screen-to-face-distance, it works great but it doesn't happen to use a preview of the frontal camera (Really, I tested it on at least 10 people, all measurements were REALLY close or perfect). My app already had a selfie camera part made by me, but using the old camera API, and I didn't find a solution to have both camera preview and the face distance to work together on that, always would receive an error that the camera was already in use.

I decided to move to the camera2 to use more than one camera stream, and I'm still learning this process of having two streams at the same time for different things. Btw documentation on this seems to be so scarce, I'm really lost on it.

Now, am I on the right path to this? Also, on his project,Ivan uses this:

        Camera camera = frontCam();
        Camera.Parameters campar = camera.getParameters();
        F = campar.getFocalLength();
        angleX = campar.getHorizontalViewAngle();
        angleY = campar.getVerticalViewAngle();
        sensorX = (float) (Math.tan(Math.toRadians(angleX / 2)) * 2 * F);
        sensorY = (float) (Math.tan(Math.toRadians(angleY / 2)) * 2 * F);

This is the old camera API, how can I call this on the new one? Judging from this answer: Android camera2 API get focus distance in AF mode

Do I need to get min and max focal lenghts?

For the horizontal and vertical angles I found this one: What is the Android Camera2 API equivalent of Camera.Parameters.getHorizontalViewAngle() and Camera.Parameters.getVerticalViewAngle()?

The rest I believe is done by Google's Cloud Vision API

EDIT:

I got it to work on camera2, using GMS's own example, CameraSourcePreview and GraphicOverlay to display whatever I want to display together the preview and detect faces.

Now to get the camera characteristics:

CameraManager manager = (CameraManager) this.getSystemService(Context.CAMERA_SERVICE);
            try {
                character = manager.getCameraCharacteristics(String.valueOf(1));
            } catch (CameraAccessException e) {
                Log.e(TAG, "CamAcc1Error.", e);
            }

            angleX = character.get(CameraCharacteristics.SENSOR_INFO_PHYSICAL_SIZE).getWidth();
            angleY = character.get(CameraCharacteristics.SENSOR_INFO_PHYSICAL_SIZE).getHeight();
            sensorX = (float)(Math.tan(Math.toRadians(angleX / 2)) * 2 * F);
            sensorY = (float)(Math.tan(Math.toRadians(angleY / 2)) * 2 * F);

This pretty much gives me mm accuracy to face distance, which is exactly what I needed. Now what is left is getting a picture from this preview with GMS's CameraSourcePreview, so that I can use later.

Final Edit here:

I solved the picture issue, but I forgot to edit here. The thing is, all the examples using camera2 to take a picture are really complicated (rightly so, it's a better API than camera, has a lot of options), but it can be really simplyfied to what I did here:

mCameraSource.takePicture(null, bytes -> {
                Bitmap bitmap;
                bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
                if (bitmap != null) {
                    Matrix matrix = new Matrix();
                    matrix.postRotate(180);
                    matrix.postScale(1, -1);
                    rotateBmp = Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(),
                            bitmap.getHeight(), matrix, false);
                    saveBmp2SD(STORAGE_PATH, rotateBmp);
                    rotateBmp.recycle();
                    bitmap.recycle();
                }
            });

That's all I needed to take a picture and save to a location I specified, don't mind the recycling here, it's not right, I'm working on it

Hey Have you completed this task, I am seeking some help in it. I am not able to get the Focal length. And It will be good if you could share the answer or code. Thank you — Pushpendra, Jul 05 '22 at 06:09
Hi @Pushpendra, specifically about the focal lenght, you may use .get(CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) to get a value from the system, and using this answer here [link](https://stackoverflow.com/a/67030241/1085701), you are able to get a real Focal Lenght. Hope this helps you as it helped me! — Luis, Jul 06 '22 at 13:36

score 1 · Answer 1 · answered Apr 16 '21 at 21:58

It looks like that bit of math is calculating the physical dimensions of the image sensor, via the angle-of-view equation:

The camera2 API has the sensor dimensions as part of the camera characteristics directly: SENSOR_INFO_PHYSICAL_SIZE.

In fact, if you want to get the field of view in camera2, you have to use the same equation in the other direction, since FOVs are not part of camera characteristics.

Beyond that, it looks like the example you linked just uses the old camera API to fetch that FOV information, and then closes the camera and uses the Vision API to actually drive the camera. So you'd have to look at the vision API docs to see how you can give it camera input instead of having it drive everything. Or you could use the camera API's built-in face detector, which on many devices gives you eye locations as well.

Thanks for the answer, I'm gonna look into it right now, also I read somewhere that Vision API and camera2 had to have some workarounds to work together. About the built in face detector, seems to be an easier way to implement what I want, and it doesn't seem to be too many devices without eye location really. — Luis, Apr 19 '21 at 12:20

AndroidCamera2 for face detection and distance measurement

1 Answers1