1

What would be the right approach to extract a separate image of a specific part of the user's face using ARCore? Specifically, I'm interested in an area covering an eye and an eyebrow, as shown in the image below.

I've searched everything and checked the ARCore documentation, but none of the information describes how this can be achieved.

The closest thing I've found is adding accessories like glasses to the face, but couldn't find any samples on how to separate out a specific part of the face.

enter image description here

SharpAffair
  • 5,558
  • 13
  • 78
  • 158
  • I don't think you could do this with only `ARCore`. You would need something like `OpenCV` to crop the image along with [`getRegionPose()`](https://developers.google.com/ar/reference/java/com/google/ar/core/AugmentedFace#getRegionPose(com.google.ar.core.AugmentedFace.RegionType)) to get the portion of the face that you want. – PCDSandwichMan Jul 23 '23 at 07:51

1 Answers1

0

Yes.There's no direct way to extract a separate image of a specific part of the user's face using ARCore.But you can use Augmented Faces API, which provides a 468-point dense 3D face mesh that allows you to paint adaptable, detailed textures that accurately follow a face. Use this information to create a mask for the specific part of the face you are interested in, such as an eye and an eyebrow, and then use OpenCV to crop the image along with the mask.

You can use OpenCV to crop an image by specifying the height and width (in pixels) of the area to be cropped. In Python, you can use NumPy slicing to crop an image:

crop_img = img[y:y+h, x:x+w]

user1874594
  • 2,277
  • 1
  • 25
  • 49
  • You have mentioned "create a mask for the specific part of the face you are interested in". How should the mask be created? I think your answer is missing this step. – SharpAffair Jul 24 '23 at 18:04
  • I have awarded the bounty to you, but please provide more detail in your answer. Thank you. – SharpAffair Jul 24 '23 at 20:35
  • Thx for that. I have outlined the steps here https://paste.c-net.org/ChurningSkulls – user1874594 Jul 25 '23 at 08:45
  • [1 Introduction to ARCore Augmented Faces, Unity - YouTube](https://www.youtube.com/watch?v=1rZ-JorHJEY) [2 Introduction to ARCore Augmented Faces, Android](https://www.youtube.com/watch?v=1rZ-JorHJEY) and [3 Augmented Faces introduction | ARCore | Google for Developers](https://developers.google.com/ar/develop/augmented-faces) – user1874594 Jul 25 '23 at 08:56