How to create texture to overlay on Augmented Face mesh?

Question

I am working on an AR application using ARCore and Sceneform. I want to add texture to face landmarks like nose, lips, face and eyes. I want to know how to create texture so that I can overlay it on Augmented Face mesh?

Texture.builder()
   .setSource(this, R.drawable.makeupforlips)
   .setUsage(Texture.Usage.COLOR)
   .build()
   .thenAccept(texture -> faceMeshTexture = texture);

addOnUpdateListener:

for (AugmentedFace face: faceList) {

    if (!faceNodeMap.containsKey(face)) {

        AugmentedFaceNode faceNode = new AugmentedFaceNode(face);
        faceNode.setParent(scene);
        // faceNode.setFaceRegionsRenderable(faceRegionsRenderable);
        faceNode.setFaceMeshTexture(faceMeshTexture);
        faceNodeMap.put(face, faceNode);
    }
}

Andy Jazz · Answer 1 · 2022-04-11T14:12:07.303

3

When you're building ARCore app with Augmented Faces functionality you need to use a canonical face mesh that was saved in .fbx, .obj, or .glTF file format. Import this canonical face into Autodesk Maya and using UV Texture Editor, create a UV-mapped texture that can be repainted in Adobe Photoshop or Pixelmator.

edited Apr 11 '22 at 14:12

answered Aug 18 '20 at 09:56

Andy Jazz

49,178
17
136
220

1

Thanks for your reply . It works fine.But the texture doesn't blend to the face so well.It looks like a 3d effect . please guide me how to fix that – Preethi Aug 24 '20 at 05:57
1

I'm sure, all the issues in this texture look – is due to the lack of shadows. You need include shadows in your texture. These shadows are named `baked shadows` or `fake shadows`. https://catlikecoding.com/unity/tutorials/scriptable-render-pipeline/baked-shadows/ Also, to get rid of computer-looking texture – use color-corrected textures with resolution 2048x2048 (aka 2K textures) – Andy Jazz Aug 24 '20 at 06:02
1

Thanks for your reply. am working on an android application and am creating textures using photoshop . Could you please guide me how to make real textures that blend to the skin in photoshop . also please explain what is meant by color-corrected textures. – Preethi Aug 25 '20 at 05:15
There are too many resources about drawing a skin for 3D character. Just google it. Color-corrected skin is a one that has natural tone and it has a real-world shading features – like moles, pores, scars, etc. – Andy Jazz Aug 25 '20 at 05:21
I used [canonical_face_mesh.psd](https://github.com/google-ar/arcore-android-sdk/blob/master/assets/canonical_face_texture.psd) to apply texture to lips . A gap is seen in-between the lips even when the lips are closed.Do you have any idea about this issue? – Preethi Sep 02 '20 at 09:35
1

You need precisely UV-map this texture to geometry. – Andy Jazz Sep 02 '20 at 09:40
Can you share more information about it . I couldn't figure how to implement it . – Preethi Sep 02 '20 at 10:37
1

Watch this video – https://www.youtube.com/watch?v=O5Efs6TZIdQ – and this one – https://www.youtube.com/watch?v=uMUQeLStdpk – Andy Jazz Sep 02 '20 at 10:42

How to create texture to overlay on Augmented Face mesh?

1 Answers1