Augmented Faces API (ARCore) - How to replace face mask with another mask by using button in Android Studio

Question

I'm developing AR android app with using ARCore-Augmented Faces API in Android Studio. For now, I have finish set up the app (5 buttons and its associate 3d model) and it is work well when I click one of those buttons for the first time launched the app. The problem is when I click another button to change the current face mask, it won't replace the current mask with new mask.

However, to fix this problem. I have tried to use restart() method and tried to remove the facenode. But it seems, those solutions don't work at all.

This is partial of my code that currently I work on it (not the fix one) where I still stuck with the problem I encounter.


        alchemist_ovalbtn.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                alchemistOval();
            }
        });

        scene.addOnUpdateListener(
                (FrameTime frameTime) -> {
                    if (faceRegionsRenderable == null) {
                        return;
                    }

                    Collection<AugmentedFace> faceList =
                            sceneView.getSession().getAllTrackables(AugmentedFace.class);

                    // Make new AugmentedFaceNodes for any new faces.
                    for (AugmentedFace face : faceList) {
                        if (!faceNodeMap.containsKey(face)) {
                            AugmentedFaceNode faceNode = new AugmentedFaceNode(face);
                            faceNode.setParent(scene);
                            faceNode.setFaceRegionsRenderable(faceRegionsRenderable);
                            faceNode.setFaceMeshTexture(faceMeshTexture);
                            faceNodeMap.put(face, faceNode);
                        }
                    }

                });

// Remove any AugmentedFaceNodes associated with an AugmentedFace that stopped tracking.

        Iterator<Map.Entry<AugmentedFace, AugmentedFaceNode>> iter =
                faceNodeMap.entrySet().iterator();
        while (iter.hasNext()) {
            Map.Entry<AugmentedFace, AugmentedFaceNode> entry = iter.next();
            AugmentedFace face = entry.getKey();
            if (face.getTrackingState() == TrackingState.STOPPED) {
                AugmentedFaceNode faceNode = entry.getValue();
                faceNode.setParent(null);
                iter.remove();
            }
        }

public void alchemistOval() {

        ModelRenderable.builder()
                .setSource(this, R.raw.alchemist_oval)
                .build()
                .thenAccept(
                        modelRenderable -> {
                            faceRegionsRenderable = modelRenderable;
                            modelRenderable.setShadowCaster(false);
                            modelRenderable.setShadowReceiver(false);
                        });

    }

So, I expect that the current face mask will be replace by new mask when user click another button.

@khairudin did u find solution? I'm also looking for the same but not getting any solution, please help me if u got any solution — Mallikarjun Hampannavar, Oct 23 '19 at 11:38

score 0 · Answer 1 · edited Jan 23 '20 at 13:04

This code swaps between 3 face masks on click, hope it helps.

package com.example.faceplay;

import androidx.appcompat.app.AppCompatActivity;

import android.os.Build;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.widget.ImageView;

import com.google.ar.core.AugmentedFace;
import com.google.ar.core.TrackingState;
import com.google.ar.sceneform.ArSceneView;
import com.google.ar.sceneform.FrameTime;
import com.google.ar.sceneform.Scene;
import com.google.ar.sceneform.rendering.ModelRenderable;
import com.google.ar.sceneform.rendering.Renderable;
import com.google.ar.sceneform.rendering.Texture;
import com.google.ar.sceneform.ux.AugmentedFaceNode;

import java.util.Collection;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;

public class MainActivity extends AppCompatActivity implements View.OnClickListener {


    private FaceFragment arFragment;

    private Texture faceMeshTexture;
    Scene scene;
    ArSceneView sceneView;
    AugmentedFaceNode faceNode;

    private final HashMap<AugmentedFace, AugmentedFaceNode> faceNodeMap = new HashMap<>();

    ImageView mask, eyemask, skull_1;
    View arrayView[];

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        arFragment = (FaceFragment) getSupportFragmentManager().findFragmentById(R.id.face_fragment);



        mask = (ImageView)findViewById(R.id.mask);
        eyemask = (ImageView)findViewById(R.id.eyemask);
        skull_1 = (ImageView)findViewById(R.id.skull_1);

        setArrayView();
        setClickListener();


        sceneView = arFragment.getArSceneView();
        sceneView.setCameraStreamRenderPriority(Renderable.RENDER_PRIORITY_FIRST);
        scene = sceneView.getScene();



        scene.addOnUpdateListener(

                (FrameTime frameTime) -> {

                    if ( faceMeshTexture == null) {
                        return;
                    }

                    Collection<AugmentedFace> faceList =
                            sceneView.getSession().getAllTrackables(AugmentedFace.class);

                    for (AugmentedFace face : faceList) {

                        if (!faceNodeMap.containsKey(face)) {

                           faceNode = new AugmentedFaceNode(face);

                            faceNode.setParent(scene);
                            faceNode.setFaceMeshTexture(faceMeshTexture);

                            faceNodeMap.put(face, faceNode);

                        }

                    }

                });
    }

    private void AugmentedFaceDisplay(int resource){

        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
            Texture.builder()
                    .setSource(this, resource)
                    .build()
                    .thenAccept(texture -> faceMeshTexture = texture);
        }


    }

    private void setArrayView(){
        arrayView = new View[]{
                mask, eyemask, skull_1
        };
    }

    private void setClickListener(){
        for ( int i = 0; i < arrayView.length; i++){
            arrayView[i].setOnClickListener(this);
        }
    }

    @Override
    public void onClick(View v) {

        // First face mask 
        if( v.getId() == R.id.mask )
        {
            faceNodeMap.clear();
            try{
                faceNode.setParent(null);
            }
            catch (Exception e){

            }

            AugmentedFaceDisplay(R.drawable.mask);
        }

        // Second face mask      
        else  if( v.getId() == R.id.eyemask)
        {
            faceNodeMap.clear();
            try{
                faceNode.setParent(null);
            }
            catch (Exception e){

            }
            AugmentedFaceDisplay(R.drawable.eyemask);
        }

        // Third face mask 
        else  if( v.getId() == R.id.skull_1 )
        {
            faceNodeMap.clear();
            try{
                faceNode.setParent(null);
            }
            catch (Exception e){

            }
            AugmentedFaceDisplay(R.drawable.skull1);
        }
    }
}

This code swaps between 3 face masks on click, hope it helps. — Hur Ali, Jan 23 '20 at 12:11

Augmented Faces API (ARCore) - How to replace face mask with another mask by using button in Android Studio

1 Answers1