ARKit – How to use ARImageTrackingConfiguration with ARFaceTrackingConfiguration?

Question

I have built an app that uses image tracking and swaps flat images. I am also using people occlusion (now) so people can get photos in front of those images. I really want this app to have a selfie mode, so people can take their own pictures in front of image swapped areas.

I'm reading the features on ARKit 3.5, but as far as I can tell, the only front-facing camera support is with ARFaceTrackingConfiguration, which doesn't support image tracking. ARImageTrackingConfiguration and ARWorldTrackingConfiguration only use the back camera.

Is there any way to make a selfie mode with people occlusion (and image tracking) using the front-facing camera?

Answer 1

The answer is NO, you can't use any ARConfiguration except ARFaceTrackingConfiguration for front camera. Although, you can simultaneously use ARFaceTrackingConfiguration on the front camera and ARWorldTrackingConfiguration on the rear camera. This allows users interact with AR content in the rear camera using their face as certain controller.

Look at this docs page to find out what config to what camera (rear or front) corresponds to.

Here's a table containing ARKit 5.0 eight tracking configurations:

ARConfiguration	What Camera?
ARWorldTrackingConfiguration	Rear
ARBodyTrackingConfiguration	Rear
AROrientationTrackingConfiguration	Rear
ARImageTrackingConfiguration	Rear
ARFaceTrackingConfiguration	FRONT
ARObjectScanningConfiguration	Rear
ARPositionalTrackingConfiguration	Rear
ARGeoTrackingConfiguration	Rear

Simultaneous World and Face configs

To use driven World Tracking depending on driver Face Tracking use the following code:

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
    
    guard ARFaceTrackingConfiguration.isSupported,
          ARFaceTrackingConfiguration.supportsWorldTracking
    else {
        fatalError("We can't do face tracking")
    }        
    let config = ARFaceTrackingConfiguration()
    config.isWorldTrackingEnabled = true
    sceneView.session.run(config)
}

Or you can use Face Tracking as a secondary configuration:

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
    
    let config = ARWorldTrackingConfiguration()
    
    if ARFaceTrackingConfiguration.isSupported {
        config.userFaceTrackingEnabled = true
    }
    sceneView.session.run(config)
}

Pay attention that both properties are available on iOS 13 and higher.

var userFaceTrackingEnabled: Bool { get set }

var isWorldTrackingEnabled: Bool { get set }

P.S.

At the moment .userFaceTrackingEnabled = true still doesn't work. I tested it in Xcode 13.2.1 and iPad Pro 4th Gen with iPadOS 15.3 installed.