Keeping model anchored in place when a tracking image is hidden

Question

I am doing the following:

• detect an image with an ARWorldTrackingConfiguration with maximumNumberOfTrackedImages set to 1
• when the image disappears and didUpdate is called, create a plane with a VideoMaterial and add it to an anchor created with AnchorEntity(.world(transform: imageAnchor.transform))

As the video plays, the plane is more or less anchored at the spot where the image was, but not perfectly. It still moves around a bit as the phone moves.

Is there anything I can do to make it stay perfectly still? didUpdate is never called again after the tracked image disappears, so the usual technique of updating the transform there is not available to me.

Answer 1

Re-anchoring technique

You can place your model in the scene using image tracking and then reanchor it preserving model's current world space position. For that, implement reanchor(_:preservingWorldTransform:) instance method inside session(_:didUpdate:) delegate's method.

import ARKit
import RealityKit

extension ViewController : ARSessionDelegate {
    
    func session(_ session: ARSession, didUpdate frame: ARFrame) {

        if anchor.isActive {
            anchor.reanchor(.world(transform: sphere.transform.matrix), 
                    preservingWorldTransform: true)
        }
    }
}

class ViewController : UIViewController {
    
    @IBOutlet var arView: ARView!
    var anchor = AnchorEntity(.image(group: "AR", name: "image"))
    let sphere = ModelEntity(mesh: .generateSphere(radius: 0.05))

    override func viewDidLoad() {
        super.viewDidLoad()
        
        arView.session.delegate = self
        anchor.addChild(sphere)
        arView.scene.anchors.append(anchor)
    }
}