How do I specify a child node location relative to ARImageAnchor?

Question

I have a RealityKit app that is doing some basic AR image tracking. It detects a rectangular-shaped image, and I am looking to place some spherical dots at each corner of the image. I know I can create the spheres themselves using a ModelEntity, but I haven't been able to figure out how to specify the position of these relative to the established ARImageAnchor from the reference image.

I think I just need a counterpart to SceneKit's addChildNode(SCNNode) function, which uses SCNVector3Make() to specify a position. I just haven't been able to find a way to establish a relative position and assign a child node to the ARImageAnchor outside of these SceneKit functions. Is there something built into RealityKit that would accomplish this, or is there a way to use SceneKit to place the corner dots while still using my current setup with RealityKit for the AR reference image tracking?

Andy Jazz · Accepted Answer · 2022-03-26T12:12:58.607

Try the following approach:

import ARKit
import RealityKit
 
class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    var anchorEntity = AnchorEntity()
    let model_01 = ModelEntity(mesh: .generateSphere(radius: 0.03))
    let model_02 = ModelEntity(mesh: .generateSphere(radius: 0.03))
    let model_03 = ModelEntity(mesh: .generateSphere(radius: 0.03))
    let model_04 = ModelEntity(mesh: .generateSphere(radius: 0.03))
    
    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)            
        arView.session.delegate = self
        
        guard let reference = ARReferenceImage.referenceImages(
                                                inGroupNamed: "AR Resources",
                                                      bundle: nil)
        else { return }
        
        let config = ARImageTrackingConfiguration()
        config.trackingImages = reference
        arView.session.run(config)
        
        self.anchorEntity.addChild(model_01)
        self.anchorEntity.addChild(model_02)
        self.anchorEntity.addChild(model_03)
        self.anchorEntity.addChild(model_04)    
        arView.scene.anchors.append(self.anchorEntity)
    }
}

Then implement session(_:didUpdate:) method:

extension ViewController: ARSessionDelegate {
    
    func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
        
        guard let imageAnchor = anchors.first as? ARImageAnchor
        else { return }
        
        let width = Float(imageAnchor.referenceImage.physicalSize.width)
        let height = Float(imageAnchor.referenceImage.physicalSize.height)
        let x = imageAnchor.transform.columns.3.x
        let y = imageAnchor.transform.columns.3.y
        let z = imageAnchor.transform.columns.3.z
        
        let lowerLeft = SIMD3<Float>(x - width/2, y - height/2, z)
        let lowerRight = SIMD3<Float>(x + width/2, y - height/2, z)
        let upperRight = SIMD3<Float>(x + width/2, y + height/2, z)
        let upperLeft = SIMD3<Float>(x - width/2, y + height/2, z)
        
        self.model_01.position = lowerLeft
        self.model_02.position = lowerRight
        self.model_03.position = upperRight
        self.model_04.position = upperLeft
        
        self.anchorEntity = AnchorEntity(anchor: imageAnchor)
    }
}

Thanks for the info! Would it also work to set this up so the positions are added to the anchor children within `session(_:didAdd:)` instead of `didUpdate`? Or would they functionally work the same way since the anchor is only "updated" when it is initially placed (when the program finds the reference image)? — Charlie, Mar 26 '22 at 18:40
In this particular case, you need to use `session(_:didUpdate:)`. — Andy Jazz, Mar 26 '22 at 18:43
Okay, I think that makes sense. So in your example, `session(_:didUpdate:)` would only be called once whenever the reference image is found, correct? — Charlie, Mar 26 '22 at 18:50
`session(_:didUpdate:)` is called for every ARFrame, i.e. 60 times per sec. — Andy Jazz, Mar 26 '22 at 18:52
Does that mean that all of the positions would be reassigned every frame? Or would it not since no anchors are being "updated" after the image is initially found, so the `guard let` would fall through to the return? — Charlie, Mar 26 '22 at 18:57
You can track a stationary image (a.k.a. camera tracking), or you can track an image on a moving bus (a.k.a. object tracking). ARKit/RealityKit tracks the pose of an image (or rather its transform) in every frame. So, its anchor is being updated in every frame. — Andy Jazz, Mar 26 '22 at 19:06
And, YES, if there's no image – `guard let` statement falls through to the `void return`. — Andy Jazz, Mar 26 '22 at 19:09
ARKit and RealityKit are modules for tracking specific Anchors. — Andy Jazz, Mar 26 '22 at 19:11

How do I specify a child node location relative to ARImageAnchor?

1 Answers1

Linked