Using ARKit anchor to place RealityKit Scene

Question

I am currently struggling to add an RealityKit Scene to an ARKit anchor. As my previous question, it is not possible to directly add the RealityKit Scene to an ARKit Anchor Node.

But how can I use the ARKit Anchor to place my Scene? For example, I want to find an object in the room, and depending on which object I found, I want to use different scenes and anchor them in the room depending on the recognised objects position.

Thanks for the help.

Current try:

func makeUIView(context: Context) -> ARView {
        
        let arView = ARView(frame: .zero)
        
        // Load the "Box" scene from the "Experience" Reality File
        let boxAnchor = try! Experience.loadBox()
        
        // Add the box anchor to the scene
        arView.scene.anchors.append(boxAnchor)
        
        let configuration = ARWorldTrackingConfiguration()
        guard let referenceObjects = ARReferenceObject.referenceObjects(
                inGroupNamed: "AR Objects", bundle: nil) else {
            fatalError("Missing expected asset catalog resources.")
        }
        
        configuration.detectionObjects = referenceObjects
        arView.session.run(configuration) // not possible at arview, but i need arview for the .rcproject :-/
        
        return arView
        
    }

EDIT: add code

i am really a bit confused. [Here](https://developer.apple.com/documentation/realitykit/creating_3d_content_with_reality_composer/selecting_an_anchor_for_a_reality_composer_scene) is apple referencing to [this](https://developer.apple.com/documentation/arkit/scanning_and_detecting_3d_objects) as anchor point. but i cannot see how they archive it. — Auryn, Dec 15 '20 at 15:30

Andy Jazz · Accepted Answer · 2021-06-03T20:43:12.747

ARKit has no nodes. SceneKit does.

However, if you need to place RealityKit's model into AR scene using ARKit's anchor, it's as simple as this:

import ARKit
import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    let boxScene = try! Experience.loadBox()
    
    override func viewDidLoad() {
        super.viewDidLoad()
        arView.session.delegate = self
        
        let entity = boxScene.steelBox
        
        var transform = simd_float4x4(diagonal: [1,1,1,1])
        transform.columns.3.z = -3.0  // three meters away
        
        let arkitAnchor = ARAnchor(name: "ARKit anchor",
                              transform: transform)
        
        let anchorEntity = AnchorEntity(anchor: arkitAnchor)
        anchorEntity.name = "RealityKit anchor"
        
        arView.session.add(anchor: arkitAnchor)
        anchor.addChild(entity)
        arView.scene.anchors.append(anchorEntity)
    }
}

It's possible thanks to the AnchorEntity convenience initializer:

convenience init(anchor: ARAnchor)

Also you can use session(_:didAdd:) and session(_:didUpdate:) instance methods:

extension ViewController: ARSessionDelegate {
        
    func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
            
        guard let objectAnchor = anchors.first as? ARObjectAnchor
        else { return }
            
        let anchor = AnchorEntity(anchor: objectAnchor)
        let entity = boxScene.steelBox
        anchor.addChild(entity)
        arView.session.add(anchor: objectAnchor)
        arView.scene.anchors.append(anchor)
    }
}

thanks a lot for your help! It works. I totally miss, that i also can add the scanned object as object reference in the reality composer, it was not explained in any tutorial. this does the trick for me :-/ — Auryn, Dec 15 '20 at 16:07

Using ARKit anchor to place RealityKit Scene

1 Answers1

Linked