First Person Experience in RealityKit

Question

I am fairly new to Swift and just started playing with RealityKit and ARKit. I am working on a personal project where I would like to have a 3D object stick to the camera in first person. Similar to the AR Angry birds or any FPS game. I have seen a few examples in SceneKit or SpriteKit, I am sure it is just a misunderstanding of how anchoring entities works.

My main question is:

How would I go about sticking a reality object I created in Reality Composer to the camera in first person? I want to create a Reality Scene, in this case an arm cannon and upon tapping it shots.

Below is the code for my ViewController

extension ViewController: ARSessionDelegate
{
    func session(_ session: ARSession, didUpdate frame: ARFrame)
    {
          guard let arCamera = session.currentFrame?.camera else { return }
          // Probably where I update the location of my reality experience
    }
}

class ViewController: UIViewController
{
    @IBOutlet var arView: ARView!

    override func viewDidLoad()
    {
        super.viewDidLoad()
        arView.session.delegate = self

        // Load the "ArmCannon" scene from the "Experience" Reality File
        let armCannonAnim = try! Experience.loadArmcannon()

        // Create Anchor to anchor arm cannon to
        let anchor = AnchorEntity(.camera)
        anchor.transform = arView.cameraTransform

        // Add the anchor to the scene
        arView.scene.addAnchor(anchor)

        // Setup tap gesture on arm cannon
        let tapGesture = UITapGestureRecognizer(target: self, action:#selector(onTap))
        arView.addGestureRecognizer(tapGesture)

        // Add the the cannon animation to arView
        arView.scene.anchors.append(armCannonAnim)

    }

    @IBAction func onTap(_ sender: UITapGestureRecognizer)
    {

        let tapLocation = sender.location(in: arView)

        // Get the entity at the location we've tapped, if one exists
        if let cannonFiring = arView.entity(at: tapLocation)
        {
            print(cannonFiring.name)
            print("firing Cannon")
        }
    }
}

I have looked at and read Track camera position with RealityKit and Where is the .camera AnchorEntity located?

You want to fix cannons position as camera position? – Abdul Karim Khan Mar 29 '20 at 15:53 — Abdul Karim Khan, Mar 29 '20 at 15:53

score 2 · Answer 1 · answered Mar 30 '20 at 15:30

2

Instead of:

arView.scene.anchors.append(armCannonAnim)

put:

anchor.addChild(armCannonAnim)

You need this armCannonAnim to be a child of the camera, and the anchor object is an anchor at the camera transform. This is equivalent to in SceneKit adding a child to the cameraNode.

answered Mar 30 '20 at 15:30

maxxfrazer

1,173
6
15

Unfortunately, that did not work. The Reality Scene will not load at all. – Jav Solo Mar 30 '20 at 23:04

score 0 · Answer 2 · answered Mar 09 '23 at 05:06

Don't manually modify the transform of the camera anchor (AnchorEntity(.camera)).
You should add your model entities to the camera anchor, otherwise they are unrelated.
When you load your model from a Reality Composer file, you should load an entity instead of an anchor. Because you want to anchor your model to the camera anchor by code instead of any anchor you defined in the Reality Composer file.
Set the correct position. The z axis points backward, so to put entities in front of you, you typically use negative z values. If you don't see your models, you may try different z values to see if they appear.

The following code is modified from the Xcode RealityKit template project of augmented reality app, you should see a steel box always in front of you:

class ViewController: UIViewController {

    @IBOutlet var arView: ARView!

    override func viewDidLoad() {
        super.viewDidLoad()

        let cameraAnchor = AnchorEntity(.camera)
        arView.scene.addAnchor(cameraAnchor)

        // Experience.loadBox() returns an anchor which is not what we want
        let boxEntity = try! Experience.loadBox().steelBox!
        // Put it 0.5 meters in front of the camera
        boxEntity.transform.translation = simd_float3(0, 0, -0.5)
        cameraAnchor.addChild(boxEntity)
    }
}

One more thing if you want to find your entities using the entity(at:) method or the scene.raycast methods: your entities must have collision shapes.

boxEntity.generateCollisionShapes(recursive: true)

First Person Experience in RealityKit

2 Answers2