How do I use SwiftUI gestures to control a camera in ARView?

Question

I have an ARView contained in a UIViewRepresentable. A USDZ model is loaded. All of this displays in SwiftUI view. The ARView is .nonAR.

I want a DragGesture and a PinchGesture in SwiftUI to control the camera position.

Is this possible? ARView seems not to have a direct equivalent of allowsCameraControl like you get in SceneKit.

Answer 1

NonAR Camera Control

To control RealityKit's nonAR camera position with SwiftUI gestures, use following code. I used a SwiftUI's pinch gesture (MagnifyGesture struct) to simulate dolly in and dolly out camera behavior. You can tweak the dolly formula until you get the perfect camera movement.

import SwiftUI
import RealityKit

class GestureDrivenCamera: Entity, HasPerspectiveCamera {
    required init() {
        super.init()
        self.position.z = 4.0
        self.camera.fieldOfViewInDegrees = 50.0
    }
}

---

struct ContentView : View {
    
    @GestureState var magnifyBy: CGFloat = 0.001
    @State var state: CGFloat = 1.0
    @State var arView = ARView(frame: .zero)
    @State var camera = GestureDrivenCamera()
    
    var body: some View {
        ARViewContainer(arView: $arView, camera: $camera)
            .ignoresSafeArea()
            .gesture(
                MagnifyGesture()
                    .updating($magnifyBy) { (value, _, _) in
                        Task {
                            camera.position.z = Float(state + value.magnification)
                        }
                    }
                    .onEnded { value in
                        state = value.magnification
                    }
            )
    }
}

---

struct ARViewContainer: UIViewRepresentable {
    
    @Binding var arView: ARView
    @Binding var camera: GestureDrivenCamera
    
    func makeUIView(context: Context) -> ARView {
        arView.cameraMode = .nonAR
        let mesh = MeshResource.generateBox(size: 0.5)
        let model = ModelEntity(mesh: mesh)

        let anchor = AnchorEntity()
        anchor.children.append(model)
        arView.scene.anchors.append(anchor)
        
        anchor.addChild(camera)
        return arView
    }
    func updateUIView(_ view: ARView, context: Context) { }
}