37

I'm in the process of learning both ARKit and Scenekit concurrently, and it's been a bit of a challenge.

With a ARWorldTrackingSessionConfiguration session created, I was wondering if anyone knew of a way to get the position of the user's 'camera' in the scene session. The idea is I want to animate an object towards the user's current position. 

let reaperScene = SCNScene(named: "reaper.dae")!
let reaperNode = reaperScene.rootNode.childNode(withName: "reaper", recursively: true)!
reaperNode.position = SCNVector3Make(0, 0, -1)
let scene = SCNScene()
scene.rootNode.addChildNode(reaperNode)

// some unknown amount of time later   
let currentCameraPosition = sceneView.pointOfView?.position
let moveAction = SCNAction.move(to: currentCameraPosition!, duration: 1.0)
reaperNode.runAction(moveAction)

However, it seems that currentCameraPosition is always [0,0,0], even though I am moving the camera around. Any idea what I'm doing wrong? Eventually the idea is I would rotate the object around an invisible sphere until it is in front of the camera and then animate it in, doing something similar to this: Rotate SCNCamera node looking at an object around an imaginary sphere (that way the user sees the object animate towards them)

Thanks for any help.

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
Ryan Pfister
  • 3,176
  • 4
  • 24
  • 26

3 Answers3

28

Set yourself as the ARSession.delegate. Than you can implement session(_:didUpdate:) which will give you an ARFrame for every frame processed in your session. The frame has an camera property that holds information on the cameras transform, rotation and position. 

func session(_ session: ARSession, didUpdate frame: ARFrame) {
    // Do something with the new transform
    let currentTransform = frame.camera.transform
    doSomething(with: currentTransform)
}

As rickster pointed out you always can get the current ARFrame and the camera position through it by calling session.currentFrame. This is useful if you need the position just once, eg to move a node where the camera has been but you should use the delegate method if you want to get updates on the camera's position.

BSMP
  • 4,596
  • 8
  • 33
  • 44
jlsiewert
  • 3,494
  • 18
  • 41
  • 5
    This works, but you can also get frames without being the session delegate by getting the session's `currentFrame` property. – rickster Jul 14 '17 at 03:51
  • 1
    @rickster you are tight and I have edited the answer, but usually you want to have other nodes react to you changing your position, eg by following you around. Then you would want constant updates on new camera poses anyway. – jlsiewert Jul 14 '17 at 05:14
  • 2
    True. Point is, depending on how your app is otherwise set up, you could either have ARKit delivering you frames at 60 fps and you rendering as you get them, or you could be running your own render loop at (up to) 60 fps and ask ARKit for a frame each time you need one. It supports both ways, so use whichever you like. – rickster Jul 14 '17 at 05:18
  • Great, thanks for your help! I'm using frame.camera.transform.columns.3.x ... any reason why that shouldn't work? I'm not clear on what the difference between the columns are, but 3 seems to do it for me. – Ryan Pfister Jul 17 '17 at 01:03
  • 1
    This is not working consistently for me. I'm frequently getting (0,0,0) as xyz coordinates. – dmr07 Aug 16 '17 at 19:09
  • Check the [`trackingState`](https://developer.apple.com/documentation/arkit/arcamera.trackingstate) of the camera, maybe you are moving to fast for the camera – jlsiewert Aug 16 '17 at 20:28
  • I get "[DeviceMotion] Event ref invalid" whenever I run an ARSCNView.session. Anyone have an idea why I get this? – unknown_jy Oct 02 '17 at 22:47
  • Based on @rickster 's [answer](https://stackoverflow.com/questions/45084187/arkit-get-current-position-of-camera-in-scene/51972161#comment77162624_45089244) How can this be made? I need the frames position for a rate of 25 fps – cconur Aug 24 '18 at 14:21
  • How would you get the camera position from transform? – Jacobo Koenig Nov 25 '18 at 18:50
  • Is there a way to achieve the same result if the tracking configuration is: ARImageTrackingConfiguration? In my case ARCamera is always in position 0,0,0! – Jacobo Koenig Nov 26 '18 at 19:42
  • 1
    Does anyone know what all the values in the transform matrix are? They are very hard to figure out. – Adalex3 Jul 15 '19 at 22:35
13

I know it had been solved but i have a little neat solution for it .. I would prefere adding a renderer delegate method.. it's a method in ARSCNViewDelegate

func renderer(_ renderer: SCNSceneRenderer, willRenderScene scene: SCNScene, atTime time: TimeInterval) {
    guard let pointOfView = sceneView.pointOfView else { return }
    let transform = pointOfView.transform
    let orientation = SCNVector3(-transform.m31, -transform.m32, transform.m33)
    let location = SCNVector3(transform.m41, transform.m42, transform.m43)
    let currentPositionOfCamera = orientation + location
    print(currentPositionOfCamera)
}

of course you can't by default add the two SCNVector3 out of the box.. so you need to paste out of the class the following 

func +(lhv:SCNVector3, rhv:SCNVector3) -> SCNVector3 {
     return SCNVector3(lhv.x + rhv.x, lhv.y + rhv.y, lhv.z + rhv.z)
}
Mohamed Emad Hegab
  • 2,665
  • 6
  • 39
  • 64
  • 1
    Why are you just taking the diagonal entries of the rotation matrix? And why is "position" equal to "orientation + location". The position is the location, i.e. the last column of the transform. The upper left 3x3 matrix contains the orientation (and possibly scale and shear). There exist different methods to extract pitch, yaw and roll from it, but the diagonal is not one of them. – kjyv Mar 07 '18 at 18:06
  • @kjyv where did you see the location is the last column of the transform? thanks for sharing! – Crashalot Apr 03 '18 at 00:55
  • The transformation matrix is a 4x4 matrix, hence homogenous coordinates. The documentation might not explicitly say this but it is pretty standard. Have a look e.g. here for the 2D case https://en.wikipedia.org/wiki/Transformation_matrix#Affine_transformations – kjyv Apr 03 '18 at 09:45
  • 1
    @kjyv is `transform.m33` for `orientation` supposed to be `-transform.m33`? – Crashalot Apr 05 '18 at 22:58
  • 2
    For orientation it's probably better to just use sceneView.pointOfView.presentation.worldOrientation. If you really need to get it from the transform, have a look e.g. here https://gamedev.stackexchange.com/questions/50963/how-to-extract-euler-angles-from-transformation-matrix You should prefer a quaternion over ambiguous euler angles. – kjyv Apr 26 '18 at 09:20
  • 1
    This is wrong on so many levels - the transform's orientation is contained in a 3x3 matrix, not just the diagonal entries. The position is the location – oneiros Mar 14 '19 at 21:08
  • Can someone explain why "position = orientation + location"? What does adding the orientation here do exactly? I was under the impression that the position should just be the location? – Friendly King Aug 14 '19 at 07:50
7

ARKit + SceneKit

For your convenience you can create a ViewController extension with an instance method session(_:didUpdate:) where an update will be occured.

import ARKit
import SceneKit

extension ViewController: ARSessionDelegate {

    func session(_ session: ARSession, didUpdate frame: ARFrame) {
        let transform = frame.camera.transform
        let position = transform.columns.3
        print(position.x, position.y, position.z)     // UPDATING        
    }
}

class ViewController: UIViewController {

    @IBOutlet var sceneView: ARSCNView!

    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)
        sceneView.session.delegate = self             // ARSESSION DELEGATE

        let config = ARWorldTrackingConfiguration()
        sceneView.session.run(config)
    }
}

RealityKit

In RealityKit, ARView's object contains a camera's transform as well:

import RealityKit
import UIKit
import Combine

class ViewController: UIViewController {

    @IBOutlet var arView: ARView!
    var subs: [AnyCancellable] = []

    override func viewDidLoad() {
        super.viewDidLoad()

        arView.scene.subscribe(to: SceneEvents.Update.self) { _ in            
            let camTransform = self.arView.cameraTransform.matrix
            print(camTransform)                             // UPDATING  
        }.store(in: &subs)
    }
}
Andy Jazz
  • 49,178
  • 17
  • 136
  • 220