ARKit – Raycasting using a world ray instead of a screen point

Question

I want to achieve something similar like ARCore's raycast method which takes an arbitrary ray in world space coordinates instead of a screen-space point:

List<HitResult> hitTest (float[] origin3, int originOffset, float[] direction3, int directionOffset)

I see ARKit itself has not that method like that, but in any way maybe someone has an idea!

Thanks.

Answer 1

In Apple RealityKit and ARKit frameworks you can find three main types of Raycast methods: ARView Raycast, ARSession Raycast and Scene Raycast (or World Raycast). All methods written in Swift:

ARView.raycast(from:allowing:alignment:)

This instance method performs a ray cast, where a ray is cast into the scene from the center of the camera through a point in the view, and the results are immediately returned. You can use this type of raycast in ARKit.

func raycast(from point: CGPoint, 
        allowing target: ARRaycastQuery.Target, 
              alignment: ARRaycastQuery.TargetAlignment) -> [ARRaycastResult]

ARView.scene.raycast(origin:direction:query:mask:relativeTo:)

WORLD RAYCAST

This instance method performs a convex ray cast against all the geometry in the scene for a ray of a given origin, direction, and length.

func raycast(origin: SIMD3<Float>, 
          direction: SIMD3<Float>,
              query: CollisionCastQueryType, 
               mask: CollisionGroup, 
         relativeTo: Entity) -> [CollisionCastHit]

ARView.session.trackedRaycast(_:updateHandler:)

This instance method repeats a ray-cast query over time to notify you of updated surfaces in the physical environment. You can use this type of raycast in ARKit 3.5.

func trackedRaycast(_ query: ARRaycastQuery, 
              updateHandler: @escaping ([ARRaycastResult]) -> Void) -> ARTrackedRaycast?

ARView.trackedRaycast(from:allowing:alignment:updateHandler:)

This RealityKit's instance method also performs a tracked ray cast, but here a ray is cast into the scene from the center of the camera through a point in the view.

func trackedRaycast(from point: CGPoint, 
               allowing target: ARRaycastQuery.Target, 
                     alignment: ARRaycastQuery.TargetAlignment, 
                 updateHandler: @escaping ([ARRaycastResult]) -> Void) -> ARTrackedRaycast?

Code snippet 01:

import RealityKit

let startPosition: SIMD3<Float> = [3,-2,0]
let endPosition: SIMD3<Float> = [10,7,-5]
let query: CollisionCastQueryType = .all
let mask: CollisionGroup = .all

let raycasts: [CollisionCastHit] = arView.scene.raycast(from: startPosition, 
                                                          to: endPosition, 
                                                       query: query,  
                                                        mask: mask, 
                                                  relativeTo: nil)

guard let rayCast: CollisionCastHit = raycasts.first
else { 
    return 
}

Code snippet 02:

import ARKit

let query = arView.raycastQuery(from: screenCenter,
                            allowing: .estimatedPlane,
                           alignment: .any)

let raycast = session.trackedRaycast(query) { results in

    if let result = results.first {
        object.transform = result.transform
    } 
}

raycast.stop()