How to move and rotate SCNode using ARKit and Gesture Recognizer?

Question

I am working on an AR based iOS app using ARKit(SceneKit). I used the Apple sample code https://developer.apple.com/documentation/arkit/handling_3d_interaction_and_ui_controls_in_augmented_reality as base for this. Using this i am able to move or rotate the whole Virtual Object.

But i want to select and move/rotate a Child Node in Virtual object using user finger, similar to how we move/rotate the whole Virtual Object itself.

I tried the below two links but it is only moving the child node in particular axis but not freely moving anywhere as the user moves the finger.

ARKit - Drag a node along a specific axis (not on a plane)

Dragging SCNNode in ARKit Using SceneKit

Also i tried replacing the Virtual Object which is a SCNReferenceNode with SCNode so that whatever functionality present for existing Virtual Object applies to Child Node as well, it is not working.

Can anyone please help me on how to freely move/rotate not only the Virtual Object but also the child node of a Virtual Object?

Please find the code i am currently using below,

       let tapPoint: CGPoint = gesture.location(in: sceneView)
        let result = sceneView.hitTest(tapPoint, options: nil)
        if result.count == 0 {
            return
        }
        let scnHitResult: SCNHitTestResult? = result.first
        movedObject = scnHitResult?.node //.parent?.parent

        let hitResults = self.sceneView.hitTest(tapPoint, types: .existingPlane)
        if !hitResults.isEmpty{
            guard let hitResult = hitResults.last else { return }
            movedObject?.position = SCNVector3Make(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
        }

Answer 1

To move an object:

Perform a hitTest to check where you have touched, and detect which plane you touched and get a position. Move your SCNNode to that position by changing the node.position value with an SCNVector3.

Code:

@objc func panDetected(recognizer: UIPanGestureRecognizer){
let hitResult = self.arSceneView.hitTest(loc, types: .existingPlane)
if !hitResult.isEmpty{
guard let hitResult = hitResult.last else { return }
self.yourNode.position = SCNVector3Make(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
}

The above code is enough to move your node over a detected plane, anywhere you touch, and not just in a single axis.

Rotating a node according to your gesture is a very difficult task and I have worked on a solution for quite sometime, never reaching a perfect output. But, I have come across this repository in GitHub which allows you to do just that with a very impressive result. https://github.com/Xartec/ScreenSpaceRotationAndPan

The Swift version of the code you require to rotate your node using your gesture would be :

var previousLoc: CGPoint?
var touchCount: Int?

@objc func panDetected(recognizer: UIPanGestureRecognizer){

    let loc = recognizer.location(in: self.view)
    var delta = recognizer.translation(in: self.view)

    if recognizer.state == .began {
        previousLoc = loc
        touchCount = recognizer.numberOfTouches
    }
    else if gestureRecognizer.state == .changed {
        delta = CGPoint.init(x: 2 * (loc.x - previousLoc.x), y: 2 * (loc.y - previousLoc.y))
        previousLoc = loc
        if touchCount != recognizer.numberOfTouches {
            return
        }
        var rotMatrix: SCNMatrix4!
        let rotX = SCNMatrix4Rotate(SCNMatrix4Identity, Float((1.0/100) * delta.y), 1, 0, 0)
        let rotY = SCNMatrix4Rotate(SCNMatrix4Identity, Float((1.0 / 100) * delta.x), 0, 1, 0)
        rotMatrix = SCNMatrix4Mult(rotX, rotY)

        let transMatrix = SCNMatrix4MakeTranslation(yourNode.position.x, yourNode.position.y, yourNode.position.z)
        self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, SCNMatrix4Invert(transMatrix))
        let parentNoderanslationMatrix = SCNMatrix4MakeTranslation((self.yourNode.parent?.worldPosition.x)!, (self.yourNode.parent?.worldPosition.y)!, (self.yourNode.parent?.worldPosition.z)!)
        let parentNodeMatWOTrans = SCNMatrix4Mult((self.yourNode.parent?.worldTransform)!, SCNMatrix4Invert(parentNoderanslationMatrix))
        self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, parentNodeMatWOTrans)
        let camorbitNodeTransMat = SCNMatrix4MakeTranslation((self.arSceneView.pointOfView?.worldPosition.x)!, (self.arSceneView.pointOfView?.worldPosition.y)!, (self.arSceneView.pointOfView?.worldPosition.z)!)
        let camorbitNodeMatWOTrans = SCNMatrix4Mult((self.arSceneView.pointOfView?.worldTransform)!, SCNMatrix4Invert(camorbitNodeTransMat))
        self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, SCNMatrix4Invert(camorbitNodeMatWOTrans))
        self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, rotMatrix)
        self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, camorbitNodeMatWOTrans)
        self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, SCNMatrix4Invert(parentNodeMatWOTrans))
        self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, transMatrix)
    }
}