Scene Kit - Dragging an Object

Question

I’m trying to drag a chess piece with the mouse/trackpad using Scene Kit. All objects (board and pieces) are children of the root node, loaded from a Collada file.

I found a helpful description of the process elsewhere on Stack Overflow. Using that description I wrote the initial version of the code below. My problem is the disparity between the click coordinates and the piece node position — their coordinates are different orders of magnitude. I remain unclear on how to match them up — put them in the “same universe”. I’ve tried a number of suggestions from the Apple forums along with flights of fancy of my own.

Here’s my current attempt, mostly reverted back to the original version based on the link above, along with logged coordinate values along the way. The result is that dragging a chess piece causes it to abruptly jump off screen:

- (NSPoint)
viewPointForEvent: (NSEvent *) event_
{
    NSPoint   windowPoint    = [event_ locationInWindow];
    NSPoint   viewPoint        = [self.view convertPoint: windowPoint
                                             fromView: nil];
    return viewPoint;
}

- (SCNHitTestResult *)
hitTestResultForEvent: (NSEvent *) event_
{
    NSPoint      viewPoint        = [self viewPointForEvent: event_];
    CGPoint      cgPoint        = CGPointMake (viewPoint.x, viewPoint.y);
    NSArray * points        = [(SCNView *) self.view hitTest: cgPoint
                                                     options: @{}];
    return points.firstObject;
}

- (void)
mouseDown: (NSEvent *) theEvent
{
    SCNHitTestResult * result = [self hitTestResultForEvent: theEvent];

    SCNVector3 clickWorldCoordinates = result.worldCoordinates;
      log output: clickWorldCoordinates x 208.124578, y -12827.223365, z 3163.659073
    SCNVector3 screenCoordinates = [(SCNView *) self.view projectPoint: clickWorldCoordinates];
    log output: screenCoordinates x 245.128906, y 149.335938, z 0.985565
    // save the z coordinate for use in mouseDragged
    mouseDownClickOnObjectZCoordinate = screenCoordinates.z;

    selectedPiece = result.node;  // save selected piece for use in mouseDragged

    SCNVector3    piecePosition = selectedPiece.position;
      log output: piecePosition x -18.200000, y 6.483060, z 2.350000

    offsetOfMouseClickFromPiece.x = clickWorldCoordinates.x - piecePosition.x;
    offsetOfMouseClickFromPiece.y = clickWorldCoordinates.y - piecePosition.y;
    offsetOfMouseClickFromPiece.z = clickWorldCoordinates.z - piecePosition.z;
    log output: offsetOfMouseClickFromPiece x 226.324578, y -12833.706425, z 3161.309073  
}

- (void)
mouseDragged: (NSEvent *) theEvent;
{
    NSPoint   viewClickPoint        = [self viewPointForEvent: theEvent];

    SCNVector3 clickCoordinates;
    clickCoordinates.x = viewClickPoint.x;
    clickCoordinates.y = viewClickPoint.y;
    clickCoordinates.z = mouseDownClickOnObjectZCoordinate;
      log output:  clickCoordinates x 246.128906, y 0.000000, z 0.985565

      log output:  pieceWorldTransform = 
      m11 = 242.15889219510001, m12 = -0.000045609300002524833, m13 = -0.00000721691076126, m14 = 0, 
      m21 = 0.0000072168760805499971, m22 = -0.000039452697396149999, m23 = 242.15890446329999, m24 = 0, 
      m31 = -0.000045609300002524833, m32 = -242.15889219510001, m33 = -0.000039452676995750002, m34 = 0, 
      m41 = -4268.2349924762348, m42 = -12724.050221935429, m43 = 4852.6652710104272, m44 = 1)

    SCNVector3 newPiecePosition;
    newPiecePosition.x = offsetOfMouseClickFromPiece.x + clickCoordinates.x;
    newPiecePosition.y = offsetOfMouseClickFromPiece.y + clickCoordinates.y;
    newPiecePosition.z = offsetOfMouseClickFromPiece.z + clickCoordinates.z;
      log output: newPiecePosition x 472.453484, y -12833.706425, z 3162.294639

    selectedPiece.position = newPiecePosition;
}

Up to this point, I’ve gotten a lot of interesting and useful comments and advice. But I’ve realized that to move forward, I’m probably going to need a working code sample which shows the secret sauce which allows clicks and vectors to exist in the “same universe”.

score 1 · Answer 1 · answered Jan 27 '15 at 22:56

1

You don't need to "play around" to find the origin. You can do code like this:

let originZ = sceneView.projectPoint(SCNVector3Zero).z
let viewLocation = /* Location in SCNView coordinate system */
let viewLocationWithOriginZ = SCNVector3(
  x: Float(viewLocation.x), // X & Y are in view coordinate system
  y: Float(viewLocation.y),
  z: originZ                // Z is in scene coordinate system
)
var position = sceneView.unprojectPoint(viewLocationWithOriginZ)
/* "position" is in the scene's coordinate system with z of 0 */

answered Jan 27 '15 at 22:56

Donovan Voss

1,450
13
12

What if you want to unproject a point with a different depth? For instance instead of on the Z=0 plane, what if you wanted to convert the touch coordinates to a point 15 units in front of the camera? – Crashalot Sep 10 '16 at 22:57
Like the docs say, the z of the vector that you pass to unprojectPoint (the parameter vector) controls what z position you get back. To map the touch to a visible scene coordinate, the parameter vector's z should be between 0 and 1, corresponding to the near and far clipping panes. Figure out where 15 units in front of the camera is, in those terms, and that gives you the value you should use for originZ in the parameter vector. – Donovan Voss Sep 23 '16 at 03:48

score 0 · Answer 2 · answered Nov 08 '14 at 05:30

You have to use the method unprojectPoint. The key is to play around with z-values to find the correct depth of your scene relative to your click point (at least that's how I understood it). I had a similar problem trying to do touch-dragging on iOS. This is what I did to solve it:

let convertedTranslation = scnView.unprojectPoint(SCNVector3Make(Float(translation.x), Float(translation.y), 1.0))

Notice that the z-value for the method is 1. From the documentation, you can see that it can be a value between -1 and 1. -1 and 0 did not work for me, but 1 did.

What if you want to unproject a point with a different depth? For instance instead of on the Z=0 plane, what if you wanted to convert the touch coordinates to a point 15 units in front of the camera? — Crashalot, Sep 10 '16 at 22:57

Scene Kit - Dragging an Object

2 Answers2