how to use skeletal joint to act as cursor using bounds (No gestures)

Question

I just want to be able to do something when my skeletal joint (x,y,z) coordinates are over the x,y,z coordinates of the button . . I have the following code but somehow it doesnt work properly . .as soon as my hand moves it will do something without my hand reaching the button

    if (skeletonFrame != null)
                {
                    //int skeletonSlot = 0;
                    Skeleton[] skeletonData = new Skeleton[skeletonFrame.SkeletonArrayLength];
                    skeletonFrame.CopySkeletonDataTo(skeletonData);

                    Skeleton playerSkeleton = (from s in skeletonData where s.TrackingState == SkeletonTrackingState.Tracked select s).FirstOrDefault();

                    if (playerSkeleton != null)
                    {
                        Joint rightHand = playerSkeleton.Joints[JointType.HandRight];

                        handPosition = new Vector2((((0.5f * rightHand.Position.X) + 0.5f) * (640)), (((-0.5f * rightHand.Position.Y) + 0.5f) * (480)));

                        var rightHands = playerSkeleton.Joints[JointType.HandRight];
                        var rightHandsX = rightHands.Position.X;
                        var rightHandsY = rightHands.Position.Y;
                        var rightHandsZ = rightHands.Position.Z;


                        if (Math.Sqrt(Math.Pow(rightHandsX - position.X, 2) + Math.Pow(rightHandsY - position.Y, 2)) < 20)
                        {

                           // Exit();
                        }



if (Math.Sqrt(Math.Pow(rightHandsX - start_bttn.Bounds.X, 1) + Math.Pow(rightHandsY - start_bttn.Bounds.Y, 1)) < 10)
                        {

                            currentGameState = GameState.Selection;
                            // Exit();
                        }

if ((rightHandsX < GraphicsDevice.Viewport.Width / 2 + 150 && rightHandsX > GraphicsDevice.Viewport.Width / 2 - 75) && (rightHandsY > GraphicsDevice.Viewport.Height / 2 && rightHandsY < GraphicsDevice.Viewport.Height / 2 + 50))
                        {
                            currentGameState = GameState.Selection;

                        }

}

score 1 · Answer 1 · answered Sep 26 '12 at 21:44

Here is my hand tracking function. See if it does what you want, or gets you closer...

    private void TrackHandMovement(Skeleton skeleton)
    {
        Joint leftHand = skeleton.Joints[JointType.HandLeft];
        Joint rightHand = skeleton.Joints[JointType.HandRight];

        Joint leftShoulder = skeleton.Joints[JointType.ShoulderLeft];
        Joint rightShoulder = skeleton.Joints[JointType.ShoulderRight];

        Joint rightHip = skeleton.Joints[JointType.HipRight];

        // the right hand joint is being tracked
        if (rightHand.TrackingState == JointTrackingState.Tracked)
        {
            // the hand is sufficiently in front of the shoulder
            if (rightShoulder.Position.Z - rightHand.Position.Z > 0.4)
            {
                double xScaled = (rightHand.Position.X - leftShoulder.Position.X) / ((rightShoulder.Position.X - leftShoulder.Position.X) * 2) * SystemParameters.PrimaryScreenWidth;
                double yScaled = (rightHand.Position.Y - rightShoulder.Position.Y) / (rightHip.Position.Y - rightShoulder.Position.Y) * SystemParameters.PrimaryScreenHeight;

                // the hand has moved enough to update screen position (jitter control / smoothing)
                if (Math.Abs(rightHand.Position.X - xPrevious) > MoveThreshold || Math.Abs(rightHand.Position.Y - yPrevious) > MoveThreshold)
                {
                    RightHandX = xScaled;
                    RightHandY = yScaled;

                    xPrevious = rightHand.Position.X;
                    yPrevious = rightHand.Position.Y;

                    // reset the tracking timer
                    trackingTimerCounter = 10;
                }
            }
        }
    }

There is a bit of math in there to translate the hand position to the screen position. Different strokes for different folks, but my logic is:

Shoulders = top of screen
Hips = bottom of screen
Left Should = left most on screen

To get the right most screen position, I take the distance between the left & right shoulder and add it to the right shoulder.

Hey , Thank you so much for answering..and this mite be a really stupid question but i dont understand what the double xScaled & yScaled is doing? Why do we need it? — Sweta Dwivedi, Sep 27 '12 at 11:17
That is the piece of code that is translating the hand position to the screen position, relative to the primary display's resolution. Mapping the shoulders to the top of the screen, hips to the bottom, left shoulder to the left screen, and a short distance outside the right shoulder to the right of the screen. It produces a "hit box" for the user's arm, so they don't have to stretch wildly in any given direction to get the cursor somewhere on the screen. — Nicholas Pappas, Sep 27 '12 at 15:10

how to use skeletal joint to act as cursor using bounds (No gestures)

1 Answers1

Linked