A kinect cursor in unity

Question

I wish to be able to get a curser working in unity using Kinect. its essential that i do this by using the kinect sdk as a plugin into unity and not to use openni , ive managed to get it half working however its not really perfected. At the moment ive got it working along the x axis but it doesnt cover the whole screen. It would be more ideal to have the cursor start at the center of the screen the move accordingly. At the moment when moving more towards the right side of the screen it only goes halfway. I rather not hard code in a float number, but to rather calculate according to screen. In unity there was a function where you could scale to world

void Update () {


    if (sw.pollSkeleton())
    {
        distance.x=sw.bonePos[0,2].x - sw.bonePos[0,7].x;//bonpos 0,2 is the spine 0,7 is the left hand

        //spine position will usually be around (0.1,1.2,0.3)
        /*
        Debug.Log("the distance is"+distance.x);

        */
        /*
         * (-0.2,1.4,0.7) hand over head on left side// essentially by this trime
         * (-0.3,1.0,0.6) hand near shoulders resonably high (-0.4,1.0,0.7) shoulders meaning anything 1.0 region
         * (-0.2,0.6,0.7) hand lower torso level curser should then be alive
         * (-)
         * (-0.1,0.7,0.3)hand at the side shouldnt record curser (-0.1,0.5,0.3) 0.0 -0.1
         * 
         */

        /*
         * (0.3,1.3,0.8)left hand over head on right side
         * (0.2,1.0,0.8) hand higher up on the other side right shoulder (0.4,1.0,0.7) further right
         * (0.3,0.9,0.7) hand lower then shoullder but pointing towards the right
         * (0.3,0.7,0.6) hand lower right below waist
         * (0.2,0.7,0.5) hand close to groin and spine
         */



        //if the left hands x axis is less than that of the spine
        //
        if(sw.bonePos[0,7].x<sw.bonePos[0,2].x)
        {
            //distance being 0.2053766 * -0.5f
             difference = -0.5f*distance.x;
            //difference being -0.1026883
            Debug.Log("hand is over by the left");

        }
        if(sw.bonePos[0,7].x>=sw.bonePos[0,2].x)
        {
             difference = -0.8f*distance.x;
            Debug.Log("hand is over by the right");

        }

        Debug.Log("Dist: " + distance.x + ", Diff: " + difference);


        Debug.Log("The spine is positioned at :" +sw.bonePos[0,2]+"The left hand is at :" +sw.bonePos[0,7]+" the position of the cursor"+transform.position);
    }


}

void OnGUI() {
    //left top width height
    Rect r = new Rect(Screen.width * (difference + 0.3f),0.5f*Screen.height,100,100);
    /*screen.width 480 * 0.1973117,0.5f*1280,100,100
     * 
     * 
     */
    GUI.Label(r,cursor);
    //GUI.DrawTexture();
}
}

score 0 · Answer 1 · edited May 23 '17 at 12:28

I've not dealt with Unity, so I can unfortunately not speak directly to the code for Unity -- but I've posted code before for the Kinect SDK. I've included source code that creates a bounding box around the player to represent the screen in my answer to this question:

how to use skeletal joint to act as cursor using bounds (No gestures)

The code in the above post creates a box that is bounded by the following points around the player's body:

Shoulders = top of screen
Hips = bottom of screen
Left Shoulder = left most on screen

To get the right most screen position, I take the distance between the left & right shoulder and add it to the right shoulder position.

This provides a comfortable area that allows the player to move the cursor around the screen. It also easily scales to different players of different sizes, and will also work no matter where the player is standing within the Kinect's FOV.

The concept should be adaptable to the Unity code.

:) thank you @EvilClosetMonkey for the help and having such a cool name — j bel, Mar 21 '13 at 16:46

score 0 · Accepted Answer · edited May 23 '17 at 11:49

Just incase anybody comes across this problem i found a work around that one can play with which i found on this very site Mapping a range of values to another

float translate(float value, float leftMin, float leftMax, float rightMin,float rightMax)
{
    float leftSpan = leftMax - leftMin;
    float rightSpan= rightMax - rightMin;

    float valueScaled = (float)(value-leftMin)/(float)(leftSpan);
    return rightMin+(valueScaled * rightSpan);
}

in my update method i would do this

differnetx=translate(distance.x,.6f,0,0,1);
differencey=translate(distance.y,.5f,0,0,1);

This creates a rectangle from the furthest stretch of the hand to the spine, and maps it to the screen. The kinect sometimes get confused when the hand goes across the body so i thought it would be better to instead do all the navigation away from crossing the spine

A kinect cursor in unity

2 Answers2