mapping an ellipse to a joint in kinect sdk 1.5

Question

i want to map an ellipse to the hand joint.And the ellipse has to move as my hand joint will move.

Please provide me some reference links that help me in doing programs using kinect Sdk 1.5. Thank you

Answer 1

Although what @Heisenbug would work, there is a much simpler way in WPF. You can find a tutorial on it at Channel 9's Skeleton Fundamentals. Basically you need a Canvas, and however many ellipses you want. Here is the code XAML

 <Window x:Class="SkeletalTracking.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="MainWindow" Height="600" Width="800" Loaded="Window_Loaded"  
    Closing="Window_Closing" WindowState="Maximized">       
<Canvas Name="MainCanvas">
    <Ellipse Canvas.Left="0" Canvas.Top="0" Height="50" Name="leftEllipse"  Width="50" Fill="#FF4D298D" Opacity="1" Stroke="White" />
    <Ellipse Canvas.Left="100" Canvas.Top="0" Fill="#FF2CACE3" Height="50" Name="rightEllipse" Width="50" Opacity="1" Stroke="White" />
    <Image Canvas.Left="66" Canvas.Top="90" Height="87" Name="headImage" Stretch="Fill" Width="84" Source="/SkeletalTracking;component/c4f-color.png" />
    <Ellipse Canvas.Left="283" Canvas.Top="233" Height="23" Name="leftknee" Stroke="Black" Width="29" />
    <Ellipse Canvas.Left="232" Canvas.Top="233" Height="23" Name="rightknee" Stroke="Black" Width="30" />
</Canvas>
</Window>

Here I am using 4 ellipses (knees, hands) and an image (head). The essential code for this is right here:

    private void ScalePosition(FrameworkElement element, Joint joint)
    {
        //convert the value to X/Y
        //Joint scaledJoint = joint.ScaleTo(1280, 720); 

        //convert & scale (.3 = means 1/3 of joint distance)
        //note you need to have Coding4Fun
        Joint scaledJoint = joint.ScaleTo(1280, 720, .3f, .3f);

        Canvas.SetLeft(element, scaledJoint.Position.X);
        Canvas.SetTop(element, scaledJoint.Position.Y);
    }

This is the key to this program. It takes the location from the joint and changes the elements location to there. The next part is also important, you will do something like this:

    void GetCameraPoint(Skeleton first, AllFramesReadyEventArgs e)
    {

        using (DepthImageFrame depth = e.OpenDepthImageFrame())
        {
            if (depth == null ||
                kinectSensorChooser1.Kinect == null)
            {
                return;
            }


            //Map a joint location to a point on the depth map
            //head
            DepthImagePoint headDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.Head].Position);
            //left hand
            DepthImagePoint leftDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.HandLeft].Position);
            //right hand
            DepthImagePoint rightDepthPoint =
                depth.MapFromSkeletonPoint(first.Joints[JointType.HandRight].Position);


            //Map a depth point to a point on the color image
            //head
            ColorImagePoint headColorPoint =
                depth.MapToColorImagePoint(headDepthPoint.X, headDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);
            //left hand
            ColorImagePoint leftColorPoint =
                depth.MapToColorImagePoint(leftDepthPoint.X, leftDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);
            //right hand
            ColorImagePoint rightColorPoint =
                depth.MapToColorImagePoint(rightDepthPoint.X, rightDepthPoint.Y,
                ColorImageFormat.RgbResolution640x480Fps30);


            //Set location
            CameraPosition(headImage, headColorPoint);
            CameraPosition(leftEllipse, leftColorPoint);
            CameraPosition(rightEllipse, rightColorPoint);
        }
    }

The above code maps the the skeleton point to the depth frame then to the color frame.

    Skeleton GetFirstSkeleton(AllFramesReadyEventArgs e)
    {
        using (SkeletonFrame skeletonFrameData = e.OpenSkeletonFrame())
        {
            if (skeletonFrameData == null)
            {
                return null;
            }


            skeletonFrameData.CopySkeletonDataTo(allSkeletons);

            //get the first tracked skeleton
            Skeleton first = (from s in allSkeletons
                              where s.TrackingState == SkeletonTrackingState.Tracked
                              select s).FirstOrDefault();

            return first;

        }
    }

This just gets the code for the skeleton you will select. Then in AllFrameReadyEventArgs you will do this to bring it all together.

    void sensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
    {
        if (closing)
        {
            return;
        }

        //Get a skeleton
        Skeleton first = GetFirstSkeleton(e);

        if (first == null)
        {
            return;
        }



        //set scaled position
        ScalePosition(headImage, first.Joints[JointType.Head]);
        ScalePosition(leftEllipse, first.Joints[JointType.HandLeft]);
        ScalePosition(rightEllipse, first.Joints[JointType.HandRight]);
        ScalePosition(leftknee, first.Joints[JointType.KneeLeft]);
        ScalePosition(rightknee, first.Joints[JointType.KneeRight]);

        GetCameraPoint(first, e);
    }

Your results will look the best if you have a color image behind, but I decided to skip that part for now. Hope this helps!

Answer 2

Basically somewhere in the code you will have a class to interact with Kinect SDK:

private KinectSensor kinectSensor;

You can initilazied KinectSensor this way:

public void kinectInit()
{
    KinectSensor.KinectSensors.StatusChanged += (object sender, StatusChangedEventArgs e) =>
    {
        if (e.Sensor == kinectSensor)
        {
            if (e.Status != KinectStatus.Connected)
            {
                SetSensor(null);
            }
        }else if ((kinectSensor == null) && (e.Status == KinectStatus.Connected))
        {
            SetSensor(e.Sensor);
        }
    };

    foreach (var sensor in KinectSensor.KinectSensors)
    {
        if (sensor.Status == KinectStatus.Connected)
        {
            SetSensor(sensor);
         }
    }
}

Basically you are defining a delegate to handle KinectSensor's status changed. Where SetSensor method could be something like this:

private void SetSensor(KinectSensor newSensor)
{
    if (kinectSensor != null)
    {
        kinectSensor.Stop();
    }

    kinectSensor = newSensor;

    if (kinectSensor != null)
    {
        kinectSensor.SkeletonStream.Enable();
        kinectSensor.SkeletonFrameReady += OnSkeletonFrameReady;
        kinectSensor.Start();
     }
}

Now, OnSkeletonFrameReady it's the "update function". It will be called continuosly at each Kinect sensor update. From inside it you can retrieve information about the joints and render what you want.

private void OnSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{

    skelFrame = e.OpenSkeletonFrame();
    skeletons = new Skeleton[kinectSensor.SkeletonStream.FrameSkeletonArrayLength];          

    if (skelFrame != null)
    {
        skelFrame.CopySkeletonDataTo(skeletons);
        foreach (Skeleton skel in skeletons) {
        if (skel.TrackingState >= SkeletonTrackingState.Tracked)
        {
            //here's get the joints for each tracked skeleton
            SkeletonPoint rightHand = skel.Joints[JointType.HandRight].Position;
            ....    
         }
    }
}

Since your are using C# and Kinect, the simple library you could use for rendering you ellipse is XNA.