Kinect sideways skeleton display

Question

I am developing a game using Kinect with Windows Kinect SDK 1.5. The Kinect camera is put on the desk and I am standing sideways towards Kinect. I want to display skeleton of the player, like figure 1 and 2. enter image description here Figure 1 Figure 2

These need the position of joint detected by Kinect Skeleton Stream. But in most case, some joints hidden behind cannot be detected by Kinect when the player is standing sideways. As you see in figure 1, when I overlap my arms, I don’t want the joints of the hidden arm to display on the screen. I just want to display the joints in my left arm. The other situation is that, when I put my arms in normal ways, as you see in figure 2, I want to display both of them.

My idea is to use the position information and tracked state of joints to decide whether one arm is behind another part of your body (There are many kinds of overlap, arm-arm, arm-body, arm-head, -arm to leg). But I don’t think it is enough just do not display joints which are not tracked by Kinect. Maybe we can detect whether two arms overlap or not by calculating distance between the two wrist, the two elbows and the two shoulders. If the distance between the two wrists and the two elbows are all within in the threshold, there is overlap of both arms and we can just display the joints of the tracked arm skeleton.

But there is another problem that there are many ways for an arm to overlap with anther parts of body, and so does legs. So this solution needs so many threshold and checks before display. Besides, I am not sure about the effects of this solution.

Do you guys have cooler ideas to settle this problem?

exact duplicate http://stackoverflow.com/questions/10192476/kinect-sideways-skeleton-tracking — Kinected, Nov 04 '12 at 00:03

score 0 · Answer 1 · answered Oct 26 '12 at 22:01

0

Have a look at the SkeletonBasics-WPF example that comes with the v1.5 SDK Toolkit examples. It will show you a similar action of what you're wanting to do.

The block of code that will likely be of most interest is this one:

foreach (Joint joint in skeleton.Joints)
{
    Brush drawBrush = null;

    if (joint.TrackingState == JointTrackingState.Tracked)
    {
        drawBrush = this.trackedJointBrush;
    }
    else if (joint.TrackingState == JointTrackingState.Inferred)
    {
        drawBrush = this.inferredJointBrush;
    }

    if (drawBrush != null)
    {
        drawingContext.DrawEllipse(drawBrush, null, this.SkeletonPointToScreen(joint.Position), JointThickness, JointThickness);
    }
}

Note the JointTrackingState.Inferred check. In this demo, the joints continue to show but they turn yellow. In your case, you can just hide them. JointTracingState also has NotTracked enumeration, which may also be of use depending on your situation.

Bones, in the case of the example, are also only drawn between joints that are Tracked.

By the way - the official Kinect for Windows SDK is v1.6, as of this writing. You may want to look into updating.

answered Oct 26 '12 at 22:01

Nicholas Pappas

10,439
12
54
87

Thanks for your comment. I have the function that draw different color for tracked, referred and not-tracked joints. See grey points in my figure are referred joints(blue points are tracked joints). – Badger Oct 26 '12 at 22:40
Your question is hiding joints and bones that are hidden. Inferred joints are those the Kinect can not see (i.e., they're blocked) but is guessing at, while NotTracked joints are just plain out of view. Looking at Inferred (and their z-index compared to known joints) will tell you where and how they are blocked. Or am I missing your question? – Nicholas Pappas Oct 26 '12 at 22:45
Thanks for your comment. So inferred joint also has x,y,z position information but they are can't be seen by Kinect. Do you know any other situation that may cause a inferred joint except they are blocked?
I found that sometimes joints are inferred even there are no barrier between the joint and Kinect. – Badger Oct 29 '12 at 19:23
2

An Inferred joint can happen any time the Kinect loses track of a particular joint. It is not necessarily because it is hidden, just that the Kinect lost track of it. The skeleton tracker "guesses" where it is (hence "inferred") and continues to provide x/y/z coordinates for where it thinks the joint is. – Nicholas Pappas Oct 29 '12 at 22:19

score 0 · Answer 2 · edited May 23 '17 at 12:21

This is an exact duplicate of Kinect sideways skeleton tracking, so just look at Outlaw Lemur's answer, which explains the capabilities of Kinect and why this doesn't work and some attempts at work-arounds of how to get it to work. Essentially he is saying that Kinect is made to track people face on or their back, but will interpret that as their front. Sideways tracking doesn't work because Kinect can only see half of your joints, and has to estimate for the rest, leading to bad results.

Kinect sideways skeleton display

2 Answers2