27

I've recently started hacking on my Kinect and I want to remove the depth shadow. The shadow is caused by the IR emitter being positioned slightly to the side of the camera, so any close object will get a big shadow and distant object less or no shadow.

The shadow length is related to the distance between the closest and the farthest spot on each side of the shadow.

My goal is to be able to map the color image correctly onto the depth. This doesn't work without processing the shadow as this picture shows:
enter image description here

Tedd Hansen
  • 12,074
  • 14
  • 61
  • 97
  • Maybe some kind of trapezoid resize will work? – chriszero Jan 31 '11 at 14:18
  • 2
    It'd help to be more explicit about what data you have to work with, for the Kinectless like me. It sounds like you have a grid of pixels from the camera and a corresponding grid of distances from the IR emitter. Right? – Darius Bacon Jan 31 '11 at 20:36
  • I have the data you see in the two pictures. The black area is shadow with color 0. The rest has a color value depending on distance. – Tedd Hansen Feb 02 '11 at 16:32
  • 1
    maybe you're too close to the sensor? MS does advice you to be a few feet away from it, I guess this is the reason. – fejesjoco Feb 02 '11 at 20:39
  • @Tedd I notice you tagged this with C#, which libraries are you using for this? I've been waiting for the official MS SDK to come out so that I can use C# in a University project. – Darren Young May 27 '11 at 09:45
  • @Darren: http://blog.tedd.no/2011/02/01/early-alpha-release-kinect-c-starter-kit/ ... its come a long way since then, but that is the latest public release. ... I'm waiting for MS SDK too. :) – Tedd Hansen May 29 '11 at 14:18
  • @Tedd Thanks. Hopefully the offical SDk shall be released soon. MS did say it would be released in Spring, so hopefully anytime now. – Darren Young May 31 '11 at 10:30
  • @TeddHansen did you solve the problem in the meantime? – memyself Nov 12 '12 at 18:25
  • I never found a solution to this, but also haven't looked at the MS SDK yet. – Tedd Hansen Nov 12 '12 at 20:41
  • Was the Temporal Median of no help? – Nick Udell Aug 14 '14 at 12:00

6 Answers6

3

Does the depth shadow always come out black?

If so you could use a simple method like a temporal median to calculate the background of the image (more info here: http://www.roborealm.com/help/Temporal_Median.php) and then whenever a pixel is black, set it to the background value at that pixel location.

Nick Udell
  • 2,420
  • 5
  • 44
  • 83
3

I did some preliminary work on this problem a few weeks ago. My code works directly on a WriteableBitmap rather than the depth data, but if you're only doing image processing, it should work. The algorithm isn't perfect and would benefit with some more tweaking. If you update the code at all, let me know; I'd be very interested to see what you're doing!

The source code is posted on my blog:

http://richardpianka.com/2011/02/trackingni-depth-correction/

Richard Pianka
  • 3,317
  • 2
  • 28
  • 36
2

I don't know how it is with c# but openni c++ has a function called xnSetViewPoint() the only problem is, you lose the top 20 or so rows of imagedata due to the transformation.

the reason is due to using two different sensors, which are placed close by each other but not exactly at the same position.

Jan
  • 1,827
  • 2
  • 16
  • 29
1

Kinect Method - MapDepthFrametoColorFrame

Get the [x,y] positions in the depth frame, and use that method to fill in

Shai
  • 111,146
  • 38
  • 238
  • 371
16per9
  • 502
  • 4
  • 17
1

The color camera and kinect depth camera dont have the same dimensions, and origin of the infra red dots are not from the same cam, its a IR projector a few cm aside from it (as that displacement is used to calculate depth).

However the solution seams easy here, your shadow data is on the left side. so you need to extend the last known color data before it went black.

And to fit it better move translate the color cam data to the right.

Peter
  • 2,043
  • 1
  • 21
  • 45
0

I'm sorry to say but that shadow is caused by your body blocking the inferred dots from hitting that spot of the room so it creates a black spot... Nothing you can do but change the base background to a different color other than black so it won't be a noticeable shadow

Trevor Rudolph
  • 1,055
  • 4
  • 18
  • 42
  • That I understand, but if you look at the image of me then you see that the shadow is a perfect match, but wrongly aligned and sized. Size and alignment depends on proximity. if it was calibrated I would be able to repaint myself perfectly in the foreground while removing the background. (It is easy to see that all the required data is there.) – Tedd Hansen Aug 30 '11 at 16:04
  • oh, i see what you meen... i can't help – Trevor Rudolph Sep 03 '11 at 02:58