Aligning Kinect V2 RGB picture and depth map using java

Question

I would like to know if there is a way to align the RGB picture and the depth data of a Kinect V2 using the colour data as a starting point using Java, I am actually using java for Kinect as a wrapper and it does not seem to give me the possibility for that. Is there any way doing that?

Answer 1

• Finally got around it by using @Spektre answer here, I got to play around with the formulas to make it work but it seem fine to me. Rectified for my needs, it gives:

          int alignx= (((x-512)<<8)/241)+Width;
          int aligny= (((y-424)<<8)/240)+25+Height;
  

It works fine as long as the Kinect is at the same level that the object you want to target (ie: no pitch used).

Answer 2

I don't quite agree with Alex Acquier’s answer, it is not the right approach I feel. I have faced the same issue and I know I am doing this 8 months late, but in the interest of others who came here searching for the solution, I now present it here:

The thing is, you don't have to manually align the RGB and the Depth frames. There is a class already available that can do that for you, "IMultiSourceFrameReader". Using this as a source, you can sure to be making the point cloud in the right way.

Now this is okay if you just want to use the feeds. But if somewhere down your code, if you are going to be using some kind of a coordinate system and if you are going to be needing the coordinates of the RGB and the depth pixels, then you would expect them them to be the same, right? Because you are using the aligned coordinates after all, right? But you won't get the coordinates aligned till to use the "ICoordinateMapper" class. This class will make all the coordinates from all the different sensors, RGB and Infra, to align as well and will return the aligned coordinates.

Please refer to this source, it has been my go to source for Kinect V2 since a long time.