how to get depth values and then convert to millimetres in OpenCV with Kinect?

Question

I have VS2010 running with OpenCV 3.2.1 with kniect.

i'm using the tutorial from here

int main( int argc, char* argv[] ){

VideoCapture capture(CV_CAP_OPENNI); // or CV_CAP_OPENNI
for(;;)
{
    Mat depthMap;
    Mat bgrImage;

    capture.grab();

    capture.retrieve( depthMap, CV_CAP_OPENNI_DEPTH_MAP ); // Depth values in mm (CV_16UC1)
    capture.retrieve( bgrImage, CV_CAP_OPENNI_BGR_IMAGE );

    cout << "rows: " << depthMap.rows << " cols: " << depthMap.cols << endl;
    cout << "depth: " << depthMap.at<int>(0,0) << endl;

    imshow("RGB image", bgrImage);

    if( waitKey( 30 ) >= 0 )
        break;
}h
return 0;

after running the code I get the following results:

close to the wall about a meter away:

rows: 480 cols: 640
depth: 1157
rows: 480 cols: 640
depth: 1157
rows: 480 cols: 640
depth: 1157
rows: 480 cols: 640
depth: 1157

(Kinect facing the wall, just over a meter away)

rows: 480 cols: 640
depth: 83690629
rows: 480 cols: 640
depth: 83690629
rows: 480 cols: 640
depth: 83690629
rows: 480 cols: 640
depth: 83690629
rows: 480 cols: 640
depth: 83690629

I have been told that these values are in fact two pixels? which I really don't understand.

So here is my understanding so far:

I grabbed a depth frame and stored it inside a matrix called depthMap. the size of the matrix is now 640*480. I am using the code depthMap.at(0,0) to grab the first depth value from row 0, column 0. but instead of getting back a result on millimetres i get back 83690629! which is way off the max value of 10000 which i'm expecting

How can I convert these values into millimetres so I can make use of them? thank you

Perhaps the Kinect is too close to the wall on under a meter? I know the restrictions are somewhere around there, if it's not that exactly. Also see [here](http://stackoverflow.com/questions/7696436/precision-of-the-kinect-depth-camera/7709746#7709746) — Liam McInroy, Apr 28 '12 at 13:10
na its not under a meter because I get the same values but slightly higher when taking it much further away like 2-3 meters. Someone has said to me that me to change 'depthMap.at(0,0)' to 'depthMap.at(0,0)' and the resulting value is (83690629) in fact two pixels and I should convert to hex? (489102879 = 0x1d271e1f) dont have a clue what he was on about. but I still get the same values and have no idea on how to best interpret them. — user1361518, Apr 29 '12 at 08:35

score 1 · Answer 1 · answered Jun 12 '12 at 08:02

1

The depth map data is a single channel unsigned 16Bit. Try to replace int to CV_16UC1

answered Jun 12 '12 at 08:02

Eran W

1,696
15
20

score 1 · Answer 2 · answered Jun 27 '12 at 18:42

I'm currently working with the kinect (+OpenNi + OpenCv) and I don't know if you're still stuggling with your issue.

I tried the exact same code but with depthMap.at<unsigned short>(i,j) and it worked very well, giving me the value in mm (502 when I'm 50 cm away from the kinect)

The problem is : I don't have a clue about how the depth value is delivered and I saw a lot of contradictory informations on the internet. Why would it be 2 pixels? Why convert to Hex?

I'm sure I'm not the one to struggle with understanding the underlying principles here. So, does someone has some solid documentation abour the kinect depth??

(I read about stephane Magnenat/Nicolas Burrus formulas : kinect raw data on 11 bits, adding some geometrical stuff and so on... But does OpenNi deal with that already and provide the depth as is??)

Feel free to comment. Cheers.

score 1 · Answer 3 · answered Sep 21 '16 at 09:59

the depth values are stored in 11 bit, encapsulated in a 16 bit variable. Thus, when you access your matrix you must provide the correct element data-type. As answered before, you can use depthMap.at<unsigned short>(i,j) but you must be sure that unsigned short will be 16 bit long in all compilers. I prefer to use uint16_t data type instead.

Furthermore, depending on the openni PIXEL_FORMAT that you have selected, the depth value stored inside the matrix can be:

in millimiters - openni::PIXEL_FORMAT_DEPTH_1_MM
in tenths of millimiters - openni::PIXEL_FORMAT_DEPTH_100_UM

you can set the pixel_format with void openni::VideoMode::setPixelFormat(PixelFormat) function http://www.openni.ru/wp-content/doxygen/html/classopenni_1_1_video_mode.html#af7fd37bba526fced9c7dbbbf1ab419ea

if you want the more accurated data (PIXEL_FORMAT_DEPTH_100_UM) in millimiters, you can cast it to float and divide it by 10:

uint16_t redValue = depthMap.at<uint16_t>(i,j);
float myVal = ( (float) redValue ) / 10.0f;

score 0 · Answer 4 · answered Nov 18 '15 at 23:49

0

check for the correct datatype, debug mode and then breakpoint having the matrix, then check the file datatype

answered Nov 18 '15 at 23:49

Martijn van Wezel

1,120
14
25

how to get depth values and then convert to millimetres in OpenCV with Kinect?

4 Answers4