Convertion Between cv::Mat and XnDepthPixel*

Question

i am working with OpenNI 1.5.4.0 and OpenCV 2.4.5, plus Qt for visualization purpose (only RGB images). Basically, i am retrieving the depth and rgb frames from a Kinect and store them on the hard drive, using the conversion:

/* Depth conversion */

cv::Mat depth = cv::Mat(2, sizes, CV_16UC1, (void*) pDepthMap); //XnDepthPixel *pDepthMap 

/* RGB conversion */

///image is a QImage* , pImageMap is a XnUInt8*

for(int i = 0; i < height; ++i)
{
    for (unsigned y = 0; y < height; ++y)
    {
       uchar *imagePtr = (*image).scanLine(y);
       for (unsigned x=0; x < width; ++x)
       {
          imagePtr[0] = pImageMap[2];
          imagePtr[1] = pImageMap[1];
          imagePtr[2] = pImageMap[0];
          imagePtr[3] = 0xff;
          imagePtr+=4;
          pImageMap+=3;
       }
    }
}

Now, i want to load those images from hard drive, in order to compute the 3D pointclouds as a post-processing computation. I am loading the Depth maps as:

depth_image = cv::imread(m_rgb_files.at(w).toStdString(), CV_LOAD_IMAGE_ANYDEPTH | CV_LOAD_IMAGE_ANYCOLOR );

but applying the formulas:

depth = depth_image.at<int>(j,i);  //depth_image is stored as 16bit
p.z = (float)depth * 0.001f;    //converting from millimeters to meters
p.x = (float)(i - m_params.center_x) * depth * m_params.focal_length; //focal_length read using OpenNI function
p.y = (float)(j - m_params.center_y) * depth * m_params.focal_length;

the pointcloud obtained is a mess.

If i do the "online" processing, using directly the native XnDepthPixel* data, the result is perfect, using the formulas written before. Can anybody give me a "hint" about my fault?

thanks in advance

EDIT: i was following also this resource, but it doesn't work for me, because XnDepthPixel contains the real world data in millimeters

ok sorry..i am still new to the platform – madduci May 13 '13 at 14:52 — madduci, May 13 '13 at 14:52

score 0 · Accepted Answer · answered May 11 '13 at 16:19

0

I think there is a possible problem here:

depth = depth_image.at<int>(j,i);  //depth_image is stored as 16bit

If the depth image really is 16-bit, should it really be:

depth = depth_image.at<short>(j,i);  //depth_image is stored as 16bit

Because int is 32-bits, not 16.

answered May 11 '13 at 16:19

Roger Rowland

25,885
11
72
113

the depth map is actually 10 bit + 1 bit for validity check, but it is stored into a unsigned int of 16 bit structure (called XnDepthMap, which is a renaming of uint16_t). I will try to use as suggested. I had the doubt about the conversion (on saving step) between uint16_t* and unsigned char* (unfortunately, it's the default constructor of cv::Mat), which could affect the result. Thanks for the suggestion – madduci May 12 '13 at 08:44
1

depth = depth_image.at(j,i); did the trick. Many thanks – madduci May 13 '13 at 14:49

score 0 · Answer 2 · answered May 12 '13 at 01:14

0

Just as an addition, if you've built OpenCV with OpenNI support, you can retrieve the depth image as a cv::Mat which you can easily save with cv::imwrite Here's a minimal example:

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"

#include <iostream>

using namespace cv;
using namespace std;

int main(){
    VideoCapture capture;
    capture.open(CV_CAP_OPENNI);

    if( !capture.isOpened() ){
        cout << "Can not open a capture object." << endl;
        return -1;
    }
    cout << "ready" << endl;

    for(;;){
        Mat depthMap,depthShow;
        if( !capture.grab() ){
            cout << "Can not grab images." << endl;
            return -1;
        }else{
            if( capture.retrieve( depthMap, CV_CAP_OPENNI_DEPTH_MAP ) ){
                const float scaleFactor = 0.05f;
                depthMap.convertTo( depthShow, CV_8UC1, scaleFactor );
                imshow("depth",depthShow);
                if( waitKey( 30 ) == 115)    {//s to save
                    imwrite("your_depth_naming_convention_here.png",depthShow);
                }
            }
        }
        if( waitKey( 30 ) == 27 )    break;//esc to exit
    }

}

answered May 12 '13 at 01:14

George Profenza

50,687
19
144
218

i am not using directly OpenCV for retrieving the depth map because i am using multiple Kinects and it doesn't enumerate the additional devices that's why i switched to OpenNI pure APIs. Anyway, thanks for the suggestion – madduci May 12 '13 at 08:40
You can use multiple kinects with OpenCV, I tried [this](http://stackoverflow.com/questions/14983248/how-to-initialize-multiple-openni-sensors-with-opencv) before – George Profenza May 12 '13 at 10:28
thanks for the hint, but i've tried it so far and didn't work for me. It is failing on enumerating the devices (i have the rev. 1473 and neither PointCloudLibrary can detect more than one kinect). – madduci May 12 '13 at 14:48
That's a shame :( How can I check what revision I have ? I remember building OpenCV 2.4.4 from source at some point in Feb. using the archive supplied in the main download links. – George Profenza May 12 '13 at 17:08
you should check under the Kinect or with lsusb -vv (ultra verbose) – madduci May 12 '13 at 21:10

Convertion Between cv::Mat and XnDepthPixel*

2 Answers2

Linked