9

I have a numpy array which is the color image "img" of shape (height, width, 3), and also a depth numpy array of shape (height, width). And I want to create an RGBD image and display it, for which I'm doing the following:

o3d.geometry.RGBDImage.create_from_color_and_depth(img, depth)

But I'm getting the error:

TypeError: create_from_color_and_depth(): incompatible function arguments. The following argument types are supported:
    1. (color: open3d.open3d_pybind.geometry.Image, depth: open3d.open3d_pybind.geometry.Image, depth_scale: float = 1000.0, depth_trunc: float = 3.0, convert_rgb_to_intensity: bool = True) -> open3d.open3d_pybind.geometry.RGBDImage

How to fix this? If it requires an Image type, then how to convert the numpy array to an Image type?

Even when I pass in the numpy arrays into the o3d.geometry.Image constructors like so:

o3d.geometry.RGBDImage.create_from_color_and_depth(o3d.geometry.Image(img), o3d.geometry.Image(depth))

I get the error:

TypeError: create_from_color_and_depth(): incompatible function arguments. The following argument types are supported:
    1. (color: open3d.open3d_pybind.geometry.Image, depth: open3d.open3d_pybind.geometry.Image, depth_scale: float = 1000.0, depth_trunc: float = 3.0, convert_rgb_to_intensity: bool = True) -> open3d.open3d_pybind.geometry.RGBDImage

How to fix this and create the RGBD image from the rgb numpy array and the depth numpy array?

zendevil
  • 899
  • 4
  • 13
  • 26
  • I am having a very similar problem. Did you manage to solve it and save the depth image? – M.K Feb 09 '22 at 07:31

3 Answers3

3

Would be great if you shared reproducible example, but I think that creating Image from np.array isn't as simple as calling the ctor, basing on signature. This is not necessarily an array of uint8, right?

Basing on this article, you have to create it as follows:

depth_as_img = o3d.geometry.Image((depth).astype(np.uint8))

and pass further to create_from_color_and_depth. So, you have to explicitly specify that it's an array of uint8s.

Marek Piotrowski
  • 2,988
  • 3
  • 11
  • 16
  • 3
    In my case, the depth image is of type `np.float32` which works fine, too. However, you have to make sure it's a c-type array as well. This is the default, so you don't need to worry to much about it, but if it's not c-type, you can use `o3d.geometry.Image(np.ascontiguousarray(np_array).astype(np.float32))` to avoid errors. – ketza Mar 25 '21 at 15:08
1

You should check the convert_rgb_to_intensity parameter in this function. By default, it will use the greyscale image. In this way, your color image should have only one channel. If you want RGB, set that parameter to false and see if it solves.

In addition, extra comments for from numpy to open3d image: https://github.com/intel-isl/Open3D/issues/957

Shawn Jiang
  • 80
  • 11
1

I am with @ketza on this one. You have to convert your images to the contiguous arrays. you can do this with numpy arrays as following!

depth_map = np.array(depth_map, dtype=np.float32)
Achyut Sarma
  • 119
  • 7