4

I am trying to receive a depth image from an Orbbec Astra Pro camera connected to a Windows 10 machine. I have therfor installed opencv-python 4.0.0.21 and primesense 2.2.0.30.post5 which seems to be latest stable python packages available.

This is the code snippet I am experimenting with: 

import numpy as np
import cv2
from primesense import openni2
from primesense import _openni2 as c_api

openni2.initialize("./OpenNI-Windows-x64-2.3/Redist")    

if openni2.is_initialized():
    print('openni2 ready')
else:
    print('openni2 not ready')

dev = openni2.Device.open_any()

depth_stream = dev.create_depth_stream()
depth_stream.set_video_mode(c_api.OniVideoMode(pixelFormat = c_api.OniPixelFormat.ONI_PIXEL_FORMAT_DEPTH_100_UM, resolutionX = 640, resolutionY = 480, fps = 30))
depth_stream.start()

while(True):
    frame = depth_stream.read_frame()
    frame_data = frame.get_buffer_as_uint16()
    img = np.frombuffer(frame_data, dtype=np.uint16)
    img.shape = (1, 480, 640)
    img = np.concatenate((img, img, img), axis=0)
    img = np.swapaxes(img, 0, 2)
    img = np.swapaxes(img, 0, 1)

    cv2.imshow("image", img)
    cv2.waitKey(34)

depth_stream.stop()
openni2.unload()

However when cv2.imshow() is getting called I am receiving:

cv2.error: OpenCV(4.0.0) C:\projects\opencv-python\opencv\modules\highgui\src\window_w32.cpp:1230: error: (-215:Assertion failed) dst.data == (uchar*)dst_ptr in function 'cvShowImage'

I have no idea on how to convert an OpenNI frame to an OpenCV mat data structure and why cv2.imshow() refuses to show an image. OpenNI seems to correctly initialize, at least it prints openni2 ready... What am I doing wrong here?

Edit

This seems to be a bug which I have reported here https://github.com/skvark/opencv-python/issues/167

Solution (kind of)

Downgrading OpenCV version to the latest 3.x release made it work!

pip install --upgrade opencv-python==3.4.5.20

Matthias Güntert
  • 4,013
  • 6
  • 41
  • 89
  • What is the shape and dtype of `img` right before the `cv2.imshow(...)`? – Berriel Jan 23 '19 at 23:34
  • Thanks for replying. The shape of `img` is `{tuple} (480, 640, 3)` and dtype is `{dtype} uint16` – Matthias Güntert Jan 24 '19 at 18:46
  • I have found the following which may seem related: https://github.com/opencv/opencv/issues/13216 – Matthias Güntert Jan 24 '19 at 19:00
  • 1
    Looks like this is the case. I asked, because I tried in my machine (Ubuntu, OpenCV 4.0) with `np.uint16` and it worked. Maybe Windows build is messed up. Which version are you using? Maybe you should jump into this issue and report the bug if you cannot find the solution over there. – Berriel Jan 24 '19 at 20:03
  • Thanks again. Opened up an issue, see my updated question – Matthias Güntert Jan 24 '19 at 20:23
  • 1
    If I were you, I would see if the error happens when you try to run `cv2.imshow('test', np.ones((50, 50), dtype=np.uint16)); cv2.waitKey()`, because this probably has nothing to do with the `openni` and adding this requirement on the sample code only decreases the probability of someone testing your code. – Berriel Jan 24 '19 at 20:31
  • 1
    Yields the same assertion error... so opnni can be ruled out. – Matthias Güntert Jan 24 '19 at 21:08
  • I recommend you to update the sample code on the issue in order to simplify for those who might be willing to reproduce the error – Berriel Jan 24 '19 at 21:21

1 Answers1

0

Have you tried:

frame.get_buffer_as_uint32()
img = np.frombuffer(frame_data, dtype=np.uint32)

In opencv4 for some reason imshow will not work if you do not use specific dtype

Antonin GAVREL
  • 9,682
  • 8
  • 54
  • 81