I'm currently struggling on the conversion of an image captured from ARCore in YUV_420_888 format to an OpenCV Mat object. I use ARCore as a plugin for unity from which I'm able to capture the bytes that represent the image. I've furthermore written a marker detection in c++ using OpenCV that I planned on using as a native plugin. So far so good.
ARCore has the functionality of returning a GoogleARCore.CameraImageBytes object that contains the following information (taken from a single frame transmitted to the native plugin):
Width: 640
Height: 480
Y Row Stride: 640
UV Row Stride: -1008586736
UV Pixel Stride: 307200
as well as the image data as an System.IntPtr, which I'm able to convert to a byte[] and thus receive it on c++ side as unsigned char *.
These information are passed to c++ in the following functions signature:
extern "C" void findMarkersInImage(int width, int height, int y_row_stride, int uv_row_stride, int uv_pixel_stride,
unsigned char *image_data, int buffer_size)
I realize that there are many answers on this or various other platforms that suggest an algorithm for the conversion. However, all of them employ functionality to direclty gather image plane information from the image. Saying that every solution relies on calling some function called getImageplanes(image, plane_number). Furthermore all of the other solution invoke the use of an AImage which is not available to me. Also converting the gathered image bytes to an AImage, and then to a cv::Mat seems like an computational overkill for an algorithm that is supposed to run in realtime, supporting, not negatively affecting the overall performance.
So finally three main questions occur to me:
Is there a way to find out the planes positions in the
unsigned char *I get containing the whole image.The UV Row Stride value differs in every single frame, not by much but it does, is that normal behaviour?
The before-mentioned value is negative.
Maybe someone can suggest further literature that might help me. Or has a direct answer to one of the questions.
