Convert GIL image_view directly to OpenCV Mat

Question

I am getting a stream of images from a simulator in the form of a gil::image_view and need to convert them to cv::Mat for further processing. Up until now I more or less copied the code from this answer without really understanding it:

auto image_view = // data received from an API
using pixel = decltype(image_view)::value_type;
static_assert(sizeof(pixel) == 4, "RGBA");
pixel raw_data[image_view.width() * image_view.height()];
boost::gil::copy_pixels(image_view, boost::gil::interleaved_view(image_view.width(),
                                                                 image_view.height(),
                                                                 raw_data,
                                                                 image_view.width() *
                                                                 sizeof(pixel)));
auto mat = cv::Mat(image_view.height(), image_view.width(), CV_8UC4, raw_data);

I'm certain that the alpha channel is not used, so I can also define another image_view:

auto rgb_view = boost::gil::color_converted_view<boost::gil::rgb8_pixel_t>(image_view);
using pixel = decltype(rgb_view)::value_type;
static_assert(sizeof(pixel) == 3, "RGB");
...

for this case, copying pixels using boost::gil::copy_pixels(...) would make sense, since there is no way of converting interleaved rgba8 to rgb8 in constant time.

Given the nature of the application, I'm pretty sure that the image is already in the memory somewhere. So I could technically just use the pointer to the first element to create my OpenCV image at the expense of using an extra channel.

score 2 · Answer 1 · answered Nov 06 '20 at 15:22

At the time, it didn't really occur to me that the image_view could point to the memory location of the underlying raw data, despite the lack of ownership. Looking at what all those autos translated into didn't help with the confusion. Writing the question gave me an idea to solve it.

Assuming the data is already in the memory, it's pretty easy to access the raw data, which can be used by the downstream library. The following code snippets could help others deal with the issue:

Converting to another image format (here 24bpp), then to cv::Mat: requires copying all pixels

auto image_view = // data received from an API
auto rgb_view = boost::gil::color_converted_view<boost::gil::rgb8_pixel_t>(image_view);
using pixel = decltype(rgb_view)::value_type;
static_assert(sizeof(pixel) == 3, "RGB");
pixel raw_data[rgb_view.width() * rgb_view.height()];
boost::gil::copy_pixels(image_view, boost::gil::interleaved_view(rgb_view.width(),
                                                                 rgb_view.height(),
                                                                 raw_data,
                                                                 rgb_view.width() *
                                                                 sizeof(pixel)));
auto mat = cv::Mat(image_view.height(), image_view.width(), CV_8UC3, raw_data);

Directly converting to cv::Mat: no copying required

auto image_view = // data received from an API
static_assert(sizeof(decltype(image_view)::value_type) == 4, "RGBA");
auto mat = cv::Mat(image_view.height(), image_view.width(), CV_8UC4, &image_view(0, 0));

Convert GIL image_view directly to OpenCV Mat

1 Answers1