why does changing pointer to data with cv::Mat (opencv) have problems with cv::filter2D?

Question

I want to migrate to using opencv for custom imaging detector data. Datasets can be quite huge (sometime 100's of thousands of frames), and I load them with the boost memory mapping. When testing out behavior, I create a cv::mat of appropriate size (for a single frame) and change the pointer to data of the frame I am looking at. This works fine for displaying the data (cv::imshow) or applying a color map, but fails when I use something like cv::filter2D. If I clone the data or use some copy, it works, but I don't want to start copying/cloning because I think this will slow performance (maybe I'm wrong).

So - what am I doing wrong? Why doesn't cv::filter2D work here, and is there a better way?

While running (using Windows 10 right now) I get the following in the terminal:

OpenCV(4.0.1) Error: Assertion failed (data == datastart + ofs.y*step[0] + ofs.x*esz) in cv::Mat::locateROI, file c:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\matrix.cpp, line 767
OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.0.1) Error: Assertion failed (data == datastart + ofs.y*step[0] + ofs.x*esz) in cv::Mat::locateROI, file c:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\matrix.cpp, line 767

This only happens when filter2D is used with a matrix constructed from the memory mapped array.

I ran the program with the clone version un-commented and replaced the std::cout line in the loop with:

  std::cout<<"Is it continuous: " << img.isContinuous() << std::endl;

And the cv::Mat are indeed continuous.

#include <QCoreApplication>
#include<opencv2/opencv.hpp>
#include<opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>


#include<iostream>
#include <boost/iostreams/device/mapped_file.hpp>


int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    boost::iostreams::mapped_file_source mem_ifile;

    mem_ifile.open(std::string("someimage.raw"));
    std::cout<<"file size: "<<mem_ifile.size()/(396*266*sizeof (float))<<std::endl;
    cv::Mat adjMap;
    cv::Mat img(266,396,CV_32FC1);
    cv::Mat falseColorsMap;
    cv::Mat b_hist;
    cv::Mat kernel;
    cv::Mat filsMap;
    kernel = cv::Mat::ones( 2, 2,CV_32FC1)/double(4.0);
    cv::namedWindow("image", cv::WINDOW_NORMAL);
    cv::namedWindow("false color", cv::WINDOW_NORMAL);
    cv::namedWindow("filtered", cv::WINDOW_NORMAL);

    for(unsigned long long i = 0;i< mem_ifile.size()/(396*266*sizeof (float)); i++)
    {
        img.data = (reinterpret_cast<uchar *>(const_cast<char *>(mem_ifile.data()))+(i*396*266*sizeof (float)));
        cv::convertScaleAbs(img, adjMap, 255 / 500.0);
        applyColorMap(adjMap, falseColorsMap, cv::COLORMAP_JET);

// PROBLEM HERE - FIRST TWO WORK, LAST ONE DOESN'T
        //cv::filter2D(adjMap,filsMap,-1,kernel);           // works
        //cv::filter2D(img.clone(),filsMap,-1,kernel);      // works
        cv::filter2D(img,filsMap,-1,kernel);                // doesn't work


        std::cout<<"mean of adjmap: " << cv::mean(adjMap) << std::endl;
        cv::imshow("image", adjMap);
        cv::imshow("false color",falseColorsMap);
        cv::imshow("filtered", filsMap);
        cv::waitKey(20);
    }
    mem_ifile.close();
    return a.exec();
}

Working code based on Beaker's suggestion:

#include <QCoreApplication>
#include<opencv2/opencv.hpp>
#include<opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>

#include<iostream>
#include <boost/iostreams/device/mapped_file.hpp>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    boost::iostreams::mapped_file_source mem_ifile;

    mem_ifile.open(std::string("/home/hugh/Documents/APS2017April/Ti7_nr2_e_x36000.raw"));
    std::cout<<"file size: "<<mem_ifile.size()/(396*266*sizeof (float))<<std::endl;
    cv::Mat adjMap;
    cv::Mat *img;
    cv::Mat falseColorsMap;
    cv::Mat b_hist;
    cv::Mat kernel;
    cv::Mat filsMap;
    kernel = cv::Mat::ones( 2, 2,CV_32FC1)/double(4.0);
    cv::namedWindow("image", cv::WINDOW_NORMAL);
    cv::namedWindow("false color", cv::WINDOW_NORMAL);
    cv::namedWindow("filtered", cv::WINDOW_NORMAL);

    for(unsigned long long i = 0;i< mem_ifile.size()/(396*266*sizeof (float)); i++)
    {
        //img.data = (reinterpret_cast<uchar *>(const_cast<char *>(mem_ifile.data()))+(i*396*266*sizeof (float)));
        img = new cv::Mat(266,396,CV_32FC1,(reinterpret_cast<uchar *>(const_cast<char *>(mem_ifile.data()))+(i*396*266*sizeof (float))));

        cv::convertScaleAbs(*img, adjMap, 255 / 500.0);
        applyColorMap(adjMap, falseColorsMap, cv::COLORMAP_JET);

        //cv::filter2D(adjMap,filsMap,-1,kernel);           // works
        //cv::filter2D(img->clone(),filsMap,-1,kernel);      // works
        cv::filter2D(*img,filsMap,-1,kernel);                // works

        std::cout<<"Is it continuous: " << img->isContinuous() << std::endl;
        cv::imshow("image", adjMap);
        cv::imshow("false color",falseColorsMap);
        cv::imshow("filtered", filsMap);
        cv::waitKey(20);
        delete img;
    }
    mem_ifile.close();
    return a.exec();
}

Answer 1

So it looks like you're modifying the data field without updating any of the others. (See Public Attributes.) Specifically, the dataend, datalimit and datastart attributes are identified as

helper fields used in locateROI and adjustROI

locateROI being the method given in your error message.

The proper way to do this is to use the cv::Mat data* contructor. As noted in the documentation,

Matrix constructors that take data and step parameters do not allocate matrix data. Instead, they just initialize the matrix header that points to the specified data, which means that no data is copied. This operation is very efficient and can be used to process external data using OpenCV functions.

This assures that all of the header data is properly created without unnecessary copying.

Also note that when using these constructors, you should clean up your own data:

The external data is not automatically deallocated, so you should take care of it.