Qt OpenCV - SIGSEGV when displaying transformed frames

Question

I have an app that has to pull frames from video, transform one a little, transform one a lot, and simultaneously display them in GUI. In a worker thread, there's an OpenCV loop:

while(1) {
    cv::VideoCapture kalibrowanyPlik; 
    kalibrowanyPlik.open(kalibracja.toStdString()); //open file from url
    int maxFrames = kalibrowanyPlik.get(CV_CAP_PROP_FRAME_COUNT);
    for(int i=0; i<maxFrames; i++) //I thought it crashed when finished reading the first time around
    {
        cv::Mat frame;
        cv::Mat gray;
        cv::Mat color;

        kalibrowanyPlik.read(frame);
        cv::cvtColor(frame, gray, CV_BGR2GRAY);
        cv::cvtColor(frame, color, CV_BGR2RGB);

        QImage image((uchar*)color.data, color.cols, color.rows,QImage::Format_RGB888);
        QImage processedImage((uchar*)gray.data, gray.cols, gray.rows,QImage::Format_Indexed8);

        emit progressChanged(image, processedImage);
        QThread::msleep(50);
    }
}

And this is how frames are placed in GUI

void MainWindow::onProgressChagned(QImage image, QImage processedImage) {
    QPixmap processed = QPixmap::fromImage(processedImage);
    processed = processed.scaledToHeight(379);
    ui->labelHsv->clear();
    ui->labelHsv->setPixmap(processed);

    QPixmap original = QPixmap::fromImage(image); //debug points SIGSEGV here
    original = original.scaledToHeight(379);
    ui->labelKalibracja->clear();
    ui->labelKalibracja->setPixmap(original);
}

The RGB image always crashes, grayscale image never crashes (tested). Why is the RGB image crashing?

edit: I've just discovered that if I change msleep(50) to msleep(100) it executes perfectly. But I don't want that. I need at least 25 frames per second, 10 is not acceptable... why would that cause a SIGSEGV

Answer 1

Standard issue. Problem is memory management! See my other answer. In comments there is a good link.

So in your code QImage doesn't copy and doesn't take ownership of memory of matrix. And later on when matrix is destroyed and QImage tries access this memory (QImage is copied by creating shallow copy) you have a segfault.

---

Here is a code form this link (I've tweak it a bit), for some reason this site has some administration issues (some quota exceeded), that is why I'm pasting it here.

inline QImage  cvMatToQImage( const cv::Mat &inMat )
   {
      switch ( inMat.type() )
      {
         // 8-bit, 4 channel
         case CV_8UC4:
         {
            QImage image( inMat.data, inMat.cols, inMat.rows, inMat.step, QImage::Format_RGB32 );

            QImage copy(image);
            copy.bits(); //enforce deep copy
            return copy;
         }

         // 8-bit, 3 channel
         case CV_8UC3:
         {
            QImage image( inMat.data, inMat.cols, inMat.rows, inMat.step, QImage::Format_RGB888 );

            return image.rgbSwapped();
         }

         // 8-bit, 1 channel
         case CV_8UC1:
         {
            static QVector<QRgb>  sColorTable;

            // only create our color table once
            if ( sColorTable.isEmpty() )
            {
               for ( int i = 0; i < 256; ++i )
                  sColorTable.push_back( qRgb( i, i, i ) );
            }

            QImage image( inMat.data, inMat.cols, inMat.rows, inMat.step, QImage::Format_Indexed8 );
            image.setColorTable( sColorTable );

            QImage copy(image);
            copy.bits(); //enforce deep copy
            return copy;
         }

         default:
            qWarning() << "ASM::cvMatToQImage() - cv::Mat image type not handled in switch:" << inMat.type();
            break;
      }

      return QImage();
   }

Your code should utilize this functions like that:

while(1) {
    cv::VideoCapture kalibrowanyPlik; 
    kalibrowanyPlik.open(kalibracja.toStdString()); //open file from url
    int maxFrames = kalibrowanyPlik.get(CV_CAP_PROP_FRAME_COUNT);
    for(int i=0; i<maxFrames; i++) //I thought it crashed when finished reading the first time around
    {
        cv::Mat frame;
        cv::Mat gray;

        kalibrowanyPlik.read(frame);
        cv::cvtColor(frame, gray, CV_BGR2GRAY);

        QImage image(cvMatToQImage(frame));
        QImage processedImage(cvMatToQImage(gray));

        emit progressChanged(image, processedImage);
        QThread::msleep(10); // this is bad see comments below
    }
}

Use of msleep is in 95% cases bad! Remove this loop and create slot which will be invoked by signal from QTimer.

Answer 2

Another solution will be to use a timer :

void ??::timerEvent(QTimerEvent*){

 if(kalibrowanssky.isOpened()) 
        cv::Mat frame;
        cv::Mat gray;
        cv::Mat color;

        kalibrowanyPlik.read(frame);
        cv::cvtColor(frame, gray, CV_BGR2GRAY);
        cv::cvtColor(frame, color, CV_BGR2RGB);

        ui->labelHsv->setPixmap(QPixmap::fromImage(Mat2QImage(color)));
        ui->labelKalibracja->setPixmap(QPixmap::fromImage(Mat2QImage(gray)));
}

In your main :

cv::VideoCapture kalibrowanyPlik; 
startTimer(1000/25); // 25 frames by second

And the function Mat2QImage (I found it here : how to convert an opencv cv::Mat to qimage) :

QImage ??::Mat2QImage(cv::Mat const& src) {

     cv::Mat temp; 
     cvtColor(src, temp,CV_BGR2RGB); 
     QImage dest((const uchar *) temp.data, temp.cols, temp.rows, temp.step, QImage::Format_RGB888);
     dest.bits();

     return dest;
}