Display multiple image in same window with OpenCV

Question

I have three firewire cameras which can automatically synchronize via hardware and I'm trying to capture frames, save them into my hard disk and display them in a window.

Everything works fine, but at the moment, I'm only able to display frames in three different windows at the same time.

I'd like to display the frames in a single window but I do not know how to do this. At this link there is cvShowManyImages() function but you have to pass three frames at the same time but I have a for() loop which can consider one frame at a time.

This is the code which I'm using:

  for ( int j = 0; j < k_numImages; j++ )
    {
        // Display the timestamps for all cameras to show that the image
        // capture is synchronized for each image
        for ( unsigned int i = 0; i < numCameras; i++ )
        {
            Image image;
            error = ppCameras[i]->RetrieveBuffer( &image );
            if (error != PGRERROR_OK)
            {
                PrintError( error );
                return -1;
            }
            IplImage* destImage = ConvertImageToOpenCV(&image);
            char titolo[50];
            sprintf(titolo, "titolo%d", i);
            cvShowImage(titolo, destImage);
            waitKey(1);
        }
   }

It works well but creates a different windows for each camera while I want to display all cameras frames in the same window.

Can you help me, please?

EDIT: this is the ConvertImageToOpenCV() function.

IplImage* ConvertImageToOpenCV(Image* pImage)
{
    IplImage* cvImage = NULL;
    bool bColor = true;
    CvSize mySize;
    mySize.height = pImage->GetRows();
    mySize.width = pImage->GetCols();

    switch ( pImage->GetPixelFormat() )
    {
        case PIXEL_FORMAT_MONO8:     cvImage = cvCreateImageHeader(mySize, 8, 1 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 1;
                                     bColor = false;
                                     break;
        case PIXEL_FORMAT_411YUV8:   cvImage = cvCreateImageHeader(mySize, 8, 3 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_422YUV8:   cvImage = cvCreateImageHeader(mySize, 8, 3 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_444YUV8:   cvImage = cvCreateImageHeader(mySize, 8, 3 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_RGB8:      cvImage = cvCreateImageHeader(mySize, 8, 3 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_MONO16:    cvImage = cvCreateImageHeader(mySize, 16, 1 );
                                     cvImage->depth = IPL_DEPTH_16U;
                                     cvImage->nChannels = 1;
                                     bColor = false;
                                     break;
        case PIXEL_FORMAT_RGB16:     cvImage = cvCreateImageHeader(mySize, 16, 3 );
                                     cvImage->depth = IPL_DEPTH_16U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_S_MONO16:  cvImage = cvCreateImageHeader(mySize, 16, 1 );
                                     cvImage->depth = IPL_DEPTH_16U;
                                     cvImage->nChannels = 1;
                                     bColor = false;
                                     break;
        case PIXEL_FORMAT_S_RGB16:   cvImage = cvCreateImageHeader(mySize, 16, 3 );
                                     cvImage->depth = IPL_DEPTH_16U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_RAW8:      cvImage = cvCreateImageHeader(mySize, 8, 3 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_RAW16:     cvImage = cvCreateImageHeader(mySize, 8, 3 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_MONO12:    printf("Not supported by OpenCV");
                                     bColor = false;
                                     break;
        case PIXEL_FORMAT_RAW12:     printf("Not supported by OpenCV");
                                     break;
        case PIXEL_FORMAT_BGR:       cvImage = cvCreateImageHeader(mySize, 8, 3 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 3;
                                     break;
        case PIXEL_FORMAT_BGRU:      cvImage = cvCreateImageHeader(mySize, 8, 4 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 4;
                                     break;
        case PIXEL_FORMAT_RGBU:      cvImage = cvCreateImageHeader(mySize, 8, 4 );
                                     cvImage->depth = IPL_DEPTH_8U;
                                     cvImage->nChannels = 4;
                                     break;
        default: printf("Some error occured...\n");
                 return NULL;
    }

    if(bColor) {
        if(!bInitialized)
        {
            colorImage.SetData(new unsigned char[pImage->GetCols() * pImage->GetRows()*3], pImage->GetCols() * pImage->GetRows()*3);
            bInitialized = true;
        }

        pImage->Convert(PIXEL_FORMAT_BGR, &colorImage); //needs to be as BGR to be saved

        cvImage->width = colorImage.GetCols();
        cvImage->height = colorImage.GetRows();
        cvImage->widthStep = colorImage.GetStride();

        cvImage->origin = 0; //interleaved color channels

        cvImage->imageDataOrigin = (char*)colorImage.GetData(); //DataOrigin and Data same pointer, no ROI
        cvImage->imageData         = (char*)(colorImage.GetData());
        cvImage->widthStep      = colorImage.GetStride();
        cvImage->nSize = sizeof (IplImage);
        cvImage->imageSize = cvImage->height * cvImage->widthStep;
    }
    else
    {
        cvImage->imageDataOrigin = (char*)(pImage->GetData());
        cvImage->imageData         = (char*)(pImage->GetData());
        cvImage->widthStep         = pImage->GetStride();
        cvImage->nSize             = sizeof (IplImage);
        cvImage->imageSize         = cvImage->height * cvImage->widthStep;

        //at this point cvImage contains a valid IplImage
     }
    return cvImage;
}

enter image description here

karlphillip · Accepted Answer · 2012-06-01T16:44:58.027

2

I could not test the code below for obvious reasons, but it illustrates one approach:

for ( int j = 0; j < k_numImages; j++ )
{
    // Display the timestamps for all cameras to show that the image
    // capture is synchronized for each image

    IplImage* destImage[3]; // A-ha moment

    for ( unsigned int i = 0; i < numCameras; i++ )
    {
        Image image;
        error = ppCameras[i]->RetrieveBuffer( &image );
        if (error != PGRERROR_OK)
        {
            PrintError( error );
            return -1;
        }

        /* Since ConvertImageToOpenCV() doesn't copy the image data,
         * we need to do that ourselves, because when this loop is done Image 
         * is destroyed and the data is lost.
         */
        IplImage* tmp = ConvertImageToOpenCV(&image);
        destImage[i] = cvCreateImage(cvGetSize(tmp), tmp->depth, tmp->nChannels);
        cvCopy(tmp, destImage[i], NULL); 

        char titolo[50];
        sprintf(titolo, "titolo%d", i);
    }

    cvShowManyImages("all", 3, destImage[0], destImage[1], destImage[2]);
    waitKey(0);

    // when you finish using them, release the allocated resources to prevent memory leaks
    cvReleaseImage(&destImage[0]);
    cvReleaseImage(&destImage[1]);
    cvReleaseImage(&destImage[2]);                 
}

The idea is to create an array of IplImage* to store the images retrieved by the cameras, so after the loop you have access to all those 3 images and be able to display them on a single window.

EDIT:

Just to summarize out private chat, the problem is that cvShowManyImages() takes colored (3-channel) images, and his camera was returning grayscale (single-channel) images. The solution was simply to change the implementation of cvShowManyImages(), from:

DispImage = cvCreateImage( cvSize(100 + size*w, 60 + size*h), 8, 3 );

To:

DispImage = cvCreateImage( cvSize(100 + size*w, 60 + size*h), 8, 1 );

edited Jun 01 '12 at 16:44

answered Jun 01 '12 at 12:50

karlphillip

92,053
36
243
426

Thanks a lot for the tip! I tried it, it compiles but when i run it I get this error: "OpenCV Error: Assertion Failed – Marcus Barnet Jun 01 '12 at 13:21
May be it's because cvShowManyImages() use a different size for the images? At the moment, images captured from cameras are 1200x1800 px – Marcus Barnet Jun 01 '12 at 13:37
One question: does `ConvertImageToOpenCV()` copy the entire image or simply assigns the data pointer to `destImage[i]->imageData` ? My current code assumes a copy of the data is performed by `ConvertImageToOpenCV()`. – karlphillip Jun 01 '12 at 13:44
I can adjust the code in my answer to perform the copy of `IplImage` returnted by `ConvertImageToOpenCV()`, if we need it :) – karlphillip Jun 01 '12 at 13:48
I added the ConvertImageToOpenCV() function to my first post. This function return a IplImage pointer. Do you think it is possible to fix the problem in order to use the cvShowManyImages() function? – Marcus Barnet Jun 01 '12 at 14:00
You still didn't answer my question. Does `ConvertImageToOpenCV()` copy the data or does it simply copies the pointer to it? – karlphillip Jun 01 '12 at 14:01
ConvertImageToOpenCV() copy the data to a new IplImage and return a pointer to the new image. The ConvertImageToOpenCV() function code is in my first post in order to explain better how it works :) I'm sorry, I'm not a skilled programmer unfortunately :( I've a lot to learn. :( – Marcus Barnet Jun 01 '12 at 14:07
1

I just read the code and your function doesn't copy the data, it simply copies the pointer, i.e., the function copies the memory address of where the data is in memory, but not the data itself. I updated my code, you better take another look at it. – karlphillip Jun 01 '12 at 14:18
Unfortunately, it gives always the same error :( Do you think it's a memory or a image size problem? – Marcus Barnet Jun 01 '12 at 14:23
I added the screenshot error to my first post, may be it can help to better understand the problem :( – Marcus Barnet Jun 01 '12 at 14:25
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/12036/discussion-between-karlphillip-and-marcus-barnet) – karlphillip Jun 01 '12 at 14:27
Thanks Karl for all your help! I did as you suggested me, the images are 8 depth and 1 channel only so I changed the cvShowManyImages() to use just one channel and now I can see the three frames in a single window!! It works fine! The problem is that the window appears and disappears for every frame so it blinks all the time! I mean: the window appears and disappears for every capture so it's very difficult to watch it :) Is it possible to make the window stays alive for all the time? – Marcus Barnet Jun 01 '12 at 15:49
DispImage = cvCreateImage( cvSize(100 + size*w, 60 + size*h), 8, 1 ); Now it uses 1 channel. I also changed cvWaitKey(); in cvWaitKey(1); in order to let frame change during the acquisition. The problem now is the window which appears and disappears for all the time :( – Marcus Barnet Jun 01 '12 at 15:52
Do you think it's normal that my images uses just a single channel? – Marcus Barnet Jun 01 '12 at 15:54
1

It's normal if they are grayscale. The blinking effect is solved by placing the cvWaitKey() call right after cvShowManyImages(): this was a bug caused by my first updated of the answer. – karlphillip Jun 01 '12 at 16:18

Display multiple image in same window with OpenCV

1 Answers1

Linked