OpenCV Webcam feed not displaying in PictureBox visual studio 2015

Question

Hi so I used opencv to get webcam feed from my default camera and I wish to display it in a picturebox on my windows form. My webcam comes on but for some reason, the feed is never displayed on my picturebox. Please could someone help point out/solve the issue as I am stuck here right now. Thanks in advance.

In myform.h, I have this code to send the picturebox to the myform.cpp file:

System::Windows::Forms::PictureBox^ mypicbox1(void)  
{
    opencv_gui::MyForm aform;
    return aform.pictureBox1;
}

and the code to draw get the videofeed and put into my picture box in myform.cpp is:

void opencv_gui::DrawCvImage(const cv::Mat& cvImage)
{
    System::Windows::Forms::PictureBox^ pictureBox = mypicbox1();
    // only color images are supported
    assert(cvImage.type() == CV_8UC3);

    if ((pictureBox->Image == nullptr) || (pictureBox->Width != cvImage.cols) || (pictureBox->Height != cvImage.rows))
    {
        pictureBox->Width = cvImage.cols;
        pictureBox->Height = cvImage.rows;
        pictureBox->Image = gcnew System::Drawing::Bitmap(cvImage.cols, cvImage.rows);
    }

    // Create System::Drawing::Bitmap from cv::Mat
    System::Drawing::Bitmap^ bmpImage = gcnew System::Drawing::Bitmap(
        cvImage.cols, cvImage.rows, cvImage.step,
        System::Drawing::Imaging::PixelFormat::Format24bppRgb,
        System::IntPtr(cvImage.data)
    );

    // Draw Bitmap over a PictureBox
    System::Drawing::Graphics^ g = System::Drawing::Graphics::FromImage(pictureBox->Image);

    g->DrawImage(bmpImage, 0, 0, cvImage.cols, cvImage.rows);
    pictureBox->Refresh();

    delete g;
}


//camera feed
int opencv_gui::video_cap(void)
{
    VideoCapture cap;

    if (!cap.open(0)) // open the default camera (camera 0), use something different from 0 otherwise;
        return 0;
    for (;;)
    {
        Mat frame;
        cap >> frame;
        if (frame.empty()) break; // end of video stream
        DrawCvImage(frame);
        if (waitKey(10) == 27) break; // stop capturing by pressing ESC 
    }
    // the camera will be closed automatically upon exit
    // cap.close();
    return 0;
}

This is my debug log: I have used arrows like ">>>>>>>" to show important parameters and the words "RED >>>>>>>" to show error parameters.

cvImage { flags=1124024336 dims=2 rows=480 ...} cv::Mat&
allocator 0x0000000000000000 cv::MatAllocator*

cols 640 int
data 0x0000026AC2146F80 unsigned char* *(*cvImage).data 0 '' unsigned char
dataend 0x0000026AC2227F80 unsigned char*
datalimit 0x0000026AC2227F80 unsigned char*
datastart 0x0000026AC2146F80 unsigned char* dims 2 int flags 1124024336 int rows 480 int
size { p=0x000000DBDFF0DE80 } cv::MatSize
p 0x000000DBDFF0DE80 int* *(*cvImage).size.p 0 int
step { p=0x000000DBDFF0DEC8 buf={Length=2} } cv::MatStep
buf {Length=2} unsigned __int64[]
p 0x000000DBDFF0DEC8 unsigned __int64*
u 0x0000026AA4814780 cv::UMatData*
RED >>>>>>> bmpImage 0x0000026aa62a1fd0 { defaultTransparentColor= } System::Drawing::Bitmap^
RED >>>>>>> System::Drawing::Image^ 0x0000026aa62a1fd0 { nativeImage=2657511292512 rawData= userData= } System::Drawing::Image^ defaultTransparentColor System::Drawing::Color g System::Drawing::Graphics^
RED >>>>>>> pictureBox 0x0000026aa62a1388 { borderStyle=System::Windows::Forms::BorderStyle::None image=0x0000026aa62a1fa0 sizeMode=System::Windows::Forms::PictureBoxSizeMode::Normal ...} System::Windows::Forms::PictureBox^
RED >>>>>>> System::Windows::Forms::Control^ 0x0000026aa62a1388 { ControlKeyboardRouting= PaletteTracing= FocusTracing= ...} System::Windows::Forms::Control^ AllowDrop false bool BorderStyle System::Windows::Forms::BorderStyle::None System::Windows::Forms::BorderStyle CausesValidation true bool
CreateParams 0x0000026aa62a15d8 { className= caption= style=1442840576 ...} System::Windows::Forms::CreateParams^ DefaultImeMode System::Windows::Forms::ImeMode::Disable System::Windows::Forms::ImeMode
DefaultSize {Width=100 Height=50} System::Drawing::Size EVENT_SIZEMODECHANGED 0x0000026aa629af50 System::Object^
ErrorImage 0x0000026aa629b110 { nativeImage=2657511192608 rawData= userData= } System::Drawing::Image^
Font 0x0000026aa6280148 {} System::Drawing::Font^
ForeColor {RGB=0x0} System::Drawing::Color
Image 0x0000026aa62a1fa0 { nativeImage=2657511290992 rawData= userData= } System::Drawing::Image^
[System::Drawing::Bitmap^] 0x0000026aa62a1fa0 { defaultTransparentColor= } System::Drawing::Bitmap^
System::MarshalByRefObject^ 0x0000026aa62a1fa0 { __identity= } System::MarshalByRefObject^ Flags 2 int
FrameDimensionsList {Length=1} array ^ Height 480 int HorizontalResolution 96.000000 float
Palette 0x0000026aa62a25e8 { flags=1129621568 entries={Length=0} } System::Drawing::Imaging::ColorPalette^
PhysicalDimension { width=640.00000 height=480.00000 } System::Drawing::SizeF PixelFormat System::Drawing::Imaging::PixelFormat::Format32bppArgb System::Drawing::Imaging::PixelFormat PropertyIdList {Length=0} array ^ PropertyItems {Length=0} array ^
RawFormat 0x0000026aa62a3488 { ...} System::Drawing::Imaging::ImageFormat^
Size {Width=640 Height=480} System::Drawing::Size Tag System::Object^ VerticalResolution 96.000000 float Width 640 int nativeImage 2657511290992 __int64 rawData array ^ userData System::Object^ ImageLocation System::String^
ImageRectangle {X=0 Y=0 Width=640 Height=480} System::Drawing::Rectangle ImeMode System::Windows::Forms::ImeMode::Disable System::Windows::Forms::ImeMode
InitialImage 0x0000026aa629c568 { nativeImage=2657511196256 rawData= userData= } System::Drawing::Image^ PICTUREBOXSTATE_asyncOperationInProgress 1 int PICTUREBOXSTATE_cancellationPending 2 int PICTUREBOXSTATE_inInitialization 64 int PICTUREBOXSTATE_needToLoadImageLocation 32 int PICTUREBOXSTATE_useDefaultErrorImage 8 int PICTUREBOXSTATE_useDefaultInitialImage 4 int PICTUREBOXSTATE_waitOnLoad 16 int RightToLeft System::Windows::Forms::RightToLeft::No System::Windows::Forms::RightToLeft SizeMode System::Windows::Forms::PictureBoxSizeMode::Normal System::Windows::Forms::PictureBoxSizeMode TabIndex 1 int TabStop false bool Text "" System::String^ WaitOnLoad false bool borderStyle System::Windows::Forms::BorderStyle::None System::Windows::Forms::BorderStyle contentLength 0 int currentAsyncLoadOperation System::ComponentModel::AsyncOperation^ currentlyAnimating false bool
defaultErrorImage 0x0000026aa629b110 { nativeImage=2657511192608 rawData= userData= } System::Drawing::Image^
defaultErrorImageForThread 0x0000026aa629b110 { nativeImage=2657511192608 rawData= userData= } System::Drawing::Image^ defaultErrorImageKey 0x0000026aa629af08 System::Object^
defaultInitialImage 0x0000026aa629c568 { nativeImage=2657511196256 rawData= userData= } System::Drawing::Image^
defaultInitialImageForThread 0x0000026aa629c568 { nativeImage=2657511196256 rawData= userData= } System::Drawing::Image^ defaultInitialImageKey 0x0000026aa629aef0 System::Object^
errorImage 0x0000026aa629b110 { nativeImage=2657511192608 rawData= userData= } System::Drawing::Image^ handleValid false bool
image 0x0000026aa62a1fa0 { nativeImage=2657511290992 rawData= userData= } System::Drawing::Image^ imageInstallationType ImageInstallationType::DirectlySpecified ImageInstallationType imageLocation System::String^
initialImage 0x0000026aa629c568 { nativeImage=2657511196256 rawData= userData= } System::Drawing::Image^
[System::Drawing::Bitmap^] 0x0000026aa629c568 { defaultTransparentColor= } System::Drawing::Bitmap^
System::MarshalByRefObject^ 0x0000026aa629c568 { __identity= } System::MarshalByRefObject^ Flags 73744 int
FrameDimensionsList {Length=1} array ^ Height 16 int HorizontalResolution 96.000000 float
RED >>>>>>>Palette 0x0000026aa629ea60 { flags=951278840 entries={Length=0} } System::Drawing::Imaging::ColorPalette^
PhysicalDimension { width=14.000000 height=16.000000 } System::Drawing::SizeF PixelFormat System::Drawing::Imaging::PixelFormat::Format24bppRgb System::Drawing::Imaging::PixelFormat PropertyIdList {Length=0} array ^ PropertyItems {Length=0} array ^
RED >>>>>>> RawFormat 0x0000026aa629f900 { ...} System::Drawing::Imaging::ImageFormat^
Size {Width=14 Height=16} System::Drawing::Size Tag System::Object^ VerticalResolution 96.000000 float Width 14 int nativeImage 2657511196256 __int64 rawData array ^ userData System::Object^ RED >>>>>>> internalSyncObject 0x0000026aa629d900 System::Object^ loadCompletedDelegate System::Threading::SendOrPostCallback^ loadCompletedKey 0x0000026aa629af20 System::Object^ loadProgressChangedKey 0x0000026aa629af38 System::Object^ loadProgressDelegate System::Threading::SendOrPostCallback^ localImageStreamReader System::IO::StreamReader^
pictureBoxState { data=12 } System::Collections::Specialized::BitVector32 readBlockSize 4096 int readBuffer array ^
savedSize {Width=640 Height=480} System::Drawing::Size sizeMode System::Windows::Forms::PictureBoxSizeMode::Normal System::Windows::Forms::PictureBoxSizeMode tempDownloadStream System::IO::MemoryStream^ totalBytesRead 0 int uriImageStream System::IO::Stream^

Since I don't see any synchronization, I assume `opencv_gui::video_cap` runs on the main GUI thread. As such, seeing an infinite loop there is already a sign of potential trouble. While `waitKey` might kinda-sorta work in, I wouldn't rely on that for anything other than the OpenCV UI. This is a pretty awkward design in a GUI app anyway. You already wait between frames, so why not use a timer? — Dan Mašek, Feb 19 '17 at 21:42
Thanks Dan for the comment, I intend to clean up the code soon regarding the loop but removing the loop and capturing manually still doesn't display frames on the picture box which is my main issue. — Kosy Onyenso, Feb 20 '17 at 00:41
Can you try to step through that function in the debugger? What are the values of `cvImage.cols, cvImage.rows, cvImage.step` ? Does `cvImage.data` contain some reasonable values? BTW, OpenCV images are BGR, and you interpret the data as RGB. You probably need a `cvtColor` there to switch it around. — Dan Mašek, Feb 20 '17 at 02:41
Hi Dan, I have put a copy of my debug log into the question. The image size and data parameters are zero but the other parameters have reasonable value(640 and 480)...I have also signified which of the debug lines were highlighted in red for you to look at. Thanks for the help so far. — Kosy Onyenso, Feb 20 '17 at 08:59

score 0 · Accepted Answer · edited May 23 '17 at 12:25

0

Had to edit my code a bit but I pretty much solved my issue using the answer provided here: https://stackoverflow.com/a/12628861/5728859

edited May 23 '17 at 12:25

Community

1
1

answered Feb 20 '17 at 22:25

Kosy Onyenso

109
2
14

score 0 · Answer 2 · edited Apr 25 '18 at 00:49

try this

private: System::Void button2_MouseClick(System::Object^  sender, System::Windows::Forms::MouseEventArgs^  e)
{
    if (button2->Text == "Stop")
    {
        button2->Text = "Start";

    }
    else if (button2->Text == "Start")
    {
        button2->Text = "Stop";

    }

     VideoCapture capture(0);
     Mat frame;

     while (button2->Text == "Stop")
     {
         capture.read(frame);
         System::Drawing::Graphics^ graphics2 = pictureBox1->CreateGraphics();
         System::IntPtr ptr2(frame.ptr());
         System::Drawing::Bitmap^ b2 = gcnew System::Drawing::Bitmap(frame.cols,
             frame.rows, frame.step, System::Drawing::Imaging::PixelFormat::Format24bppRgb, ptr2);
         System::Drawing::RectangleF rect2(0, 0, pictureBox1->Width, pictureBox1->Height);
         graphics2->DrawImage(b2, rect2);

     }
 }

OpenCV Webcam feed not displaying in PictureBox visual studio 2015

2 Answers2