2

I've been having a tough time getting my webcam working quickly with opencv. Frames take a very long time to read, (a recorded average of 124ms across 500 frames) I've tried on three different computers (running Windows 10) with a logitech C922 webcam. The most recent machine I tested on has a Ryzen 9 3950X, with 32gbs of ram; no lack of power.

Here is the code:

cv::VideoCapture cap = cv::VideoCapture(m_cameraNum);

// Check if camera opened successfully
if (!cap.isOpened())
{
    m_logger->critical("Error opening video stream or file\n\r");
    return -1;
}

bool result = true;
result &= cap.set(cv::CAP_PROP_FRAME_WIDTH, 1280);
result &= cap.set(cv::CAP_PROP_FRAME_HEIGHT, 720);

bool ready = false;
std::vector<string> timeLog;
timeLog.reserve(50000);
int i = 0;

while (i < 500)
{
    auto start = std::chrono::system_clock::now();
    
    cv::Mat img;
    ready = cap.read(img);

    // If the frame is empty, break immediately
    if (!ready)
    {
        timeLog.push_back("continue");
        continue;
    }

    i++;
    auto end = std::chrono::system_clock::now();
    timeLog.push_back(std::to_string(std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count()));
}

for (auto& entry : timeLog)
    m_logger->info(entry);

cap.release();
return 0;

Notice that I write the elapsed time to a log file at the end of execution. The average time is 124ms for debug and release, and not one instance of "continue" after half a dozen runs.

It doesn't matter if I use USB 2 or USB 3 ports (the camera is USB2) or if I run a debug build or a release build, the log file will show anywhere from 110ms to 130ms of time for each frame. The camera works fine in other app, OBS can get a smooth 1080@30fps or 720@60fps.

Stepping through the debugger and doing a lot of Googling, I've learned the following about my system:

  • The backend chosen by default is DSHOW. GStreamer and FFMPEG are also available.
  • DSHOW uses FFMPEG somehow (it needs the FFMPEG dll) but I cannot use FFMPEG directly through opencv. Attempting to use cv::VideoCapture(m_cameraNum, cv::CAP_FFMPEG) always fails. It seems like Opencv's interface to FFMPEG is only capable of opening video files.
  • Microsoft really screwed up camera devices in Windows a few years back, not sure if this is related to my problem.

Here's a short list of the fixes I have tried, most taken from older SO posts:

  • result &= cap.set(cv::CAP_PROP_FRAME_COUNT, 30); // Returns false, does nothing
  • result &= cap.set(cv::CAP_PROP_CONVERT_RGB, 0); // Returns true, does nothing
  • result &= cap.set(cv::CAP_PROP_MODE, cv::VideoWriter::fourcc('M', 'J', 'P', 'G')); // Returns false, does nothing
  • Set registry key from http://alax.info/blog/1693 that should disable the new Windows camera server.
  • Updated from 4.5.0 to 4.5.2, no change.
  • Asked device manager to find a newer driver, no newer driver found.

I'm out of ideas. Any help?

Mustard Tiger
  • 49
  • 1
  • 3
  • Whats happening if you do not set the height and width. Can you delete those lines and retry ? Also can you try to output of `int FPS = cap.get(CV_CAP_PROP_FPS);` – Yunus Temurlenk Jun 09 '21 at 04:33
  • I think [this](https://stackoverflow.com/a/54444910/11048887) is an answer to this question. – Yunus Temurlenk Jun 09 '21 at 04:40
  • @YunusTemurlenk, the value of FPS is 0. If I don't set the width and height then the video stream defaults to 640 x 480, and the time to retrieve each frame falls to around ~30ms, which is still far too high. – Mustard Tiger Jun 09 '21 at 05:06
  • You can check the link I referred. Even your camera is capable to get 60 fps, the backend sometimes doesn't support as you see 0 fps which is not expected – Yunus Temurlenk Jun 09 '21 at 05:41
  • maybe the camera is in some low light mode. they sometimes lower their frame rate to reduce noise. you should switch a light on in the room. – Christoph Rackwitz Jun 09 '21 at 11:49
  • Maybe [this](https://stackoverflow.com/a/28220490/6347620) – Mike Jun 09 '21 at 19:42
  • Have you ever achived(i mean with older openCV versions) faster reading? – Deumaudit Jun 10 '21 at 13:25
  • @Deumaudit, yes. When I first worked on this project a year or two ago, I was using an older version of opencv in python. On an ancient desktop with an i3 and 4GB of ram, I was reading 720p frames in just a few milliseconds. I never measured it directly, but the time was negligible. I had 25+ ms worth of processing to do on each frame, so that means I was reading frames in 8ms or less. I think it may be worth trying this in python. It would only take a few minutes to whip up a test.py file..... – Mustard Tiger Jun 10 '21 at 14:05
  • Well I've (sort of) solved my problem. I think the issue is caused by OpenCV using YUV2 instead of MJPEG. If I set the camera to MJPEG as well as set the exposure, frame rate, etc. in OBS, then opencv retrieves the frame in about ~28ms, leaving me 4-5 ms to process the frame (which is plenty for what I am doing). If I set any video capture settings in OpenCV, then the format changes back to YUV2, (I think, based on the time it takes). The work around is to remove all capture settings from my program. Then, before run time, use OBS to configure the camera. Not ideal but at least it works. – Mustard Tiger Jun 24 '21 at 01:48

0 Answers0