0

I am investigating a memory pile-up issue in my code.

Code description:
(1) Read images from webcam in real-time.
(2) Based upon some conditions, add these images to a std::vector
(3) std::vector Passed onto another thread for writing a video file. (4) Resolutions tried at 720p, 540p, 360p.

Issue:
- Memory keeps piling up, inspite of me clearing the std::vector after every .avi file write.
- Already verified that reference count of these images goes down to zero once I clear the vector.
- The above happens only when running at 720p. Not seen at 540p, 360p.

After investigating a lot for memory leaks, I haven't found any.

I have noticed that this problem happens only when CPU is at 100%. Is it possible that OpenCV memory cleanup is unable to run in cases when CPU usage is very high? Has anyboy seen similar problems?

Any insights would be useful.
Also, would be useful to know how OpenCV triggers the routine which does memory cleanup.

Vishal
  • 3,178
  • 2
  • 34
  • 47
  • interestingly I had some very similar behaviour. Couldnt always reconstruct the memory consumption though... which funtcionality did you use? I used Haar Cascade Classifiers. – Micka Apr 27 '15 at 08:14
  • Mine is Background Subraction + tracking in the main thread. And in the consumer thread, it's merely writing the frames using cv::VideoWriter. – Vishal Apr 27 '15 at 21:59
  • @Micka: How did you solve this issue.. Any workarounds you had used? – Vishal Apr 27 '15 at 22:16
  • no solution yet, but didnt get 100% memory consumption yet... – Micka Apr 28 '15 at 05:08
  • For me 100% memory consumption happens when I increase the resolution. That might be the case for you too – Vishal Apr 29 '15 at 06:47
  • are you using OpenCV with TBB support? – Micka Apr 29 '15 at 08:43
  • 1
    Yes. It's with TBB support – Vishal Apr 29 '15 at 18:17
  • for me too... but didn't hit 100% cpu load without it – Micka Apr 29 '15 at 20:20
  • your webcam is accessed by cv::VideoCapture? according to the answer of http://stackoverflow.com/questions/30032063/opencv-videocapture-lag-due-to-the-capture-buffer there is a buffer within VideoCapture. Probably, the videocapture buffer is growing up all the time? Don't know details about that VideoCapture image buffer, but you might want to give it a try to reduce that buffer size?!? – Micka May 04 '15 at 14:45

1 Answers1

2

C++ generally has deterministic memory management. That is to say, there isn't a fundamental need for a garbage collection thread. Every action which makes memory eligible for reuse is already associated with a thread - usually destructors running when a object goes out of scope.

There might perhaps be a thread running which simplifies the free memory list, but that's more theoretical than a real concern. OpenCV doesn't have such a thread, AFAICT, and common compilers don't use extra threads either.

So what I suspect is that you have the cause and effect swapped. The CPU hits 100% because of all the delete calls running.

It's also possible that the problem is caused by a design fault. Say that you have a producer thread which creates images and adds them to a queue, and a consumer thread which takes the images out of the queue. Trivially, if the consumer thread runs at lower priority, it will be starved first when the CPU hits 100%. That also means the queue will fill up. But at lower resolutions, there's plenty of CPU to go around so the consumer thread can keep the queue empty.

MSalters
  • 173,980
  • 10
  • 155
  • 350
  • Thanks for your inputs. I checked, it's not a problem with consumer thread getting piled up. Any additional inputs could be quite helpful. – Vishal Apr 27 '15 at 21:57