8

I'm receiving an H.264 stream from a DVR using its SDK. There were memory leaks and i thought it was the SDK causing all the leaks. But when i recorded the stream and played the frames one by one reading from the disk (without any 3rd party dlls involved), i noticed that the problem is not the dll but the stream itself.

Strange enough, DivX H264 Decoder is the only codec which doesn't cause a memory leak but when the stream runs for a long time, sometimes DivX decoder crashes as well. I'd prefer to use Microsoft DTV-DVD Video Decoder but it causes huge memory leaks and drops a lot of frames. Many other H.264 decoders I've tried behaves the same way.

I examined the h.264 frames using some h.264 parsers comparing with some other problem-free streams but i didn't notice anything obvious from the logs.

Since my problem is about the h.264 frames structure, i've prepared a source filter named FramesFromFileSourceFilter which you can download below.

http://www.akaydin.com/directshow/FramesFromFileSourceFilter.zip

It's a Visual Studio 2008 project and all dependencies are included in the zip file in relatively located folders (including the h.264 frames). So, all you need to do is to compile the project, register the output with regsvr32.exe and run the filter with any h.264 decoder you want from GraphEdit or GraphStudio. Example graphs are below.

FramesFromFileSourceFilter with DivX

FramesFromFileSourceFilter with Microsoft DTV-DVD Video Decoder

Also h264 frames are available as a single raw h264 file at the link below which is playable by VLC (with wrong FPS since original was 12 FPS).

http://www.akaydin.com/directshow/stream.zip

Question:

What might be causing the memory leak problems with many famous H264 Decoders except DivX decoder. What is wrong with this stream?

Update 1

Reading data thread is removed and functionality moved into FillBuffer without using any buffers and flags. Problem remains the same.

http://www.akaydin.com/directshow/FramesFromFileSourceFilterUpdate1.zip

Update 2

Update1 was using Sleep() in FillBuffer() function which was causing some problems. Now i removed the Sleep() and used SetTime() to have ~12 FPS. This also solved Microsoft DTV-DVD Video Decoder's dropping frame issues but didn't solve memory problems.

http://www.akaydin.com/directshow/FramesFromFileSourceFilterUpdate2.zip

Memory increase occurs at Working Set only. Virtual Bytes and Private Bytes seem to be stable. What might be causing continuous Working Set memory increment which only happens with Microsoft DTV-DVD Video Decoder?

Emir Akaydın
  • 5,708
  • 1
  • 29
  • 57

1 Answers1

3

You don't do any synchronization around your variables

BYTE* m_buffer;
DWORD m_bufferSize;
bool isFrameReady;

And they are used from two concurrent threads. You just leak your memory by this inaccurate allocation/deallocation AND/OR letting your code crash on access violation. Debug build of your DLL indicates this by showing you "heap corruption" alert as you run your test. The runtime behavoir might vary with decoders and environment, however this is definitely a severe bug to be fixed.

For instance, you can use CAutoLock cAutoLock(m_pLock); in your thread that fills buffer to prevent streaming thread access while you are reading data from file.

Note that you read next frame into the same buffer pointer without a check whether previously allocated memory is freed or not, you just overwrite the pointer possibly leaving a leak.

Memory Leak/Working Set Update: Now, when code issues are sorted out, the unwanted runtime behavior is an increase in Working Set size. This is not a leak. It is an indication that Windows considers the process as a sort of priority (why not? it is active and works with memory) and throws more real pages towards this process to facilitate its performance. See this answer on good explanation of how process memory metrics correspond to memory leaks in an application.

The difference between decoders you might be seeing is likely to be caused by the fact that some decoders are good with smaller amount of buffers, or reuse them more actively, e.g. prefer to take the same buffer out of a pool rather than picking one by one through all available.

Community
  • 1
  • 1
Roman R.
  • 68,205
  • 6
  • 94
  • 158
  • Actually, my real filter has a better mechanism, this is the simplified one. but since there is a sleep like 83 miliseconds between frames, concurrency shouldn't be the case here. plus, DivX Decoder doesn't cause any memory leaks. i still believe streams are causing this problem. additionally, same filter works fine with other h.264 streams from other devices without any memory leaks with the same decoders. – Emir Akaydın Jun 28 '12 at 13:09
  • Maybe your real filter does it better, but this code causes heap corruption. On another run when it managed to pass through, I did not notice any leak (with MS DTV-DVD Decoder), how do you tell if the leak took place? – Roman R. Jun 28 '12 at 13:12
  • memory keeps increasing during the process as long as the filter runs. of course in this example there are limited number of frames. but when working with a live stream, memory usage reachs gigabytes after a few hours. didn't you notice any memory increase of GraphEdit/GraphStudio when you run the filter with MS DTV-DVD Video Decoder? if so, what is your OS? i've treid this on Win7 32 bits. – Emir Akaydın Jun 28 '12 at 13:26
  • I checked with Win 7 x64 but it does not matter. The leaking only takes place around your `m_buffer`, I don't think there is any problem with decoder itself. The difference you might be seeing follows from behaviors of decoders, sometimes they might consume data immediately, sometimes a delay involved - this causes the bug on your side to come up or not. – Roman R. Jun 28 '12 at 13:39
  • I just added a critical section around the reading data section. But problem remains the same here. Btw, with MS DTV-DVD Video Decoder, a lot of frames are dropped here. So, i'm unable to get a smooth video. it starts smooth, then drops to 1-2 FPS. with DivX decoder, i'm able to get solid 12 FPS video. don't you have this problem as well? is the video playing smoothly at your side? – Emir Akaydın Jun 28 '12 at 14:00
  • See last paragraph above in my answer. Your thread that fills `m_buffer` overwrites pointer without freeing. That is, when you have drops on presentation, chances are high that your reading thread is reading next frame without previous being consumed, without freeing previously loaded data. If this is not a leak, then what is it. Your reading thread should wait your streaming thread. Why don't you read from files right in `FillBuffer` in first place. Do this and you will see decoder works neatly. – Roman R. Jun 28 '12 at 14:07
  • I updated the question (added Update 1 section to the end). Please download and replace the new files with the old ones. No buffers, no flags, no additional threads. Everything is done in the FillBuffer with no synch problems. Problem remains the same here. – Emir Akaydın Jun 28 '12 at 14:33
  • Runs fine now. First of all, no waste for CPU cycles anymore. Steady memory usage until end of the stream http://i.imgur.com/PICQA.png – Roman R. Jun 28 '12 at 14:42
  • Roman, i tried on 5 different PCs now, 3 Win7 32 bit and 2 Win7 64 bit. They all have the same problem. Can you please check the memory usage from Windows Task Manager? Memory should be growing. But with DivX decoder, there is no such thing. If you can try with DivX h264 decoder, you should also notice video runs much smoother as it should be. If you only tried with MS's decoder, you might have not noticed the difference. Even if you are unable to install/try with DivX decoder, please check the memory usage during the process from Windows Task Manager with MS's decoder at least. – Emir Akaydın Jun 28 '12 at 15:50
  • Maybe memory leak is the wrong definition. I've tried some memory leak detection utilities before and they don't get this as a leak since they think memory will be freed at the end of the process. but memory usage keeps growing when you check it from the Task Manager. – Emir Akaydın Jun 28 '12 at 15:54
  • Define "Memory Usage". My screenshot shows you "Private Bytes" and it's fine. "Virtual Size" is constant also. What you might be referring to is "Working Set Size" and it does not show the value you can use for telling a leak. – Roman R. Jun 28 '12 at 16:15
  • Check this out: [How to know if it is memory leak or not if Mem Usage in Task Manager keep increasing](http://stackoverflow.com/questions/6676415/how-to-know-if-it-is-memory-leak-or-not-if-mem-usage-in-task-manager-keep-increa) – Roman R. Jun 28 '12 at 16:19
  • Maybe even better: http://stackoverflow.com/a/2938501/868014 and this http://stackoverflow.com/questions/1984186/what-is-private-bytes-virtual-bytes-working-set – Roman R. Jun 28 '12 at 16:25
  • the problem is, i try my source filter with DivX decoder and there is no memory increment problems. i try my source filter with another h.264 streams and both DivX and MS Decoder works fine. only case memory increment occurs is this streams combined with MS Decoder. also when i try the same streams with Elecard's H.264 decoder and set "don't show frames with errors" flag and i got many dropped frames + memory increment. so, i was 100% sure that the problem was caused by the streams. after getting your first answer, i started questining my own filter again. but now, i still believe... – Emir Akaydın Jun 29 '12 at 07:20
  • So how you measure memory usage? What memory metrics (per links above) indicate a leak? – Roman R. Jun 29 '12 at 07:24
  • it's caused by the h.264 stream. because my filter works fine with all decoders with other h.264 streams. it also works fine with this stream (which comes with the source code package as binary files) using DivX decoder. if there anything i can do at source filter side, please tell me. do you see anything wrong in FillBuffer code for example? it just reads a binary file, send it to next filter and doesn't cause a memory leak. so, why does the memory keeps increasing in MS Decoder case and how can i solve it? that's the problem here. – Emir Akaydın Jun 29 '12 at 07:24
  • `Working Set` increase is not a memory leak, see update above. – Roman R. Jun 29 '12 at 10:00
  • Ok, since you've corrected me and put me at the right direction, i've accepted your answer. But "Working Set" or not, i want to find a solution of this unwanted memory increment problem. Because it causes memory inc. by 100 kb/sec. If i use 64 camera viewports at the same time it will be 6.4 MB/sec. In ~10 minutes, it will start using 4GB of memory if it doesn't free the memory at some point. Do you know if "Working Set" causes an out of memory error or free up memory when too much memory is spent? – Emir Akaydın Jun 29 '12 at 11:10
  • Working set is limited, it will reach some point and won't go beyond that (there is a quota in first place). This was a test project, and you have real one - now you know which metrics you need to look at (not just Task Manager) and you will be able to see if you have real memory leak (such as putting up your Private Bytes) or just prioritization of available resources towards your application. – Roman R. Jun 29 '12 at 11:19