ffmpeg decoder seems to drop frames

Question

I am using avcodec (c++) api to decode video file frames and process them. I noticed that the decoded frames count is less than the expected\actual number of frames. The difference is 16 frames. While x frames were decoded with ffmpeg, VirtualDub says the same file has x+16 frames. Using ffprobe, I saw that the file contains x frames:

$ ffprobe -v error -count_frames -select_streams v:0   -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 inputFile.avi
10875

Furthermore, when I tried to extract the x+1 frame using ffmpeg utility, I got a failure:

$  ffmpeg -i inputFile.avi -vf "select=eq(n\,10875)" -vframes 1 outPic.jpg
ffmpeg version N-80801-gc0cb53c Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 27.100 / 55. 27.100
  libavcodec     57. 48.101 / 57. 48.101
  libavformat    57. 40.101 / 57. 40.101
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 46.102 /  6. 46.102
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, avi, from 'inputFile.avi':
  Metadata:
    comment         : Recorder-v2
    encoder         : Lavf57.40.101
  Duration: 00:18:09.10, start: 0.000000, bitrate: 3421 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 1280x960 [SAR 1:1 DAR 4:3], 3420 kb/s, 10 fps, 10 tbr, 10 tbn, 10 tbc
[swscaler @ 00000000021100a0] deprecated pixel format used, make sure you did set range correctly
[image2 @ 0000000001f10540] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, image2, to 'outPic.jpg':
  Metadata:
    comment         : Recorder-v2
    encoder         : Lavf57.40.101
    Stream #0:0: Video: mjpeg, yuvj420p(pc), 1280x960 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 10 fps, 10 tbn, 10 tbc
    Metadata:
      encoder         : Lavc57.48.101 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed=   0x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

The interesting and weird part is that saving all frames of this movie in a sequence produced x+16 files (same frame number VirutalDub claims):

$ ffmpeg -i inputFile.avi "frames/out-%03d.jpg"
ffmpeg version N-80801-gc0cb53c Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 27.100 / 55. 27.100
  libavcodec     57. 48.101 / 57. 48.101
  libavformat    57. 40.101 / 57. 40.101
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 46.102 /  6. 46.102
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, avi, from 'inputFile.avi':
  Metadata:
    comment         : Recorder-v2
    encoder         : Lavf57.40.101
  Duration: 00:18:09.10, start: 0.000000, bitrate: 3421 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) (xvid / 0x64697678), yuv420p, 1280x960 [SAR 1:1 DAR 4:3], 3420 kb/s, 10 fps, 10 tbr, 10 tbn, 10 tbc
[swscaler @ 00000000020600a0] deprecated pixel format used, make sure you did set range correctly
[image2 @ 0000000001d20540] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, image2, to 'frames/out-%03d.jpg':
  Metadata:
    comment         : Recorder-v2
    encoder         : Lavf57.40.101
    Stream #0:0: Video: mjpeg, yuvj420p(pc), 1280x960 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 10 fps, 10 tbn, 10 tbc
    Metadata:
      encoder         : Lavc57.48.101 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=10891 fps=120 q=24.8 Lsize=N/A time=00:18:09.10 bitrate=N/A dup=16 drop=0 speed=  12x
video:384325kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

counted number of files/frames:

$ ls -l frames/* | wc -l
    10891

I am pretty much lost here. Any clue why avcodec decodes x frames and not x+16? Is ffmpeg misses frames? Is VirtualDub maybe duplicate (16) frames?

Thanks

Does this video have pure black frames? – user7860670 Aug 08 '17 at 08:31 — user7860670, Aug 08 '17 at 08:31

score 0 · Answer 1 · answered Aug 08 '17 at 21:00

0

Possible reason is "drop frames", you can check this with VirtualDub hotkeys "Go -> Next drop frame". FFmpeg does not count them as independent frames but you can notice jumps in timecode.

answered Aug 08 '17 at 21:00

shekh

116
8

Yep, this is it! Looks like Vdub "completing" dropped frame, by filling in a copy of the last frame. Also, ffmpeg can imitate this behavior when dumping all frames: ffmpeg -i inFile.avi "%03d.jpg" – Aviv Sharon Aug 09 '17 at 07:33
Does anyone know a way to tell ffmpeg using avcodec (c++) api to fill duplicated frames where there are 'holes', like virtualDub? In ffmpeg utility, for instance, it can be made with '-vsync cfr' switch But I need to do it in my decode c++ code Thanks – Aviv Sharon Aug 15 '17 at 12:34
afaik there is no simple switch. You can watch for gaps in pts and insert duplicates yourself. btw, this is implemented in my ffmpeg-based driver for virtualdub https://sourceforge.net/p/vdfiltermod/cch_input/ci/master/tree/ – shekh Aug 16 '17 at 13:57

score 0 · Answer 2 · edited Nov 21 '18 at 12:21

0

You may forget to flush your encoder after last frame is sent.

Or you may forget to rescale timestamps before writing frame:

av_packet_rescale_ts(&pkt, codec_context->time_base, stream->time_base);

edited Nov 21 '18 at 12:21

Suraj Rao

29,388
11
94
103

answered Nov 21 '18 at 12:06

Kuznets

46
4

ffmpeg decoder seems to drop frames

2 Answers2