How can I seek to frame No. X with ffmpeg?

Question

I'm writing a video editor, and I need to seek to exact frame, knowing the frame number.

Other posts on stackoverflow told me that ffmpeg may give me a few broken frames after seeking, which is not a problem for playback but a big problem for video editors.

And I need to seek by frame number, not by time, which will become inaccurate when converted to frame number.

I've read dranger's tuts (which is outdated now), and end up with:

av_seek_frame(fmt_ctx, video_stream_id, frame, AVSEEK_FLAG_ANY);

It always seek to frame No. 0, and always return 0 which means success. Then I tried to read Blender's source code and found it really complex(maybe I should implement an image buffer?).

So, is there any simple way to seek to a frame with just a simple call like seek(context, frame_number)(while getting a full frame, not a broken one)? Or, is there any lightweight library that simplifies this?

EDIT: Thanks to praks411，I found the solution:

void AV_seek(AV * av, size_t frame)
{
    int frame_delta = frame - av->frame_id;
    if (frame_delta < 0 || frame_delta > 5)
        av_seek_frame(av->fmt_ctx, av->video_stream_id,
                frame, AVSEEK_FLAG_BACKWARD);
    while (av->frame_id != frame)
        AV_read_frame(av);
}

void AV_read_frame(AV * av)
{
    AVPacket packet;
    int frame_done;

    while (av_read_frame(av->fmt_ctx, &packet) >= 0) {
        if (packet.stream_index == av->video_stream_id) {
            avcodec_decode_video2(av->codec_ctx, av->frame, &frame_done, &packet);
            if (frame_done) {
                ...
                av->frame_id = packet.dts;
                av_free_packet(&packet);
                return;
            }
        }
        av_free_packet(&packet);
    }
}

EDIT2: Turns out there is a library for this: FFMS2. It is "an FFmpeg based source library [...] for easy frame accurate access", and is portable (at least across Windows and Linux).

are you guaranteed a fixed stable fps? Then you can seek by time. See also the discussion on "key -frame" — rogerdpack, Feb 12 '14 at 22:35

score 12 · Accepted Answer · edited Feb 12 '14 at 21:17

12

av_seek_frame will only seek based on timestamp to the key-frame. Since it seeks to the keyframe, you may not get what you want. Hence it is recommended to seek to nearest keyframe and then read frame by frame util you reach the desired frame.

However, if you are dealing with fixed FPS value, then you can easily map timestamp to frame index.

Before seeking you will need to convert your time to AVStream.time_base units if you have specified stream. Read ffmpeg documentation of av_seek_frame in avformat.h.

For example, if you want to seek to 1.23 seconds of clip:

 double m_out_start_time = 1.23;
 int flgs = AVSEEK_FLAG_ANY;
 int seek_ts = (m_out_start_time*(m_in_vid_strm->time_base.den))/(m_in_vid_strm->time_base.num);
 if(av_seek_frame(m_informat, m_in_vid_strm_idx,seek_ts, flgs) < 0)
 {
     PRINT_MSG("Failed to seek Video ")
 }

edited Feb 12 '14 at 21:17

shybovycha

11,556
6
52
82

answered Jul 09 '13 at 13:50

praks411

1,972
16
23

2

Oh, and I finally know the documentation is in the header files, no wonder I can't find any online. – Giumo Jul 10 '13 at 02:12
Then, how can I know where the nearest key-frame is? – Giumo Jul 10 '13 at 02:12
Isn't time_base.den/time_base.num the fps? Isn't seek_ts=time*fps the frame number? – Giumo Jul 10 '13 at 02:17
You are talking about codec context time base, I've taken stream time which is different. For seeking to nearest time key frame. I think you should keep some tolerance in seeking that is try to seek let say 3 frame before your desire frame so you will not miss your frame if it is not key frame. Then from there read frame by frame. – praks411 Jul 10 '13 at 07:32
After some trying, I found that I have to set flags to AVSEEK_FLAG_BACKWARD to seek to the keyframe. And seek_ts *is* the frame number. I also found that the video file I tried before has only 4 keyframes (a WebM, 360 frames in total, 30fps, don't know why), which really interfered my experiments on seeking. – Giumo Jul 11 '13 at 03:16
Shouldn't you also add the `m_in_vid_strm->start_time` to seek_ts? – Sam Sep 08 '15 at 11:31

How can I seek to frame No. X with ffmpeg?

1 Answers1

Linked