How to make Gstreamer return only keyframes?

Question

In gstreamer pipeline, I'm trying to figure out if there's a way to specify that I only want key frames from a RTSP stream.

In ffmpeg you can do this with with -skip_frame nokey flag. E.g.:

ffmpeg -skip_frame nokey -i rtsp://184.72.239.149/vod/mp4:BigBuckBunny_175k.mov -qscale 0 -r 1/1 frame%03d.jpg

The corresponding gstreamer command to read the RTSP feed looks like this:

gst-launch-1.0 rtspsrc location=rtsp://184.72.239.149/vod/mp4:BigBuckBunny_175k.mov ! decodebin ! videorate ! "video/x-raw,framerate=1/1" ! videoconvert ! autovideosink

Does anyone know if it is possible to ask gstreamer to only return keyframes?

score 5 · Answer 1 · edited Feb 27 '20 at 20:54

After spending days looking for a complete answer to this question, I eventually ended up with a solution that gave me the rtsp processing boost I was looking for.

Here is the diff of a pipeline in Python that transitioned from processing every RTSP frame to only processing key frames.

https://github.com/ambianic/ambianic-edge/pull/171/files#diff-f89415777c559bba294250e788230c5e

First register for the stream start bus event:

Gst.MessageType.STREAM_START

This is triggered when the stream processing starts. When this event occurs, request seek to the next keyframe.

When the request completes, the pipeline triggers the next bus event we need to listen for:

Gst.MessageType.ASYNC_DONE

Finally, here is the keyframe seek request itself:

    def _gst_seek_next_keyframe(self):
        found, pos_int = self.gst_pipeline.query_position(Gst.Format.TIME)
        if not found:
            log.warning('Gst current pipeline position not found.')
            return
        rate = 1.0  # keep rate close to real time
        flags = \
            Gst.SeekFlags.FLUSH | Gst.SeekFlags.KEY_UNIT | \
            Gst.SeekFlags.TRICKMODE | Gst.SeekFlags.SNAP_AFTER | \
            Gst.SeekFlags.TRICKMODE_KEY_UNITS | \
            Gst.SeekFlags.TRICKMODE_NO_AUDIO
        is_event_handled = self.gst_pipeline.seek(
            rate,
            Gst.Format.TIME,
            flags,
            Gst.SeekType.SET, pos_int,
            Gst.SeekType.END, 0)

score 4 · Accepted Answer · answered Jul 13 '18 at 03:12

4

I think you could try add GST_PAD_PROBE_TYPE_BUFFER pad probe and return GST_PAD_PROBE_DROP on buffers with GST_BUFFER_FLAG_DELTA_UNIT flag set.

answered Jul 13 '18 at 03:12

RSATom

817
6
16

Thanks, do you know how do i add it to the gst-launch pipeline? – Anton Fedorov Jul 16 '18 at 09:11
unfortunately it's possible only in code I think. gst-launch is intended to be only some sort of quick test tool, not fully functional application as ffmpeg. – RSATom Jul 17 '18 at 01:37

A. Binzxxxxxx · Answer 3 · 2018-10-18T12:24:34.953

You can use a new seek event gst_event_new_seek with the flag GstSeekFlags for trickmode GST_SEEK_FLAG_TRICKMODE, skip frames GST_SEEK_FLAG_SKIP and keyframes only GST_SEEK_FLAG_TRICKMODE_KEY_UNITS.

You can also use identity and its property drop-buffer-flags to filter for GST_BUFFER_FLAG_DELTA_UNIT and maybe GST_BUFFER_FLAG_DROPPABLE.

see trickmodes, seeking and GstSeekFlags in the documentation for the seeking and identity:drop-buffer-flags and GstBufferFlags for identity.

How to make Gstreamer return only keyframes?

3 Answers3