3

I need to encode an mpegts video using the ffmpeg C++ API. The output video shall have two streams: the first one shall be of type AVMEDIA_TYPE_VIDEO; the second one shall be of type AVMEDIA_TYPE_DATA and shall contain a set of KLV data.

I have written my own KLV library to manage the KLV format.

However I'm not able to create "from scratch" a new video by combining the two streams. Following the implementation as in FFMPEG C api h.264 encoding / MPEG2 ts streaming problems I can successfully encode a mpegts video with a single video stream.

However I'm not able to add a new AVMEDIA_TYPE_DATA stream to the output video since, as soon as I add a new data stream using methods like avformat_new_stream(...) the output video is empty: neither the data stream nor the video one are produced and the output file is empty.

Can anyone suggest me a tutorial page or a sample on how to properly add a data stream to my output video in mpegts format?

Thanks a lot!

Community
  • 1
  • 1
arms
  • 163
  • 1
  • 3
  • 9

1 Answers1

3

I was able to get a KLV stream added to a muxed output by starting with the "muxing.c" example that comes with the FFmpeg source, and modifying it as follows.

First, I created the AVStream as follows, where "oc" is the AVFormatContext (muxer) variable:

AVStream *klv_stream = klv_stream = avformat_new_stream(oc, NULL);
klv_stream->codec->codec_type = AVMEDIA_TYPE_DATA;  
klv_stream->codec->codec_id = AV_CODEC_ID_TIMED_ID3;
klv_stream->time_base = AVRational{ 1, 30 };
klv_stream->id = oc->nb_streams - 1;

Then, during the encoding/muxing loop:

AVPacket pkt;
av_init_packet(&pkt);
pkt.data = (uint8_t*)GetKlv(pkt.size);
auto res = write_frame(oc, &video_st.st->time_base, klv_stream, &pkt);
free(pkt.data);

(The GetKlv() function returns a malloc()'ed array of binary data that would be replaced by whatever you're using to get your encoded KLV. It sets pkt.size to the length of the data.)

With this modification, and specifying a ".ts" target file, I get a three-stream file that plays just fine in VLC. The KLV stream has a stream_type of 0x15, indicating synchronous KLV.

Note the codec_id value of AV_CODEC_ID_TIMED_ID3. According to the libavformat source file "mpegtsenc.c", a value of AV_CODEC_ID_OPUS should result in stream_type 6, for asynchronous KLV (no accompanying PTS or DTS). This is actually important for my application, but I'm unable to get it to work -- the call to avformat_write_header() throws a division by zero error. If I get that figured out, I'll add an update here.

Ted W
  • 239
  • 4
  • 11
  • For initializing the KLV stream I set the codec_id to "AV_CODEC_ID_SMPTE_KLV" and I also set the codec_tag to "1096174667". Doing these two things made the data stream recognizable as a KLV stream for me. – Nikkisixx2 Dec 16 '19 at 14:49
  • this is amazing, I've been trying to get it in Python for a long time but without success and I think this could help me. this is the project https://github.com/All4Gis/QGISFMV – Fran Raga Feb 04 '20 at 10:10