Reading RTSP stream with FFMpeg library - how to use avcodec_open2?

Question

While trying to read rtsp stream I get some problems, with code and documentation alike. Short description: whatever I do, avcodec_open2 either fails (saying "codec type or id mismatches") or width and height of codec context after the call are 0 (thus making further code useless). Stream itself can be opened normally by VLC player and av_dump_format() displays correct info. My code is based on technique answer to this question.

Long description: my code is in C#, but here is C++-equivalent of FFMpeg calls (I actually reduced my code to this minimum and problem persists):

av_register_all();
avformat_network_init(); //return code ignored

AVFormatContext* formatContext = avformat_alloc_context();
if (avformat_open_input(&formatContext, stream_path, null, null) != 0) {
    return;
}

if (avformat_find_stream_info(formatContext, null) < 0) {
    return;
}

int videoStreamIndex = 0;
for (int i = 0; i < formatContext->nb_streams; ++i) {
    AVStream* s = formatContext->streams[i];
    if (s->codec == null) continue;
    AVCodecContext c = *(s->codec);
    if (c.codec_type == AVMEDIA_TYPE_VIDEO) videoStreamIndex = i;
}

//start reading packets from stream and write them to file
//av_read_play(formatContext); //return code ignored
//this call would print "method PLAY failed: 455 Method Not Valid in This State"
//seems to be the case that for rtsp stream it isn't needed

AVCodec* codec = null;
codec = avcodec_find_decoder(AV_CODEC_ID_H264);
if (codec == null) {
    return;
}

AVCodecContext* codecContext = avcodec_alloc_context3(null);
avcodec_get_context_defaults3(codecContext, codec);//return code ignored
avcodec_copy_context(codecContext, formatContext->streams[videoStreamIndex]->codec); //return code ignored

av_dump_format(formatContext, videoStreamIndex, stream_path, 0);

if (avcodec_open2(codecContext, codec, null) < 0) {
    return;
}

The code actually uses DLL version of FFMpeg library; avcodec-55.dll and avformat-55.dll are used.

Documentation says something weird about which calls can be made in which succession (that copy_context should be called before get_context_defaults), current code is left close as possible to technique version. As written, it results in non-zero return from avcodec_open2 with "codec type or id mismatches" message. Changing the order does little good: now avcodec_open2 executes successfully, but both codecContext->width and codecContext->height are 0 afterwards.

Also documentation doesn't mention which is default value for the third argument of avcodec_open2 should be, but source code seems to taking into account that options can be NULL.

Output of av_dump_format is as follows:

Input #0, rtsp, from 'rtsp://xx.xx.xx.xx:xx/video.pro1':
  Metadata:
    title           : QStream
    comment         : QStreaming Media
  Duration: N/A, start: 0.000000, bitrate: 64 kb/s
    Stream #0:0: Video: h264 (Baseline), yuvj420p(pc), 1920x1080, 30 fps, 25 tbr, 90k tbn, 60 tbc
    Stream #0:1: Audio: pcm_mulaw, 8000 Hz, 1 channels, s16, 64 kb/s

score 7 · Accepted Answer · answered Aug 21 '14 at 09:36

First, what does the av_dump_format shows? Are you sure your video stream codec is h264, because you try to open the codec as if it were H264.

In order to open any codec, change your avcodec_find_decoder to pass it the source codec id:

codec = avcodec_find_decoder(formatContext->streams[videoStreamIndex]->codec->codec_id);

By the way, (forget this one if you do not use the c++ code but stick with c#): you do not need to make a copy of the initial AVCodecContext when you are looking for the video stream. You can do: (note that you may want to keep a pointer to the inital codec context, see below).

AVCodecContext* c = s->codec;
if (c->codec_type == AVMEDIA_TYPE_VIDEO) {
    videoStreamIndex = i;
    initialVideoCodecCtx = c;
}

Next point, not really relevant in this case: instead of looping through all the steams, FFmpeg has a helper function for it:

int videoStreamIndex = av_find_best_stream(formatContext, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);

Last point: I think only the first point should do the trick to make avcodec_open2 work, but you might not be able to decode your stream. You opened the codec for the new codec context, but no codec is opened for the inital context. Why did you make a copy of the initial codec context? It is usefull if you want to record your stream in another file (i.e. transcode), but if you only want to decode your stream, it is much easier to use the initial context, and use it instead of the new one as a parameter for avcodec_decode_video2.

To sum it up, replace your code after avformat_find_stream_info by (warning: no error check):

int videoStreamIndex = av_find_best_stream(formatContext, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
AVCodecContext* codecCtx = formatContext->streams[videoStreamIndex]->codec;
AVCodec* codec = avcodec_find_decoder(codecCtx->codec_id);
// tune codecCtx if you want special decoding options. See FFmpeg docs for a list of members
if (avcodec_open2(codecCtx, codec, null) < 0) {
    return;
}

// use av_read_frame(formatContext, ...) to read packets
// use avcodec_decode_video2(codecCtx, ...) to decode packets

Added av_dump_format output. Your version fails for following reason: `codecCtx->codec_id` is 0 (and thus `avcodec_find_decoder` returns NULL). Initially I thought this is why `get_context_defaults` call was necessary. — Abstraction, Aug 21 '14 at 10:13
The problem was my fault, in the end. I used header without `FF_API_CODEC_NAME` defined (and `AVCodecContext` has 32 bytes field in the beginning of the struct if this macros is enabled). Once I defined it, I successfully got my pictures. Thanks for the help and sorry for wasted time. — Abstraction, Aug 21 '14 at 12:57
Hi, @Abstraction, could you explain a bit more this last bit? Do I just define FF_API_CODEC_NAME before including libav? — Sergio Basurco, Mar 19 '15 at 07:51
@chuckleplant : yes and no. I had to re-write structs in C#, so I just added `byte padding[32]` in the beginning of `AVCodecContext`; when using original C++ headers, macro definition before including struct definition should do the same. But the main problem is that source code contains a number of `ifdef` branches, with no info which of them should be enabled, so for me it was basically trial-and-error solution. Thus, for other applications of the library other problems of the same kind may arise. — Abstraction, Mar 20 '15 at 08:54
Then the problem might be somewhere else, for me the cause is not the C# usage, but the redefinition of some structs to solve [this other topic](http://stackoverflow.com/q/29075467/2628257). Thanks! — Sergio Basurco, Mar 20 '15 at 12:10

score 0 · Answer 2 · answered Apr 29 '15 at 09:16

If avcodec_open2 does not fail, and you still see width and height being 0 this might be expected. Notice that the stream (frame) dimensions are not always known until you actually start decoding.

You should use the AVFrame values in order to initialize your decoding buffers, after your first avcodec_decode_video2 decoding call.

Reading RTSP stream with FFMpeg library - how to use avcodec_open2?

2 Answers2