1

I am currently trying to learn the FFmpeg API, following this tutorial. However, I already have issues with the first lesson on video decoding. My code is basically the same as the one from the tutorial except I am using C++. My issue is that the video stream does not match the one from the packet returned by av_read_frame.

The video stream is obtained looping on the available streams until the video stream is found.

for(int i = 0; i < pFormatCtx->nb_streams; i++) { // nb_streams == 2

    if(pFormatCtx->streams[i]->codec->codec_type==AVMEDIA_TYPE_VIDEO) {
        videoStream = i; 
        break; // videoStream == 0
    }
}

Then when retrieving the frame data, it seams grabbing the audio channel. 

while(av_read_frame(pFormatCtx, &packet) >= 0) { // read returns 0

    // Is this a packet from the video stream?
    if(packet.stream_index == videoStream) { 
        //packet.stream_index == 1, which correspond to the audio stream
    } 
}

I have not found examples online where this test is actually failing. Have I miss some way to specify the stream_index that is not in the tutorial? Maybe the tutorial is not up to date and is doing something wrong? If so, what is the correct way to extract the frame data? In case that matters, I am using the latest FFmpeg 4.0.2 build, on Windows 64-bits, compiling with Visual Studio 2017.

On videos with no sound, the two streams match and I am able to decode and display the frames correctly.

Student
  • 805
  • 1
  • 8
  • 11
yultan
  • 239
  • 1
  • 9
  • 1
    I'm not sure I understand. If the frame read is for a stream you're not interested in then just ignore it and move on to the next frame. Apologies if I've missed the point. – G.M. Jul 30 '18 at 14:29
  • 1
    Packets are stored interleaved in the container, so you decide to retain them or not post-demux. Unref the packet if it belongs to another stream. – Gyan Jul 30 '18 at 14:55
  • @G.M. You are completely on point. I actually already tried moving to the next frames but not far enough ! I guess I was expecting a more regular pattern between the streams. For my video it was : 0-46 -> audio, 47 -> video, 48-49 -> audio, 50-81 -> video and I stopped earlier than the 46-47 transition. Thank you for your help. Should I remove the post ? – yultan Jul 30 '18 at 15:10
  • Follow this link for a sample program which coded in a simple way that beginners can understand the basics. https://stackoverflow.com/a/37484145/6180077. Also this github link: https://github.com/abdullahfarwees/screen-recorder-ffmpeg-cpp – Abdullah Farweez Aug 02 '18 at 06:36

1 Answers1

0

Try like this:

while(av_read_frame(pFormatCtx, &packet) == 0) {

    AVStream *st = pFormatCtx->streams[packet.stream_index];
    switch (st->codecpar->codec_type)
    {
        case AVMEDIA_TYPE_AUDIO:
            /* handle audio */
            break;

        case AVMEDIA_TYPE_VIDEO:
            /* handle video */
            break;

        ...
    }
}
the kamilz
  • 1,860
  • 1
  • 15
  • 19