Why decoding frames from avi container and encode them to h264/mp4 doesn't work?

Question

I started using ffmpeg and I want to convert avi file to mp4/h264 file. I've read many posts including this, but I couldn't find any good example how to save frames to mp4 file. The code below is simplified one that decodes frames from avi file and encode it to H264/mp4 file, but when I save the frames the mp4 file cannot be played. I think I do somethinkg wrong in encoding

I will appreciate if you could tell me what is wrong and how to fix it.

const char* aviFileName = "aviFrom.avi";
const char* mp4FileName = "mp4To.mp4";

// Filling pFormatCtx by open video file and Retrieve stream information
// ...
// Retrieving codecCtxDecode and opening codecDecode 
//...


// Get encoder
codecCtxEncode = avcodec_alloc_context();   
codecCtxEncode->qmax = 69; 
codecCtxEncode->max_qdiff = 4;
codecCtxEncode->bit_rate = 400000;
codecCtxEncode->width = codecCtxDecode->width;
codecCtxEncode->height = codecCtxDecode->height;
codecCtxEncode->pix_fmt = AV_PIX_FMT_YUV420P;   
codecEncode = avcodec_find_encoder(CODEC_ID_H264);
if(codecEncode == NULL)
    return -1;  
if(avcodec_open2(codecCtxEncode, codecEncode, NULL))
    return -1;

SwsContext *sws_ctx = sws_getContext(codecCtxDecode->width, codecCtxDecode->height, codecCtxDecode->pix_fmt,
                            codecCtxDecode->width, codecCtxDecode->height, AV_PIX_FMT_YUV420P,
                            SWS_BILINEAR, NULL, NULL, NULL);

// Allocate an AVFrame structure    
frameDecoded = avcodec_alloc_frame();
frameEncoded = avcodec_alloc_frame();    

avpicture_alloc((AVPicture *)frameEncoded, AV_PIX_FMT_YUV420P, codecCtxDecode->width, codecCtxDecode->height);

while(av_read_frame(pFormatCtx, &packet)>=0)
{       
    // Is this a packet from the video stream?
    if(packet.stream_index==videoStreamIndex) {
        avcodec_decode_video2(codecCtxDecode, frameDecoded, &frameFinished, &packet);
        // Did we get a video frame?
        if(frameFinished)
        {           
            fwrite(packet.data, packet.size,
            sws_scale(sws_ctx, frameDecoded->data, frameDecoded->linesize, 0, codecCtxDecode->height, 
                        frameEncoded->data, frameEncoded->linesize);



            int64_t pts = packet.pts;
            av_free_packet(&packet);
            av_init_packet(&packet);
            packet.data = NULL;
            packet.size = 0;    
            frameEncoded->pts = pts;                

            int failed = avcodec_encode_video2(codecCtxEncode, &packet, frameEncoded, &got_output);
            if(failed)
            {
                exit(1);
            }
            fwrite(packet.data,1,packet.size, mp4File);
        }
    }

    av_free_packet(&packet);
}

score 8 · Accepted Answer · edited Nov 28 '13 at 16:59

You have to use ffmpeg output context, instead of writing raw packets directly.

The basic steps you need to perform:

// find output format
AVOutputFormat * outputFormat = av_guess_format("mp4", NULL, NULL);
AVFormatContext *outFmtCtx = NULL;

// create output cotext
avformat_alloc_output_context2(&outFmtCtx, outputFormat, NULL, NULL);

// create new stream
AVStream * outStrm = avformat_new_stream(outFmtCtx, codecEncode);
avcodec_get_context_defaults3(outStrm->codec, *codec);

outStrm->codec->codec_id = codec_id;
outStrm->codec->coder_type = AVMEDIA_TYPE_VIDEO;
/// outStrm->codec-> ... 
/// set all fields marked as "MUST be set by user" in avcodec.h
/// ....

// create file
avio_open2(&outFmtCtx->pb, file_name, AVIO_FLAG_WRITE, NULL, NULL);
avformat_write_header(outFmtCtx, NULL);

/// write packets
/// for ( )
av_interleaved_write_frame(outFmtCtx, packet);

/// finish
av_write_trailer(outFmtCtx);
avio_close(outFmtCtx->pb);
avformat_free_context(outFmtCtx);

UPD: The full code that copies the video without re-encoding:

extern "C" {
#include <libavformat/avformat.h>
#include <libavcodec/avcodec.h>
#include <libavutil/avutil.h>
}

int main(int argc, char* argv[])
{
    const char * kInputFileName = "f:/Projects/Temp/testFFMPEG2/test/test_in.avi";
    const char * kOutputFileName = "f:/Projects/Temp/testFFMPEG2/test/text_out.avi";
    const char * kOutputFileType = "avi";

    av_register_all();

    AVFormatContext * inCtx = NULL;
    int err = avformat_open_input(&inCtx, kInputFileName, NULL, NULL);
    if (err < 0)
        exit(1);

    err = av_find_stream_info(inCtx);
    if (err < 0)
        exit(1);


    int vs = -1;
    for (unsigned int s = 0; s < inCtx->nb_streams; ++s)
    {
        if (inCtx->streams[s] &&
            inCtx->streams[s]->codec &&
            inCtx->streams[s]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
        {
            vs = s;
            break;
        }        
    }

    if (vs == -1)
        exit(1);

    AVOutputFormat * outFmt = av_guess_format(kOutputFileType, NULL, NULL);
    if (!outFmt)
        exit(1);

    AVFormatContext *outCtx = NULL;
    err = avformat_alloc_output_context2(&outCtx, outFmt, NULL, NULL);

    if (err < 0 || !outCtx)
        exit(1);

    AVStream * outStrm = av_new_stream(outCtx, 0);
    AVStream const * const inStrm = inCtx->streams[vs];
    AVCodec * codec = NULL;
    avcodec_get_context_defaults3(outStrm->codec, codec);
    outStrm->codec->thread_count = 1;

#if (LIBAVFORMAT_VERSION_MAJOR == 53)
    outStrm->stream_copy = 1;
#endif

    outStrm->codec->coder_type = AVMEDIA_TYPE_VIDEO;
    if(outCtx->oformat->flags & AVFMT_GLOBALHEADER) 
        outStrm->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;

    outStrm->codec->sample_aspect_ratio = outStrm->sample_aspect_ratio = inStrm->sample_aspect_ratio; 

#if (LIBAVFORMAT_VERSION_MAJOR == 53)
    outCtx->timestamp = 0;
#endif

    err = avio_open(&outCtx->pb, kOutputFileName, AVIO_FLAG_WRITE);
    if (err < 0)
        exit(1);

#if (LIBAVFORMAT_VERSION_MAJOR == 53)
    AVFormatParameters params = {0};
    err = av_set_parameters(outCtx, &params);
    if (err < 0)
        exit(1);
#endif

    outStrm->disposition = inStrm->disposition;
    outStrm->codec->bits_per_raw_sample = inStrm->codec->bits_per_raw_sample;
    outStrm->codec->chroma_sample_location = inStrm->codec->chroma_sample_location;
    outStrm->codec->codec_id = inStrm->codec->codec_id;
    outStrm->codec->codec_type = inStrm->codec->codec_type;

    if (!outStrm->codec->codec_tag)
    {
        if (! outCtx->oformat->codec_tag
            || av_codec_get_id (outCtx->oformat->codec_tag, inStrm->codec->codec_tag) == outStrm->codec->codec_id
            || av_codec_get_tag(outCtx->oformat->codec_tag, inStrm->codec->codec_id) <= 0)
                    outStrm->codec->codec_tag = inStrm->codec->codec_tag;
    }

    outStrm->codec->bit_rate = inStrm->codec->bit_rate;
    outStrm->codec->rc_max_rate = inStrm->codec->rc_max_rate;
    outStrm->codec->rc_buffer_size = inStrm->codec->rc_buffer_size;

    const size_t extra_size_alloc = (inStrm->codec->extradata_size > 0) ?
                                    (inStrm->codec->extradata_size + FF_INPUT_BUFFER_PADDING_SIZE) :
                                     0;

    if (extra_size_alloc)
    {
        outStrm->codec->extradata = (uint8_t*)av_mallocz(extra_size_alloc);    
        memcpy( outStrm->codec->extradata, inStrm->codec->extradata, inStrm->codec->extradata_size);
    }
    outStrm->codec->extradata_size = inStrm->codec->extradata_size;

    AVRational input_time_base = inStrm->time_base;
    AVRational frameRate = {25, 1};
    if (inStrm->r_frame_rate.num && inStrm->r_frame_rate.den 
        && (1.0 * inStrm->r_frame_rate.num / inStrm->r_frame_rate.den < 1000.0))
    {
        frameRate.num = inStrm->r_frame_rate.num;
        frameRate.den = inStrm->r_frame_rate.den;
    }

    outStrm->r_frame_rate = frameRate;
    outStrm->codec->time_base = inStrm->codec->time_base;

    outStrm->codec->pix_fmt = inStrm->codec->pix_fmt;
    outStrm->codec->width =  inStrm->codec->width;
    outStrm->codec->height =  inStrm->codec->height;
    outStrm->codec->has_b_frames =  inStrm->codec->has_b_frames;
    if (! outStrm->codec->sample_aspect_ratio.num) {
        AVRational r0 = {0, 1};
        outStrm->codec->sample_aspect_ratio =
            outStrm->sample_aspect_ratio =
            inStrm->sample_aspect_ratio.num ? inStrm->sample_aspect_ratio :
            inStrm->codec->sample_aspect_ratio.num ?
            inStrm->codec->sample_aspect_ratio : r0;
    }
#if LIBAVFORMAT_VERSION_MAJOR == 53
    av_write_header(outFmtCtx);
#else
    avformat_write_header(outCtx, NULL);
#endif


    for (;;)
    {
        AVPacket packet = {0};
        av_init_packet(&packet);

        err = AVERROR(EAGAIN);
        while (AVERROR(EAGAIN) == err) 
            err = av_read_frame(inCtx, &packet);

        if (err < 0)
        {
            if (AVERROR_EOF != err && AVERROR(EIO) != err)
            {
                // error
                exit(1);            
            }
            else
            {
                // end of file
                break;
            }            
        }


        if (packet.stream_index == vs)
        {

            err = av_interleaved_write_frame(outCtx, &packet);
            if (err < 0)
                exit(1);
        }            

        av_free_packet(&packet);        

    }

    av_write_trailer(outCtx);
    if (!(outCtx->oformat->flags & AVFMT_NOFILE) && outCtx->pb)
        avio_close(outCtx->pb);

    avformat_free_context(outCtx);
    av_close_input_file(inCtx);
    return 0;
}

Hypothetically, if I would manually add the header and footer to the file, would it be OK? — theateist, Nov 08 '12 at 15:32
For some formats, it can work, but in general, this is the wrong way. — pogorskiy, Nov 08 '12 at 16:14
as for experiment with what you wrote I tried to read from avi file and write the same packets I've read(without decoding) to new avi file. I expected to get the same file but the new file is bigger on 3KB and media player cannot open it — theateist, Nov 08 '12 at 17:22
I did 2 things 1 - instead of "mp4" I wrote "avi" and the filename of-course is also changed to "newavi.avi". 2 - for every field for `outStrm->codec->` I took the value from `codecCtxDecode->`. And the package writing `while(av_read_frame(pFormatCtx, &packet)>=0) { av_interleaved_write_frame(outFmtCtx, &packet); }`. What is the problem in your opinion? — theateist, Nov 08 '12 at 17:26
I tried your code that copies the video without re-encoding, but the output was the same - the size of output file was was bigger on some KB, overall bitrate was larger on 1Kbps, stream size was 99% and not 100%. Do you have any suggestions? — theateist, Nov 10 '12 at 19:57
The file size may not match exactly. It depends on how ffmpeg write the file headers — pogorskiy, Nov 11 '12 at 12:47
ok, but after using your code Windows Media Player fails to play the file. VLC player succeeded to open the file, but instead of video it show some blinking stripes. Another question is if the headers are different how the Players would play the file? — theateist, Nov 11 '12 at 13:25
Eventually it worked, but there are some avi files that still fails and I don't can figure out why — theateist, Nov 13 '12 at 11:09
In each case, may have a different reason. In general, copy packets from one container to another is not always possible to perform correctly — pogorskiy, Nov 13 '12 at 12:06
@pogorskiy , thank you for your answer. I need to decode the packet into a frame, process it with opencv and transfer it back to AVPacket. I use avcodec_encode_video2 but it comes with an error. Would you please help me how to decode a frame to AVFrame and encode it to AVPacket? — Davood Falahati, Jul 02 '17 at 14:32
@DavoodFalahati see decoding_encoding.c example from ffmpeg source — pogorskiy, Jul 03 '17 at 09:01

Why decoding frames from avi container and encode them to h264/mp4 doesn't work?

1 Answers1

Linked