3

Marked question as outdated as using the deprecated avcodec_decode_video2

I'm currently experiencing artifacts when decoding video using ffmpegs api. On what I would assume to be intermediate frames, artifacts build slowly only from active movement in the frame. These artifacts build for 50-100 frames until I assume a keyframe resets them. Frames are then decoded correctly and the artifacts proceed to build again.

One thing that is bothering me is I have a few video samples that are 30fps(h264) that work correctly, but all of my 60fps videos(h264) experience the problem.

I don't currently have enough reputation to post an image, so hopefully this link will work. https://i.stack.imgur.com/1bv4X.jpg

int numBytes;
int frameFinished;
AVFrame* decodedRawFrame;
AVFrame* rgbFrame;

    //Enum class for decoding results, used to break decode loop when a frame is gathered
DecodeResult retResult = DecodeResult::Fail;

decodedRawFrame = av_frame_alloc();
rgbFrame = av_frame_alloc();
if (!decodedRawFrame) {
    fprintf(stderr, "Could not allocate video frame\n");
    return DecodeResult::Fail;
}

numBytes = avpicture_get_size(PIX_FMT_RGBA, mCodecCtx->width,mCodecCtx->height);
uint8_t* buffer = (uint8_t *)av_malloc(numBytes*sizeof(uint8_t));

avpicture_fill((AVPicture *) rgbFrame, buffer, PIX_FMT_RGBA, mCodecCtx->width, mCodecCtx->height);

AVPacket packet;

while(av_read_frame(mFormatCtx, &packet) >= 0 && retResult != DecodeResult::Success)
{
    // Is this a packet from the video stream?
    if (packet.stream_index == mVideoStreamIndex)
    {
        // Decode video frame
        int decodeValue = avcodec_decode_video2(mCodecCtx, decodedRawFrame, &frameFinished, &packet);

        // Did we get a video frame?
        if (frameFinished)// && rgbFrame->pict_type != AV_PICTURE_TYPE_NONE )
        {
            // Convert the image from its native format to RGB
            int SwsFlags = SWS_BILINEAR;
            // Accurate round clears up a problem where the start
                            // of videos have green bars on them
            SwsFlags |= SWS_ACCURATE_RND;
            struct SwsContext *ctx = sws_getCachedContext(NULL, mCodecCtx->width, mCodecCtx->height, mCodecCtx->pix_fmt, mCodecCtx->width, mCodecCtx->height, 
                PIX_FMT_RGBA, SwsFlags, NULL, NULL, NULL);
            sws_scale(ctx, decodedRawFrame->data, decodedRawFrame->linesize, 0, mCodecCtx->height, rgbFrame->data, rgbFrame->linesize);
            
            //if(count%5 == 0 && count < 105)
            //  DebugSavePPMImage(rgbFrame, mCodecCtx->width, mCodecCtx->height, count);
            
            ++count;
            // Viewable frame is a struct to hold buffer and frame together in a queue
            ViewableFrame frame;
            frame.buffer = buffer;
            frame.frame = rgbFrame;
            mFrameQueue.push(frame);
            
            
            retResult = DecodeResult::Success;
                        
            sws_freeContext(ctx);
        }
    }
    
    // Free the packet that was allocated by av_read_frame
    av_free_packet(&packet);
}

// Check for end of file leftover frames
if(retResult != DecodeResult::Success)
{
    int result = av_read_frame(mFormatCtx, &packet);
    if(result < 0)
        isEoF = true;
    av_free_packet(&packet); 
}   

// Free the YUV frame
av_frame_free(&decodedRawFrame);

I'm attempting to build a queue of the decoded frames that I then use and free as needed. Is my seperation of the frames causing the intermediate frames to be decoded incorrectly? I also break the decoding loop once I've successfully gathered a frame(Decode::Success, most examples I've seen tend to loop through the whole video.

All codec contect, video stream information, and format contexts are setup up exactly as shown in the main function of https://github.com/chelyaev/ffmpeg-tutorial/blob/master/tutorial01.c

Any suggestions would be greatly appreciated.

Antonio
  • 19,451
  • 13
  • 99
  • 197
  • Due diligence: Do these problem videos decode correctly with any other players? If not, then the streams might be corrupted. Does FFmpeg's own player (ffplay) show the frames correctly? Also, where do these videos come from? Do you know what encoder was used? – Multimedia Mike Dec 31 '13 at 02:13
  • Your decoding code looks fine to me. I agree with Mike above. Note that video DECODER doesn't care about FPS at all - only a video PLAYER should - so it is irrelevant if you have 1 frame a second or 1200. – George Y. Dec 31 '13 at 03:54

1 Answers1

2

For reference if someone finds themselves in a similar position. Apparently with some of the older versions of FFMPEG there's an issue when using sws_scale to convert an image and not changing the actual dimensions of the final frame. If instead you create a flag for the SwsContext using:

int SwsFlags = SWS_BILINEAR; //Whatever you want SwsFlags |= SWS_ACCURATE_RND; // Under the hood forces ffmpeg to use the same logic as if scaled

SWS_ACCURATE_RND has a performance penalty but for regular video it's probably not that noticeable. This will remove the splash of green, or green bars along the edges of textures if present.

I wanted to thank Multimedia Mike, and George Y, they were also right in that the way I was decoding the frame wasn't preserving the packets correctly and that was what caused the video artifacts building from previous frames.