How to decode MP4 file use GPU use ffmpeg.autogen?

Question

How to decode MP4 file use GPU?

I use the demo of FFmpeg.AutoGen. It is good for code: private static unsafe void DecodeAllFramesToImages() But this is use CPU to decode. I want get a demo use GPU to decode. How can i do this ?

This is FFmpeg.AutoGen demo:

private static unsafe void DecodeAllFramesToImages()
       {
           // decode all frames from url, please not it might local resorce, e.g. string url = "../../sample_mpeg4.mp4";
           var url = "https://rtmp-luzhi.oss-cn-beijing.aliyuncs.com/eryuan2019/%E5%84%BF%E7%AB%A5%E6%89%8B%E8%B6%B3%E5%8F%A3%E7%97%85%E7%9A%84%E9%98%B2%E6%AD%A2%E4%B8%8E%E6%B2%BB%E7%96%97.mp4"; // be advised this file holds 1440 frames
           using (var vsd = new VideoStreamDecoder(url))
           {
               Console.WriteLine($"codec name: {vsd.CodecName}");

               var info = vsd.GetContextInfo();
               info.ToList().ForEach(x => Console.WriteLine($"{x.Key} = {x.Value}"));

               var sourceSize = vsd.FrameSize;
               var sourcePixelFormat = vsd.PixelFormat;
               var destinationSize = sourceSize;
               var destinationPixelFormat = AVPixelFormat.AV_PIX_FMT_BGR24;
               using (var vfc = new VideoFrameConverter(sourceSize, sourcePixelFormat, destinationSize, destinationPixelFormat))
               {
                   var frameNumber = 0;
                   while (vsd.TryDecodeNextFrame(out var frame))
                   {
                       var convertedFrame = vfc.Convert(frame);

                       using (var bitmap = new Bitmap(convertedFrame.width, convertedFrame.height, convertedFrame.linesize[0], PixelFormat.Format24bppRgb, (IntPtr) convertedFrame.data[0]))
                           bitmap.Save($"frame.{frameNumber:D8}.jpg", ImageFormat.Jpeg);

                       Console.WriteLine($"frame: {frameNumber}");
                       frameNumber++;
                   }
               }
           }
       }

First step, does your GPU havre a decoder that support the necessary codecs? — szatmary, Sep 02 '19 at 16:14
See [FFmpeg Wiki: NVDEC](https://trac.ffmpeg.org/wiki/HWAccelIntro#NVDECCUVID). — llogan, Sep 03 '19 at 17:12

vakym · Answer 1 · 2019-11-05T09:54:02.187

You need change VideoStreamDecoder class:

First you need to configurate your AVCodecContext with using HWDevice of your machine. Something like this:

  AVCodecHWConfig* config = ffmpeg.avcodec_get_hw_config(codec, 0);
  ffmpeg.av_hwdevice_ctx_create(&pCodecContext->hw_device_ctx, HWType, null, null, 0);

The HWType is your type of hardware decoder.

Then when you open context and start decoding frames you need take decoded frame from hardware device:

ffmpeg.av_hwframe_transfer_data(cpuFrame, pFrame, 0)

pFrame - the frame recived from hardware hardware decoder

cpuFrame - the frame into which the data is copied.

UPDATE:

Now in example at FFMPEG.AutoGet repository you can use some hardware decoders.

How to decode MP4 file use GPU use ffmpeg.autogen?

1 Answers1