10

Story

I'm working on a smooth 60 FPS 1080p (Full HD) video transfer application that encodes in x264, sends the encoded data via LAN to a receiving device, which then decodes it using the OpenH264's decoder. I managed to get it working, and it works fine and is stable, but I found it to be very slow (around 20 FPS as opposed to the desired 60 FPS).

Problem

I did extensive testing and found that the issue lies with the OpenH264 decoder.

The decoder makes use of a full core (25% CPU usage total) of my i5-2500 @ 3.9Ghz, which is way too high. Even though the decoder is single-threaded, I tested the raw data on a Media Player Classic, and its playback (at 60 FPS) resulted in mere 0.3% CPU usage. (When switching the render engine to 'Old Video Render' it increased to 12.8-14.4% CPU usage--see comments)

So my question is: What optimizations can I do to speed up the decoding process and what am I doing wrong? I can't possibly imagine OpenH264 is just this slow.

Extra Info

  • The encoder is easily able to push out 60 FPS 1080p using about 20% CPU.
  • The connection is wired LAN and can push > 10MB/s, so no problem there.
  • Both sender and receiver PCs have 8GB RAM.

Code

Below is all the C++ code related to the decoder:

ISVCDecoder *decoder;
SBufferInfo bufferInfo;
SDecodingParam decodingParam;
uint8_t** yuvData;

void init(int width, int height) {
    WelsCreateDecoder(&decoder);
    decodingParam = { 0 };
    decodingParam.sVideoProperty.eVideoBsType = VIDEO_BITSTREAM_AVC;
    decoder->Initialize(&decodingParam);
    bufferInfo = { 0 };

    yuvData = new uint8_t*[3];
    yuvData[0] = new uint8_t[width*height];
    yuvData[1] = new uint8_t[width*height / 4];
    yuvData[2] = new uint8_t[width*height / 4];
}

bool decode(cont unsigned char* rawEncodedData, int rawEncodedDataLength, uint8_t** yuvData) {
    int err = decoder->DecodeFrameNoDelay(rawEncodedData, rawEncodedDataLength, yuvData, &bufferInfo);
    if(err != 0) {
        std::cout << "H264 decoding failed. Error code: " << err << "." << std::endl;
        return false;
    }
    return true;
}
Roland
  • 612
  • 5
  • 17
  • 1
    Have you tried profiling the code? – EvilTeach Sep 02 '16 at 20:42
  • 4
    If Media Player Classic is using 1.2% of a core, it is almost certainly offloading the job to a hardware video decoder. – Dietrich Epp Sep 02 '16 at 20:46
  • Try this on media player then measure the performance of media player : https://www.pixelmetrics.com/Tips/VidBlank/MediaPlayer.php – Humam Helfawi Sep 02 '16 at 20:52
  • @DietrichEpp Good point, I changed the render engine to "Old video render" and it bumped up the CPU usage to 12.8 - 14.4%. http://pasteboard.co/gcibDaq8F.png – Roland Sep 02 '16 at 21:05
  • you can try to be sure the 3 chunks pointed by `yuvData` are contiguous, ie, `uint8_t * p = new uint8_t[width*height*3/2]; *yuvData=p; *(yuvData+1)=p+width*height; *(yuvData+2)=p+(width*height*5)/4;` – norisknofun Sep 07 '16 at 19:21
  • OpenH264 decoder is a software decoder using the cpu. Media Player Classic can use DXVA2 to decode h264 using the GPU. You need to use a h264 GPU decoder. – mofo77 Sep 07 '16 at 21:33

1 Answers1

1

A relatively un-optimized CPU based H.264 decoder can easily be that slow. If you are on a PC and you have a hardware H.264 decoder - you might as well use it.

I'd try: https://software.intel.com/en-us/media-sdk-support/code-samples

Markus Schumann
  • 7,636
  • 1
  • 21
  • 27
  • Switching over to FFmpeg's H.264 decoder solved the problem. I still have between 20-35% CPU usage but that also incorporates a few other processes. It turns out the OpenH264 decoder is in fact just that slow... – Roland May 24 '17 at 11:13