67

I've been struggling with this issue for more than a week, and most likely it is a bug in the Qualcomm GPU/hardware video encoder. Since we are pressed to release the application, and their developer forums did not provide any feedback, I am posting it here too, hoping that someone is able to provide some clues, or even better, a workaround so that the bug in the encoder is not triggered.

The application encodes from a Surface. When certain images are rendered to the surface, the encoder fails (100%) in the same place (when the frame currently being encoded is a key frame). The Logcat output from the encoder at the point of the crash is(with the last lines repeated at a very high rate):

11-26 11:41:33.312: E/OMX-VENC-720p(25949): ERROR: async_msg_process() - Error statuscode = 1
11-26 11:41:33.312: E/ACodec(29317): [OMX.qcom.video.encoder.avc] ERROR(0x80001009)
11-26 11:41:33.312: E/MediaCodec(29317): Codec reported an error. (omx error 0x80001009, internalError -2147483648)
11-26 11:41:33.362: E/OMX-VENC-720p(25949): ERROR: venc_use_buf:set input buffer failed 
11-26 11:41:33.362: E/OMX-VENC-720p(25949): ERROR: in dev_use_buf
11-26 11:41:33.362: E/OMX-VENC-720p(25949): ERROR: empty_buffer_done() failed!
11-26 11:41:33.372: E/OMX-VENC-720p(25949): m_fbd_count at o/p flush = 306
11-26 11:41:33.372: E/OMX-VENC-720p(25949): m_etb_count at i/p flush = 313
11-26 11:41:33.372: E/OMX-VENC-720p(25949): ERROR: ioctl VEN_IOCTL_CMD_FILL_OUTPUT_BUFFER failed
11-26 11:41:33.372: E/OMX-VENC-720p(25949): ERROR: dev_fill_buf() Failed
11-26 11:41:33.372: E/OMX-VENC-720p(25949): ERROR: FTBProxy() failed!

I can replicate it also on Grafika by simply replacing in the Record GL app the drawing of two geometric forms with drawing one of the problematic images, full screen, in landscape mode, and changing the bitrate to a higher value (7Mbps). The crash of the encoder happens sooner for larger bitrate.

Here is one image that seems to break it easily and here is RecordFBOActivity.java with the required changes.

With Grafika, I tested and the encoder crashes on both Samsung S4, international version, and on the original Nexus 4. With our software, which is a little more complex than rendering the plain image, it still crashes on both. Did not test on any other Adreno devices. On Samsung S3, with a Mali 400 GPU, it works fine.

At 4Mbps, the encoder in our application still crashes on both S4 and N4, but later. Grafika crashes on N4 in the same place, but not on S4.

EDIT: According to comments bellow, it can be reproduced also when encoding the same image from the buffer. Various tests seem to narrow the conditions to reproduce it at: h264 hw encoder on Qualcomm devices, encoding a still image for many frames (this determines very low bitrates in the encoder due to similar frames), error appearing while encoding a key frame (the error appears only when encoding certain images, which seem to have more detail, i.e. need many bits for intra coding).

user1592546
  • 1,480
  • 1
  • 14
  • 30
  • 1
    The `omx` error code of `0x80001009` is `OMX_ErrorHardware` which means that component is returning an error related to underlying hardware. On surface recording, can you quickly check the color format of the frame being provided to the underlying HW? Can you try by converting from RGB to YUV space say `YUV420 Planar` or `YUV420 Interleaved` and then calling the encoder to encode? – Ganesh Dec 03 '14 at 00:18
  • 3
    Please try INDE Media for Mobile, GLCapture sample from: https://github.com/INDExOS/media-for-mobile . Does it crash? It was definetly checked to work on S4. Also I remember there were issues with high bitrates on some devices as a result of too small (more optimal) memory buffers allocations for encoded frames, so we had to limit max bitrate – Marlon Dec 03 '14 at 05:53
  • @Ganesh: mEncoder.getCodecInfo().getCapabilitiesForType("video/avc") returns 0x7FA30C06(not found in the defintions from static class CodecCapabilities, only found COLOR_QCOM_FormatYUV420SemiPlanar which is actually 0x7FA30C00), 0x7F000789 (COLOR_FormatSurface), 21(COLOR_FormatYUV420SemiPlanar), -1320330260 (could not identify it). I am using COLOR_FormatSurface when the format of the encoder is configured, which "indicates that the data will be a GraphicBuffer metadata reference". By RGB->YUV and then encode, you mean to use glReadPixels, perform conversion and then encode from buffer? Or? – user1592546 Dec 03 '14 at 12:09
  • @Marlon: In order to try it, do I need to download and install the Intel® INDE, the free edition? So the media-for-mobile does not work on its own. Isn't inde available as a jar? – user1592546 Dec 03 '14 at 12:35
  • you need INDE free edition, you will get .jar's there. sorry, there is no was to get .jar files in the other way – Marlon Dec 03 '14 at 13:02
  • @Marlon: It seems it really is a Qualcomm bug, so probably INDE is affected too(I did not have time so far to try it). The question remains open, as I did not find a workaround so far. BTW, is INDE able to save playable mp4v-es files, or just h264? – user1592546 Dec 08 '14 at 10:07
  • this is great that you have found a workaroud! INDE supports only H.264, since official reqs from Google does not have mp4v-es section for econder, so there are no tests for hw vendors to check if codec works correctly – Marlon Dec 08 '14 at 10:18
  • @Marlon: I did NOT find. mp4v-es does not seem to have the same error on Qualcomm devices, but I could not find any working example of it. – user1592546 Dec 08 '14 at 11:44
  • ooops:) i do not think this is safe to use mp4v-es encoder at this point - untill Google does not put it as a reqs for hw vendors and does not create certification tests the behavior will be very different from device to device – Marlon Dec 08 '14 at 11:54
  • why do you need to set 7Mbps bitrate? what if you set 3Mbps for example, it can work on your device, isn't quality is good enough in this case? – Marlon Dec 08 '14 at 12:13
  • @Marlon: 7Mbps is just to make the error happen sooner. The product would be released with 3-4 Mbps, but it only determines that the error happens later for selected images(at a later keyframe). For most images, it does not happen at all. If the user will record short enough, the error may not appear even for the images known to produce it. But I think it is not acceptable to release an application like this. – user1592546 Dec 08 '14 at 13:24
  • @user1592546.. You mentioned that the error happens earlier with higher bitrate. Did you try moving the output location from `/sdcard` to `/data` or vice-versa? i.e. try changing the location of output file from card to internal memory or vice-versa. – Ganesh Dec 21 '14 at 13:18
  • @Ganesh: The encoder works fine at all kind of bitrates if the input of the encoder has animation from frame to frame, so I did not test this situation. I rather suspect the part of the encoder responsible of bitrate computation, or something related to it, like quantization factors, etc, during a keyframe. Something that happens when bitrate was very low for last several frames, even with low quantization(due to lack of motion in frame), and suddenly has to manage the same frame as an intra, i.e. use a lot of bits to keep similar quality. – user1592546 Dec 22 '14 at 18:30
  • @user1592546... In the past I have seen errors cropping up due to slower device speed i.e. speed of `/sdcard` and the associated file system along with a synchronous handling in `GingerBread` gave a lot of headache for me. Hence, suggested the expt. – Ganesh Dec 22 '14 at 23:24
  • I tried reproducing this issue on a Galaxy S4 (international version), with checking out Grafika, replacing RecordFBOActivity.java with the linked version of the file, adding the test JPG at the right place, and running the "Record GL app" mode (is there any difference in reproducing the issue between the three modes within this activity?), but didn't see any issue while encoding over 8000 frames. Can you provide an APK for testing to see if there's a difference in the code I've built, or help me reproduce the issue? – mstorsjo Jan 01 '15 at 20:16
  • @mstorsjo: Did you use the attached image? Start the application, select "Record GL app", turn it in landscape mode, select "Rec: FBO blit x2" and then Start recording. The issue was reported also here: https://code.google.com/p/android/issues/detail?id=81488&colspec=ID%20Type%20Status%20Owner%20Summary%20Stars and here: https://developer.qualcomm.com/forum/qdn-forums/maximize-hardware/mobile-gaming-graphics-adreno/28307 with more details about how it can be reproduced. I'll attach also an apk. – user1592546 Jan 05 '15 at 14:53
  • I was using the attached image, yes, but I missed running it in landscape mode. Now I can reproduce the issue as well. – mstorsjo Jan 05 '15 at 15:00
  • Then I'll skip the apk. – user1592546 Jan 05 '15 at 15:08
  • FWIW, I managed to reproduce the same crash when encoding the same image (scaled down to 1280x720 and flipped vertically, although that doesn't seem to matter) via buffer input instead of a surface. So the core issue (which probably isn't a surprise) isn't related to the input but to encoding in general (currently the title of the issue reports sounds like Surface input would be the issue, while it is only an extra detail). Not really sure if I can help debugging it any further though. – mstorsjo Jan 05 '15 at 16:26
  • @mstorsjo: Please check my edit in the original question, maybe it adds some clues. – user1592546 Jan 08 '15 at 10:58
  • Thanks, those are useful details, to know what to avoid or where to expect issues. I don't think I can debug it any further (since debugging it would require source to the whole DSP codec), but it's at least useful to have it narrowed down like this. – mstorsjo Jan 08 '15 at 11:02
  • As a possible workaround, if you can reliably tell where the encoder crashes, could you just `stop()` the encoder and `start()` it again so that it refreshes its internal states? – Saran Tunyasuvunakool Feb 16 '15 at 11:58
  • Did u check here, https://code.google.com/p/android/issues/detail?id=81488 , seems this issue is already reported as a bug. – Sazzad Hissain Khan Dec 25 '15 at 08:12
  • That is my report on the issue, on android side. Thanks. There is another one on Qualcomm forums. – user1592546 Dec 28 '15 at 12:37

1 Answers1

1

It seems to be a bug as previously stated above.

DaveJone359
  • 37
  • 3