Decoding AAC using MediaCodec API on Android

Question

I'm trying to used the MediaCodec API on Android to decode an AAC stream. (It's raw AAC.) I tried using the MediaFormat.createAudioFormat() to create the format object to pass to MediaCodec.configure(), but I kept getting errors when using AAC (audio/mp4a-latm). (It works with MP3 (audio/mpeg) though...)

Finally I created a MediaExtractor for an AAC file and looked at the format object it was producing. I saw that it included the key "csd-0" for a ByteBuffer composed of two bytes both with the value 0x12. If I include that key and value in the format object that I used to configure the AAC codec, everything works.

Does anyone have an idea what is going on? The documentation states that I shouldn't configure that key. Does anyone have a pointer to MediaCodec examples to decode AAC files without using MediaExtractor to generate the format object?

Further investigation has lead me to believe that csd-0 contains the ESDS (Elementary Stream Descriptor). Looking at MakeAACCodecSpecificData in avc_utils.cpp in the Android sources I see that the first 5 bits are the object type (2), the next 4 are the frequency index (4 = 48000), the next 4 are the channel config (2). [This wiki page](http://wiki.multimedia.cx/index.php?title=Understanding_AAC) indicates that the remaining bits are for frame length, depends on core decoder, and extension flag. I'm using MediaFormat.createAudioFormat() which should setup the important elements of the ESDS. — Benjamin Reed, Oct 18 '12 at 21:47

score 4 · Answer 1 · edited Jun 28 '13 at 04:16

Yes, the codec config 2 bytes are the initial you receive. And yes it is raw aac data blocks. You can see how I derive the format below while encoding. I initially tried to follow the documentation, which said they are in latm format, and tried to parse that. I then found some 'diff' on the android documentation that said the output was indeed raw blocks. Knowing that then, it was simply a matter of choosing a container for my needs. In particular, I needed the adts container rather than flv or mp4.

Copying the payload data into an array that is large enough for your container, just add on your bits. So after scouring the internet for my solution I produced the following code:

profile = (configParams[0]>>3)&0x1f;

frequency_index = (this.configParams[0]&0x7) <<1 | (this.configParams[1]>>7) &0x1;

channel_config = (this.configParams[1]>>3) &0xf;

int finallength = encoded_length + 7;       
ENCodedByteArray[0] = (byte) 0xff;
ENCodedByteArray[1] = (byte) 0xf1;
ENCodedByteArray[2] = (byte) ( ((profile - 1) << 6) + (frequency_index << 2) +(channel_config >> 2));
ENCodedByteArray[3] = (byte) (((channel_config & 0x3) << 6) + (finallength >> 11));
ENCodedByteArray[4] = (byte)( (finallength & 0x7ff) >> 3);
ENCodedByteArray[5] = (byte) (((finallength & 7) << 5) + 0x1f) ;
ENCodedByteArray[6] = (byte) 0xfc;

Using something like this:

byte chunkADTS[]=new byte[info.size + 7];
fillInADTSHeader(chunkADTS,info.size);
outputBuffers[bR].get(chunkADTS,7,info.size);
buffer.pushData(chunkADTS);

I have a follow-up question: what is the "configParams" value above? Is this the config=xxxx value from the a=fmtp:nn line in an SDP message (for RTSP streams)? — David Schwartz, Jun 09 '14 at 18:31
THe config params are the specific audio config, and typically the first two bytes output byt the latm media codec. The BufferINfo object should have Codec_info flag set. — Andy--S, Sep 12 '14 at 18:36

score 2 · Answer 2 · answered Mar 12 '14 at 03:10

I used the follow code and add ES which removed ADTS header, it could work well, but I really do not know why should set "csd-0", or codec will occur error

      decoder = MediaCodec.createDecoderByType("audio/mp4a-latm");
      mMediaFormat = MediaFormat.createAudioFormat("audio/mp4a-latm", 44100,2);
      byte[] bytes = new byte[]{(byte) 0x12, (byte)0x12};
      ByteBuffer bb = ByteBuffer.wrap(bytes);
      mMediaFormat.setByteBuffer("csd-0", bb);

score 0 · Answer 3 · edited Oct 20 '20 at 08:39

0

I had no issues decoding/playing AAC content on the few devices I tested. My approach was to use the MediaExtractor first to set the Data source, then initialise the MediaFormat and finally do the work in a loop with buffers sent in/out to/from the MediaCodec. For the surface I used null, as this is just an audio player so there is nothing to display.

edited Oct 20 '20 at 08:39

Martijn Pieters

1,048,767
296
4,058
3,343

answered Jun 01 '14 at 18:49

radhoo

2,877
26
27

An example would have been nice :) – Christian Aug 16 '21 at 20:49

Decoding AAC using MediaCodec API on Android

3 Answers3

Linked