Android MediaExtractor and mp3 stream

Question

I am trying to play an mp3 stream using MediaExtractor/MediaCodec. MediaPlayer is out of the question due to latency and long buffer size.

The only sample code i have found is this: http://dpsm.wordpress.com/category/android/

The code samples are only parcial (?) and use a File instead of a stream.

I have been trying to adapt this example to play an Audio Stream but i can't get my head around how this is supposed to work. The Android documentation as usual is no help.

I understand that first we get information about the stream, presumably setup the AudioTrack with this information ( code sample does include AudioTrack initialization ?) and then open an input buffer and output buffer.

I have recreated code for this, with what i can guess would be the missing parts, but no audio comes out of this.

Can someone point me in the right direction to understand how this is supposed to work?

public final String LOG_TAG = "mediadecoderexample";
private static int TIMEOUT_US = -1;
MediaCodec codec;
MediaExtractor extractor;

MediaFormat format;
ByteBuffer[] codecInputBuffers;
ByteBuffer[] codecOutputBuffers;
Boolean sawInputEOS = false;
Boolean sawOutputEOS = false;
AudioTrack mAudioTrack;
BufferInfo info;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    String url = "http://82.201.100.9:8000/RADIO538_WEB_MP3";
    extractor = new MediaExtractor();

    try {
        extractor.setDataSource(url);
    } catch (IOException e) {
    }

    format = extractor.getTrackFormat(0);
    String mime = format.getString(MediaFormat.KEY_MIME);
    int sampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);

    Log.i(LOG_TAG, "===========================");
    Log.i(LOG_TAG, "url "+url);
    Log.i(LOG_TAG, "mime type : "+mime);
    Log.i(LOG_TAG, "sample rate : "+sampleRate);
    Log.i(LOG_TAG, "===========================");

    codec = MediaCodec.createDecoderByType(mime);
    codec.configure(format, null , null , 0);
    codec.start();

    codecInputBuffers = codec.getInputBuffers();
    codecOutputBuffers = codec.getOutputBuffers();

    extractor.selectTrack(0); 

    mAudioTrack = new AudioTrack(
            AudioManager.STREAM_MUSIC, 
            sampleRate, 
            AudioFormat.CHANNEL_OUT_STEREO, 
            AudioFormat.ENCODING_PCM_16BIT, 
            AudioTrack.getMinBufferSize (
                    sampleRate, 
                    AudioFormat.CHANNEL_OUT_STEREO, 
                    AudioFormat.ENCODING_PCM_16BIT
                    ), 
            AudioTrack.MODE_STREAM
            );

    info = new BufferInfo();


    input();
    output();


}

private void output()
{
    final int res = codec.dequeueOutputBuffer(info, TIMEOUT_US);
    if (res >= 0) {
        int outputBufIndex = res;
        ByteBuffer buf = codecOutputBuffers[outputBufIndex];

        final byte[] chunk = new byte[info.size];
        buf.get(chunk); // Read the buffer all at once
        buf.clear(); // ** MUST DO!!! OTHERWISE THE NEXT TIME YOU GET THIS SAME BUFFER BAD THINGS WILL HAPPEN

        if (chunk.length > 0) {
            mAudioTrack.write(chunk, 0, chunk.length);
        }
        codec.releaseOutputBuffer(outputBufIndex, false /* render */);

        if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
            sawOutputEOS = true;
        }
    } else if (res == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
        codecOutputBuffers = codec.getOutputBuffers();
    } else if (res == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
        final MediaFormat oformat = codec.getOutputFormat();
        Log.d(LOG_TAG, "Output format has changed to " + oformat);
        mAudioTrack.setPlaybackRate(oformat.getInteger(MediaFormat.KEY_SAMPLE_RATE));
    }

}

private void input()
{
    Log.i(LOG_TAG, "inputLoop()");
    int inputBufIndex = codec.dequeueInputBuffer(TIMEOUT_US);
    Log.i(LOG_TAG, "inputBufIndex : "+inputBufIndex);

    if (inputBufIndex >= 0) {   
        ByteBuffer dstBuf = codecInputBuffers[inputBufIndex];

        int sampleSize = extractor.readSampleData(dstBuf, 0);
        Log.i(LOG_TAG, "sampleSize : "+sampleSize);
        long presentationTimeUs = 0;
        if (sampleSize < 0) {
            Log.i(LOG_TAG, "Saw input end of stream!");
            sawInputEOS = true;
            sampleSize = 0;
        } else {
            presentationTimeUs = extractor.getSampleTime();
            Log.i(LOG_TAG, "presentationTimeUs "+presentationTimeUs);
        }

        codec.queueInputBuffer(inputBufIndex,
                               0, //offset
                               sampleSize,
                               presentationTimeUs,
                               sawInputEOS ? MediaCodec.BUFFER_FLAG_END_OF_STREAM : 0);
        if (!sawInputEOS) {
            Log.i(LOG_TAG, "extractor.advance()");
            extractor.advance();

        }
     }

}
}

Edit: adding logcat output for extra ideas.

03-10 16:47:54.115: I/mediadecoderexample(24643): ===========================
03-10 16:47:54.115: I/mediadecoderexample(24643): url ....
03-10 16:47:54.115: I/mediadecoderexample(24643): mime type : audio/mpeg
03-10 16:47:54.115: I/mediadecoderexample(24643): sample rate : 32000
03-10 16:47:54.115: I/mediadecoderexample(24643): ===========================
03-10 16:47:54.120: I/OMXClient(24643): Using client-side OMX mux.
03-10 16:47:54.150: I/Reverb(24643):  getpid() 24643, IPCThreadState::self()->getCallingPid() 24643
03-10 16:47:54.150: I/mediadecoderexample(24643): inputLoop()
03-10 16:47:54.155: I/mediadecoderexample(24643): inputBufIndex : 0
03-10 16:47:54.155: I/mediadecoderexample(24643): sampleSize : 432
03-10 16:47:54.155: I/mediadecoderexample(24643): presentationTimeUs 0
03-10 16:47:54.155: I/mediadecoderexample(24643): extractor.advance()
03-10 16:47:59.085: D/HTTPBase(24643): [2] Network BandWidth = 187 Kbps
03-10 16:47:59.085: D/NuCachedSource2(24643): Remaining (64K), HighWaterThreshold (20480K)
03-10 16:48:04.635: D/HTTPBase(24643): [3] Network BandWidth = 141 Kbps
03-10 16:48:04.635: D/NuCachedSource2(24643): Remaining (128K), HighWaterThreshold (20480K)
03-10 16:48:09.930: D/HTTPBase(24643): [4] Network BandWidth = 127 Kbps
03-10 16:48:09.930: D/NuCachedSource2(24643): Remaining (192K), HighWaterThreshold (20480K)
03-10 16:48:15.255: D/HTTPBase(24643): [5] Network BandWidth = 120 Kbps
03-10 16:48:15.255: D/NuCachedSource2(24643): Remaining (256K), HighWaterThreshold (20480K)
03-10 16:48:20.775: D/HTTPBase(24643): [6] Network BandWidth = 115 Kbps
03-10 16:48:20.775: D/NuCachedSource2(24643): Remaining (320K), HighWaterThreshold (20480K)
03-10 16:48:26.510: D/HTTPBase(24643): [7] Network BandWidth = 111 Kbps
03-10 16:48:26.510: D/NuCachedSource2(24643): Remaining (384K), HighWaterThreshold (20480K)
03-10 16:48:31.740: D/HTTPBase(24643): [8] Network BandWidth = 109 Kbps
03-10 16:48:31.740: D/NuCachedSource2(24643): Remaining (448K), HighWaterThreshold (20480K)
03-10 16:48:37.260: D/HTTPBase(24643): [9] Network BandWidth = 107 Kbps
03-10 16:48:37.260: D/NuCachedSource2(24643): Remaining (512K), HighWaterThreshold (20480K)
03-10 16:48:42.620: D/HTTPBase(24643): [10] Network BandWidth = 106 Kbps
03-10 16:48:42.620: D/NuCachedSource2(24643): Remaining (576K), HighWaterThreshold (20480K)
03-10 16:48:48.295: D/HTTPBase(24643): [11] Network BandWidth = 105 Kbps
03-10 16:48:48.295: D/NuCachedSource2(24643): Remaining (640K), HighWaterThreshold (20480K)
03-10 16:48:53.735: D/HTTPBase(24643): [12] Network BandWidth = 104 Kbps
03-10 16:48:53.735: D/NuCachedSource2(24643): Remaining (704K), HighWaterThreshold (20480K)
03-10 16:48:59.115: D/HTTPBase(24643): [13] Network BandWidth = 103 Kbps
03-10 16:48:59.115: D/NuCachedSource2(24643): Remaining (768K), HighWaterThreshold (20480K)
03-10 16:49:04.480: D/HTTPBase(24643): [14] Network BandWidth = 103 Kbps
03-10 16:49:04.480: D/NuCachedSource2(24643): Remaining (832K), HighWaterThreshold (20480K)
03-10 16:49:09.955: D/HTTPBase(24643): [15] Network BandWidth = 102 Kbps

FWIW, additional examples can be found at http://bigflake.com/mediacodec/, but they're primarily for video and are only really useful to the extent that the MediaCodec API is the same. `DecoderTest` is for audio. Do you see anything "interesting" in logcat? — fadden, Mar 10 '14 at 15:15
I have also seen those examples but i think that the subject is too complicated for me to figure out how to "port them" to my specific scenario. The logcat (ill add it to the question) doesn't out anything too interesting and the phone just seems to 'hang', i can't event turn volume up or down with the side buttons. The app doesn't crash though — Juan Carlos Ospina Gonzalez, Mar 10 '14 at 15:49
there is the openmxplayer for audio, released as opensource. — radhoo, Aug 22 '14 at 13:32
Thanks, it seems one [needs](https://github.com/google/ExoPlayer/issues/1086) a custom AssetDataSource-like wrapper for the InputStream. Is that how it's done? — rraallvv, Apr 20 '18 at 17:58

score 4 · Answer 1 · answered Sep 02 '14 at 07:54

4

For anyone still looking for an answer to the problem of playing streamin audio reliably, you might want to have a look at this project (based on the MediaCodec API)

https://code.google.com/p/android-openmxplayer/

answered Sep 02 '14 at 07:54

Juan Carlos Ospina Gonzalez

4,153
3
34
43

This code also does not deal with network streams on many devices due to . – osxdirk Mar 23 '17 at 17:02

fadden · Accepted Answer · 2014-03-10T17:14:26.223

The code in onCreate() suggests you have a misconception about how MediaCodec works. Your code is currently:

onCreate() {
    ...setup...
    input();
    output();
}

MediaCodec operates on access units. For video, each call to input/output would get you a single frame of video. I haven't worked with audio, but my understanding is that it behaves similarly. You don't get the entire file loaded into an input buffer, and it doesn't play the stream for you; you take one small piece of the file, hand it to the decoder, and it hands back decoded data (e.g. a YUV video buffer or PCM audio data). You then do whatever is necessary to play that data.

So your example would, at best, decode a fraction of a second of audio. You need to be doing submit-input-get-output in a loop with proper handling of end-of-stream. You can see this done for video in the various bigflake examples. It looks like your code has the necessary pieces.

You're using a timeout of -1 (infinite), so you're going to supply one buffer of input and wait forever for a buffer of output. In video this wouldn't work -- the decoders I've tested seem to want about four buffers of input before they'll produce any output -- but again I haven't worked with audio, so I'm not sure if this is expected to work. Since your code is hanging I'm guessing it's not. It might be useful to change the timeout to (say) 10000 and see if the hang goes away.

I'm assuming this is an experiment and you're not really going to do all this in onCreate(). :-)

Thank you, yes i am very confused. The documentation is minimal and the examples apply to video usually (not audio). This is very helpful. — Juan Carlos Ospina Gonzalez, Mar 11 '14 at 08:18
I tried modifying OPs code to using a do/while: ```do { input(); output(); } while (!sawInputEOS);``` but I still don't get sound... — IcedDante, Mar 03 '17 at 19:04
Hello! I have similar question, but still there is difference https://stackoverflow.com/q/57956915/5709159 , could you please tell me what you think about it? — Sirop4ik, Sep 16 '19 at 12:29

IcedDante · Answer 3 · 2017-12-18T17:35:43.607

2

There are two issues with the above code. First, as the accepted answer states, only one read is done from the input stream. However, secondly, a call to .play() is needed on the AudioTrack.

This modification fixes OPs code:

mAudioTrack.play();

do {
    input();
    output();
} while (!sawInputEOS);

edited Dec 18 '17 at 17:35

answered Mar 05 '17 at 03:32

IcedDante

6,145
12
57
100

Android MediaExtractor and mp3 stream

3 Answers3

Linked