Capturing audio data and save to raw PCM file in AudioFlinger

Question

After some research, I find out this is possible to capture the audio data in libaudioflinger of Android.

I think the audio data is being written to HAL in here:

ssize_t framesWritten = mNormalSink->write((char *)mSinkBuffer + offset, count);

Full code: https://android.googlesource.com/platform/frameworks/av/+/lollipop-release/services/audioflinger/Threads.cpp#2118

So, I would like to save the mSinkBuffer + offset to a file (which I expected it will be raw PCM audio file). I using those stream to write it to file:

std::ofstream audioData ("/data/audiodata.raw", std::fstream::app);
audioData.write((char *)mSinkBuffer + offset, count);
audioData.close();

The file is successfully written and it has data in it. But, when I play the PCM file (audiodata.raw) with aplay or ffplay, the only sound I got is noise.

aplay -t raw -c 2 -f S16_LE -r 48000 audiodata.raw

I was worry about the config of aplay. So I print some log of the libaudioflinger:

10-07 10:14:54.575  1300  1366 I AudioFlinger: I/O handle: 13
10-07 10:14:54.575  1300  1366 I AudioFlinger: Standby: no
10-07 10:14:54.575  1300  1366 I AudioFlinger: Sample rate: 48000 Hz
10-07 10:14:54.575  1300  1366 I AudioFlinger: HAL frame count: 512
10-07 10:14:54.575  1300  1366 I AudioFlinger: HAL format: 0x1 (AUDIO_FORMAT_PCM_16_BIT)
10-07 10:14:54.575  1300  1366 I AudioFlinger: HAL buffer size: 2048 bytes
10-07 10:14:54.575  1300  1366 I AudioFlinger: Channel count: 2
10-07 10:14:54.575  1300  1366 I AudioFlinger: Channel mask: 0x00000003 (front-left, front-right)
10-07 10:14:54.575  1300  1366 I AudioFlinger: Processing format: 0x5 (AUDIO_FORMAT_PCM_FLOAT)
10-07 10:14:54.576  1300  1366 I AudioFlinger: Processing frame size: 8 bytes
10-07 10:14:54.576  1300  1366 I AudioFlinger: Pending config events:
10-07 10:14:54.576  1300  1366 I AudioFlinger:  none
10-07 10:14:54.576  1300  1366 I AudioFlinger: Output device: 0x2 (AUDIO_DEVICE_OUT_SPEAKER)
10-07 10:14:54.576  1300  1366 I AudioFlinger: Input device: 0 (AUDIO_DEVICE_NONE)
10-07 10:14:54.576  1300  1366 I AudioFlinger: Audio source: 0 (default)

I don't know what I did wrong. Please help me!

Thank you in advanced!

mail2subhajit · Accepted Answer · 2019-10-24T19:35:21.927

1

Open The file in append |binary mode

std::ofstream audioData ("/data/audiodata.raw", std::fstream::app | std::fstream::binary);

binary - binary - Operations are performed in binary mode rather than text.

Raw PCM buffer should be written in binary mode.

Please check the code for the difference between the bytes and count ( added some comments for the reference , I hope this will solve your problem)

if (mNormalSink != 0) {

    /* Count is the number of Frames or sample written != bytes  */
    const size_t count = mBytesRemaining / mFrameSize;

    ATRACE_BEGIN("write");
    // update the setpoint when AudioFlinger::mScreenState changes
    uint32_t screenState = AudioFlinger::mScreenState;
    if (screenState != mScreenState) {
        mScreenState = screenState;
        MonoPipe *pipe = (MonoPipe *)mPipeSink.get();
        if (pipe != NULL) {
            pipe->setAvgFrames((mScreenState & 1) ?
                    (pipe->maxFrames() * 7) / 8 : mNormalFrameCount * 2);
        }
    }
    ssize_t framesWritten = mNormalSink->write((char *)mSinkBuffer + offset, count);

    ATRACE_END();
    if (framesWritten > 0) {
        bytesWritten = framesWritten * mFrameSize;
        // std::ofstream audioData ("/data/audiodata.raw", std::fstream::binary);
        /* File write or stream write is the number of bytes written to the file */
        audioData.write((char *)mSinkBuffer + offset, bytesWritten);
        // audioData.close();
    } else {
        bytesWritten = framesWritten;
    }
// otherwise use the HAL / AudioStreamOut directly
}

Audacity - Open a raw file

File -> Import -> raw data

Select the raw file path

Based on your raw file attached.

Use these Setting

Encoding : 32 -bit float

Byte order: Little Endian

Channels : 2 channels ( stereo)

Start offset : 0

Amount to import : 100

Sample Rate : 48000

File plays properly without any glitches/noise.

edited Oct 24 '19 at 19:35

answered Oct 10 '19 at 20:39

mail2subhajit

1,106
5
16

Thanks for your comment! Unfortunately, it's still contains only noise when I tried to play it. Do you have any other idea? – Banana droid Oct 11 '19 at 08:12
Can you open the raw file in audacity (import option) and check if the wave form or audio data is present. What is the count size you are getting for each execution? – mail2subhajit Oct 11 '19 at 10:57
I can only Import > Raw Data. And if I use std::fstream::binary and then I can't hear anything but noise. If I use only std::fstream::app, I can hear some of the music with noise. – Banana droid Oct 11 '19 at 13:41
and, the count is 1024 – Banana droid Oct 11 '19 at 13:49
Can you try only binary mode , open the file 1 time at beginning and then close it at the end of your program. you don't need to use append mode . – mail2subhajit Oct 11 '19 at 16:31
audioflinger is about to write the data to HAL again and again with different data, like a stream. So I think I can't open it just one time and close it. If I do that, it will contain very less data (maybe only a pulse, I think) – Banana droid Oct 12 '19 at 03:44
No audio don't work that way, there will be a stream of buffer that will be submitted to the Audio driver continuously. ( 1024 is very few millisecond data ) So try and open the file once and then close it once done . – mail2subhajit Oct 12 '19 at 17:28
you are right. It's a stream of buffer. Please take a look at this function: https://android.googlesource.com/platform/frameworks/av/+/lollipop-release/services/audioflinger/Threads.cpp#2094. This function is continuously called to submit that stream of buffer to the audio driver. So I really don't know when to open and when to close the file.. – Banana droid Oct 13 '19 at 17:30
You can check these option : open the raw pcm file in AudioFlinger::PlaybackThread::createTrack_l, close the raw pcm file in AudioFlinger::PlaybackThread::destroyTrack_l, Write the audio buffer in AudioFlinger::PlaybackThread::threadLoop_write. – mail2subhajit Oct 14 '19 at 23:40
Thanks for your dedicated support! I'm really really appreciate it!. Unfortunately, I modified as your suggestion but the output still contains only noise. Code: https://gist.github.com/namchuai/e782ec8006fc0bc79b774a1d2e34cab5 I think I should upload the raw PCM I recorded. Here's the link: https://github.com/namchuai/stock-scraping/blob/master/audiodata.raw – Banana droid Oct 15 '19 at 17:23
Thx for sharing the code, this helps - make the changes as update above. – mail2subhajit Oct 15 '19 at 19:16
I tried your code: audioData.write((char *)mSinkBuffer + offset, (count*mFrameSize)); But the output is quite same, I think. Only noise. – Banana droid Oct 15 '19 at 19:41
I hope the mSinkbuffer is having the proper data pointer, please check again, made some more changes. – mail2subhajit Oct 15 '19 at 20:03
Did you check with these changes once .. let me know. – mail2subhajit Oct 16 '19 at 18:55
Sorry for took you so long. I tried with your but still contains noise. I uploaded it to https://github.com/namchuai/stock-scraping/blob/master/audiodata.raw. I tried with audacity and change some configuration like channel, sample rate but no luck. – Banana droid Oct 18 '19 at 09:34
FYI, framesWritten = 1024 and mFrameSize = 8 – Banana droid Oct 18 '19 at 09:36
do you have any other idea? If yes, please let me know :( – Banana droid Oct 21 '19 at 03:24
I placed some logs and see that threadloop_write() is called by multiple thread.. I think it might be the cause. – Banana droid Oct 21 '19 at 06:03
Can you share the log path and the modified file. – mail2subhajit Oct 23 '19 at 20:40
I modified as your change on Oct 15 at 20:02. I can hear the sound with Audacity now. But it still contains so much noise and only right channel is working. Left channel is only noise. I played the raw file in audacity with Signed 16-bit PCM, No endianess, 2 channesl, 96000Hz. The file is located at: https://drive.google.com/file/d/1PIT_HNxjN9Y739Jmygm_UTiDbNFLXPfG/view – Banana droid Oct 24 '19 at 09:29
Play with the setting I have added in my answer edited . – mail2subhajit Oct 24 '19 at 19:36
I have no word to say! you are my hero now. I'm really appreciated your support! Thank you very much! – Banana droid Oct 25 '19 at 02:35
Thanks for the kind words.. Just out of curiosity, want to know what are you developing and what is your final product. – mail2subhajit Oct 25 '19 at 12:28
I'm capturing the android's audio output and streaming it to another android device :). just a small proof of concept project – Banana droid Oct 25 '19 at 14:16

score 0 · Answer 2 · answered Oct 10 '19 at 09:31

0

When you have to play your raw file you must use a codec format for the sound frames,such as wav or mp3 etc. Try to pass the raw data to a wav WAVEFORMAT defined in winmm and get audio from the devices that can be opened and closed via the WAVEFORMAT types in C++.

answered Oct 10 '19 at 09:31

SteveTheGrk

352
1
7

actually, the raw file is contains only audio data. That's why I have to specify the sampling rate, channel, etc to the aplay for it to play the file. Unlike WAV, which has a header define those needed information. – Banana droid Oct 10 '19 at 15:21

Capturing audio data and save to raw PCM file in AudioFlinger

2 Answers2