Reading audio samples via AVAssetReader

Question

How do you read audio samples via AVAssetReader? I've found examples of duplicating or mixing using AVAssetReader, but those loops are always controlled by the AVAssetWriter loop. Is it possible just to create an AVAssetReader and read through it, getting each sample and throwing the int32 of each audio sample into an array?

Thanks.

score 27 · Accepted Answer · edited May 12 '11 at 13:09

27

To expand on @amrox's answer, you can get an AudioBufferList from the CMBlockBufferRef, e.g.

CMItemCount numSamplesInBuffer = CMSampleBufferGetNumSamples(buffer);

AudioBufferList audioBufferList;

CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
      buffer,
      NULL,
      &audioBufferList,
      sizeof(audioBufferList),
      NULL,
      NULL,
      kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
      &buffer
    );

for (int bufferCount=0; bufferCount < audioBufferList.mNumberBuffers; bufferCount++) {
  SInt16* samples = (SInt16 *)audioBufferList.mBuffers[bufferCount].mData;
  for (int i=0; i < numSamplesInBuffer; i++) {
    // amplitude for the sample is samples[i], assuming you have linear pcm to start with
  }
}

//Release the buffer when done with the samples 
//(retained by CMSampleBufferGetAudioBufferListWithRetainedblockBuffer)
CFRelease(buffer);

edited May 12 '11 at 13:09

badgerr

7,802
2
28
43

answered Feb 17 '11 at 16:40

Roger

563
5
12

Thanks so much! That was the part that I was missing. – Eric Christensen Feb 20 '11 at 05:49
When I use this method my sound gets choppy, when I use @amrox's method, which uses a char*, everything works fine when altering the volume by doing something like `samples[i] *= 0.5;`Any reason for this? – Dex Mar 22 '12 at 00:51
Is it possible to get the samples and then write it into another audio file? I'm trying to reverse an audio file. Please take a look at [my question](http://stackoverflow.com/questions/41310944/reverse-an-audio-file-swift-objective-c). Any help would be appreciated. – Chan Jing Hong Dec 28 '16 at 07:27
but then how do you play that audio from the speakers ? – omarojo Jul 31 '17 at 23:17
1

Swift 4 I get the error "Type 'AudioBuffer' has no subscript members" when declaring the variable "samples" in the for loop. – JCutting8 Aug 10 '18 at 15:28

score 18 · Answer 2 · edited Jan 13 '17 at 22:33

AVAssetReader *reader   = [[AVAssetReader alloc] initWithAsset:asset error:&error];
AVAssetTrack  *track    = [[asset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
NSDictionary  *settings = @{ AVFormatIDKey : [NSNumber numberWithInt:kAudioFormatLinearPCM] };

AVAssetReaderTrackOutput *readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track
                                                                                    outputSettings:settings];

[reader addOutput:readerOutput]; 
[reader startReading];

CMSampleBufferRef sample = [readerOutput copyNextSampleBuffer];

while ( sample )
{
   sample = [readerOutput copyNextSampleBuffer];

    if ( ! sample )
    {
       continue;
    }

    CMBlockBufferRef buffer = CMSampleBufferGetDataBuffer(sample);

    size_t  lengthAtOffset;
    size_t  totalLength;
    char   *data;

    if ( CMBlockBufferGetDataPointer( buffer, 0, &lengthAtOffset, &totalLength, &data ) != noErr )
    {
        NSLog(@"error!");
        break;
    }

    // do something with data...

    CFRelease(sample);
}

Thanks @amrox. This helps some. But I think it's missing adding the output and starting the reading. That can just be added with: [reader addOutput:readerOutput]; [reader startReading]; after you create readerOutput, right? Once I add that, this doesn't read through a whole audio file, but only once or twice. Is that how it's supposed to work? If so, I don't understand how this gets me to the individual samples. Each sample in an audio file is an int, with the amount and range defined by the bit depth and sample rate. How can I get to those ints? Thanks again. — Eric Christensen, Feb 15 '11 at 05:43
They're in that char* data. Set more keys in the settings dictionary if you want more control: AVNumberOfChannelsKey, AVLinearPCMBitDepthKey. And don't forget to award the bounty to amrox. — Rhythmic Fistman, Feb 18 '11 at 21:57

score 7 · Answer 3 · answered Mar 27 '15 at 22:23

The answers here are not generic. The call to CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer might fail when the AudioBufferList needs to be sized differently. When having non-intererleaved samples as example.

The correct way is to call CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer twice. The first call queries the size needed for the AudioBufferList and second one actually fills the AudioBufferList.

size_t bufferSize = 0;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
    sampleBuffer,
    &bufferSize,
    NULL,
    0,
    NULL,
    NULL,
    0,
    NULL
);

AudioBufferList *bufferList = malloc(bufferSize);
CMBlockBufferRef blockBuffer = NULL;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
    sampleBuffer,
    NULL,
    bufferList,
    bufferSize,
    NULL,
    NULL,
    kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
    &blockBuffer
);

// handle audio here

free(bufferList);
CFRelease(blockBuffer);

In a real world example you must perform error handling and also you should not malloc every frame, instead cache the AudioBufferList.

Reading audio samples via AVAssetReader

3 Answers3

Linked