How to convert CMSampleBuffer to Data in Swift?

Question

I need to convert CMSampleBuffer to Data format. I am using one Third party framework for audio related task. That framework gives me the streaming (i.e Real Time audio) audio in CMSampleBuffer object.

Like this:

func didAudioStreaming(audioSample: CMSampleBuffer!) {
    //Here I need to conver this to Data format. 
    //Because I am using GRPC framework for Audio Recognization, 
}

Please provide me the steps to convert the CMSampleBuffer to Data.

FYI

    let formatDesc:CMFormatDescription? = CMSampleBufferGetFormatDescription(audioSample)

    <CMAudioFormatDescription 0x17010d890 [0x1b453ebb8]> {
    mediaType:'soun' 
    mediaSubType:'lpcm' 
    mediaSpecific: {
        ASBD: {
            mSampleRate: 16000.000000 
            mFormatID: 'lpcm' 
            mFormatFlags: 0xc 
            mBytesPerPacket: 2 
            mFramesPerPacket: 1 
            mBytesPerFrame: 2 
            mChannelsPerFrame: 1 
            mBitsPerChannel: 16     } 
        cookie: {(null)} 
        ACL: {(null)}
        FormatList Array: {(null)} 
    } 
    extensions: {(null)}
}

May i ask how did you record at a sample rate of 16000?? I have called ```session.setPreferredSampleRate``` but i keep getting the sample rate in 44100 — Ibrahim Yildirim, Sep 21 '17 at 14:22

Nilesh · Answer 1 · 2017-07-12T11:45:27.867

16

Try below code to convert CMSampleBuffer to NSData.

let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
CVPixelBufferLockBaseAddress(imageBuffer!, CVPixelBufferLockFlags(rawValue: 0))
let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer!)
let height = CVPixelBufferGetHeight(imageBuffer!)
let src_buff = CVPixelBufferGetBaseAddress(imageBuffer!)
let data = NSData(bytes: src_buff, length: bytesPerRow * height)
CVPixelBufferUnlockBaseAddress(imageBuffer!, CVPixelBufferLockFlags(rawValue: 0))

EDIT-

For AudioBuffer use below code -

var audioBufferList = AudioBufferList()
var data = Data()
var blockBuffer : CMBlockBuffer?

CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, nil, &audioBufferList, MemoryLayout<AudioBufferList>.size, nil, nil, 0, &blockBuffer)

let buffers = UnsafeBufferPointer<AudioBuffer>(start: &audioBufferList.mBuffers, count: Int(audioBufferList.mNumberBuffers))

for audioBuffer in buffers {
    let frame = audioBuffer.mData?.assumingMemoryBound(to: UInt8.self)
    data.append(frame!, count: Int(audioBuffer.mDataByteSize))
}

edited Jul 12 '17 at 11:45

answered Jul 12 '17 at 10:50

Nilesh

533
7
19

Thanks Nilesh, But I am receiving audio samples not video or image . Will it work for my case? – Sridhar Jul 12 '17 at 10:52
@Sridhar I have edited my answer, please let know whether work for you or not. – Nilesh Jul 12 '17 at 11:46
Thanks Nileash, I used your updated answer, I am getting - Error Domain=NSOSStatusErrorDomain Code=1954115647 "(null)" this error coming from AudioPlayer – Sridhar Jul 12 '17 at 12:07
let formatDesc:CMFormatDescription? = CMSampleBufferGetFormatDescription(audioSample) { mediaType:'soun' mediaSubType:'lpcm' mediaSpecific: { ASBD: { mSampleRate: 48000.000000 mFormatID: 'lpcm' mFormatFlags: 0xc mBytesPerPacket: 4 mFramesPerPacket: 1 mBytesPerFrame: 4 mChannelsPerFrame: 2 mBitsPerChannel: 16 } cookie: {(null)} ACL: {(null)} FormatList Array: {(null)} } extensions: {(null)} – Sridhar Jul 12 '17 at 12:10
The operation couldn’t be completed. (OSStatus error 1954115647.) – Sridhar Jul 12 '17 at 12:17
Where you getting an error, and why are you fetch formatDesc, you should directly pass your data in your audio player like this - let player = try! AVAudioPlayer(data: data) \n player.play() – Nilesh Jul 12 '17 at 12:25
I am getting error at playing the let player = try! AVAudioPlayer(data: data) here – Sridhar Jul 12 '17 at 12:27
Okay, Instead of initializing the audio player with the Data object directly, can you please save the file to the Documents folder, and then initialized the player with the file URL. – Nilesh Jul 12 '17 at 12:36
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/149004/discussion-between-sridhar-and-nilesh). – Sridhar Jul 12 '17 at 12:50
@Sridhar is it resolved? I am facing the same issue. – ossamacpp Feb 04 '19 at 15:27
I'm getting a warning for the let buffers = statement. I receive "Initialization of 'UnsafeBufferPointer' results in a dangling buffer pointer" – Dustin Nielson May 14 '20 at 00:20
I'm having the same problem. – Nickkk Feb 06 '21 at 02:51

score 3 · Answer 2 · answered Apr 09 '19 at 13:58

Using CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer will require to call at some point CFRelease(blockBuffer) because the buffer is retained and if not released the pool of buffers will become eventually empty and no new CMSampleBuffer will be generated.

I'd suggest to get directly the data using the following:

CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
size_t lengthAtOffset;
size_t totalLength;
char *data;
CMBlockBufferGetDataPointer(blockBuffer, 0, &lengthAtOffset, &totalLength, &data);

NSData *audioData = [NSData dataWithBytes:data length:totalLength];

This is objective-C, question was asked for Swift :) You could also add swift version — iSpain17, Aug 15 '19 at 10:51

How to convert CMSampleBuffer to Data in Swift?

2 Answers2

Linked

Related