How to manually release CMSampleBuffer

Question

This code leads to memory leak and app crash:

    var outputSamples = [Float]()

    assetReader.startReading()
    while assetReader.status == .reading {
        let trackOutput = assetReader.outputs.first!

        if let sampleBuffer = trackOutput.copyNextSampleBuffer(),
            let blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) {
            let blockBufferLength = CMBlockBufferGetDataLength(blockBuffer)
            let sampleLength = CMSampleBufferGetNumSamples(sampleBuffer) * channelCount(from: assetReader)
            var data = Data(capacity: blockBufferLength)
            data.withUnsafeMutableBytes { (blockSamples: UnsafeMutablePointer<Int16>) in
                CMBlockBufferCopyDataBytes(blockBuffer, atOffset: 0, dataLength: blockBufferLength, destination: blockSamples)
                CMSampleBufferInvalidate(sampleBuffer)

                let processedSamples = process(blockSamples,
                                               ofLength: sampleLength,
                                               from: assetReader,
                                               downsampledTo: targetSampleCount)
                outputSamples += processedSamples
            }
        }
    }
    var paddedSamples = [Float](repeating: silenceDbThreshold, count: targetSampleCount)
    paddedSamples.replaceSubrange(0..<min(targetSampleCount, outputSamples.count), with: outputSamples)

This is due to copyNextSampleBuffer() and The Create Rule.

In turn, we can not use CFRelease() in Swift. The reason why a link to the Objective-C only rule is there is beyond my understanding.

Is there a way to release CMSampleBuffer manually in Swift?

let sampleBuffer: CMSampleBuffer? = trackOutput.copyNextSampleBuffer(), if let smBuf = sampleBuffer { // Do operations } sampleBuffer = nil — Sachin Vas, Feb 27 '19 at 11:18
@SachinVas You have successfully set sampleBuffer to nil with your code. How do you release smBuf now? :) — DmitryoN, Feb 27 '19 at 11:49
The core foundation types are memory managed automatically in Swift. https://developer.apple.com/documentation/swift/imported_c_and_objective-c_apis/working_with_core_foundation_types — Sachin Vas, Mar 01 '19 at 04:26
@SachinVas However reading 13 minute long audio file fills the memory with 500MB after which the app crashes. CoreMedia library allocates memory without releasing it. — DmitryoN, Mar 01 '19 at 05:07
@SachinVas I ran the code you've suggested yesterday, there was no change at all. — DmitryoN, Mar 01 '19 at 05:20
Now I'm leaning towards the problem is not with the buffer, but with the ```assetReader.startReading()``` line. — DmitryoN, Mar 01 '19 at 05:22
@SachinVas Invalidation is in my original code, you can see it. — DmitryoN, Mar 01 '19 at 05:23

score 4 · Answer 1 · answered Jun 01 '20 at 20:16

I recently solved a similar issue by using an autoreleasepool

Try wrapping the area where sampleBuffer is used in an autoreleasepool. Something like this:

var outputSamples = [Float]()

assetReader.startReading()
while assetReader.status == .reading {
    let trackOutput = assetReader.outputs.first!

    autoreleasepool {
        if let sampleBuffer = trackOutput.copyNextSampleBuffer(),
            let blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) {
            let blockBufferLength = CMBlockBufferGetDataLength(blockBuffer)
            let sampleLength = CMSampleBufferGetNumSamples(sampleBuffer) * channelCount(from: assetReader)
            var data = Data(capacity: blockBufferLength)
            data.withUnsafeMutableBytes { (blockSamples: UnsafeMutablePointer<Int16>) in
                CMBlockBufferCopyDataBytes(blockBuffer, atOffset: 0, dataLength: blockBufferLength, destination: blockSamples)
                CMSampleBufferInvalidate(sampleBuffer)

                let processedSamples = process(blockSamples,
                                               ofLength: sampleLength,
                                               from: assetReader,
                                               downsampledTo: targetSampleCount)
                outputSamples += processedSamples
            }
        }
    }
}
var paddedSamples = [Float](repeating: silenceDbThreshold, count: targetSampleCount)
paddedSamples.replaceSubrange(0..<min(targetSampleCount, outputSamples.count), with: outputSamples)

If I understand correctly, once it moves out of the scope of autoreleasepool, the sampleBuffer will be released

score 1 · Accepted Answer · answered Mar 05 '19 at 10:11

This is not really a solution, because it seems that releasing memory manually is impossible and using while loop in conjunction with assetReader results in memory not being released when unsafe mutable bytes are read.

The problem was solved by a workaround: converting the audio file into CAF format before exposing it to the while loop.

Downside: it takes a hot second, the longer the audio file - the more time it takes.

Upside: it only used minuscule amount of memory, which was the problem in the first place.

Inspired by: https://stackoverflow.com/users/2907715/carpsen90 answer in Extract meter levels from audio file

How to manually release CMSampleBuffer

2 Answers2