Swift 3 CGImage Memory issue

Question

I have this function in my code in Swift3, that I actually almost literally translated from Apple sample code.

My app processes sampleBuffers that comes from capturing live feed from the camera.

This function correctly create and returns an image from a CMSampleBuffer. It works fine, but the memory keeps growing until the apps crash.

With Instruments I saw that there is some image data that doesn't get released. If I comment the line where I do "context.makeImage", the memory stay down. Reading that func doc, it says that it copies the data from the context. So I'm thinking that there is some data that get copied and one copy is not released.

The 'problem' is that Swift automatically handle the CoreFoundations memory retains/release, so I have no way to handle it.

As you can see I tried with an autoreleasepool, but it's not working.

Any idea?

Thanks

func image(from sampleBuffer:CMSampleBuffer) -> UIImage?{

    /* https://developer.apple.com/library/content/qa/qa1702/_index.html */
    let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)

    guard imageBuffer != nil else{
        return nil
    }

    return autoreleasepool{ () -> UIImage in

        CVPixelBufferLockBaseAddress(imageBuffer!, .readOnly);

        let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer!)

        // Get the number of bytes per row for the pixel buffer
        let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer!);
        // Get the pixel buffer width and height
        let width = CVPixelBufferGetWidth(imageBuffer!);
        let height = CVPixelBufferGetHeight(imageBuffer!);

        let colorSpace = CGColorSpaceCreateDeviceRGB();

        let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)
            .union(.byteOrder32Little)

        // Create a bitmap graphics context with the sample buffer data
        let context = CGContext(data: baseAddress, width: width, height: height, bitsPerComponent: 8,
                                bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo.rawValue);

        // Create a Quartz image from the pixel data in the bitmap graphics context
        let quartzImage = context!.makeImage();
        // Unlock the pixel buffer
        CVPixelBufferUnlockBaseAddress(imageBuffer!,.readOnly);

        let image = UIImage(cgImage: quartzImage!)

        return image
    }
}

Are you 100% sure that the issue is here and not, for example, in your subsequent use of `image`? Sure, the leaked image may be created here, but that doesn't mean the source of the problem is here. Have you use the "debug memory graph" feature of Xcode and looked to see if the `UIImage` is there? That will show you what's retaining `UIImage`. — Rob, Feb 06 '17 at 12:19
Try to inspect you code using Instrument specifically Allocation, as Rob said, maybe the problem is somewhere else. Are you storing the images into an array? — Andrea, Feb 06 '17 at 13:40
@Rob You were right, the code here works just fine. The problem was that I was doing this `tempImageView.layer.render(in: context!)` from a backgroundThread, and UIKit is not thread safe. So that left some memory hanging somewhere. Thanks for pointing me in the right direction. — Axy, Feb 07 '17 at 09:22

score 4 · Accepted Answer · answered Apr 27 '17 at 14:00

4

I found the solution to my problem.

I didn't know that UIKit is not thread safe. It appears that executing that code in a thread different from the main, does not allow the system to free the memory as it should.

On the main thread, the memory is correctly released and managed.

answered Apr 27 '17 at 14:00

Axy

374
2
15

1

what you did exactly? – Pooja M. Bohora Oct 30 '18 at 10:55
I dispatched on the main thread the code that was calling this function and the memory got released correctly. – Axy Oct 30 '18 at 14:07
ok.. but my code is already on main thread :( any other solution – Pooja M. Bohora Oct 31 '18 at 05:14
It may be some other problem, can you post some code? – Axy Nov 07 '18 at 00:10
I'm not an Operating System expert, but I play one on StackOverflow. It appears the main thread clears auto-release heap allocations regularly, where as background threads do not unless you use the autorelease {} closure, or the thread terminates. See my answer below. – Stickley Sep 16 '19 at 15:49

score 3 · Answer 2 · edited Sep 13 '20 at 08:23

Like other Core* libraries which are not ARC compliant, you probably need to wrap the CM call inside the autorelease.

func image(from sampleBuffer:CMSampleBuffer) -> UIImage?{
 

    var ret: UIImage?
    autoreleasepool {

        /* https://developer.apple.com/library/content/qa/qa1702/_index.html */
        let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)


        CVPixelBufferLockBaseAddress(imageBuffer!, .readOnly);

        let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer!)

        // Get the number of bytes per row for the pixel buffer
        let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer!);
        // Get the pixel buffer width and height
        let width = CVPixelBufferGetWidth(imageBuffer!);
        let height = CVPixelBufferGetHeight(imageBuffer!);

        let colorSpace = CGColorSpaceCreateDeviceRGB();

        let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)
            .union(.byteOrder32Little)

        // Create a bitmap graphics context with the sample buffer data
        let context = CGContext(data: baseAddress, width: width, height: height, bitsPerComponent: 8,
                                bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo.rawValue);

        // Create a Quartz image from the pixel data in the bitmap graphics context
        let quartzImage = context!.makeImage();
        // Unlock the pixel buffer
        CVPixelBufferUnlockBaseAddress(imageBuffer!,.readOnly);

        let image = UIImage(cgImage: quartzImage!)

        ret = image
    }
    return ret
}

See this answer for details.

Nice, had a huge memoryleak due to using CGContext and CGImage and related classes on a background thread loop. You can simplify it by writing `return autoreleasepool { ... }` — Peterdk, Feb 13 '20 at 17:20

Swift 3 CGImage Memory issue

2 Answers2