OS X objective-C app uses excessive memory

Question

I am writing an OS X application that will create a video using a series of images. It was developed using code from here: Make movie file with picture Array and song file, using AVAsset, but not including the audio portion.

The code functions and creates an mpg file.

The problem is the memory pressure. It doesn't appear to free up any memory. Using XCode Instruments I found the biggest culprits are:

CVPixelBufferCreate
[image TIFFRepresentation];
CGImageSourceCreateWithData
CGImageSourceCreateImageAtIndex

I tried adding code to release, but the ARC should already being doing that. Eventually OS X will hang and or crash.

Not sure how to handle the memory issue. There are no mallocs in the code. I'm open to suggestions. It appears that many others have used this same code.

This is the code that is based on the link above:

- (void)ProcessImagesToVideoFile:(NSError **)error_p size:(NSSize)size videoFilePath:(NSString *)videoFilePath jpegs:(NSMutableArray *)jpegs fileLocation:(NSString *)fileLocation
{

    AVAssetWriter *videoWriter = [[AVAssetWriter alloc] initWithURL:
                              [NSURL fileURLWithPath:videoFilePath]
                                                       fileType:AVFileTypeMPEG4
                                                          error:&(*error_p)];
    NSParameterAssert(videoWriter);

    NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                               AVVideoCodecH264, AVVideoCodecKey,
                               [NSNumber numberWithInt:size.width], AVVideoWidthKey,
                               [NSNumber numberWithInt:size.height], AVVideoHeightKey,
                               nil];
     AVAssetWriterInput* videoWriterInput = [AVAssetWriterInput
                                        assetWriterInputWithMediaType:AVMediaTypeVideo
                                        outputSettings:videoSettings];


        AVAssetWriterInputPixelBufferAdaptor *adaptor = [AVAssetWriterInputPixelBufferAdaptor
                                                     assetWriterInputPixelBufferAdaptorWithAssetWriterInput:videoWriterInput
                                                 sourcePixelBufferAttributes:nil];



    NSParameterAssert(videoWriterInput);
    NSParameterAssert([videoWriter canAddInput:videoWriterInput]);

    videoWriterInput.expectsMediaDataInRealTime = YES;

    [videoWriter addInput:videoWriterInput];
    //Start a session:
    [videoWriter startWriting];
    [videoWriter startSessionAtSourceTime:kCMTimeZero];


    CVPixelBufferRef buffer = NULL;

    //Write all picture array in movie file.

    int frameCount = 0;

    for(int i = 0; i<[jpegs count]; i++)
    {
        NSString *filePath = [NSString stringWithFormat:@"%@%@", fileLocation, [jpegs objectAtIndex:i]];
        NSImage *jpegImage = [[NSImage alloc ]initWithContentsOfFile:filePath];
        CMTime frameTime = CMTimeMake(frameCount,(int32_t) 24);

        BOOL append_ok = NO;
        int j = 0;
        while (!append_ok && j < 30)
        {
            if (adaptor.assetWriterInput.readyForMoreMediaData)
            {
                if ((frameCount % 25) == 0)
                {
                    NSLog(@"appending %d to %@ attemp %d\n", frameCount, videoFilePath, j);
                }


                buffer = [self pixelBufferFromCGImage:jpegImage  andSize:size];
                append_ok = [adaptor appendPixelBuffer:buffer withPresentationTime:frameTime];
                if (append_ok == NO) //failes on 3GS, but works on iphone 4
                {
                    NSLog(@"failed to append buffer");
                    NSLog(@"The error is %@", [videoWriter error]);
                }
                //CVPixelBufferPoolRef bufferPool = adaptor.pixelBufferPool;
                //NSParameterAssert(bufferPool != NULL);

                if(buffer)
                {
                    CVPixelBufferRelease(buffer);
                    //CVBufferRelease(buffer);
                }
            }
            else
            {
                printf("adaptor not ready %d, %d\n", frameCount, j);
                [NSThread sleepForTimeInterval:0.1];
            }
            j++;
        }

        if (!append_ok)
        {
            printf("error appending image %d times %d\n", frameCount, j);
        }

        frameCount++;
            //CVBufferRelease(buffer);
        jpegImage = nil;
        buffer = nil;
    }

    //Finish writing picture:
    [videoWriterInput markAsFinished];
    [videoWriter finishWritingWithCompletionHandler:^(){
    NSLog (@"finished writing");

    }];
}

- (CVPixelBufferRef) pixelBufferFromCGImage: (NSImage *) image andSize:(CGSize) size
{
    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                         [NSNumber numberWithBool:YES],     kCVPixelBufferCGImageCompatibilityKey,
                         [NSNumber numberWithBool:YES],     kCVPixelBufferCGBitmapContextCompatibilityKey,
                         nil];
    CVPixelBufferRef pxbuffer = NULL;

    CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault,
                                      size.width,
                                      size.height,
                                      kCVPixelFormatType_32ARGB,
                                      (__bridge CFDictionaryRef) options,
                                      &pxbuffer);

    NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);

    CVPixelBufferLockBaseAddress(pxbuffer, 0);
    void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
    NSParameterAssert(pxdata != NULL);

    CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
    CGContextRef context = CGBitmapContextCreate(pxdata, size.width, size.height,
                                             8, 4*size.width, rgbColorSpace,
                                             kCGImageAlphaPremultipliedFirst);
    NSParameterAssert(context);
    CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));
    CGImageRef imageRef = [self nsImageToCGImageRef:image];
    CGRect imageRect = CGRectMake(0, 0, CGImageGetWidth(imageRef),     CGImageGetHeight(imageRef));
    CGContextDrawImage(context, imageRect, imageRef);


    CGColorSpaceRelease(rgbColorSpace);
    CGContextRelease(context);

    CVPixelBufferUnlockBaseAddress(pxbuffer, 0);
    imageRef = nil;
    context = nil;
    rgbColorSpace = nil;
    return pxbuffer;
}

- (CGImageRef)nsImageToCGImageRef:(NSImage*)image;
{
    NSData * imageData = [image TIFFRepresentation];// memory hog
    CGImageRef imageRef;
    if(!imageData) return nil;
    CGImageSourceRef imageSource = CGImageSourceCreateWithData((__bridge CFDataRef)imageData, NULL);
    imageRef = CGImageSourceCreateImageAtIndex(imageSource, 0, NULL);

    imageData = nil;
    imageSource = nil;
    return imageRef;
}

Answer 1

ARC works only for retainable object pointers. ARC documentation defines them as

A retainable object pointer (or “retainable pointer”) is a value of a retainable object pointer type (“retainable type”). There are three kinds of retainable object pointer types:

1. block pointers (formed by applying the caret (^) declarator sigil to a function type)

2. Objective-C object pointers (id, Class, NSFoo*, etc.)

3. typedefs marked with attribute((NSObject)) Other pointer types, such as int* and CFStringRef, are not subject to ARC’s semantics and restrictions.

You already explicitly call release here

CGContextRelease(context);

You should do the same for other objects. Like

CVPixelBufferRelease(pxbuffer);

for pxbuffer

Answer 2

Your code is using ARC but the libraries you are calling might not be using ARC. They might be relying on the older autorelease pool system to free up memory.

You should have a read how it works, this is fundamental stuff that every Obj-C developer needs to memorise, but basically any object can be added to the current "pool" of objects, which will be released when the pool is released.

By default, the pool on the main thread is emptied each time the app enters an idle state. This usually works fine, since the main thread should never be busy for more than few hundredths of a second and you can't really build up much memory in that amount of time.

When you do a lengthy and memory intensive operation you need to manually setup an autorelease pool, which is most commonly put inside a for or while loop (although you can actually put them anywhere you want, that's just the most useful scenario):

for ( ... ) {
  @autoreleasepool {
    // do somestuff
  }
}

Also, ARC is only for Objective C code. It does not apply to objects created by C functions like CGColorSpaceCreateDeviceRGB() and CVPixelBufferCreate(). Make sure you are manually releasing all of those.