How to properly release an AVCaptureSession

Question

I'm using the AVFoundation classes to capture the live video stream from the camera and to process the video samples. This works nicely. However, I do have problems properly releasing the AVFoundation instances (capture session, preview layer, input and output) once I'm done.

When I no longer need the session and all associated objects, I stop the capture session and release it. This works most of the time. However, sometimes the app crashes with an EXEC_BAD_ACCESS signal raised in the second thread that was created by the dispatch queue (and where the video samples are processed). The crash is mainly due to my own class instance, which serves as the sample buffer delegate and is freed after I've stopped the capture session.

The Apple documentation mentions the problem: Stopping the capture session is an asynchronous operation. That is: it doesn't happen immediately. In particular, the second thread continues to process video samples and access different instances like the capture session or the input and output devices.

So how do I properly release the AVCaptureSession and all related instances? Is there a notification that reliably tells me that the AVCaptureSession has finished?

Here's my code:

Declarations:

AVCaptureSession* session;
AVCaptureVideoPreviewLayer* previewLayer;
UIView* view;

Setup of instances:

AVCaptureDevice* camera = [AVCaptureDevice defaultDeviceWithMediaType: AVMediaTypeVideo];
session = [[AVCaptureSession alloc] init];

AVCaptureDeviceInput* input = [AVCaptureDeviceInput deviceInputWithDevice: camera error: &error];
[session addInput: input];
AVCaptureVideoDataOutput* output = [[[AVCaptureVideoDataOutput alloc] init] autorelease];
[session addOutput: output];

dispatch_queue_t queue = dispatch_queue_create("augm_reality", NULL);
[output setSampleBufferDelegate: self queue: queue];
dispatch_release(queue);

previewLayer = [[AVCaptureVideoPreviewLayer layerWithSession: session] retain];
previewLayer.frame = view.bounds;
[view.layer addSublayer: previewLayer];

[session startRunning];

Cleanup:

[previewLayer removeFromSuperlayer];
[previewLayer release];
[session stopRunning];
[session release];

Answer 1

Here's the best solution I've found so far. The basic idea is to use the finalizer of the dispatch queue. When the dispatch queue quits, we can be sure that there won't be any more action in the second thread where the sample buffers are processed.

static void capture_cleanup(void* p)
{
    AugmReality* ar = (AugmReality *)p; // cast to original context instance
    [ar release];  // releases capture session if dealloc is called
}

...

dispatch_queue_t queue = dispatch_queue_create("augm_reality", NULL);
dispatch_set_context(queue, self);
dispatch_set_finalizer_f(queue, capture_cleanup);
[output setSampleBufferDelegate: self queue: queue];
dispatch_release(queue);
[self retain];

...

Unfortunately, I now have to explicitly stop capturing. Otherwise releasing my instance won't free it because the second thread now increments and decrements the counter as well.

A further problem is that my class is now released from two different threads. Is this reliable or is it the next problem causing crashes?

Answer 2

I've posted a very similar question in the Apple Developer Forum and got an answer from an Apple employee. He says it's a known problem:

This is a problem with the AVCaptureSession / VideoDataOutput in iOS 4.0-4.1 that has been fixed and will appear in a future update. For the time being, you can work around it by waiting for a short period after stopping the AVCaptureSession, e.g. half a second, before disposing of the session and data output.

He/she proposes the following code:

dispatch_after(
    dispatch_time(0, 500000000),
    dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), // or main queue, or your own
    ^{
        // Do your work here.
        [session release];
        // etc.
    }
);

I still like the approach with the dispatch queue finalizer better because this code just guesses when the second thread might have finished.

Answer 3

As per current apple docs(1) [AVCaptureSession stopRunning] is a synchronous operation which blocks until the receiver has completely stopped running. So all these issues shouldn't happen any more.

Answer 4

 -(void)deallocSession
{
[captureVideoPreviewLayer removeFromSuperlayer];
for(AVCaptureInput *input1 in session.inputs) {
    [session removeInput:input1];
}

for(AVCaptureOutput *output1 in session.outputs) {
    [session removeOutput:output1];
}
[session stopRunning];
session=nil;
outputSettings=nil;
device=nil;
input=nil;
captureVideoPreviewLayer=nil;
stillImageOutput=nil;
self.vImagePreview=nil;

}

i called this function before popping and pushing any other view. It solved my issue of low memory warning.

Answer 5

Solved! Perhaps it is the sequence of acions on initializing the session. This one works for me:

NSError *error = nil;

if(session)
    [session release];

// Create the session
session = [[AVCaptureSession alloc] init];


// Configure the session to produce lower resolution video frames, if your 
// processing algorithm can cope. We'll specify medium quality for the
// chosen device.
session.sessionPreset = AVCaptureSessionPresetMedium;

// Find a suitable AVCaptureDevice
AVCaptureDevice *device = [AVCaptureDevice
                           defaultDeviceWithMediaType:AVMediaTypeVideo];

// Create a device input with the device and add it to the session.
AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:device 
                                                                    error:&error];
if (!input) {
    // Handling the error appropriately.
}
[session addInput:input];

// Create a VideoDataOutput and add it to the session
AVCaptureVideoDataOutput *output = [[[AVCaptureVideoDataOutput alloc] init] autorelease];
[session addOutput:output];


// Configure your output.
dispatch_queue_t queue = dispatch_queue_create("myQueue", NULL);
[output setSampleBufferDelegate:self queue:queue];
dispatch_release(queue);

// Specify the pixel format
output.videoSettings = 
[NSDictionary dictionaryWithObject:
 [NSNumber numberWithInt:kCVPixelFormatType_32BGRA] 
                            forKey:(id)kCVPixelBufferPixelFormatTypeKey];

// If you wish to cap the frame rate to a known value, such as 15 fps, set 
// minFrameDuration.
output.minFrameDuration = CMTimeMake(1, 15);

previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:session];
[delegate layerArrived:previewLayer];

NSNotificationCenter *notify =
[NSNotificationCenter defaultCenter];
[notify addObserver: self
            selector: @selector(onVideoError:)
            name: AVCaptureSessionRuntimeErrorNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStart:)
            name: AVCaptureSessionDidStartRunningNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStop:)
            name: AVCaptureSessionDidStopRunningNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStop:)
            name: AVCaptureSessionWasInterruptedNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStart:)
            name: AVCaptureSessionInterruptionEndedNotification
            object: session];

// Start the session running to start the flow of data
[session startRunning];

Btw this sequence seems to resolve the synchronous notifications problem :)

Answer 6

With the queue finalizers, you can use a dispatch_semaphore for each queue and then continue with your cleanup routine once your done.

#define GCD_TIME(delayInSeconds) dispatch_time(DISPATCH_TIME_NOW, delayInSeconds * NSEC_PER_SEC)

static void vQueueCleanup(void* context) {
  VideoRecordingViewController *vc = (VideoRecordingViewController*)context;
  if (vc.vSema) dispatch_semaphore_signal(vc.vSema);
}

static void aQueueCleanup(void* context) {
  VideoRecordingViewController *vc = (VideoRecordingViewController*)context;
  if (vc.aSema) dispatch_semaphore_signal(vc.aSema);
}

//In your cleanup method:
vSema = dispatch_semaphore_create(0);
aSema = dispatch_semaphore_create(0);
self.avSession = nil;
if (vSema) dispatch_semaphore_wait(vSema, GCD_TIME(0.5));
if (aSema) dispatch_semaphore_wait(aSema, GCD_TIME(0.5));
[self.navigationController popViewControllerAnimated:YES];

Remember that you have to set your AVCaptureVideoDataOutput/AVCaptureAudioDataOutput objects sample buffer delegates to nil or they will never release their associated queues and thus never call their finalizers when you release your AVCaptureSession.

[avs removeOutput:vOut];
[vOut setSampleBufferDelegate:nil queue:NULL];

Answer 7

After AVCaptureSession allocation you may use:

NSNotificationCenter *notify =
[NSNotificationCenter defaultCenter];
[notify addObserver: self
            selector: @selector(onVideoError:)
            name: AVCaptureSessionRuntimeErrorNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStart:)
            name: AVCaptureSessionDidStartRunningNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStop:)
            name: AVCaptureSessionDidStopRunningNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStop:)
            name: AVCaptureSessionWasInterruptedNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStart:)
            name: AVCaptureSessionInterruptionEndedNotification
            object: session];

These are calling back the relevant methods upon session.stopRunning, session.startRunning etc.

There you should also implement some undocumented cleanup block:

AVCaptureInput* input = [session.inputs objectAtIndex:0];
[session removeInput:input];
AVCaptureVideoDataOutput* output = (AVCaptureVideoDataOutput*)[session.outputs objectAtIndex:0];
[session removeOutput:output];  

What I found confusing though is that upon calling seesion.stopRunning, onVideoStop: is called synchronously! despite Apple's asynchronous assumption on the case.

Its working but please let me know in case you see any trick. I would prefer working with it asynchronously.

Thanks