Can't continue reading from AVAssetReaderOutput after going to background and back to foreground

Question

I'm using an AVAssetReaderOutput to read samples from an AVAsset, do some processing on them, and play the result using a RemoteIO AU.

The problem is that after calling AudioOutputUnitStop to pause the playback, then after going to the background and back to the foreground the audio won't start again after calling AudioOutputUnitStart. This is due to an error returned from the copyNextSampleBuffer method of AVAssetReaderOutput, which is called as part of the rendering pipeline.

The status property of AVAssetReader after calling copyNextSampleBuffer is AVAssetReaderStatusFailed, and its error property is Error Domain=AVFoundationErrorDomain Code=-11847 "Operation Interrupted" UserInfo=0x1d8b6100 {NSLocalizedRecoverySuggestion=Stop other operations and try again., NSLocalizedDescription=Operation Interrupted}

I'm looking for a solution which won't force me to reinitialize the entire pipeline after coming back to the foreground - Hoping there is such a solution, that AVAssetReaders can survive the app going to background and back...

Notes

• The app is entitled to play audio in the background.
• I'm handling audio interruptions - Setting my AVAudioSession as the active one both on AVAudioSessionDelegates endInterruptionWithFlags: event and whenever the app becomes active. Doesn't make a difference whether I do this or not, getting the same error.

Some code:

AudioPlayer

@implementation AudioPlayer
    ...
// Audio Unit Setup
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_RemoteIO;
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;

AudioComponent defaultOutput = AudioComponentFindNext(NULL, &desc);

AudioComponentInstanceNew(defaultOutput, &_audioUnit);

AudioStreamBasicDescription audioFormat;
    FillOutASBDForLPCM(audioFormat, 44100, 2, 16, 16, false, false);

AudioUnitSetProperty(self.audioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, kOutputBus, &audioFormat, sizeof(audioFormat));

AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc = RenderCallback;
callbackStruct.inputProcRefCon = (__bridge void*)self;
AudioUnitSetProperty(self.audioUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, kOutputBus, &callbackStruct, sizeof(callbackStruct));

AudioUnitInitialize(self.audioUnit);

AudioReader Setup

@implementation AudioReader
    ...
NSError* error = nil;
self.reader = [AVAssetReader assetReaderWithAsset:self.asset error:&error];
NSDictionary *outputSettings = ...
self.readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:[self.asset.tracks objectAtIndex:0] outputSettings:outputSettings];
[self.reader addOutput:self.readerOutput];
[self.reader startReading];

AudioReader Render Method, called eventually by the RenderCallback function

-(BOOL)readChunkIntoBuffer
{
     CMSampleBufferRef sampleBuffer = [self.readerOutput copyNextSampleBuffer];
     if ( sampleBuffer == NULL )
     {
         NSLog(@"Couldn't copy next sample buffer, reader status=%d error=%@, self.reader.status, self.reader.error);
         return NO;
     }
 ...
}

Answer 1

The graph and AVReader underpinnings are not connected/linked whatsoever. The confusion perhaps comes from that iOS won't "background" (hibernate) a process if it sees the audio graph running (because then audio data would be unable to be generated). That's why when you stop the audio graph, 2-3 minutes later, iOS will background your process (as of iOS 9). Presumably, iOS looks at what's going on in your process and decides when your process should be forced to background (via beginBackgroundTaskWithName:expirationHandler).

Apple devs decided for whatever reason to make AVReader stop when put into background mode (my best guess is QA). The good news is that you can detect it and recover when your process exits background mode, but you WILL need to restart your reader and readerOutput again. First, when AVAssetReaderOutput's copyNextSampleBuffer returns NULL, check AVAssetReader.error.code for AVErrorOperationInterrupted.

The way we pull off a gapless recovery here is when we get to that point, we first calculate the exact time we left off at (easy to do -- just maintain a counter of the total samples already output). Then, you WILL need to restart your AVAssetReader and AVAssetReaderOutput flow. But that's no problem since your good development practices would have encapsulated that in a single function that has a seek time as an argument. In that function, use AVAssetReader.timeRange to seek to that time you need to resume at and presto!

Answer 2

There first thing I would try is:

Not calling AudioOutputUnitStop? Why don't you leave the graph running and when the audio is paused simply not call

[self.readerOutput copyNextSampleBuffer]

Instead just fill the buffer with 0's and/or use a render action of kAudioUnitRenderAction_OutputIsSilence for the buffers. Starting and stopping the graph takes time and leads to an unresponsive graph. I never stop the graph while my apps are running. (Unless some serious interruption happens) This may keep the asset reader around and happy.

But I have a hunch that as the AVAssetReader is not directly connected to the audio graph (you just happen to request samples and place them in the buffer supplied by the render callback) the problem does not lie with the audio graph being stopped and restarted at all. So the next line of attack I would try is to request samples from an AVAsset, put the app into the background and bring it back to the foreground without using an AUGraph at all. This will determine if the problem is related to the AUGraph or the app entering a background state.

If something happens that forces you to tear down your AVAssetReader you will have to reconnect to the asset. So as you have to write that code another the option I would look at is the one you don't want to do. Reconnect to the AVAssetReader, seek to the position you left off at and keep going.