5

I'm using an AVSampleBufferDisplayLayer to decode and display H.264 video streamed from a server. When my app goes into the background and then returns to the foreground, the decoding process gets screwed up and the AVSampleBufferDisplayLayer fails. The error I'm seeing is:

H.264 decoding layer has failed: Error Domain=AVFoundationErrorDomain
  Code=-11847 "Operation Interrupted" UserInfo=0x17426c500
  {NSUnderlyingError=0x17805fe90 "The operation couldn’t be completed.
    (OSStatus error -12084.)",
   NSLocalizedRecoverySuggestion=Stop other operations and try again.,
   NSLocalizedDescription=Operation Interrupted}

Has anybody else run into issues like this with AVSampleBufferDisplayLayer? What does this mean?

I have tried destroying the AVSampleBufferDisplayLayer and creating a new one when I get the error, but then I start receiving other errors from the H.264 decoder:

Error Domain=AVFoundationErrorDomain Code=-11821 "Cannot Decode"
UserInfo=0x1740e9700 {AVErrorMediaSubTypeKey=(1635148593),
  NSLocalizedFailureReason=The media data could not be decoded. It may be damaged.,
  NSUnderlyingError=0x174247680 "The operation couldn’t be completed. (OSStatus error -12909.)",
  AVErrorMediaTypeKey=vide,
  AVErrorPresentationTimeStampKey=CMTime: {7/30 = 0.233},
  NSLocalizedDescription=Cannot Decode}

I was not receiving any of those errors before the AVSampleBufferDisplayLayer failed.

Greg
  • 10,360
  • 6
  • 44
  • 67
  • 1
    Did you find a solution to this? Why does it stop decoding after coming from background? I'm having the same issue. I have to create a new `AVSampleBufferDisplayLayer ` every time it foregrounds the app. But that gives me a few seconds of black screen which is not ok. – Mihai Jul 29 '15 at 13:28
  • 1
    I never found a solution for this. Even worse, I started getting crashes from the internals of `AVSampleBufferDisplayLayer` where it looked like it was overreleasing some internal object. I've stopped using `AVSampleBufferDisplayLayer` entirely. Currently trying to figure out how to use `VTDecompressionSession`'s output directly. – Greg Jul 29 '15 at 17:19
  • 1
    Oh that's bad news. I filed a ticket on Apple Support. I'll keep you updated if something useful comes back. – Mihai Jul 30 '15 at 12:05
  • 1
    same error occurred, When we run project on simulator, NSLocalizedFailureReason=The encoder required for this media cannot be found., NSLocalizedDescription=Cannot Encode, AVErrorMediaTypeKey=vide},{ AVErrorMediaSubTypeKey = ( 1785750887 ); AVErrorMediaTypeKey = vide; NSLocalizedDescription = "Cannot Encode"; NSLocalizedFailureReason = "The encoder required for this media cannot be found."; }. – Jaywant Khedkar Mar 18 '16 at 12:21
  • any updates on this issue? – stanley Oct 13 '16 at 03:53
  • 2
    I haven't tried using `AVSampleBufferDisplayLayer` again. We're having success using `VTDecompressionSession` and rendering into a `GLKView`. – Greg Oct 14 '16 at 14:26

3 Answers3

0

After you rebuild a new AVSampleBufferDisplayLayer, you should enqueue it with the last nearest IDR frame except current frame is the IDR, which means, you should save nalus in one GOP when decoding and delete them when next IDR is coming.

0

I solved it

// create renderer
let renderingLayer = AVSampleBufferDisplayLayer()

// when enqueue data
if renderingLayer.status == .failed {
    // this way
    renderingLayer.flushAndRemoveImage()
}
0

I encountered the same issue, and was able to solve with the following code. I needed to use pubDidBecomeActive event instead of willEnterForeground event to deal with the case where Siri was activated as well.

class RoomState: NSObject, ObservableObject  {
    private var subs = [AnyCancellable]()
    @Published var displayLayer = AVSampleBufferDisplayLayer()

    override init() {
        super.init()
        let pubDidBecomeActive = NotificationCenter.default.publisher(for: UIApplication.didBecomeActiveNotification)
        subs.append(pubDidBecomeActive.sink { [weak self] _ in
            self?.displayLayer = AVSampleBufferDisplayLayer()
        })
    }
}

Satoshi Nakajima
  • 1,863
  • 18
  • 29