13

I'm building a video export feature for one of my apps. In essence, the video is a series of one of six different images lasting for different (short) durations.

The export works fine when I export something containing 283 images of varying durations, but when I try to export one of 803, I get the dreaded "The operation could not be completed" error (A.K.A. "we have no idea what just blew up because AVFoundation error reporting is awful").

When I try to add the 754th frame (always the 754th frame) using my AVAssetWriterInputPixelBufferAdaptor, appendPixelBuffer:withPresentationTime: returns NO, the AVAssetWriter's status is failed and its error is this:

Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x17ab2050 {Error Domain=NSOSStatusErrorDomain Code=-16364 "(null)"}, NSLocalizedFailureReason=An unknown error occurred (-16364)}

I can't for the life of me figure out what that underlying error (OSStatus -16364) is. www.osstatus.com has no idea, macerror says no such thing exists, and this Python script for searching the SDK headers finds nothing. It's also not a four character code like some OSStatus errors (unless I messed up checking this).

I've ruled out every common cause of "The operation cannot be completed" errors I've found. It's not related to filesystem permissions or overwriting, no two calls of appendPixelBuffer have the same presentation time.

It's not a memory thing (memory usage stays flat at 165MB during video export), and CPU stays near 3%.

If it's of any importance, I reuse the same 6 CVPixelBuffers over and over again for the 6 images instead of creating new ones from UIImages each time. This seems to help performance, and changing it to new ones each time doesn't seem to change anything (except make it fail on frame 753 instead), but who knows.

Does anyone have any idea what this could be?

Linuxios
  • 34,849
  • 13
  • 91
  • 116

2 Answers2

24

OK. Finally figured this out.

Because of rounding (rounding small duration values into the timescale of 30 FPS, which caused them to become 0/30), appendPixelBuffer:withPresentationTime: was being called twice with the same presentationTime under specific circumstances. AVFoundation didn't notice the problem till 7 frames later when it throws the error:

Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x17ab2050 {Error Domain=NSOSStatusErrorDomain Code=-16364 "(null)"}, NSLocalizedFailureReason=An unknown error occurred (-16364)}

Using 60 FPS in place of 30 FPS prevented this particular situation from rounding to a zero duration, but the general solution was to drop frames with durations that rounded to zero.

Linuxios
  • 34,849
  • 13
  • 91
  • 116
  • nice - boo to `AVFoundation`'s error messages. can you accept your answer? – Rhythmic Fistman Dec 28 '15 at 01:26
  • @RhythmicFistman: Done. You know, maintaining two apps using AVFoundation is a nightmare. – Linuxios Dec 28 '15 at 01:41
  • what kind of problems are you having? – Rhythmic Fistman Dec 28 '15 at 01:42
  • @RhythmicFistman: In essence, I'm trying to export a series of still images that last for different durations. If I just `appendPixelBuffer` for each still image, instead of AVFoundation making something sensible out of that, it creates a video file with *variable frame rate*. QuickTime and iOS devices understand it fine (I'd have hoped), and generally nothing else borks too badly that I ve settled for it, but really? Variable frame rate? If I start exporting every frame (i.e. calling `appendPixelBuffer` 30 times/sec), performance is abysmal: 52s export time to 16s content at 60FPS, 32s at 30. – Linuxios Dec 28 '15 at 02:54
  • @RhythmicFistman: The only better solution I could think of would be to manually generate video using a simple codec (something like qtrle animation), and just insert a bunch of "repeat last frame" frames instead of doing this stupid dance where I insert 24 identical frames and then AVFoundation spends a bunch of time figuring out that they're identical and compressing them. That would work, but I've spent so much time on this feature already that its just not feasible. – Linuxios Dec 28 '15 at 02:57
  • variable frame can be desirable! why bloat your bitrate by inserting redundant frames? anyway, if you want a constant framerate, pass your variable bitrate file through an `AVAssetExportSession`. – Rhythmic Fistman Dec 28 '15 at 02:58
  • @RhythmicFistman: I'd be fine with it if it didn't make problems in 1/3d of the players I've tested with. But, it's good enough, so it's staying! – Linuxios Dec 28 '15 at 02:59
  • @RhythmicFistman: Tried that too. Bad performance. Not as bad (something like 30s for 16s of content at 60FPS), but that was still just a little too much. – Linuxios Dec 28 '15 at 03:00
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/99080/discussion-between-rhythmic-fistman-and-linuxios). – Rhythmic Fistman Dec 28 '15 at 03:09
  • @xaphod: :D glad it helped – Linuxios Aug 12 '19 at 20:36
0

In my case it happened then append(buffer: buffer, with: time) was called with time (taken as CACurrentMediaTime() in one thread) that is lower than time used for previously added frame. It happen when image was generated in concurrent thread and seems was completed in wrong order.

I added a check self!.lastTime! < time and it helped.

Alexander Volkov
  • 7,904
  • 1
  • 47
  • 44