9

After implementing the solution to encoding video (with audio) in this question, Video Encoding using AVAssetWriter - CRASHES, I found that the code works correctly in the iPhone Simulator. Unfortunately, certain videos fail to encode their audio while running on an actual iPhone 5 (and other devices).

For example, videos generated from the WWDC 2011 sample code RosyWriter (https://developer.apple.com/library/IOS/samplecode/RosyWriter/Introduction/Intro.html) do not completely encode because the function -[AVAssetReaderOutput copyNextSampleBuffer] never returns.

The video buffers come in correctly, but as soon as it tries to copy the first audio CMSampleBufferRef, the call hangs. When I try this on videos that come from other sources, like those recorded in the native iOS Camera app, the audio imports correctly.

This thread, https://groups.google.com/forum/#!topic/coreaudio-api/F4cqCu99nUI, makes note of the copyNextSampleBuffer function hanging when being used with AudioQueues, and suggests keeping the operations on a single thread. I've tried keeping everything on a separate thread, on the main thread, but had no luck.

Did anyone else experience this and have a possible solution?

EDIT: It appears that videos generated from RosyWriter have their tracks reversed relative to videos from the native Camera app, i.e. audio stream as stream 0, and video stream as stream 1.

Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 60 kb/s
Metadata:
  creation_time   : 2013-10-28 16:13:05
  handler_name    : Core Media Data Handler
Stream #0:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1920x1080, 8716 kb/s, 28.99 fps, 29.97 tbr, 600 tbn, 1200 tbc
Metadata:
  rotate          : 90
  creation_time   : 2013-10-28 16:13:05
  handler_name    : Core Media Data Handler

Not sure if this makes a difference to the AVAssetReader.

Community
  • 1
  • 1
jlw
  • 3,166
  • 1
  • 19
  • 24
  • Did you ever find a solution for this? I'm experiencing the same issue. I have been reverse-engineering the RosyWriter to record my own videos, and when I try to encode/export them later, they 'sometimes' fail. It's very very inconsistent.. – Sti Sep 20 '14 at 14:59
  • It seemed to be an issue with the way AVFoundation was encoding the audio track. It's fixed in iOS8. I'll add an answer. – jlw Sep 20 '14 at 19:25
  • Is there any recent development on this issue? – pearl7721 Mar 04 '20 at 02:47

4 Answers4

4

I was still experiencing this issue on iOS 9.3.2, and the thing that resolved it was making sure that the AVAssetReaderAudioMixOutput* was initially set with options rather than nil when calling -[AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks].

Example:

NSDictionary *outputSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                                    [NSNumber numberWithInt:kAudioFormatLinearPCM], AVFormatIDKey,
                                    [NSNumber numberWithFloat:44100.0], AVSampleRateKey,
                                    [NSNumber numberWithInt:16], AVLinearPCMBitDepthKey,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsNonInterleaved,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsFloatKey,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsBigEndianKey,
                                    nil];

// create an AVAssetReaderOutput for the audio tracks
NSArray* audioTracks = [asset tracksWithMediaType:AVMediaTypeAudio];
AVAssetReaderAudioMixOutput* _audioReaderOutput = [AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks:audioTracks audioSettings:outputSettings];

This prevented later calls to -[AVAssetReaderOutput copyNextSampleBuffer] from hanging when they otherwise were doing so.

Dan G
  • 232
  • 1
  • 8
0

Check the time range on the audio tracks using something like this:

NSLog(@"audioTrack timeRange: %lld, %lld", audioTrack.timeRange.start.value, audioTrack.timeRange.duration.value);

Empty time ranges (0, 0) can cause copyNextSampleBuffer() to hang.

John Lemberger
  • 2,689
  • 26
  • 25
-1

This was an AVFoundation bug in iOS7. It is now fixed in iOS8.

jlw
  • 3,166
  • 1
  • 19
  • 24
  • It's good news that it's fixed in iOS8, but for those of us who want to support earlier iOS-versions, it's still a problem :( I too have Audio as the first stream, video as the second, in my files. Did you find out if this had any significance to the error? – Sti Sep 20 '14 at 19:32
  • Not sure if it's related to the error, but the videos that record correctly in iOS8 now have video stream first, and audio stream second. – jlw Sep 21 '14 at 21:47
  • Did anybody find a workaround for this issue? Is there maybe a way to change the order of the streams when setting up the AssetWriter? I'd really like to be able to re-process a video I recorded before – also in iOS 7. – Frank Rupprecht May 16 '15 at 17:34
  • I tested this on iOS8 on my iPhone 5 and it is NOT WORKING. Works fine on simulator (NOT on the iPhone 5). Perhaps it works on iPhone 6? – etayluz Jul 29 '15 at 20:38
  • 1
    I'm seeing copyNextBuffer occasionally hang on iOS 9.3.3 as well. For me, it's occurring sporadically with an AVAssetReaderAudioMixOutput with an asset that has 5 audio tracks, all with non-empty time ranges. Anyone else still seeing this? – Greg Jul 25 '16 at 23:51
  • 3
    Still a problem on iOS 13, new hardware. – xaphod Oct 23 '19 at 11:55
-1

This is a a hardware specific issue. I tested it on the simulator, iPhone5, and iPhone 6. The problem occurs 100% with certain video files, but ONLY on the iPhone 5. I can't speak for the iPhone 5S though. iOS8 does NOT fix this issue. The fix may be having to limit your iPhone 5 users from certain features. Another work around is to have the user select a video from their camera roll, thus forcing Apple's special compression on it - and baptizing that video into a format that the iPhone 5 hardware can work with.

etayluz
  • 15,920
  • 23
  • 106
  • 151