Near Real Time Video Upload from iPhone

Question

I am trying to find the best way to upload video from an iPhone (iOS5) as fast as possible - real time if possible.

I found this previous question and answer very useful.
streaming video FROM an iPhone

But it has left me with several unanswered questions. I dont have enough rep to post comments in that question- and I think my questions are getting beyond the scope of the original question anyway.

So:

1. Is using AVCaptureSession/AVAssetWriter and chopping the video into short clips the best way to rapidly move (compressed) video off of the iPhone - in near realtime?

2. If so could someone supply more details on how to use two AVAssetWriters and a background queue to avoid dropouts (as user Steve McFarlin mentions in the referenced question above)? I am unclear how the handoff from one AVAssetWriter to another would work...

3. (Critical) Is there an easy way to append the chopped video files back into a full length video... or at least be able to play them as if they were one complete video? I would need to merge the smaller files to look like one file both on the server AND on the iPhone (for preview).

Thanks for any help...

Answer 1

Well you can try to do the buffering on the phone but that seems counter-productive to me, given that it has limited memory. I would try setting up an AVCaptureSession and use the AVCaptureVideoDataOutput which will vend the frames to you on a separate dispatch_queue thread (if setup it will vend them as MPEG frames). That thread can hand the frames off to an async socket to transmit, possibly with a small header that indicates the frame number and video format. Alternately you can hand the data off to a sending thread via a queue which would let you monitor how many frames are waiting to be transmitted.

On the receiving server, you'd want to deal with creating a small buffer (say a few seconds) and doing the frame reordering if they arrive out of order.

The big issue will be detecting the bandwidth and knowing when to drop the quality down so you don't end up with a backlog of packets waiting to go out. That's an entirely different and complicated topic :) The key will be in your selection if codec, quality, and video size... that is going to directly determine the bandwidth required to transmit the frames in real-time. AVVideoCodecH264 is supported in hardware in certain modes and is probably the only realistic option for real-time encoding.

I don't think you are going to find a ready-made example for this though as it represents a lot of work to get working just right.

Answer 2

2) Here's how I chunked the files without dropping too many frames:

- (void) segmentRecording:(NSTimer*)timer {
    AVAssetWriter *tempAssetWriter = self.assetWriter;
    AVAssetWriterInput *tempAudioEncoder = self.audioEncoder;
    AVAssetWriterInput *tempVideoEncoder = self.videoEncoder;
    self.assetWriter = queuedAssetWriter;
    self.audioEncoder = queuedAudioEncoder;
    self.videoEncoder = queuedVideoEncoder;
    //NSLog(@"Switching encoders");

    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
        [tempAudioEncoder markAsFinished];
        [tempVideoEncoder markAsFinished];
        if (tempAssetWriter.status == AVAssetWriterStatusWriting) {
            if(![tempAssetWriter finishWriting]) {
                [self showError:[tempAssetWriter error]];
            }
        }
        if (self.readyToRecordAudio && self.readyToRecordVideo) {
            NSError *error = nil;
            self.queuedAssetWriter = [[AVAssetWriter alloc] initWithURL:[self newMovieURL] fileType:(NSString *)kUTTypeMPEG4 error:&error];
            if (error) {
                [self showError:error];
            }
            self.queuedVideoEncoder = [self setupVideoEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:videoFormatDescription bitsPerSecond:videoBPS];
            self.queuedAudioEncoder = [self setupAudioEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:audioFormatDescription bitsPerSecond:audioBPS];
            //NSLog(@"Encoder switch finished");

        }
    });
}

https://github.com/chrisballinger/FFmpeg-iOS-Encoder/blob/master/AVSegmentingAppleEncoder.m

3) Here's a script to concatenate the files on the server

import glob
import os
run = os.system  # convenience alias


files = glob.glob('*.mp4')
out_files = []

n = 0
for file in files:
    out_file = "out-{0}.ts".format(n)
    out_files.append(out_file)
    full_command = "ffmpeg -i {0} -f mpegts -vcodec copy -acodec copy -vbsf h264_mp4toannexb {1}".format(file, out_file)
    run(full_command)
    n += 1

out_file_concat = ''
for out_file in out_files:
    out_file_concat += ' {0} '.format(out_file)

cat_command = 'cat {0} > full.ts'.format(out_file_concat)
print cat_command
run(cat_command)
run("ffmpeg -i full.ts -f mp4 -vcodec copy -acodec copy -absf aac_adtstoasc full.mp4")

https://github.com/chrisballinger/FFmpeg-iOS-Encoder/blob/master/concat-mp4.py