How to merge Audio and video using AVMutableCompositionTrack

Question

In my application I need to merge audio and video and then I need to play the audio file in the Mediaplayer. How can I merge the audio and video in IOS. Is there is any source code for this. Please suggest me some ideas

Thanks in advance

Do some research - for example view the WWDC video + sample code. — spring, Mar 02 '13 at 02:04

score 10 · Answer 1 · answered Mar 02 '13 at 01:09

Use this

AVURLAsset* audioAsset = [[AVURLAsset alloc]initWithURL:audioUrl options:nil];
AVURLAsset* videoAsset = [[AVURLAsset alloc]initWithURL:videoUrl options:nil];

AVMutableComposition* mixComposition = [AVMutableComposition composition];

AVMutableCompositionTrack *compositionCommentaryTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeAudio 
                                                                                    preferredTrackID:kCMPersistentTrackID_Invalid];
[compositionCommentaryTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, audioAsset.duration) 
                                    ofTrack:[[audioAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] 
                                     atTime:kCMTimeZero error:nil];

AVMutableCompositionTrack *compositionVideoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo 
                                                                                    preferredTrackID:kCMPersistentTrackID_Invalid];
[compositionVideoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) 
                               ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] 
                                atTime:kCMTimeZero error:nil];

AVAssetExportSession* _assetExport = [[AVAssetExportSession alloc] initWithAsset:mixComposition 
                                                                      presetName:AVAssetExportPresetHighestQuality];   

NSString* videoName = @"export.mov";

NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:videoName];
NSURL    *exportUrl = [NSURL fileURLWithPath:exportPath];

if ([[NSFileManager defaultManager] fileExistsAtPath:exportPath]) 
{
    [[NSFileManager defaultManager] removeItemAtPath:exportPath error:nil];
}

_assetExport.outputFileType = @"com.apple.quicktime-movie";
_assetExport.outputURL = exportUrl;
_assetExport.shouldOptimizeForNetworkUse = YES;

[_assetExport exportAsynchronouslyWithCompletionHandler:
 ^(void ) {      
            // your completion code here
     }       
 }
 ];

I have try this code. It is not working and also there is one bug in the above coding. Replace initWithURL with URLAssetWithURL — btmanikandan, Mar 02 '13 at 17:38
I used this code with a 1s audio file and a 3s video file. The result was a video with 1s of audio with a black screen, followed by 2s of black screen, followed by the 3s of video. Do you know what could be causing this problem? I just want to add the audio to the beginning of the video. — ill_always_be_a_warriors, Jul 01 '14 at 18:30
I'm only merging a single audio/video file. The problem is that, my video file of 40 seconds and audio file is of 28 seconds. So for remaining 12 (40-28) seconds – I want repeat it from 0 seconds in Audio file. How do I that? Is there a direct way to do this? — Hemang, Jun 18 '15 at 11:23
@BrandonA : NSString *videoPath = [[NSBundle mainBundle] pathForResource:@"name of resource" ofType:@"MOV"]; NSURL *videoURl = [NSURL fileURLWithPath:videoPath]; — Sandeep Singh, Aug 29 '16 at 05:30

CodeCracker · Answer 2 · 2015-04-21T13:44:35.453

You can merge videos by creating the Mutable composition.

AVMutableComposition* composition = [[AVMutableComposition alloc]init];
AVURLAsset* video1 = [[AVURLAsset alloc]initWithURL:[NSURL fileURLWithPath:path1]options:nil];

NSArray *pathComponents = [NSArray arrayWithObjects:
[NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) lastObject],@"MyAudio.m4a",nil];

  NSURL *outputFileURL = [NSURL fileURLWithPathComponents:pathComponents];
AVAsset *audioAsset = [AVAsset assetWithURL:outputFileURL];

//Create mutable composition of audio type
 AVMutableCompositionTrack *audioTrack = [composition    addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];

[audioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero,video1.duration)
ofTrack:[[audioAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:kCMTimeZero error:nil];

  AVMutableCompositionTrack* composedTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];

[composedTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, video1.duration)
ofTrack:[[video1 tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0]
atTime:kCMTimeZero error:nil];


AVAssetExportSession*exporter = [[AVAssetExportSession alloc]initWithAsset:composition presetName:AVAssetExportPresetHighestQuality];

[exporter exportAsynchronouslyWithCompletionHandler:^{
           case AVAssetExportSessionStatusFailed:
                 NSLog(@"Failed to export video");
                 break;
           case AVAssetExportSessionStatusCancelled:
                 NSLog(@"export cancelled");
                 break;

}

For Video merging visit this tutorial. http://iosbucket.blogspot.in/2015/04/mp4-conversion-and-video-merging-in-ios.html

You can also find the sample project for merging videos.

score 1 · Answer 3 · answered Nov 01 '13 at 11:25

1

http://www.raywenderlich.com/13418/how-to-play-record-edit-videos-in-ios visit this tutorial for merging audio and video files

answered Nov 01 '13 at 11:25

Ankita

469
4
15

Note that [link-only answers](http://meta.stackoverflow.com/tags/link-only-answers/info) are discouraged, SO answers should be the end-point of a search for a solution (vs. yet another stopover of references, which tend to get stale over time). Please consider adding a stand-alone synopsis here, keeping the link as a reference. – kleopatra Nov 01 '13 at 11:44

score 0 · Answer 4 · answered Dec 17 '20 at 10:11

It’s a bit late to answer, but it can help someone in the future. repeats audio if video duration is larger than audio.

+ (void)mergeVideoWithAudio:(NSURL *)videoUrl audioUrl:(NSURL *)audioUrl success:(void (^)(NSURL *url))success failure:(void (^)(NSError *error))failure {
    AVMutableComposition *mixComposition = [AVMutableComposition new];
    NSMutableArray<AVMutableCompositionTrack *> *mutableCompositionVideoTrack = [NSMutableArray new];
    NSMutableArray<AVMutableCompositionTrack *> *mutableCompositionAudioTrack = [NSMutableArray new];
    AVMutableVideoCompositionInstruction *totalVideoCompositionInstruction = [AVMutableVideoCompositionInstruction new];
    
    AVAsset *aVideoAsset = [AVAsset assetWithURL:videoUrl];
    AVAsset *aAudioAsset = [AVAsset assetWithURL:audioUrl];
    
    AVMutableCompositionTrack *videoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
    AVMutableCompositionTrack *audioTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
    if (videoTrack && audioTrack) {
        [mutableCompositionVideoTrack addObject:videoTrack];
        [mutableCompositionAudioTrack addObject:audioTrack];
        
        AVAssetTrack *aVideoAssetTrack = [aVideoAsset tracksWithMediaType:AVMediaTypeVideo].firstObject;
        AVAssetTrack *aAudioAssetTrack = [aAudioAsset tracksWithMediaType:AVMediaTypeAudio].firstObject;
        
        if (aVideoAssetTrack && aAudioAssetTrack) {
            [mutableCompositionVideoTrack.firstObject insertTimeRange:CMTimeRangeMake(kCMTimeZero, aVideoAssetTrack.timeRange.duration) ofTrack:aVideoAssetTrack atTime:kCMTimeZero error:nil];
            
            CMTime videoDuration = aVideoAsset.duration;
            if (CMTimeCompare(videoDuration, aAudioAsset.duration) == -1) {
                [mutableCompositionAudioTrack.firstObject insertTimeRange:CMTimeRangeMake(kCMTimeZero, aVideoAssetTrack.timeRange.duration) ofTrack:aAudioAssetTrack atTime:kCMTimeZero error:nil];
            } else if (CMTimeCompare(videoDuration, aAudioAsset.duration) == 1) {
                CMTime currentDuration = kCMTimeZero;
                while (CMTimeCompare(currentDuration, videoDuration) == -1) {
                    // repeats audio
                    CMTime restTime = CMTimeSubtract(videoDuration, currentDuration);
                    CMTime maxTime = CMTimeMinimum(aAudioAsset.duration, restTime);
                    [mutableCompositionAudioTrack.firstObject insertTimeRange:CMTimeRangeMake(kCMTimeZero, maxTime) ofTrack:aAudioAssetTrack atTime:currentDuration error:nil];
                    currentDuration = CMTimeAdd(currentDuration, aAudioAsset.duration);
                }
            }
            videoTrack.preferredTransform = aVideoAssetTrack.preferredTransform;
            totalVideoCompositionInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, aVideoAssetTrack.timeRange.duration);
        }
    }
    
    NSString *outputPath = [NSHomeDirectory() stringByAppendingPathComponent:@"tmp/screenCapture.mp4"];
    if ([[NSFileManager defaultManager] fileExistsAtPath:outputPath]) {
        [[NSFileManager defaultManager] removeItemAtPath:outputPath error:nil];
    }
    NSURL *outputURL = [NSURL fileURLWithPath:outputPath];
    
    AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:mixComposition presetName:AVAssetExportPresetHighestQuality];
    exportSession.outputURL = outputURL;
    exportSession.outputFileType = AVFileTypeMPEG4;
    exportSession.shouldOptimizeForNetworkUse = YES;
    
    // try to export the file and handle the status cases
    [exportSession exportAsynchronouslyWithCompletionHandler:^{
        switch (exportSession.status) {
            case AVAssetExportSessionStatusFailed:
                failure(exportSession.error);
                break;
            case AVAssetExportSessionStatusCancelled:
                failure(exportSession.error);
                break;
            default:
                success(outputURL);
                break;
        }
    }];
}

How to merge Audio and video using AVMutableCompositionTrack

4 Answers4

Linked