How to merge multiple videos side by side swift 4

Question

I am trying to merge multiple videos in one video playing simultaneously. My first question is, is it better to do the merging on server or on user's phone? And how can i merge videos on swift? I tried to find the solution by googling it but i couldn't find any on swift. Any help would be appreciated

Answer 1

Is it better to do the merging on server or on user's phone?

It depends, really. Servers are generally costly but they give you more agility when it comes to support plus you only need to create the functionality once instead of once per platform. I would always go with doing it on device if possible. So in your case it seems "device" is the way to go.

How can i merge videos on Swift? I tried to find the solution by googling it but i couldn't find any on Swift.

Swift itself will not be able to do what you ask. You will need tools/frameworks that allow such things. And you can find them for iOS.

First you should try to specify what exactly you are doing. A single video consists (in most cases) of video and audio track. From title I can deduct that video tracks should be placed side by side while there is no info about audio. Further more there is a question on how the "side by side" is done for videos that do not have same video resolution. There are a few possibilities for that. Also there is a question to what happens when not all videos have the same length?

The video part in your case should probably boil down to images at certain frame rate. For iOS that should be getting a UIImage for each of the videos at a given time, stitch images together, insert the new image into your stitched video file. So what you are looking for:

• Get UIImage from video at certain time (try this)
• Merge images together looks promising here
• Create video from UIImages may need a bit more work but check this out

Naturally you still need to decide what output video you will choose. What to do with videos of different sizes and what to do with videos of different lengths. But these are all just decisions and all can be done on the level of UIImage.

You will need to decide what frame-rate to use. I guess there is no reason not to just use a fixed one like 30FPS which means you are iterating time as CMTimeMake(i, 30) where i is in range [0, videoDuration*30].

The only thing left is the audio part for which you may find some clues here.

So overall high level code should be something like:

func mergeVideos(urls: [URL]) -> URL {
    let overallSize: CGSize = sizeForStichedVideos(urls: urls)
    let overallDuration: TimeInterval = longestVideoFrom(urls: urls)
    let frameRate: Int = decideFrameRateForVideos(urls: urls)

    let stichedVideo: VideoObject = VideoObject(size: overallSize, duration: overallDuration, frameRate: frameRate)

    for frameIndex in 0..<Int(ceil(overallDuration))*frameRate {
        let time = CMTime(value: frameIndex, timescale: frameRate)
        let images: [UIImage?] = imagesFromVideos(urls: urls, at: time)
        let stichedImage: UIImage = stitchImages(images)
        stichedVideo.appendVideoSample(withImage: stichedImage, at: time)
        // TODO: append audio samples
    }
    stichedVideo.finalize()

    return stichedVideo.outputURL
}

Have fun.