0

What would be the simplest approach to re-encoding arbitrary video files chosen from a user's photo library on their iPhone to match desired parameters? Namely: 720p (aspect fill), portrait (with no transforms), 30fps, h264.

Example: Given a video with the following parameters: 1080x1920, 90d rotation (even though its width is < height it is meant to be rotated and played sideways), 24fps, hvec. It would become: 720p (In this case the aspect ratio is the same. If it were different, it would have cropped/scaled aspect fill), portrait (with no transforms), 30fps, h264.

This is what I have tried so far:

// Make an asset object from the input video url
let asset = AVURLAsset(url: videoUrl)

// We will be using the AVAssetWriter class to output the video and it requires an AVAssetReader as input
var reader: AVAssetReader? = nil
do {
    reader = try AVAssetReader(asset: asset)
} catch {
    error.record(with: "Unable to create AVAssetReader from asset")
    completion(.failure(error))
    return
}

// Get the video track and make sure we have a valid reader
guard
    let assetReader = reader,
    let videoTrack = asset.tracks(withMediaType: AVMediaType.video).first
else {
    completion(.failure(VideoHelperError.unknown))
    return
}

// Set the video properties that we want
let videoReaderSettings: [String:Any] = [kCVPixelBufferPixelFormatTypeKey as String:kCVPixelFormatType_32ARGB ]
let videoSettings: [String:Any] = [
    AVVideoScalingModeKey: AVVideoScalingModeResizeAspectFill,
    AVVideoCodecKey: AVVideoCodecType.h264,
    AVVideoHeightKey: 1280,
    AVVideoWidthKey: 720
]

let assetReaderVideoOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: videoReaderSettings)

if assetReader.canAdd(assetReaderVideoOutput) {
    assetReader.add(assetReaderVideoOutput)
} else {
    completion(.failure(VideoHelperError.unknown))
    return
}

// Start preparing the writer
let videoInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: videoSettings)
videoInput.transform = videoTrack.preferredTransform

// This is the queue we will be using for feeding the video into the writer
let videoInputQueue = DispatchQueue(label: "videoQueue")

// Create the temporary file onto which we will export our result
guard let outputUrl = try? FileManager.tempFileURL(withFileExtension: "mp4") else {
    completion(.failure(VideoHelperError.fileSystemFailure))
    return
}

// Create the writer that will generate the output file
var writer: AVAssetWriter? = nil
do {
    writer = try AVAssetWriter(outputURL: outputUrl, fileType: AVFileType.mp4)
} catch {
    error.record(with: "Unable to create AVAssetWriter")
    completion(.failure(error))
    return
}

guard let assetWriter = writer else{
    completion(.failure(VideoHelperError.unknown))
    return
}

// Add the writer input to the writer and start read/write of the video file
assetWriter.shouldOptimizeForNetworkUse = true
assetWriter.add(videoInput)
assetWriter.startWriting()
assetReader.startReading()
assetWriter.startSession(atSourceTime: CMTime.zero)

// As data becomes available, we should pass it through the writer
videoInput.requestMediaDataWhenReady(on: videoInputQueue) {
    while(videoInput.isReadyForMoreMediaData){
        let sample = assetReaderVideoOutput.copyNextSampleBuffer()
        if (sample != nil) {
            videoInput.append(sample!)
        } else {
            videoInput.markAsFinished()
            DispatchQueue.main.async {
                assetWriter.finishWriting(completionHandler: {
                    completion(.success(assetWriter.outputURL))
                })
                
                assetReader.cancelReading()
            }
            
            break;
        }
    }
}

The code above achieves 2 of the 4 objectives. It scales/crops to 720p (aspect fill) and it encodes in the correct format. However, it does not adjust for the FPS or remove the rotations correctly causing some videos to end up outside of the frame an appear as blank.

  1. If the fps is higher it would be decimated and adjusted (60fps -> 30fps). If lower, it would would have new filler frames programmatically generated (24fps -> 30fps). Without altering the length (would playback in natural speed). Is there a built in approach to do this?

  2. What would be the best approach to apply the video's transforms before aspectfill fitting to ensure the content always falls within the viewport of the exported asset?

References

  1. How do I control AVAssetWriter to write at the correct FPS
  2. https://medium.com/samkirkiles/swift-using-avassetwriter-to-compress-video-files-for-network-transfer-4dcc7b4288c5
Paulo C
  • 374
  • 1
  • 5
  • 11
  • Did you ever figure out a solution for this? I'm having trouble and have very similar needs, except going from e.g. 24 to 30fps (staying at 24fps is fine in my use case). – inspector-g Nov 01 '22 at 22:38
  • @inspector-g I did not. We moved onto something else. – Paulo C Nov 10 '22 at 20:24

0 Answers0