When reading frames from a video on an iPad with AVAssetReader, the images are not properly oriented

Question

A few things I want to establish first:

This works properly on multiple iPhones (iOS 10.3 & 11.x)
This works properly on any iPad simulator (iOS 11.x)

What I am left with is a situation where when I run the following code (condensed from my application to remove unrelated code), I am getting an image that is upside down (landscape) or rotated 90 degrees (portrait). Viewing the video that is processed just prior to this step shows that it is properly oriented. All testing has been done on iOS 11.2.5.

* UPDATED *

I did some further testing and found a few more interesting items:

If the video was imported from a phone, or an external source, it is properly processed
If the video was recorded on the iPad in portrait orientation, then the reader extracts it rotated 90 degrees left
If the video was recorded on the iPad in landscape orientation, then the reader extracts it upside down
In the two scenarios above, UIImage reports an orientation of portrait

A condensed version of the code involved:

import UIKit
import AVFoundation

let asset = ...
let assetReader = try? AVAssetReader(asset: asset)

if let assetTrack = asset.tracks(withMediaType: .video).first,
    let assetReader = assetReader {
    let assetReaderOutputSettings = [
        kCVPixelBufferPixelFormatTypeKey as String : NSNumber(value: kCVPixelFormatType_32BGRA)
    ]
    let assetReaderOutput = AVAssetReaderTrackOutput(track: assetTrack,
                                                     outputSettings: assetReaderOutputSettings)
    assetReaderOutput.alwaysCopiesSampleData = false
    assetReader.add(assetReaderOutput)

    var images = [UIImage]()
    assetReader.startReading()

    var sample = assetReaderOutput.copyNextSampleBuffer()

    while (sample != nil) {
        if let image = sample?.uiImage { // The image is inverted here
            images.append(image)
            sample = assetReaderOutput.copyNextSampleBuffer()
        }
    }
    // Continue here with array of images...
}

Duplicate of https://stackoverflow.com/questions/4627940/how-to-detect-if-a-video-file-was-recorded-in-portrait-orientation-or-landscape?rq=1 perhaps? — matt, Feb 15 '18 at 02:03
Or this? https://stackoverflow.com/questions/5347800/avassetimagegenerator-provides-images-rotated?rq=1 — matt, Feb 15 '18 at 02:04
@matt Yea, I have found something similar that gets me part way using what was described in there. Just working to get a complete fix before posting. Thanks! — CodeBender, Feb 15 '18 at 03:39

score 4 · Accepted Answer · answered Feb 15 '18 at 05:32

After some exploration, I came across the following that allowed me to obtain the video's orientation from the AVAssetTrack:

let transform = assetTrack.preferredTransform
radians = atan2(transform.b, transform.a)

Once I had that, I was able to convert it to degrees:

let degrees = (radians * 180.0) / .pi

Then, using a switch statement I could determine how to rotate the image:

Switch Int(degrees) {
   case -90, 90:
       // Rotate accordingly
   case 180:
       // Flip
   default:
       // Do nothing
}

When reading frames from a video on an iPad with AVAssetReader, the images are not properly oriented

1 Answers1