AVCapturePhoto rotated 90 counterclockwise after calling `cgImageRepresentation()!.takeUnretainedValue()`

Question

I'm working on an app that take square pictures on iOS with AVFoundation. When I save the image with the following statements in photoOutput(_:, didFinishProcessingPhoto:, error:), the image saved in my album have correct direction.

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
    PHPhotoLibrary.requestAuthorization { status in
        guard status == .authorized else { return }

        PHPhotoLibrary.shared().performChanges({
            let creationRequest = PHAssetCreationRequest.forAsset()
            creationRequest.addResource(with: .photo, data: photo.fileDataRepresentation()!, options: nil)
        }, completionHandler: nil)
    }
}

However, when I replace the code with the following in order to save the CGImage object, I found the image saved in my album was rotated by 90 counterclockwise.

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
    let image = photo.cgImageRepresentation()!.takeUnretainedValue()
    UIImageWriteToSavedPhotosAlbum(UIImage(cgImage: image), nil, nil, nil)
}

Here's my configuration of my photo capture pipeline.

private func initializeCapturePipeline() {
    captureSession = AVCaptureSession()
    captureSession?.sessionPreset = .photo
    captureDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back)
    videoInput = try! AVCaptureDeviceInput(device: captureDevice!)
    imageOutput = AVCapturePhotoOutput()
    captureSession?.addInput(videoInput!)
    captureSession?.addOutput(imageOutput!)
    imageOutput?.connection(with: .video)?.videoOrientation = .portrait
    previewLayer = AVCaptureVideoPreviewLayer(session: captureSession!)
    previewLayer?.videoGravity = .resizeAspectFill
    videoContainerView.layer.addSublayer(previewLayer!)
    captureSession?.startRunning()
}

I wonder what is causing the problem? How could I fix this problem.

gmck · Accepted Answer · 2019-10-04T08:38:27.593

My understanding is that the underlying image data is the same in both cases, and that the difference in the display is due to the lack of metadata in the photo.cgImageRepresentation(). The Photos app is aware of the image metadata and automatically applies the rotation and/or mirroring necessary based on the value of the "Orientation" property in the image metadata.

Using the PHPhotoLibrary function results in a copy of the image with embedded metadata. As described in CGImagePropertyOrientation, iOS is saving the object with the orientation .right. However, the CGImage created directly from the AVCapturePhoto object does not contain the associated metadata.

There are a couple of potential fixes (these have not been tested but should help to find a working solution):

Create the UIImage using the orientation obtained from the photo metadata.

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
    let image = photo.cgImageRepresentation()!.takeUnretainedValue()
    let orientation = CGImageProperyOrientation(rawValue: photo.metadata["Orientation"] as! UInt32)
    var uiImage: UIImage!
    if orientation == .right {
        uiImage = UIImage(cgImage: image, scale: 1.0, orientation: .right)
    } else {
        uiImage = UIImage(cgImage: image)
    }
    UIImageWriteToSavedPhotosAlbum(uiImage, nil, nil, nil)
}

Save the CGImage with metadata using a CGImageDestination and the CGImageDestinationAddImageAndMetadata function from the Image I/O framework.
Depending on the precise use case, it may be necessary to explicitly rotate the image data. Examples of how to do this can be found here.

AVCapturePhoto rotated 90 counterclockwise after calling `cgImageRepresentation()!.takeUnretainedValue()`

1 Answers1