AVVideoComposition with CIFilters crash

Question

I am creating an AVVideoComposition with CIFilters this way:

 videoComposition = AVMutableVideoComposition(asset: asset, applyingCIFiltersWithHandler: {[weak self] request in  

            // Clamp to avoid blurring transparent pixels at the image edges  
            let source = request.sourceImage.clampedToExtent()  
            let output:CIImage  

            if let filteredOutput = self?.runFilters(source, filters: filters)?.cropped(to: request.sourceImage.extent) {  
                output = filteredOutput  
            } else {  
                output = source  
            }  

            // Provide the filter output to the composition  
            request.finish(with: output, context: nil)  
        })

And then to correctly handle rotation, I create a passthrough instruction which sets identity transform on passthrough layer.

     let passThroughInstruction = AVMutableVideoCompositionInstruction()  
     passThroughInstruction.timeRange = CMTimeRangeMake(start: CMTime.zero, duration: asset.duration)  
     let passThroughLayer = AVMutableVideoCompositionLayerInstruction(assetTrack: videoTrack)  
            passThroughLayer.setTransform(CGAffineTransform.identity, at: CMTime.zero)   
     passThroughInstruction.layerInstructions = [passThroughLayer]  
            videoComposition.instructions = [passThroughInstruction]

The problem is it crashes with the error:

'*** -[AVCoreImageFilterCustomVideoCompositor startVideoCompositionRequest:] Expecting video composition to contain only AVCoreImageFilterVideoCompositionInstruction'

My issue is that if I do not specify this passThroughInstruction, output is incorrect if input asset's videotrack contains a preferredTransform which specifies rotation by 90 degrees. How do I use video composition with Core Image filters that correctly handles preferredTransform of video track?

EDIT: The question looks similar but is different from other questions that involve playback. In my case, playback is fine but it is rendering that creates distorted video.

Possible duplicate of [AVPlayer plays video composition result incorrectly](https://stackoverflow.com/questions/41580966/avplayer-plays-video-composition-result-incorrectly) — Enea Dume, Oct 12 '18 at 14:30
The issue is there is no problem in playback. Output is distorted in rendering. I will try suggestions in that answer. — Deepak Sharma, Oct 12 '18 at 14:34

score 1 · Answer 1 · answered Oct 13 '18 at 08:15

Ok I found the real issue. Issue is applying videoComposition returned by AVMutableVideoComposition(asset:, applyingCIFiltersWithHandler:) function implicitly creates a composition instruction that physically rotates the CIImages in case a rotation transform is applied through preferredTransform on video track. Whether it should be doing that or not is debatable as the preferredTransform is applied at the player level. As a workaround, in addition to what is suggested in this answer. I had to adjust width and height passed via AVVideoWidthKey, AVVideoHeightKey in videoSettings which is passed to AVAssetReader/Writer.

AVVideoComposition with CIFilters crash

1 Answers1