8

I'm using Swift to show content from an AVPlayer in a view's AVPlayerLayer. The associated AVPlayerItem has a videoComposition, and slightly simplified version of the code to create it (without error checking, etc.) looks like this:

playerItem.videoComposition = AVVideoComposition(asset: someAsset, applyingCIFiltersWithHandler: {
                [unowned self] (request: AVAsynchronousCIImageFilteringRequest) in

    let paramDict = << set up parameter dictionary based on class vars >>

    // filter the image
    let filter = self.ciFilterWithParamDict(paramDict) {
    filter.setValue(request.sourceImage, forKey: kCIInputImageKey)
    if let filteredImage = filter.outputImage {
        request.finishWithImage(filteredImage, context: nil)
    }
})

This all works as expected when the AVPlayer is playing or seeking. And if a new videoComposition is created and loaded, the AVPlayerLayer is rendered correctly.

I have not found a way, however, to "trigger" the AVPlayer/ AVPlayerItem/ AVVideoComposition to re-render when I have changed some of the values that I use to calculate filter parameters. If I change values and then play or seek, it is rendered correctly, but only if I play or seek. Is there no way to trigger a rendering "in place"?

Charlie Hitchcock
  • 287
  • 1
  • 3
  • 8

5 Answers5

6

The best way that I know to do this is to simply create a new AVVideoComposition instance for the AVPlayerItem when editing your CIFilter inputs on a paused AVPlayer. In my experience it's way faster and cleaner than swapping player items out and back into the player. You might think that creating a new video composition is slow, but really all you are doing is redefining the render path at that specific frame, which is almost as efficient as invalidating the part of the Core Image cache that was impacted by your change.

The key here that the video composition of the player item must be invalidated in some way to trigger a redraw. Simply changing the input parameters of the Core Image filters have sadly no way (that i know of) of invalidating the video composition, which is the source of the issue.

You can get even more efficient by creating a AVMutableVideoComposition instance for the AVPlayerItem and mutate it in some ways (by changing things like instructions, animationTool, frameDuration) when editing on pause.

Gilles Dezeustre
  • 61
  • 1
  • 2
  • The problem is it immediately causes AVPlayer to start playing the moment you change composition. Any known workarounds? – Deepak Sharma Oct 07 '18 at 18:12
  • @DeepakSharma I'm using this exact way to redraw AVPlayerItem as Gilles described and my AVPlayer saves it's current state, so when it's on pause, it's on pause. Maybe your case have some specific settings which cause an issue? – Eugene Alexeev Jan 30 '19 at 14:56
  • @DeepakSharma What no longer works in iOS 13? Setting it to nil and back? Or mutating something in AVMutableVideoComposition? Or pausing the player after changing the videoCOmposition? – drewster Sep 05 '19 at 03:12
  • Ok mutating the video composition and resetting works. – Deepak Sharma Sep 05 '19 at 19:45
  • confirming, it works (iOS 13.x) really good explanation, thanks for sharing – Stanislau Baranouski Jul 23 '20 at 12:10
4

I used a hack to replace the avPlayerItem entirely to force a refresh. But it would be much better if there was a way to trigger the avPlayerItem to re-render directly.

// If the video is paused, force the player to re-render the frame.
if (self.avPlayer.currentItem.asset && self.avPlayer.rate == 0) {
    CMTime currentTime = self.avPlayerItem.currentTime;

    [self.avPlayer replaceCurrentItemWithPlayerItem:nil];
    [self.avPlayer replaceCurrentItemWithPlayerItem:self.avPlayerItem];
    [self.avPlayerItem seekToTime:currentTime toleranceBefore:kCMTimeZero toleranceAfter:kCMTimeZero];
}
Matthew Self
  • 321
  • 1
  • 10
2

This one might help you. 

let currentTime = self.player.current()
self.player.play()
self.player.pause()
self.player.seek(to: currentTime, toleranceBefore:kCMTimeZero toleranceAfter:kCMTimeZero)
Tomo
  • 51
  • 3
2

Along the lines of this answer, but instead of creating a brand new AVVideoComposition and setting that as the player's videoComposition, it appears you can force a refresh of the current frame by simply setting videoComposition to nil and then immediately back to the existing videoComposition instance.

This results in the following simple workaround any time you want to force refresh the current frame:

let videoComposition = player.currentItem?.videoComposition
player.currentItem?.videoComposition = nil
player.currentItem?.videoComposition = videoComposition
Stuart
  • 36,683
  • 19
  • 101
  • 139
  • The problem is it immediately causes AVPlayer to start playing the moment you change composition. Any known workarounds? – Deepak Sharma Oct 07 '18 at 18:12
0

This one worked in my case:

let playerItem = player.currentItem!
pausePlayback()
playerItem.videoComposition = nil
playerItem.videoComposition = AVVideoComposition(asset: playerItem.asset) { request in
        let source = request.sourceImage.clampedToExtent()
        filter?.setValue(source, forKey: kCIInputImageKey) //Any CIFilter
        request.finish(with: filter?.outputImage!, context: nil)
}
resumePlayback()
Ayushi Shrivastava
  • 1
  • 2
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Oct 12 '21 at 08:24