8

I'm attempting to play a video with AVFoundation. I am using the following code for a button that advances the playback by one frame.

It works intermittently, on some executions it will do the right thing and advance one frame, but most times I will have to press the button 3 or 4 times before it will advance a frame.

This makes me think it is some kind of precision issue, but I can't figure out what it is. Each time it is run the new CMTime appears to be advancing by the same amount.

My other theory is that it could be caused by the currentTime not being set to an exact frame boundary at my frame rate (caused by seeking through the video). But I don't know how to "snap" to the nearest frame at my frame rate.

AVAssetTrack *videoTrack = ...;
Float64 frameRate = [videoTrack nominalFrameRate];

CMTime currentTime = [self.playerItem currentTime];
CMTime oneFrame = CMTimeMakeWithSeconds(1.0 / frameRate, currentTime.timescale);
CMTime added = CMTimeAdd(currentTime, oneFrame);

[self.player seekToTime:added toleranceBefore:kCMTimeZero toleranceAfter:kCMTimeZero];

Thanks for your help!

Wil Gieseler
  • 1,893
  • 1
  • 17
  • 18
  • Technically your code should work under the assumption that the video frames are evenly spaced at 1/30 of a second, and there are 30 frames for every second. Given this assumption even if current time is not on a boundary then when you advance the player will pass the PTS of the next frame and display it. Now if for some reason the hardware sampled delta between two frames is greater then 1/30 of a second then your code will fail at this point. I think a better approach is to use something like AVAssetReader to ask for the next frame. Then get the PTS for that frame and advance the play head. – Steve McFarlin Jun 24 '11 at 06:14
  • Hi Steve, Thanks for the reply. I'm not sure what you mean by PTS exactly, could you elaborate a bit? Also, the application I am working on is designed for the film industry and the majority of the clips it deals with are at frame rates other than 30, such as 23.976 or 29.97. (This is especially complicated because CMTime only uses rational numbers, so these floats for frame rates are a headache.) Therefore, I need to be frame-rate independent. (My code fails intermittently on all three of the aforementioned frame rates, by the way.) – Wil Gieseler Jun 24 '11 at 08:51
  • Presentation Time Stamp. This is a tough one, and I am not familiar enough with the editing side of AVFoundation to really give you a solid answer. It looks like AVAssetTrack has some other functions for mapping time. Not sure if this will help. I don't think using AVAssetReader as I suggested will work for you as you can not go backwards as far as I am aware. – Steve McFarlin Jun 24 '11 at 23:36

2 Answers2

6

It isn't very obvious (since strangely it's only available in AVPlayerItem rather than AVPlayer) but if the AVPlayerItem returns YES for canStepForward/canStepBackward then you can call stepByCount:(NSInteger)stepCount: to move forwards or backwards by a certain number of frames.

martinjbaker
  • 1,424
  • 15
  • 23
  • This got me the tight nudging functionality that I required. Trying to use very small time increment was not only far less accurate, but computationally expensive in comparison. – Brian Hodge Nov 12 '15 at 00:43
5

The answer to your question is in the second line of code where you remove the "nominal" from nominalFrameRate.

Your code assumes that the frame rate in your video is constant. This is incorrect.

A video can have a constant fps, but it doesn't have to. E.g. if you shoot a film with your phone in variable lighting conditions then the frame exposure time can change and your frame rate will vary inversely.

To "snap" to frame boundaries, you need to use an AVAssetReader to step through the individual frames. Each frame is tagged with a Presentation Time Stamp (its "t"), but if you're stepping that doesn't matter so much, unless you're updating a position marker.

Rhythmic Fistman
  • 34,352
  • 5
  • 87
  • 159
  • Are you sure about this? Generally, frame rate does not depend on exposure time except when exposure time exceeds the time per frame. I don't know about iPhone, but this is a general rule of thumb. In my experience, video is almost always encoded with a fixed frame rate. – angularsen Jan 02 '13 at 08:12
  • 1
    You can convince yourself by examining the timestamps on iPhone-made movie files. – Rhythmic Fistman Jan 02 '13 at 22:21
  • You are absolutely right, and I should've Googled it. I too quickly assumed it was just another misunderstanding of exposure time vs frame rate. – angularsen Jan 03 '13 at 05:45