3

Me and my team are trying to re-encode a video file to a more "gify" feeling by changing the video frame rate. We are using the following properties for the AVAssetWriterInput:

let videoSettings:[String:Any] = [
            AVVideoCodecKey: AVVideoCodecH264,
            AVVideoHeightKey: videoTrack.naturalSize.height,
            AVVideoWidthKey: videoTrack.naturalSize.width,
            AVVideoCompressionPropertiesKey: [AVVideoExpectedSourceFrameRateKey: NSNumber(value: 12)]                                         
        ]

But the output video keep playing in the normal frame rate (played using AVPlayer).

What is the right way to reduce video frame rate? (12 for example).

Any help in the right direction would be HIGHLY approcated. We stuck. Best regards, Roi

rmaddy
  • 314,917
  • 42
  • 532
  • 579
Roi Mulia
  • 5,626
  • 11
  • 54
  • 105

1 Answers1

4

You can control the timing of each sample you append to your AVAssetWriterInput directly with CMSampleBufferCreateCopyWithNewTiming.

You need to adjust the timing in the CMSampleTimingInfo you provide. Retrieve current timing info with CMSampleBufferGetOutputSampleTimingInfoArray and just go over the duration of each sample and calculate the correct duration to get 12 frames per second and adjust presentation and decode timestamps to match this new duration. You then make your copy and feed it to your writer's input.

Let's say you have existingSampleBuffer:

CMSampleBufferRef sampleBufferToWrite = NULL;
CMSampleTimingInfo sampleTimingInfo = {0};

CMSampleBufferGetSampleTimingInfo(existingSampleBuffer, 0, &sampleTimingInfo);

// modify duration & presentationTimeStamp
sampleTimingInfo.duration = CMTimeMake(1, 12) // or whatever frame rate you desire
sampleTimingInfo.presentationTimeStamp = CMTimeAdd(previousPresentationTimeStamp, sampleTimingInfo.duration);
previousPresentationTimeStamp = sampleTimingInfo.presentationTimeStamp; // should be initialised before passing here the first time

OSStatus status = CMSampleBufferCreateCopyWithNewTiming(kCFAllocatorDefault, existingSampleBuffer, 1, &sampleTimingInfo, &sampleBufferToWrite);

if (status == noErr) {
    // you can write sampleBufferToWrite
}

I'm making some assumptions in this code:

  • SampleBuffer contains only one sample
  • SampleBuffer contains uncompressed video (otherwise, you need to handle decodeTimeStamp as well)
Valérian
  • 1,068
  • 8
  • 10
  • 1
    Hey Valerian! Thank you for replying. any chance you can provide some code samples we can work with? Our brain CPU reached 150% haha. – Roi Mulia Mar 12 '18 at 13:40
  • 1
    Hey Valerian. We tried searching Google and understand what you meant. But we couldn't get it work. Please, if you can share more info or share code it'll be perfect.. – Roi Mulia Mar 12 '18 at 16:45
  • 1
    Quickly added some code to my post... it's ObjC though but I hope you get the gist :-) – Valérian Mar 12 '18 at 19:14
  • 1
    It works Valerian! AVFoundation really requires Phd. Thank you so much! We found that altho this do slows down the video to 15 fps, it's make the video longer (obviously). So an easy fix would be to remove the video frames as well. Make the video the same duration, but with less frames and less FPS. Give it that GIFY feeling. Do you happen to know if it's possible to skip frames inside the copyNextSampleBuffer() function? – Roi Mulia Mar 12 '18 at 23:02
  • 1
    Bottom line, we are trying to reduce the frame rate while keeping the same duration (meaning we need some how to drop or remove frames). Is this possible in theory? – Roi Mulia Mar 12 '18 at 23:08
  • 1
    Sure, let's say you have a 24 fps video and you want it to be a 12 fps one, you can simply double the duration (`sampleTimingInfo.duration = CMTimeMultiply(sampleTimingInfo.duration, 2)`) and drop the next sample buffer. No need to adjust the presentation time stamp in that case. – Valérian Mar 13 '18 at 08:38
  • 1
    Hey Valerian! Thank you for replying and helping us. We are trying to skip the frame buffer but it's simply not giving us the option. Any chance you can perform last edit to your code to show how to achieve this? (We are trying to keep the video total duration, while reducing frame rate) – Roi Mulia Mar 13 '18 at 09:42
  • 1
    We are trying to understand the following, If we have 180 samples (6 seconds 30 fps). And we want to create 15 fps movie (90 samples, 6 seconds), we need to "drop" every second frame, so the final movie will be continuous. For that I have to go through all the copyNextSampleBuffer() calls. And skip every second frame? – Roi Mulia Mar 13 '18 at 09:50
  • 1
    Yes and double the duration of the sample you append to the writer. – Valérian Mar 13 '18 at 10:54
  • 1
    It crashes, like the buffer is out of samples :/ If I’ll double the duration won’t it exceed the total duration at some point? – Roi Mulia Mar 13 '18 at 10:57
  • 1
    No because you're dropping every other frame. I'll try to answer on your new question later on – Valérian Mar 13 '18 at 11:24
  • 1
    Hey Valerian! Thank you so much. I'll be waiting for your respond – Roi Mulia Mar 13 '18 at 12:19
  • Roi do you happen to have the swift code... I am trying to achieve exactly what u are trying to do.. Could not figure out how to interact this with my writerinput. thanks.. – Koray Birand May 20 '19 at 22:03