Swift - get all frames from video

Question

I am following this code to get all frames from video. In this link he is trying to get a frame at a specific time. But I need to get all frames. Here is my code...

var mutableVideoURL = NSURL()
var videoFrames = [UIImage]()

let asset : AVAsset = AVAsset(url: self.mutableVideoURL as URL)
let mutableVideoDuration = CMTimeGetSeconds(asset.duration)
print("-----Mutable video duration = \(mutableVideoDuration)")
let mutableVideoDurationIntValue = Int(mutableVideoDuration)
print("-----Int value of mutable video duration = \(mutableVideoDurationIntValue)")

for index in 0..<mutableVideoDurationIntValue {
   self.generateFrames(url: self.mutableVideoURL, fromTime: Float64(index))
}

    func generateFrames(url : NSURL, fromTime:Float64) {
        let asset: AVAsset = AVAsset(url: url as URL)
        let assetImgGenerate : AVAssetImageGenerator = AVAssetImageGenerator(asset: asset)
        assetImgGenerate.appliesPreferredTrackTransform = true
        let time        : CMTime = CMTimeMakeWithSeconds(fromTime, 600)
        var img: CGImage?
        do {
            img = try assetImgGenerate.copyCGImage(at:time, actualTime: nil)
        } catch {
        }
        if img != nil {
            let frameImg: UIImage = UIImage(cgImage: img!)
            UIImageWriteToSavedPhotosAlbum(frameImg, nil, nil, nil)//I saved here to check
            videoFrames.append(frameImg)
            print("-----Array of video frames *** \(videoFrames)")
        } else {
              print("error !!!")
        }
    }

I tested this code with 2 videos(length of the videos are 5 seconds and 3.45 minutes). This code works perfectly with the small duration(video length: 5 seconds) and with long duration (video length: 3.45 minutes), NSLog shows Message from debugger: Terminated due to memory issue Any assistance would be appreciated.

what's the error? you can put `print(error.localizedDescription)` in catch — Willjay, Mar 08 '17 at 07:29
Well that means there are too many frames in memory so it's crashing. Essentially, you'd need to process and loop through the images in a less memory intensive way. — Harish, Mar 24 '17 at 16:03

vicegax · Accepted Answer · 2018-07-11T11:32:34.730

When generating more than 1 frame Apple recommends using the method: generateCGImagesAsynchronously(forTimes:completionHandler:)

Still, if you prefer to follow your current approach there are a couple of improvements you could do to reduce memory usage:

You are instantiating AVAsset and AVAssetImageGenerator inside the loop, you could instantiate them just once and send it to the method generateFrames.
Remove the line

UIImageWriteToSavedPhotosAlbum(frameImg, nil, nil, nil)//I saved here to check

because you are saving every frame in the photos album, that takes extra memory.

Final result could look like this:

var videoFrames:[UIImage] = [UIImage]
let asset:AVAsset = AVAsset(url:self.mutableVideoURL as URL)
let assetImgGenerate:AVAssetImageGenerator = AVAssetImageGenerator(asset:asset)
assetImgGenerate.appliesPreferredTrackTransform = true
let duration:Float64 = CMTimeGetSeconds(asset.duration)
let durationInt:Int = Int(mutableVideoDuration)

for index:Int in 0 ..< durationInt
{
   generateFrames(
      assetImgGenerate:assetImgGenerate,
      fromTime:Float64(index))
}

func generateFrames(
   assetImgGenerate:AVAssetImageGenerator,
   fromTime:Float64)
{
   let time:CMTime = CMTimeMakeWithSeconds(fromTime, 600)
   let cgImage:CGImage?

   do
   {
      cgImage = try assetImgGenerate.copyCGImage(at:time, actualTime:nil)
   }
   catch
   {
      cgImage = nil
   }

   guard

      let img:CGImage = cgImage

   else
   {
       continue
   }

   let frameImg:UIImage = UIImage(cgImage:img)
   videoFrames.append(frameImg)
}

Update for Swift 4.2

var videoUrl:URL // use your own url
var frames:[UIImage]
private var generator:AVAssetImageGenerator!

func getAllFrames() {
   let asset:AVAsset = AVAsset(url:self.videoUrl)
   let duration:Float64 = CMTimeGetSeconds(asset.duration)
   self.generator = AVAssetImageGenerator(asset:asset)
   self.generator.appliesPreferredTrackTransform = true
   self.frames = []
   for index:Int in 0 ..< Int(duration) {
      self.getFrame(fromTime:Float64(index))
   }
   self.generator = nil
}

private func getFrame(fromTime:Float64) {
    let time:CMTime = CMTimeMakeWithSeconds(fromTime, preferredTimescale:600)
    let image:CGImage
    do {
       try image = self.generator.copyCGImage(at:time, actualTime:nil)
    } catch {
       return
    }
    self.frames.append(UIImage(cgImage:image))
}

@BijenderSinghShekhawat I updated the code. Hope it helps you. — vicegax, Jul 12 '18 at 06:22
when I am storing a large number of images in the array. the app is crashing. — pushpank, Jan 24 '20 at 05:52
@pushpank Probably it crashes if you run out of memory. Can you show the crash log or the message in the console? — vicegax, Jan 24 '20 at 10:30
It's most likely crashing because of `fromTime` . For example `getFrame(2)` will not return the image at `2` seconds. it will return the image at `2/600` which is `0.003 seconds` *this is a lot of images to return* — Lxrd-AJ, Mar 12 '23 at 12:33

Swift - get all frames from video

1 Answers1

Linked