2

The code below was inspired by other posts on SO and extracts an image from a video. Unfortunately, the image looks blurry even though the video looks sharp and fully in focus.

Is there something wrong with the code, or is this a natural difficulty of extracting images from videos?

func getImageFromVideo(videoURL: String) -> UIImage {
    do {
        let asset = AVURLAsset(URL: NSURL(fileURLWithPath: videoURL), options: nil)
        let imgGenerator = AVAssetImageGenerator(asset: asset)
        imgGenerator.appliesPreferredTrackTransform = true
        let cgImage = try imgGenerator.copyCGImageAtTime(CMTimeMake(0, 1), actualTime: nil)
        let image = UIImage(CGImage: cgImage)
        return image
    } catch {
        ...
    }
}
Crashalot
  • 33,605
  • 61
  • 269
  • 439
  • I just tried your code with a video of mine and the extracted image is normal, not blurry. Maybe it's a problem with the framerate or encoding of your video? Try grabbing a different CMTime to see if it's still blurry. – Eric Aya Jan 20 '16 at 10:32
  • Thanks, which CMTime do you suggest? Did you try only one image or multiple images? – Crashalot Jan 20 '16 at 10:47
  • 1
    With `CMTimeMake` the first argument is the value and the second argument is the timescale. Your timescale is 1, so the value is in seconds: 0 means 1st second, 1 means 2nd second, etc. With a timescale of 4, the value would be 1/4th of a second. Try finding a CMTime that falls right on a steady frame (it depends on your video encoding and framerate, you'll have to make tests). – Eric Aya Jan 20 '16 at 10:53
  • Cool thanks! Any suggestions on how to find a steady frame based on the video encoding and framerate? The framerate is 24. – Crashalot Jan 20 '16 at 10:54
  • T clarify, based on your explanation of CMTime, this suggests the code pulls the image at the 1st second? That is odd since the video is < 0.2 seconds long. The app takes video as long as you press on a button, so if you just tap, the video will be around 0.2 seconds, and from that the goal is to extract a sharp image. – Crashalot Jan 20 '16 at 10:58
  • 1
    With your current CMTime it grabs the first frame of the first second: that's the first frame of the video (even if the video is less than 1s). // Try a timescale that is a multiple of your framerate: start with 24 and make tests, for example (with a timescale of 24, the first value represents a 1/24th second, which should be exactly one frame for you). – Eric Aya Jan 20 '16 at 11:01
  • Thanks so much! How about suggestions for accounting for the encoding value? It's a little better by using 24 as the time scale, so hopefully incorporating encoding will fix this. Though why does it work for you? If it makes any difference, we're using PBJ Vision to capture video: https://github.com/piemonte/PBJVision – Crashalot Jan 20 '16 at 11:08
  • You need to extract particular frame? Or you need all frames? Code looks ok, I use almost the same for capturing screenshots. – rkyr Jan 20 '16 at 21:25
  • Just need a frame that is sharp. Unfortunately,1 out of 2 times the frame extracted is blurry. – Crashalot Jan 20 '16 at 21:32
  • @EricD. it appears to be an issue with video stabilization? Laying the phone flat on a desk (i.e., no holding phone by hand) produces sharp images. PBJVision already uses setPreferredVideoStabilizationMode:AVCaptureVideoStabilizationModeAuto. Is there something else to ensure the captured video is not blurry at the start? – Crashalot Jan 20 '16 at 21:56
  • `Thanks so much! ` You're welcome. `Is there something else to ensure the captured video is not blurry at the start?` I don't know the answer to that. Maybe it deserves to post a new question or edit this one. – Eric Aya Jan 20 '16 at 22:01
  • @EricD. Thanks will do that. But how about suggestions to incorporate the encoding? That might help like your frame rate suggestion. – Crashalot Jan 20 '16 at 23:12
  • Thanks again for all the help @EricD. And beyond the encoding suggestion, the new question was posted here if you want to take a peek: http://stackoverflow.com/questions/34912050/avoiding-blurriness-at-start-end-of-video-even-after-using-setpreferredvideos?noredirect=1#comment57575685_34912050. – Crashalot Jan 21 '16 at 11:00
  • There was some interesting information in this discussion, so I've made an answer from my comments. – Eric Aya Jan 21 '16 at 14:03

1 Answers1

2

Your code is working without errors or problems. I've tried with a video and the grabbed image was not blurry.

I would try to debug this by using a different timescale for CMTime.

With CMTimeMake, the first argument is the value and the second argument is the timescale.

Your timescale is 1, so the value is in seconds. A value of 0 means 1st second, a value of 1 means 2nd second, etc. Actually it means the first frame after the designated location in the timeline.

With your current CMTime it grabs the first frame of the first second: that's the first frame of the video (even if the video is less than 1s).

With a timescale of 4, the value would be 1/4th of a second. Etc.

Try finding a CMTime that falls right on a steady frame (it depends on your video framerate, you'll have to make tests).

For example if your video is at 24 fps, then to grab exactly one frame of video, the timescale should be at 24 (that way each value unit would represent a whole frame):

let cgImage = try imgGenerator.copyCGImageAtTime(CMTimeMake(0, 24), actualTime: nil)

On the other hand, you mention that only the first and last frames of the video are blurry. As you rightly guessed, it's probably the actual cause of your issue and is caused by a lack of device stabilization.

A note: the encoding of the video might also play a role. Some MPG encoders create incomplete and interpolated frames that are "recreated" when the video plays, but these frames can appear blurry when grabbed with copyCGImageAtTime. The only solution I've found for this rare problem is to grab another frame just before or just after the blurry one.

Eric Aya
  • 69,473
  • 35
  • 181
  • 253
  • Thanks @EricD for all your help! We chose PBJVision because someone said capturing videos & images was complex in Swift. But looking into it more, is this true from your experience? From this SO post, however, it doesn't seem complex at all: http://stackoverflow.com/questions/33249041/how-to-record-a-video-with-avfoundation-in-swift. Is using a 3rd party library advisable for capturing videos & images? – Crashalot Jan 21 '16 at 21:06
  • 1
    You're welcome. // I'm afraid it would only be a matter of opinion. Mine would be: use what you need *now*, you can always decide to add or remove complexity later. AVFoundation and the like are indeed rather easy to use with Swift but can't do complex stuff and you can't really tweak performances. Start simple, upgrade if necessary - but again, it's only an opinion. It always *depends*. :) – Eric Aya Jan 21 '16 at 21:21