13

I need Need To extract Key Frames from Video/Stream.So is there any standard implementation. I am using open CV. (Currently i am extracting frames each second which is slower i need to improve performance.) So if any one has optimized implementation please reply in here.

Andrey Rubshtein
  • 20,795
  • 11
  • 69
  • 104
user1178178
  • 147
  • 1
  • 1
  • 6

5 Answers5

17

Using ffmpeg you can extract all key frames using the following code:

ffmpeg -vf select="eq(pict_type\,PICT_TYPE_I)" -i yourvideo.mp4 -vsync 2 -s 160x90 -f image2 thumbnails-%02d.jpeg

What follows -vf in a ffmpeg command line is a Filtergraph description. The select filter selects frames to pass in output. The constant of the filter is “pict_type” and the value “ PICT_TYPE_I”. So ffmpeg is only passing key frames to the output.

-vsync 2 prevents ffmpeg to generate more than one copy for each key frame.

-f image2 writes video frames to image files. The output filenames are specified by a pattern, which can be used to produce sequentially numbered series of files. The pattern may contain the string "%d" or "%0Nd".

Reference: http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code

Munarriz
  • 186
  • 3
  • 1
    I get `Undefined constant or missing '(' in 'PICT_TYPE_I)'` – puk Dec 18 '13 at 19:14
  • 2
    Hello ! That's a valuable command, however it's order is misplaced and you may be re-sizing the images too. You should put the -i video.mp4 first, before everything else. It goes like : ffmpeg -i "yourInputVideo.mp4" -vf select="eq(pict_type\,PICT_TYPE_I)" -vsync 2 -f image2 thumbnail_%02d.jpeg – Marcelo Ruggeri Oct 21 '14 at 00:57
  • According to [doc here](https://ffmpeg.org/ffmpeg-filters.html#select_002c-aselect), `pict_type` now takes `I` instead of `PICT_TYPE_I`. – jdhao Dec 03 '21 at 03:04
8

I will assume that a keyframe is a frame presenting content a lot different from the previous ones (it's not a formal definition, but it fits). Take frames i and i+1. Use cv2.absDiff to compute the difference between the frames and cv2.sumElems to get the sum of all pixels differences. Do this for all frames i. This will reduce your video stream to an one dimensional signal. Look for peaks in this signal and pick keyframes corresponding to these peaks. To find peaks choose a threshold on this signal either manually by finding a frame you deem to be key, and letting its error be the error threshold or automatically using statistics (e.g. any frame i+1 where the error is greater than 1 stdev from the mean error).

awiebe
  • 3,758
  • 4
  • 22
  • 33
TH.
  • 1,738
  • 1
  • 12
  • 15
4

If something wrong with the above code, try this argument order instead.

ffmpeg -i yourVideo.mp4 -vf select='eq(pict_type\,I)' -vsync 2 -s 160x90 -f image2 thumbnails-%02d.jpeg
backing
  • 127
  • 4
2

The ffmpeg solution should work well.

For someone facing problems with the select filter 'eq(pict_type\,PICT_TYPE_I)', you might want to try the filter as 'eq(pict_type\,I)'. This was broken for a while so some versions of ffmpeg might not recognize the constant. The fix can be seen here.

The final command which worked for me finally was:

ffmpeg -vf select='eq(pict_type\,I)' -i yourVideo.mp4 -vsync 2 -s 160x90 -f image2 thumbnails-%02d.jpeg
sanchitarora
  • 882
  • 9
  • 18
-1

You can use ffprobe to extract key frames. It is a tool in ffmpeg.

Use the command: 

ffprobe in.mp4 -select_streams v -show_entries frame=key_frame,pkt_pts_time -of csv=nk=1:p=0 | findstr "1,"
Nephilim
  • 494
  • 5
  • 25