8

I'm trying to measure how much a movies is "fast" (more action in the screen and quick scene chances). I don't want just a single value for the movie, but values along the movie to see how the action varies during it. After normalize the frame rate of the movies (10 fps), my idea is to compare each frame with the previous. I'm not only interest if the scene has changed, but also, if there was no cut, how much movement there is. Not only people/object movement but also, camera movement. In summary the paced (I think that the term) of the scenes.

My idea was to use the scene function from ffmpeg as a metric. But looking at the document and examples online I'm thinking I can only use the value of the Scene Change Detection as a threshold to return frames informations, but I can't get ffmpeg to return the value. Is that right? There is any way I can make it return the value?

Community
  • 1
  • 1
celacanto
  • 315
  • 2
  • 11

4 Answers4

15

Use

ffmpeg -i in.mp4 -vf "select='gte(scene,0)',metadata=print:file=scenescores.txt" -an -f null -

The text file created will have output like this:

...
frame:1440 pts:737280  pts_time:48     
lavfi.scene_score=0.003069
frame:1441 pts:737792  pts_time:48.0333
lavfi.scene_score=0.001593
frame:1442 pts:738304  pts_time:48.0667
lavfi.scene_score=0.000077
frame:1443 pts:738816  pts_time:48.1   
lavfi.scene_score=0.002219
...
William Miller
  • 9,839
  • 3
  • 25
  • 46
Gyan
  • 85,394
  • 9
  • 169
  • 201
  • It return `[AVFilterGraph @ 0x12856e0] No such filter: 'metadata'` and `Error opening filters!`. Any idea why? – celacanto Nov 22 '16 at 20:12
  • 2
    The filter was added in Feb 2016. You need a binary of a newer build. – Gyan Nov 22 '16 at 20:21
  • thanks, it work. Just so other people know. I had install ffmpeg using `apt get`. To get the most recent version I followed [this steps](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu) – celacanto Nov 23 '16 at 17:57
3

I found it hard to parse the ffmpeg output, so I created a wrapper in Python:

pip3 install scenecut_extractor

It will by default extract the scene cuts based on a threshhold, so it'll give you all that are above the parameter:

$ scenecut_extractor /path/to/file.mp4

Will output JSON by default:

[
  {
    "frame": 114,
    "pts": 114.0,
    "pts_time": 3.8,
    "score": 0.445904
  },
  {
    "frame": 159,
    "pts": 159.0,
    "pts_time": 5.3,
    "score": 0.440126
  }
]

You can set -t 0 to extract everything, i.e. the probabilities of each frame. Check -h for more options, for example for outputting to CSV:

$ scenecut_extractor test/test.mp4 -of csv
frame,pts,pts_time,score
24,12288.0,0.96,1.0
49,25088.0,1.96,1.0
74,37888.0,2.96,1.0
99,50688.0,3.96,1.0
124,63488.0,4.96,1.0
149,76288.0,5.96,1.0
174,89088.0,6.96,1.0
slhck
  • 36,575
  • 28
  • 148
  • 201
  • Similarly, I have this bash one liner to produce a csv. It certainly looks horrible but indeed it works. It may seem to hang if your video is long and this is because the ffmpeg process must finish before the results are sent to stdout. `{ echo "frame pts pts_time lavfi.scene_score"; ffmpeg -v 0 -i $INPUT -vf "select='gte(scene,0)',metadata=print:file=/dev/stdout" -f null - | awk 'ORS=NR%2?" ":"\n"' | awk '{$1=$1};1' | sed 's/fi.//' | tr -d '[:alpha:]_:=' | sed 's/ /,/g'; } > scenescores.csv` – blahreport Sep 30 '21 at 19:52
  • @blahreport You should post that as a separate answer! – slhck Dec 09 '21 at 08:11
0

As per the suggestion of @slhck reproducing my bash solution I gave in another comment.

{ echo "frame pts pts_time lavfi.scene_score"; ffmpeg -v 0 -i $INPUT -vf "select='gte(scene,0)',metadata=print:file=/dev/stdout" -f null - | awk 'ORS=NR%2?" ":"\n"' | awk '{$1=$1};1' | sed 's/fi.//' | tr -d '[:alpha:]_:=' | sed 's/ /,/g'; } > scenescores.csv
blahreport
  • 1,000
  • 6
  • 10
0

The above by @blahreport works fine until recently ffmpeg returns pts_time in scientific value (e.g. 1.0e-6). I made the following change and it managed to get over the problem.

echo "frame,pts,pts_time,score"; \
ffmpeg -hide_banner -loglevel error -i $fin \
-vf "select='eq(pict_type\,I)',select='gt(scene\,0)',metadata=print:file=/dev/stdout" \
-vsync vfr -f null - \
| awk 'ORS=NR%2?" ":"\n"' \
| awk '{$1=$1};1' \
| sed 's/fi.//' \
| awk -F'[ :=]' '{print $2","$4","$6","$8}'

Note: Based on my experience, you need to include select I-frame as the scene score of non-I frames are mostly noises.

Cee Wee
  • 11
  • 1