0

I have an ffmpeg version built with VMAF library. I can use it to calculate the VMAF scores of a distorted video against a reference video using commands like this:

ffmpeg -i distorted.mp4 -i original.mp4 -filter_complex "[0:v]scale=640:480:flags=bicubic[main];[main][1:v]libvmaf=model_path=model/vmaf_v0.6.1.json:log_path=log.json" -f null -

Now, I remember there was a way to get VMAF scores while performing regular ffmpeg encoding. How can I do that at the same time?

I want to encode a video like this, while also calulate the VMAF of the output file:

ffmpeg -i original.mp4 -crf 27 -s 640x480 out.mp4
Mary
  • 393
  • 1
  • 4
  • 17
  • Good answer. But well, I'm on Windows! – Mary Sep 08 '22 at 22:15
  • I am also on Windows. – Rotem Sep 08 '22 at 22:16
  • Didn't work: `Streamcopy requested for output stream 0:0, which is fed from a complex filtergraph. Filtering and streamcopy cannot be used together.` – Mary Sep 08 '22 at 22:28
  • Right... Try: `ffmpeg -i in.mp4 -vcodec libx264 -crf 27 -f h264 pipe: | ffmpeg -y -i in.mp4 -f h264 -i pipe: -filter_complex "[0:v][1:v]libvmaf=log_path=log.json" -map 1 -c:v:1 copy out.mp4` – Rotem Sep 08 '22 at 22:30
  • Still didn't work: `Cannot determine format of input stream 1:0 after EOF. Error marking filters as finished. Conversion failed!` – Mary Sep 08 '22 at 22:33
  • It's not working as I thought is supposed to work. Using `tee` filter is a better solution. – Rotem Sep 08 '22 at 22:35
  • Ha, @Rotem was ahead of me with the idea I commented below... how about using a streaming format? – kesh Sep 08 '22 at 22:44

1 Answers1

0

[edited]

Alright, scratch what I said earlier...

You should be able to use [the `tee` muxer](http://ffmpeg.org/ffmpeg-formats.html#tee-1) to save the file and pipe the encoded frames to another ffmpeg process. Something like this should work for you: 
ffmpeg -i original.mp4 -crf 27 -s 640x480 -f tee "out.mp4 | [f=mp4]-" \
  | ffmpeg -i - -i original.mp4 -filter_complex ...

(make them into 2 lines and remove \ for Windows)

Here is what works on my Windows PC (thanks to @Rotem for his help)

ffmpeg -i in.mp4 -vcodec libx264 -crf 27 -f nut pipe: 
| 
ffmpeg -i in.mp4 -f nut -i pipe: 
  -filter_complex "[0:v][1:v]libvmaf=log_fmt=json:log_path=log.json,nullsink" 
  -map 1 -c copy out.mp4

The main issue that @Rotem and I missed is that we need to terminate the libvmaf's output. Also, h264 raw format does not carry header info, and using `nut alleviates that issue.

There are a couple caveats:

  1. testing with the testsrc example that @Rotem suggested in the comment below does not produce any libvmaf log, at least as far as I can see, but in debug mode, you can see the filter is getting initialized.

  2. You'll likely see [nut @ 0000026b123afb80] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8) message in the log. This just means that the frames are piped in faster than the 2nd ffmpeg is processing. FFmpeg does block on both ends, so no info should be lost.

For the full disclosure, I posted my Python test script on GitHub. It just runs the shell command, so it should be easy to follow even if you don't do Python.

kesh
  • 4,515
  • 2
  • 12
  • 20
  • Good answer. But well, I'm on Windows! And I thought VMAF itself allows a same-time transcoding as well... – Mary Sep 08 '22 at 22:14
  • Wait, shouldn't it work in Windows? (I'm in Windows too but don't know because I use pipes only in Python) – kesh Sep 08 '22 at 22:22
  • hmm idk honestly. Let me try...But I wish there was a way to calculate VMAF while transcoding to save time... – Mary Sep 08 '22 at 22:24
  • @kesh The pipes are working, the \ is not (may use **^**, but better put everything in one line). I think there is also a space missing. Check it in command line. – Rotem Sep 08 '22 at 22:27
  • @Rotem - Ha good call (I liberally use "\" to indicate continuing line but I guess I need to be more sensible. Not sure about the missing space. For the `tee` output (as I added)? – kesh Sep 08 '22 at 22:36
  • @Mary - I think you are in a bind with the mp4 container. Because its MOOV header won't be written until encoding is complete, the second FFmpeg cannot start decoding till complete. So, piping actually isn't help you at all. It actually is a hinderance (needlessly hogging memory). What you may want to do is to make the first ffmpeg to output raw stream (nut maybe or any streaming format would work?) and the second ffmpeg to do the VMAF computation and MP4 packaging with `-c copy`. – kesh Sep 08 '22 at 22:42
  • @kesh The following command should work: `ffmpeg -i original.mp4 -vcodec libx264 -crf 27 -map 0 -f tee "out.mp4|[f=h264]pipe:" | ffmpeg -f h264 -i pipe: -i original.mp4 -filter_complex "[1:v][0:v]libvmaf=log_path=log.json" -f null pipe:` – Rotem Sep 08 '22 at 22:45
  • It doesn't work: `pipe:: Invalid data found when processing input` – Mary Sep 08 '22 at 23:21
  • @Rotem and your answer still error: `Cannot determine format of input stream 0:0 after EOF. Error marking filters as finished` – Mary Sep 08 '22 at 23:22
  • 1
    @Mary I tested using ffmpeg version [5.1.1-full_build-www.gyan.dev](https://www.gyan.dev/ffmpeg/builds/#release-builds). `ffmpeg-release-full.7z`. The input file used for testing is synthetic video: `ffmpeg -f lavfi -i testsrc=size=640x480:rate=1:duration=10 in.mp4`. Total VMAF score: `99.727450`. It's not an answer, I am just trying to help kesh improving the answer. – Rotem Sep 09 '22 at 07:59
  • 1
    I got a `nut` piping version to work on my PC... well sort of. `libavmaf` is not producing a log file but I can see it initialized. Try it out. – kesh Sep 09 '22 at 14:16