FFMPEG with VP9 + RTSP

Question

When we use HEVC + RTSP, ffmpeg works fine. The stream is captured by webcam.

ffmpeg -f dshow -rtbufsize 100M  -f vfwcap -i "0" -strict experimental -c:v hevc_qsv -f rtsp -rtsp_transport tcp rtsp://127.0.0.1/live/test

The output is like:

frame=  355 fps= 37 q=-0.0 size=N/A time=00:00:11.27 bitrate=N/A dup=159 drop=0 speed=1.18x

Then we switch to VP9 + RTSP.

ffmpeg -f dshow -rtbufsize 100M  -f vfwcap -i "0" -strict experimental -c:v libvpx-vp9 -f rtsp -rtsp_transport tcp rtsp://127.0.0.1/live/test

The fps and speed looks too slow:

frame=  263 fps=9.5 q=0.0 size=N/A time=00:00:07.94 bitrate=N/A dup=119 drop=0 speed=0.288x

Then we use ffplay to play the stream. There are a lot of error messages and the video seems frozen.

[vp9 @ 000001c3da86c200] Not all references are available
[vp9 @ 000001c3dffd5700] Not all references are available
[vp9 @ 000001c3da8477c0] Not all references are available
[vp9 @ 000001c3da848240] Not all references are available
[vp9 @ 000001c3da848e40] Not all references are available
[vp9 @ 000001c3e01d2fc0] Not all references are available
[vp9 @ 000001c3e01dd280] Not all references are available
[vp9 @ 000001c3e01e8980] Not all references are available
[vp9 @ 000001c3e01f1880] Not all references are available
[vp9 @ 000001c3da86c200] Not all references are available
[vp9 @ 000001c3dffd5700] Not all references are available
[vp9 @ 000001c3da8477c0] Not all references are available
[vp9 @ 000001c3da848240] Not all references are available
[vp9 @ 000001c3da848e40] Not all references are available
[vp9 @ 000001c3e01d2fc0] Not all references are available
[vp9 @ 000001c3e01dd280] Not all references are available0B f=0/0
[vp9 @ 000001c3e01e8980] Not all references are available
[vp9 @ 000001c3e01f1880] Not all references are available
[vp9 @ 000001c3da86c200] Not all references are available
[vp9 @ 000001c3dffd5700] Not all references are available
[vp9 @ 000001c3da8477c0] Not all references are available
[vp9 @ 000001c3da848240] Not all references are available
[vp9 @ 000001c3da848e40] Not all references are available
[vp9 @ 000001c3e01d2fc0] Not all references are available
[vp9 @ 000001c3e01dd280] Not all references are available
[vp9 @ 000001c3e01e8980] Not all references are available
[vp9 @ 000001c3e01f1880] Not all references are available

Finally, we try hardware VP9 encoder.

ffmpeg -f dshow -rtbufsize 100M  -f vfwcap -i "0" -strict experimental -c:v vp9_qsv -f rtsp -rtsp_transport tcp rtsp://127.0.0.1/live/test

It can't work.

[swscaler @ 0000021e88df9fc0] deprecated pixel format used, make sure you did set range correctly
[vp9_qsv @ 0000021e8687df80] Selected ratecontrol mode is unsupported
[vp9_qsv @ 0000021e8687df80] Low power mode is unsupported
[vp9_qsv @ 0000021e8687df80] Current frame rate is unsupported
[vp9_qsv @ 0000021e8687df80] Current picture structure is unsupported
[vp9_qsv @ 0000021e8687df80] Current resolution is unsupported
[vp9_qsv @ 0000021e8687df80] Current pixel format is unsupported
[vp9_qsv @ 0000021e8687df80] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

So, how can we deal with VP9 + RTSP successfully?

What CPU are you trying to encode on? [Wikipedia says](https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video) that VP9 hardware encoding is supported since Kaby Lake (7xxx) but the [ffmpeg mailing list](http://www.ffmpeg-archive.org/Is-it-vp9-qsv-encoder-work-on-i9-9900k-CPU-td4690003.html) says it requires Ice Lake (10xxx). I don't know which is correct. — Coderer, Dec 31 '20 at 15:30

score 2 · Accepted Answer · answered Jan 04 '21 at 12:03

I am by no means an authority on this, but I'm in a similar situation to yours. If I understand this GitHub issue correctly, the "gen 9" Intel GPU -- "Kaby Lake, Coffee Lake, Whiskey Lake, Comet Lake" per Wikipedia, which I think corresponds to 7xxx through 10xxx -- has hardware support for VP9 encoding, but the Windows drivers do not expose this feature for the Media SDK to take advantage of. I think that the error messages we're seeing reflect the fact that QSV encoding is not available on our particular CPU model, on our OS.

For some reason, the feature is available on Linux. I have seen several sources that say VA API (vp9_vaapi) should work on any of these "gen 9" processors; it's possible that the QSV driver (vp9_qsv) would work in Linux as well. I haven't had a chance to try it yet.

FFMPEG with VP9 + RTSP

1 Answers1