When using ffmpeg to re-encode videos, this is pretty fast. But when using the -filter_complex command the process slows down to real time. Is this process bound to the CPU or GPU? Does anyone know if there is any way to speed this up using the GPU?
The full call is the following:
ffmpeg.exe
-i video_0_0.mp4 -i video_0_1.mp4 -i video_0_2.mp4 -i video_0_3.mp4
-filter_complex
"color=c=black:size=1920x1080 [base];
[0:v] setpts=PTS-STARTPTS, scale=960x540 [upperleft];
[1:v] setpts=PTS-STARTPTS, scale=960x540 [upperright];
[2:v] setpts=PTS-STARTPTS, scale=960x540 [lowerleft];
[3:v] setpts=PTS-STARTPTS, scale=960x540 [lowerright];
[base][upperleft] overlay=shortest=1 [tmp1];
[tmp1][upperright] overlay=shortest=1:x=960 [tmp2];
[tmp2][lowerleft] overlay=shortest=1:y=540 [tmp3];
[tmp3][lowerright] overlay=shortest=1:x=960:y=540"
-q 4 -y
"test.mp4"
output of the command:
ffmpeg.exe -i video_0_0.mp4 -i video_0_1.mp4 -i video_0_2.mp4 -i video_0_3.mp4 -filter_complex "color=c=black:size=1920x1080 [base]; [0:v] setpts=PTS-STARTPTS, scale=960x540 [upperleft]; [1:v] setpts=PTS-STARTPTS, scale=960x540 [upperright]; [2:v] setpts=PTS-STARTPTS, scale=960x540 [lowerleft]; [3:v] setpts=PTS-STARTPTS, scale=960x540 [lowerright]; [base][upperleft] overlay=shortest=1 [tmp1]; [tmp1][upperright] overlay=shortest=1:x=960 [tmp2]; [tmp2][lowerleft] overlay=shortest=1:y=540 [tmp3]; [tmp3][lowerright] overlay=shortest=1:x=960:y=540" -q 4 -y "test.mp4"
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 6.3.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
libavutil 55. 34.101 / 55. 34.101
libavcodec 57. 64.101 / 57. 64.101
libavformat 57. 56.101 / 57. 56.101
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
libpostproc 54. 1.100 / 54. 1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video_0_0.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.3.100
Duration: 00:00:05.80, start: 0.000000, bitrate: 14 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc), 1280x720 [SAR 1:1 DAR 16:9], 12 kb/s, 10 fps, 10 tbr, 10240 tbn, 20 tbc (default)
Metadata:
handler_name : VideoHandler
[h264 @ 000000000275a300] left block unavailable for requested intra mode
[h264 @ 000000000275a300] error while decoding MB 0 21, bytestream 927
[h264 @ 000000000275a300] concealing 1969 DC, 1969 AC, 1969 MV errors in I frame
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'video_0_1.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Session streamed with GStreamer
encoder : Lavf57.3.100
comment : rtsp-server
Duration: 00:00:09.76, start: 0.000000, bitrate: 106 kb/s
Stream #1:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1280x720 [SAR 1:1 DAR 16:9], 104 kb/s, 24.90 fps, 25 tbr, 90k tbn, 180k tbc (default)
Metadata:
handler_name : VideoHandler
[h264 @ 0000000002a2fb40] error while decoding MB 73 20, bytestream -7
[h264 @ 0000000002a2fb40] concealing 1976 DC, 1976 AC, 1976 MV errors in I frame
Input #2, mov,mp4,m4a,3gp,3g2,mj2, from 'video_0_2.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Session streamed with GStreamer
encoder : Lavf57.3.100
comment : rtsp-server
Duration: 00:00:09.76, start: 0.000000, bitrate: 83 kb/s
Stream #2:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1280x720 [SAR 1:1 DAR 16:9], 81 kb/s, 25.21 fps, 25 tbr, 90k tbn, 180k tbc (default)
Metadata:
handler_name : VideoHandler
[h264 @ 0000000002779920] error while decoding MB 22 29, bytestream -7
[h264 @ 0000000002779920] concealing 1307 DC, 1307 AC, 1307 MV errors in I frame
Input #3, mov,mp4,m4a,3gp,3g2,mj2, from 'video_0_3.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Session streamed with GStreamer
encoder : Lavf57.3.100
comment : rtsp-server
Duration: 00:00:09.40, start: 0.000000, bitrate: 44 kb/s
Stream #3:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1280x720 [SAR 1:1 DAR 16:9], 41 kb/s, 25.21 fps, 25 tbr, 90k tbn, 180k tbc (default)
Metadata:
handler_name : VideoHandler
[swscaler @ 000000000295a5e0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 00000000029a64c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 00000000029d89c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0000000002a7a160] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 0000000002a42080] -qscale is ignored, -crf is recommended.
[libx264 @ 0000000002a42080] using SAR=1/1
[libx264 @ 0000000002a42080] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0000000002a42080] profile High, level 4.0
[libx264 @ 0000000002a42080] 264 - core 148 r2762 90a61ec - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.56.101
Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
Metadata:
encoder : Lavc57.64.101 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
Stream #0:0 (h264) -> setpts
Stream #1:0 (h264) -> setpts
Stream #2:0 (h264) -> setpts
Stream #3:0 (h264) -> setpts
overlay -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[h264 @ 0000000002b12980] left block unavailable for requested intra mode
[h264 @ 0000000002b12980] error while decoding MB 0 21, bytestream 927
[h264 @ 0000000002b12980] concealing 1969 DC, 1969 AC, 1969 MV errors in I frame
[h264 @ 0000000002b14b40] error while decoding MB 73 20, bytestream -7
[h264 @ 0000000002b14b40] concealing 1976 DC, 1976 AC, 1976 MV errors in I frame
[h264 @ 00000000029195e0] error while decoding MB 22 29, bytestream -7
[h264 @ 00000000029195e0] concealing 1307 DC, 1307 AC, 1307 MV errors in I frame
[h264 @ 0000000002b15000] left block unavailable for requested intra4x4 mode -1 0x
[h264 @ 0000000002b15000] error while decoding MB 0 18, bytestream 293
[h264 @ 0000000002b15000] concealing 2209 DC, 2209 AC, 2209 MV errors in I frame
[h264 @ 0000000002b13800] left block unavailable for requested intra modeeed= 0x
[h264 @ 0000000002b13800] error while decoding MB 0 10, bytestream 1778
[h264 @ 0000000002b13800] concealing 2849 DC, 2849 AC, 2849 MV errors in I frame
[h264 @ 000000000291c600] left block unavailable for requested intra modeeed= 0x
[h264 @ 000000000291c600] error while decoding MB 0 32, bytestream 597
[h264 @ 000000000291c600] concealing 1089 DC, 1089 AC, 1089 MV errors in I frame
[h264 @ 0000000002b12e60] error while decoding MB 9 13, bytestream -5807.7kbits/s speed=0.0985x
[h264 @ 0000000002b12e60] concealing 2600 DC, 2600 AC, 2600 MV errors in I frame
[h264 @ 0000000002b12980] left block unavailable for requested intra4x4 mode -1/s speed=0.46x
[h264 @ 0000000002b12980] error while decoding MB 0 3, bytestream 1575
[h264 @ 0000000002b12980] concealing 3409 DC, 3409 AC, 3409 MV errors in I frame
[h264 @ 0000000002b159a0] left block unavailable for requested intra mode
[h264 @ 0000000002b159a0] error while decoding MB 0 14, bytestream 213
[h264 @ 0000000002b159a0] concealing 2529 DC, 2529 AC, 2529 MV errors in I frame
[h264 @ 0000000002918c40] error while decoding MB 63 30, bytestream -761.8kbits/s speed=0.642x
[h264 @ 0000000002918c40] concealing 1186 DC, 1186 AC, 1186 MV errors in I frame
[h264 @ 0000000002b14660] error while decoding MB 58 33, bytestream 222
[h264 @ 0000000002b14660] concealing 951 DC, 951 AC, 951 MV errors in I frame
[h264 @ 0000000002b15e80] left block unavailable for requested intra4x4 mode -1
[h264 @ 0000000002b15e80] error while decoding MB 0 30, bytestream 863
[h264 @ 0000000002b15e80] concealing 1249 DC, 1249 AC, 1249 MV errors in I frame
frame= 143 fps= 25 q=-1.0 Lsize= 200kB time=00:00:05.60 bitrate= 293.2kbits/s speed=0.994x
video:198kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.268034%
[libx264 @ 0000000002a42080] frame I:1 Avg QP:14.64 size: 31429
[libx264 @ 0000000002a42080] frame P:36 Avg QP:16.42 size: 3690
[libx264 @ 0000000002a42080] frame B:106 Avg QP:18.41 size: 356
[libx264 @ 0000000002a42080] consecutive B-frames: 0.7% 1.4% 0.0% 97.9%
[libx264 @ 0000000002a42080] mb I I16..4: 33.7% 58.7% 7.6%
[libx264 @ 0000000002a42080] mb P I16..4: 0.4% 1.0% 0.2% P16..4: 6.0% 2.2% 1.5% 0.0% 0.0% skip:88.7%
[libx264 @ 0000000002a42080] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 4.3% 0.0% 0.0% direct: 0.0% skip:95.7% L0:34.4% L1:65.5% BI: 0.1%
[libx264 @ 0000000002a42080] 8x8 transform intra:59.7% inter:83.8%
[libx264 @ 0000000002a42080] coded y,uvDC,uvAC intra: 34.4% 17.3% 7.8% inter: 0.6% 0.6% 0.0%
[libx264 @ 0000000002a42080] i16 v,h,dc,p: 71% 17% 3% 10%
[libx264 @ 0000000002a42080] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 47% 22% 15% 2% 3% 3% 3% 2% 2%
[libx264 @ 0000000002a42080] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 47% 27% 11% 2% 4% 4% 3% 2% 1%
[libx264 @ 0000000002a42080] i8c dc,h,v,p: 60% 24% 15% 1%
[libx264 @ 0000000002a42080] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000000002a42080] ref P L0: 84.6% 2.6% 10.6% 2.2%
[libx264 @ 0000000002a42080] ref B L0: 58.1% 41.5% 0.5%
[libx264 @ 0000000002a42080] ref B L1: 95.8% 4.2%
[libx264 @ 0000000002a42080] kb/s:282.50
