Writing frames from camera using skvideo.io.FFmpegWriter

Question

I'm trying to finely control the video encoding of camera image frames captured on the fly using skvideo.io.FFmpegWriter and cv2.VideoCapture, e.g.

from skvideo import io
import cv2

fps = 60
stream = cv2.VideoCapture(0)                    # 0 is for /dev/video0
print("fps: {}".format(stream.set(cv2.CAP_PROP_FPS, fps)))

stream.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
stream.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
print("bit_depth: {}".format(stream.set(cv2.CAP_PROP_FORMAT, cv2.CV_8U)))

video = io.FFmpegWriter('/tmp/test_ffmpeg.avi', 
            inputdict={'-r': fps, '-width': 1920, '-height': 1080},
            outputdict={'-r': fps, '-vcodec': 'libx264', '-pix_fmt': 'h264'}
)

try:
    for i in range(fps*10):  # 10s of video
        ret, frame = stream.read()
        video.writeFrame(frame)
finally:
    stream.release()

try:
    video.close()
except:
    pass

However, I get the following exception (in Jupyter notebook):

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)

<ipython-input-33-007a547c4229> in <module>()
     18     while range(fps*10):
     19         ret, frame = stream.read()
---> 20         video.writeFrame(frame)
     21 except BaseException as err:
     22     raise err

/usr/local/lib/python3.6/site-packages/skvideo/io/ffmpeg.py in writeFrame(self, im)
    446         T, M, N, C = vid.shape
    447         if not self.warmStarted:
--> 448             self._warmStart(M, N, C)
    449 
    450         # Ensure that ndarray image is in uint8

/usr/local/lib/python3.6/site-packages/skvideo/io/ffmpeg.py in _warmStart(self, M, N, C)
    412         cmd = [_FFMPEG_PATH + "/" + _FFMPEG_APPLICATION, "-y"] + iargs + ["-i", "-"] + oargs + [self._filename]
    413 
--> 414         self._cmd = " ".join(cmd)
    415 
    416         # Launch process

TypeError: sequence item 3: expected str instance, int found

Changing this to video.writeFrame(frame.tostring()) results in ValueError: Improper data input, leaving me stumped.

How should I go about writing each frame (as returned by OpenCV) to my FFmpegWriter instance?

EDIT

The code works fine if I remove inputdict and outputdict from the io.FFmpegWriter call, however this defeats the purpose for me as I need fine control over the video encoding (I am experimenting with lossless/near-lossless compression of the raw video captured from the camera and trying to establish the best compromise in terms of compression vs fidelity for my needs).

Answer 1

A few points to note here:

• skvideo.io.FFmpegWriter uses the ffmpeg command line tool to encode videos, not the C API. So test your ffmpeg command in terminal first to make sure you've a working pipeline.
• The inputdict={},outputdict={} take string inputs not integers. If you've an integer variable then use str(var_name) to convert it to string.
• You've to make sure the pix_fmt you're using is supported by the codec. To get a list of all pix_fmts use ffmpeg -pix_fmts. Keep in mind that x264 does not accept all the pix_fmts that ffmpeg internally supports.

So for a near lossless encoding (see this and this) we can use the following command:

ffmpeg -s 1920x1080 -r 60 -i /dev/video0 -s 1920x1080 -r 60 -c:v libx264 -crf 17 -preset ultrafast -pix_fmt yuv444p test_ffmpeg.avi

The same command translated to skvideo.io.FFmpegWriter would be:

fps = 60
width = 1920
height = 1080
crf = 17
video = io.FFmpegWriter('/tmp/test_ffmpeg.avi', 
            inputdict={'-r': str(fps), '-s':'{}x{}'.format(width,height)},
            outputdict={'-r': str(fps), '-c:v': 'libx264', '-crf': str(crf), '-preset': 'ultrafast', '-pix_fmt': 'yuv444p'}
)