converting gray to RGB video python

Question

good afternoon everybody, i would like to convert video from gray format to color format, i am using as a reference two site, first one is itself stackoverflow question

from rgb to gray

and next is opencv site for cv::VideoWriter documentation, here it is cv::VideoWriter

on the based of this documentation, i have tried following code :

import cv2
import matplotlib.pyplot as plt
source =cv2.VideoCapture("test_video.mp4")
frame_width = int(source.get(3))
frame_height = int(source.get(4))
size = (frame_width, frame_height,3)
result = cv2.VideoWriter('color_version.avi',
                         cv2.VideoWriter_fourcc(*'MJPG'),
                         10, size, True)
# running the loop
while True:
    # extracting the frames
    ret, img = source.read()
    # converting to color format
    colored = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)
    # write to gray-scale
    result.write(colored)
    # displaying the video
    cv2.imshow("Live", colored)
    # exiting the loop
    key = cv2.waitKey(1)
    if key == ord("q"):
        break
# closing the window
cv2.destroyAllWindows()
source.release()

sorry some part was lost, so after running this code i am getting following error :

cv2.error: OpenCV(4.5.5) :-1: error: (-5:Bad argument) in function 'VideoWriter'
> Overload resolution failed:
>  - Can't parse 'frameSize'. Expected sequence length 2, got 3
>  - Argument 'fps' can not be treated as a double
>  - Can't parse 'frameSize'. Expected sequence length 2, got 3
>  - VideoWriter() missing required argument 'params' (pos 6)

from the official site i have indicated bool isColor=true and therefore it should accept 3D format right? please help me what is wrong with my code

Answer 1

TL:DR OpenCV's VideoCapture will always convert the image to BGR for and it's VideoWriter expects frames in BGR format. Also, the video format you chose (MJPEG) doesn't support RGB (or BGR) and will internally save images in (subsampled) YUV format. As a result, you don't have to do anything extra to process gray video; it "just works".

If you still wish to access the data in its raw pixel format, you can do so in python, but not via OpenCV. For this, you'd either use pyav or ImageIO.

import imageio.v3 as iio

# General Note: ImageIO returns 8-bit RGB frames by default.

# read frames in their raw/native format
# will fail if pixel format is not stridable
frames = iio.imread("test_video.mp4", format=None)

# force grayscale result (convert if necessary)
frames = iio.imread("test_video.mp4", format="gray")

# read frames one-by-one
for frame in iio.imiter("test_video.mp4", format="gray"):
    ...  # do something useful 

That said, the odds of finding a video that uses grayscale internally are very, very, very low. If you do find one, please reach out to me if you can share it publically. I'd love to add it to our test suite (I maintain ImageIO).

---

Long Version

The way you would handle this in OpenCV is to not worry about the pixel format of your raw input video. OpenCV will convert to BGR for you and you have no influence on this behavior in 99.9% of the cases.

There is also no reason to go from BGR to RGB unless you want to process the frames outside of OpenCV and you know that the external pipeline expects frames in RGB. The VideoWriter expects frames in BGR format, so passing RGB to it will result in a discolored (misscolored?) result.

import cv2

source = cv2.VideoCapture("test_video.mp4", cv2.CAP_FFMPEG)
frame_width = int(source.get(3))
frame_height = int(source.get(4))
size = (frame_width, frame_height)
result = cv2.VideoWriter(
    "color_version.avi", cv2.VideoWriter_fourcc(*"MJPG"), 10, size, True
)
# running the loop
while True:
    # extracting the frames
    ret, img = source.read()
    if not ret:
        break

    # converting to color format
    colored = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    ... # external processing in RGB
    colored = cv2.cvtColor(colored, cv2.COLOR_RGB2BGR)
    # write to gray-scale
    result.write(colored)
    # displaying the video
    cv2.imshow("Live", colored)
    # exiting the loop
    key = cv2.waitKey(1)
    if key == ord("q"):
        break
# closing the window
cv2.destroyAllWindows()
source.release()

In the 0.01% of cases where you actually can have some control over the pixel format is when you are using VAPI to read frames from a generic webcam or are grabbing frames from a specialist camera using the XIEMA backend. This doesn't help much here though, because not only does this not apply to files, but it also doesn't change the fact that openCV will still convert to BGR in the end. (The mean idea behind having this feature anyway is that you can control the bandwidth of your camera stream.)

If you want to explore what is possible using the OpenCV API yourself, I invite you to check out OpenCVs source code where you can find all the config options for video IO.

---

For good measure, here is how the same program would look like using ImageIO:

import imageio.v3 as iio

# extract the FPS from the source video
fps = iio.immeta("test_video.mp4")["fps"]

with iio.imopen("color_version.avi", "w") as file:
    file.init_video_stream("mjpeg", fps=fps, pixel_format="yuvj420p")
    for frame in iio.imiter("test_video.mp4"):
        # frame is converted to RGB by default
        ... # external processing
        file.write_frame(frame)