Using FFMPEG command to read the frame and show using the inshow function in opencv

Question

I am trying to get the frame using the ffmpeg command and show using the opencv function cv2.imshow(). This snippet gives the black and white image on the RTSP Stream link . Output is given below link [ output of FFmpeg link]. I have tried the ffplay command but it gives the direct image . i am not able to access the frame or apply the image processing.

Output of FFMPEG

import cv2
import subprocess as sp
command = [ 'C:/ffmpeg/ffmpeg.exe',
            '-i', 'rtsp://192.168.1.12/media/video2',
            '-f', 'image2pipe',
            '-pix_fmt', 'rgb24',
            '-vcodec', 'rawvideo', '-']


import numpy
pipe = sp.Popen(command, stdout = sp.PIPE, bufsize=10**8)
while True:
    raw_image = pipe.stdout.read(420*360*3)
   # transform the byte read into a numpy array
    image =  numpy.fromstring(raw_image, dtype='uint8')
    image = image.reshape((360,420,3))
    cv2.imshow('hello',image)
    cv2.waitKey(1)
    # throw away the data in the pipe's buffer.
    pipe.stdout.flush()

why do it like this? OpenCV's VideoCapture does the same. it can pass the rtsp link to ffmpeg and handle the pixel format conversions. — Christoph Rackwitz, Mar 04 '22 at 10:35
VideoCapture function takes approx 40% CPU consumption as my current task when i am trying to view the rtsp frame using ffplay command its takes only 0.5% CPU for single camera whereas if cv2.VideoCapture function handle more than 70 camera it goes to 90% CPU . I need direct frame with less consumption of CPU. — HARSH BHATNAGAR, Mar 04 '22 at 12:03

score 0 · Answer 1 · answered Mar 04 '22 at 14:48

0

You're using a wrong output format, it should be -f rawvideo. This should fix your primary problem. Current -f image2pipe wraps the RGB data in an image format (donno what it is maybe BMP as rawvideo codec is being used?) thus not shown correctly.

Other tips:

If your data is grayscale, use -pix_fmt gray and read 420*360 bytes at a time.
Don't know the difference in speed, but I use np.frombuffer instead of np.fromstring
pipe.stdout.flush() is a dangerous move IMO as the buffer may have a partial frame. Consider setting bufsize to be an exact integer multiple of framesize in bytes.
If you are expecting processing to take much longer than input frame rate, you may want to reduce the output framerate -r to match the processing rate (to avoid extraneous data transfer from ffmpeg to python)

answered Mar 04 '22 at 14:48

kesh

4,515
2
12
20

I have modified the necessary points but still it shows the horizontal lines in the frame . – HARSH BHATNAGAR Mar 05 '22 at 05:59
IP camera vcodec is h264 by default and we are using the rawvideo . It can generate any problem. Correct me if i am wrong . – HARSH BHATNAGAR Mar 05 '22 at 06:02
"IP camera vcodec is h264" Your screenshot indicates h265 (not that it matters) – kesh Mar 05 '22 at 13:46
So, studying your screenshot, the horizontal lines likely comes from all these dropped packets. Can you play the RTSP stream via ffplay w/out a hitch? – kesh Mar 05 '22 at 13:47
Also, if you have a capable GPU, you can turn on hardward decoding with `-hwaccel auto` input option – kesh Mar 05 '22 at 13:48
after using the **-hwaccel auto** command is return the frame is none . Using the ffplay its working fine. – HARSH BHATNAGAR Mar 05 '22 at 14:14
Suggest me any other way to reduce the CPU utilization . Thanks in advance . – HARSH BHATNAGAR Mar 05 '22 at 14:15
Is this a CPU utilization issue or image decoding issue? – kesh Mar 05 '22 at 14:17
If you kill the subprocess immediately after the first read, do you see any warnings/errors from FFmpeg on your terminal? And does that image display correctly? – kesh Mar 05 '22 at 14:18
Yes i have seen the warnings and error after killing the process but frame is still not clear . – HARSH BHATNAGAR Mar 06 '22 at 11:03

Using FFMPEG command to read the frame and show using the inshow function in opencv

1 Answers1

Linked