Python FFmpeg query rtsp too slow

Question

Currently, I am trying to use python with FFmpeg to query rtsp data which the original format is h264.

The information of the live stream video is, fps:29; resolution: 1280*720.

I wish that I can query the data as the same format "h264" and put into a python queue in order to the future use.

Here is my code:

class CCTVReader(threading.Thread):
def __init__(self, queue, URL, fps=29):
    super().__init__()
    self.queue = queue
    self.command = ["ffmpeg",  "-y",
                    "-hwaccel",  "nvdec",
                    "-c:v",  "h264_cuvid",
                    "-vsync", "0",
                    "-max_delay", "500000",
                    "-reorder_queue_size", "10000",
                    "-i",  "rtsp://xxx.xxx.xxx.xxx:xxx/Streaming/Channels/101?transportmode=multicast",
                    "-pix_fmt", "yuv420p",
                    "-preset", "slow",
                    "-an", "-sn",
                    "-vf", "fps=29",
                    "-"]
def run(self):
    pipe = sp.Popen(self.command, stdout = sp.PIPE, bufsize=1024**3, shell=True)
    timer = time.time()
    counter = 0
    while True:
        self.queue.put(pipe.stdout.read(int(1280*720*6//4)))

However, after I run this program about 10 second, my console shows the warning message:

[rtsp @ 0000020be0fbb9c0] max delay reached. need to consume packet
[rtsp @ 0000020be0fbb9c0] RTP: missed 127 packets

It seems like my command doesn't type appropriately.

Could you kindly give me some suggestion about how to deal with this problem?

Thank you very much

If the original format is `h264`, why are you using `"-c:v", "h264_cuvid"`? Do you want to re-encode the stream? If the stream is encoded, why are you reading `int(1280*720*6//4)` bytes from `stdout`? — Rotem, Feb 05 '20 at 23:03
Can you read from a file instead of `rtsp`? You can use the following command to generate file with h264 elementary stream: `ffmpeg -y -f lavfi -i testsrc=duration=10:size=1280x720:rate=1 -c:v libx264 vid.264`. Use the file as input for testing. When reading from file, the problem is going to be reproducible and much easier to answer. (You might get some other error - unrelated to `rtsp`). — Rotem, Feb 05 '20 at 23:12
@Rotem The original format is h264. I got this command from my colleague because I am not totally understand FFmpeg. Could you give me some suggestion about what command is the best for my situation? Also, I have to read rtsp data because I am not able to change server information. — HCCY, Feb 05 '20 at 23:35
The information about the live stream I pull from URL via VLC is, Codec:H264 - MPEG-4 AVC(part 10) h264. — HCCY, Feb 05 '20 at 23:38
I am not sure what you are trying to do. Do you want to decode the video frames to raw video frames? Do you want to record the stream as is? Do you want to compress the data to take less memory space? If the `rtsp` stream is "infinite", what is the exit condition of the loop "out of memory"? — Rotem, Feb 05 '20 at 23:41
@Rotem What I am going to do is to query CCTV h264 data from server into a queue in my program. Then there is another thread call decoder will pull data from queue to decode these h264 data. The reason why I separated them into reader and decoder is because in the future I might have 300 CCTV data going to query at same time. So separate them will not cause data traffic. — HCCY, Feb 05 '20 at 23:59

score 4 · Accepted Answer · answered Feb 06 '20 at 16:23

Assuming you want to grab the video stream without modifying the data, you need to set different set of arguments to FFmpeg:

Set "-c:v", "h264" as input argument (before "-i"), for informing FFmpeg that the input is h264 video stream.
Set "-c:v", "copy" as output argument (after "-i"), so the FFmpeg copies the input video stream to the output PIPE, without modifications (without decoding and encoding).
Set "-f", "h264" as output argument, for setting PIPE output format to h264.

Here is a working code sample (please read the comments):

import ffmpeg
import numpy as np
import subprocess as sp
import threading
import queue

class CCTVReader(threading.Thread):
    def __init__(self, q, in_stream):
        super().__init__()
        self.q = q
        self.command = ["ffmpeg",
                        "-c:v", "h264",     # Tell ffmpeg that input stream codec is h264
                        "-i", in_stream,    # Read stream from file vid.264
                        "-c:v", "copy",     # Tell ffmpeg to copy the video stream as is (without decding and encoding)
                        "-an", "-sn",       # No audio an no subtites
                        "-f", "h264",       # Define pipe format to be h264
                        "-"]                # Output is a pipe

    def run(self):
        pipe = sp.Popen(self.command, stdout=sp.PIPE, bufsize=1024**3)  # Don't use shell=True (you don't need to execute the command through the shell).

        # while True:
        for i in range(100):  # Read up to 100KBytes for testing
            data = pipe.stdout.read(1024)  # Read data from pip in chunks of 1024 bytes
            self.q.put(data)

            # Break loop if less than 1024 bytes read (not going to work with CCTV, but works with input file)
            if len(data) < 1024:
                break

        try:
            pipe.wait(timeout=1)  # Wait for subprocess to finish (with timeout of 1 second).
        except sp.TimeoutExpired:
            pipe.kill()           # Kill subprocess in case of a timeout (there should be a timeout because input stream still lives).


# Build synthetic video, for testing begins:
################################################
# width, height = 1280, 720
# in_stream = "vid.264"
# sp.Popen("ffmpeg -y -f lavfi -i testsrc=size=1280x720:duration=5:rate=1 -c:v libx264 -crf 23 -pix_fmt yuv420p " + in_stream).wait()
################################################

#in_stream = "rtsp://xxx.xxx.xxx.xxx:xxx/Streaming/Channels/101?transportmode=multicast",

#Use public RTSP Streaming for testing
in_stream = "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov"

q = queue.Queue()

cctv_reader = CCTVReader(q, in_stream)
cctv_reader.start()
cctv_reader.join()

if q.empty():
    print("There is a problem (queue is empty)!!!")
else:
    # Write data from queue to file vid_from_queue.264 (for testingg)
    with open("vid_from_queue.264", "wb") as queue_save_file:
        while not q.empty():
            queue_save_file.write(q.get())

I tested the code using public RTSP Streaming and using a generated synthetic video file (code for testing with file is commented).

The code stores the output to vid_from_queue.264.
The .264 file is playable - the file format is an elementary h264 video stream.

Here is the last frame of the grabbed video stream:

Rotem, Thank you so much. It works very well. One little question about the code is about you read pip of 1024 bytes as one h264 clip. I was suffering about how many bytes I read from pip can be a complete h264 file — HCCY, Feb 06 '20 at 18:34
I don't know what is the optimal size to read from the pipe, when executing `pipe.stdout.read`. Reading small chunks reduces the chance for loosing the last frame when communication is lost, and larger chunks supposed to reduce CPU load. Assuming CCTV output bit rate is about 2Mbps at 30Hz, an **average** frame is about 8000 bytes (`2000000 / 8 / 30`). Note that H.264 video stream is not sent frame by frame (look at it as a continuous stream of bytes). — Rotem, Feb 06 '20 at 19:15

Python FFmpeg query rtsp too slow

1 Answers1

Linked