Getting current frame with OpenCV VideoCapture in Python

Question

I am using cv2.VideoCapture to read the frames of an RTSP video link in a python script. The .read() function is in a while loop which runs once every second, However, I do not get the most current frame from the stream. I get older frames and in this way my lag builds up. Is there anyway that I can get the most current frame and not older frames which have piped into the VideoCapture object?

Answer 1

I also faced the same problem. Seems that once the VideoCapture object is initialized it keeps storing the frames in some buffer of sort and returns a frame from that for every read operation. What I did is I initialized the VideoCapture object every time I wanted to read a frame and then released the stream. Following code captures 10 images at an interval of 10 seconds and stores them. Same can be done using while(True) in a loop.

for x in range(0,10):
    cap = cv2.VideoCapture(0)
    ret, frame = cap.read()
    cv2.imwrite('test'+str(x)+'.png',frame)
    cap.release()
    time.sleep(10)

Answer 2

I've encountered the same problem and found a git repository of Azure samples for their computer vision service. The relevant part is the Camera Capture module, specifically the Video Stream class.

You can see they've implemented a Queue that is being updated to keep only the latest frame:

def update(self):
    try:
        while True:
            if self.stopped:
                return

            if not self.Q.full():
                (grabbed, frame) = self.stream.read()

                # if the `grabbed` boolean is `False`, then we have
                # reached the end of the video file
                if not grabbed:
                    self.stop()
                    return

                self.Q.put(frame)

                # Clean the queue to keep only the latest frame
                while self.Q.qsize() > 1:
                    self.Q.get()

Answer 3

I'm working with a friend in a hack doing the same. We don't want to use all the frames. So far we found that very same thing: grab() (or read) tries to get you all the frames, and I guess with rtp: it will maintain a buffer and drop if you're not responsive enough.

Instead of read you can also use grab() and receive(). First one ask for the frame. Receives reads it into memory. So if you call grab several times it will effectively skip those.

We got away with doing this:

#show some initial image
while True:
    cv2.grab()
    if cv2.waitKey(10):
       im = cv2.receive()
       # process
       cv2.imshow...

Not production code but...

Answer 4

Using the following was causing a lot of issues for me. The frames being passed to the function were not sequention.

cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    function_that_uses_frame(frame)
    time.sleep(0.5)

The following also didn't work for me as suggested by other comments. I was STILL getting issues with taking the most recent frame.

cap = cv2.VideoCapture(0)

while True:
    ret = capture.grab()
    ret, frame = videocapture.retrieve()
    function_that_uses_frame(frame)
    time.sleep(0.5)

Finally, this worked but it's bloody filthy. I only need to grab a few frames per second, so it will do for the time being. For context, I was using the camera to generate some data for an ML model and my labels compared to what was being captured was out of sync.

while True:
    ret = capture.grab()
    ret, frame = videocapture.retrieve()
    ret = capture.grab()
    ret, frame = videocapture.retrieve()
    function_that_uses_frame(frame)
    time.sleep(0.5)

Answer 5

Inside the 'while' you can use:

while True:
    cap = cv2.VideoCapture()
    urlDir = 'rtsp://ip:port/h264_ulaw.sdp'
    cap.open(urlDir)
    
    # get the current frame
    _,frame = cap.read()
    cap.release() #releasing camera
    image = frame

Answer 6

I made an adaptive system as the ones the others on here posted here still resulted in somewhat inaccurate frame representation and have completely variable results depending on the hardware.

from time import time
#...
cap = cv2.VideoCapture(url)
cap_fps = cap.get(cv2.CAP_PROP_FPS)
time_start = time()
time_end = time_start
while True:
   time_difference = int((((end_time-start_time))*cap_fps)+1) #Note that the 1 might be changed to fit script bandwidth
   for i in range(0, time_difference):
      a = cap.grab()
   _, frame = cap.read()
   time_start = time()
   #Put your code here
   variable = function(frame)
   #...
   time_end = time()

This way the skipped frames adapt to the amount of frames missed in the video stream - allowing for a much smoother transition and a relatively real-time frame representation.