How to use multithreading with and cv2.VideoCapture()?

Question

I want to display two or more separate videos with slight image processing applied using cv2.

If I don't use threading/multiprocessing a lot of time is wasted just displaying frames using cv2.waitKey(timeThatCouldBeSpentFetchingAFrameForAnotherVideo).

I tried using threading, but it does not work for properly. I get this warning: WARNING: nextEventMatchingMask should only be called from the Main Thread! This will throw an exception in the future.. Which more often than not results in a plain crash.

So, I decided to try to implement multiprocessing, which in theory should be faster than threading with more complex image processing (right??). I used this as an example. However, in this case I ran into another issue this method ONLY gets me the first frame of the video. Also, I have tried putting the cv2.VideoCapture() outside of the Process, but that results in: TypeError: cannot pickle 'cv2.VideoCapture' object.

Here is my code:

class CamManager:
    def __init__(self):
        self.saving = Saving()
        self.cams = self.createCamObjList()
        self.buffers = [Buffer(cam.FPS*BUFFER_SIZE) for cam in self.cams]


    def createCamObjList(self):
        l = []
        for i, url in enumerate(STREAM_URLS):
            saved_data = self.saving.getDataFor(url)
            cam = BufferedCamera(url, i, saved_data)
            l.append(cam)
        return l


    def startStreamCapture(self):
        queue_for_cams = multiprocessing.Queue()
        processes = [multiprocessing.Process(
            target=cam.readloop, args=[queue_for_cams]).start() for cam in self.cams]
        while True:
            if not queue_for_cams.empty():
                from_queue = queue_for_cams.get()
                self.buffers[from_queue[0]].add(from_queue[1])



class BufferedCamera():
    def __init__(self, streamURL, cam_id, saved_data=None):
        self.streamURL = streamURL
        self.cam_id = cam_id

        # get current camera's framerate
        cam = cv2.VideoCapture(streamURL)
        self.FPS = cam.get(5)
        if self.FPS == 0:
            self.FPS = 24
        cam.release()
        print(f"Input res: {cam.get(3)}x{cam.get(4)} | FPS: {self.FPS}")

        # use that framerate to adjust the time between each iteration of mainloop
        self.period = int((1.0/self.FPS)*1000)


    def readloop(self, queue):
        while True:
            self.read(queue)

    
    def read(self, queue):
        cam = cv2.VideoCapture(self.streamURL)
        _, frame = cam.read()
        if frame is not None:
            queue.put((self.cam_id, frame))
        cv2.waitKey(self.period)

Sounds like `cv2` doesn't support multithreading so you cannot, and that you weren't doing the multiprocessing properly (but it's hard to say without seeing the code). — martineau, Feb 17 '21 at 16:34
I added some code snippets. Would you mind giving it another look? — Jonas, Feb 19 '21 at 22:46
Since normally no memory gets shared between processes, having to transfer large amounts of it can often make things too slow (even if you're using a `Queue` to do it). In Python 3.8 a [`multiprocessing.shared_memory`](https://docs.python.org/3/library/multiprocessing.shared_memory.html) module was added "for the allocation and management of shared memory to be accessed by one or more processes", so perhaps you could use it to (more) quickly transfer the image data. — martineau, Feb 20 '21 at 00:45

score 0 · Accepted Answer · answered Feb 21 '21 at 13:13

0

I got multiprocessing to work by creating cv2.VideoCapture object inside my Process function and prior to the readloop:

def readloop(self, queue):
    cam = cv2.VideoCapture(self.streamURL)
    while True:
        self.read(queue, cam)

answered Feb 21 '21 at 13:13

Jonas

49
7

How to use multithreading with and cv2.VideoCapture()?

1 Answers1