OpenCV and custom multiprocessing

Question

I try to build a webcam software which uses the webcam input and changes the image, then outputs this to a virtual webcam. So I got it running pretty good, but now I want to add a System Tray Icon to control it. As this is a GUI I need the my software to run on another thread.

So my (cut down) class looks like this:

import pyvirtualcam
import cv2
import effects
import numpy as np


class VirtualCam:

    def __init__(self):
        self.stopped = False
        self.paused = False
        [...]

    def run(self):
        with pyvirtualcam.Camera(width=1280, height=720, fps=self.FPS, fmt=pyvirtualcam.PixelFormat.BGR) as cam:
            while not self.stopped:
                if not self.paused:  # only copy image if NOT paused!
                    success, img = self.cap.read()
                else:
                    img = self.EMPTY_IMAGE

                cam.send(effects.zoom(img, self.zoom))
                cam.sleep_until_next_frame()

So this should be straight-forward. Now on my gui thread, which is based on this code, I added menu points, where one starts a thread and nothing more (while the others can pause, stop, etc) (again this is cut down):

if __name__ == '__main__':
    import itertools
    import glob
    import virtcam
    from multiprocessing import Process

    cam_thread = None

    vcam = virtcam.VirtualCam()

    # controlling vcam
    def pause(sysTrayIcon):
        vcam.pause()

    def cont(sysTrayIcon):
        vcam.cont()

    def start(sysTrayIcon):
        global cam_thread
        # start threading for camera capture here
        cam_thread = Process(target=vcam.run)
        cam_thread.start()

    def stop(sysTrayIcon):
        global cam_thread
        vcam.stop()
        cam_thread.join()

    menu_options = (
        ('Start', next(icons), start),
        [...]
        )

    SysTrayIcon(next(icons), hover_text, menu_options, on_quit=bye, default_menu_index=1)

Okay so this should work, shouldn't it? When I click on "Start" in the tray-menu, I get an error:

Python WNDPROC handler failed
Traceback (most recent call last):
  File "I:/Entwicklung/webcamZoomer/tray_gui.py", line 207, in command
    self.execute_menu_option(id)
  File "I:/Entwicklung/webcamZoomer/tray_gui.py", line 214, in execute_menu_option
    menu_action(self)
  File "I:/Entwicklung/webcamZoomer/tray_gui.py", line 256, in start
    cam_thread.start()
  File "H:\Programme\Python3\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)
  File "H:\Programme\Python3\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "H:\Programme\Python3\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "H:\Programme\Python3\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "H:\Programme\Python3\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle cv2.VideoCapture objects
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "H:\Programme\Python3\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "H:\Programme\Python3\lib\multiprocessing\spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Of course I used the search function, but I could only find things I don't really understand: It seems like OpenCV already uses multiprocessing. But why does it interfer with my code? Furthermore, I don't do any manual pickling; I actually need only the webcam input.

So - can someone help me out on this one? Thank you!

edit: I'm btw on Windows 10 and I only will need this software to be used on Windows systems.

"I don't do any manual pickling" -- No, but you start a new process, and want it to run a method of an object you've instantiated in the original process. In order to be able to do this, Python has to transfer that object instance to the second process. To do this, it has to send it via a queue, and to do that it needs to pickle it. | You didn't show it, but I see a `self.cap.read()`, so `VirtualCam` does have a `VideoCapture` member (quite likely already open). Even if it could be somehow pickled and transferred, you'd then have two processes trying to access same webcam (which wouldn't work) — Dan Mašek, May 31 '21 at 10:31
@DanMašek Thank you, I understand the problem now. Now should I not create the instance in the gui thread then? If not, how do I access the instance to manipulate settings I want to use on the virtualcam? — Standard, May 31 '21 at 10:35
You'd have to make sure that `VideoCapture` (and potentially other objects) are only instantiated in the second process, and you'd need to set up some IPC mechanism (e.g. Queues) to transfer the image data to the main process and commands to the capture process. | IMHO multiprocessing is an unnecessary complication. OpenCV and many numpy functions release the GIL, so in many cases you can just use `threading`. For example look at [this answer of mine](https://stackoverflow.com/a/50244613/3962537) (there is a comment with a link to pastebin that has the final code). — Dan Mašek, May 31 '21 at 11:33

OpenCV and custom multiprocessing

0 Answers0