Record OpenCV video during Tkinter mainloop

Question

I am developing an psychology experiment that analyzes facial expressions made by users while completing a behavioral task. The application mostly runs through Tkinter, and i'm using openCV to capture video.

In a minimal case, i need to start and stop recording based on user responses. For example, in the code below, i want the user to specify when to start and stop recording video using the mouse to press a button.

import Tkinter as tk
import cv2

# -------begin capturing and saving video
def startrecording():
    cap = cv2.VideoCapture(0)
    fourcc = cv2.cv.CV_FOURCC(*'XVID')
    out = cv2.VideoWriter('output.avi',fourcc,  20.0, (640,480))

    while(cap.isOpened()):
        ret, frame = cap.read()
        if ret==True:
            out.write(frame)
        else:
            break

# -------end video capture and stop tk
def stoprecording():
    cap.release()
    out.release()
    cv2.destroyAllWindows()

    root.quit()
    root.destroy()

# -------configure window
root = tk.Tk()
root.geometry("%dx%d+0+0" % (100, 100))
startbutton=tk.Button(root,width=10,height=1,text='START',command = startrecording)
stopbutton=tk.Button(root,width=10,height=1,text='STOP', command = stoprecording)
startbutton.pack()
stopbutton.pack()

# -------begin
root.mainloop()

The problem is that OpenCV uses a loop to record video, during which Tkinter is unable to listen for user responses. The program gets stuck in the OpenCV loop and the user is unable to continue. How can i simultaneously record video and listen for user responses?

I have looked into parallel processing (e.g., Display an OpenCV video in tkinter using multiprocessing), but this sounds like a larger endeavor than seems necessary.

I have also looked into using root.after command (e.g., Show webcam sequence TkInter), but using this it appears that you can only capture a single frame, whereas i want a video.

Is there another way? will i need to use two processing streams?

Answer 1

Handling this via multiprocessing is easier than you think:

import multiprocessing
import Tkinter as tk
import cv2

e = multiprocessing.Event()
p = None

# -------begin capturing and saving video
def startrecording(e):
    cap = cv2.VideoCapture(0)
    fourcc = cv2.cv.CV_FOURCC(*'XVID')
    out = cv2.VideoWriter('output.avi',fourcc,  20.0, (640,480))

    while(cap.isOpened()):
        if e.is_set():
            cap.release()
            out.release()
            cv2.destroyAllWindows()
            e.clear()
        ret, frame = cap.read()
        if ret==True:
            out.write(frame)
        else:
            break

def start_recording_proc():
    global p
    p = multiprocessing.Process(target=startrecording, args=(e,))
    p.start()

# -------end video capture and stop tk
def stoprecording():
    e.set()
    p.join()

    root.quit()
    root.destroy()

if __name__ == "__main__":
    # -------configure window
    root = tk.Tk()
    root.geometry("%dx%d+0+0" % (100, 100))
    startbutton=tk.Button(root,width=10,height=1,text='START',command=start_recording_proc)
    stopbutton=tk.Button(root,width=10,height=1,text='STOP', command=stoprecording)
    startbutton.pack()
    stopbutton.pack()

    # -------begin
    root.mainloop()

All we've done is added a call to multiprocessing.Process so that your video capture code runs in a child process, and moved the code to clean up when the capturing is done into that process as well. The only additional wrinkle compared to the single-process version is the use of a multiprocessing.Event to signal the child process when its time to shut down, which is necessary because the parent process doesn't have access to out or cap.

You could try using threading instead (just replace multiprocessing.Process with threading.Thread, and multiprocessing.Event with threading.Event), but I suspect the GIL will trip you up and hurt the performance of the GUI thread. For the same reason, I don't think it's worth trying to integrate reading/writing the streams into your event loop via root.after - it's only going to hurt performance, and since you're not trying to integrate what you're doing into the GUI itself, there's no reason to try to keep it in the same thread/process as the event loop.