opencv and multiprocessing

Question

I'm looking to use the multiprocessing module for python to create one process that continually polls a webcam via opencv's python interface, sending any resulting images to a queue from which other processes can access them. However, I'm encountering a hang (python 2.7 on Ubuntu 12.04) whenever I try to do anything with the images retrieved by other processes from the queue. Here's a minimal example:

import multiprocessing
import cv

queue_from_cam = multiprocessing.Queue()

def cam_loop(queue_from_cam):
    print 'initializing cam'
    cam = cv.CaptureFromCAM(-1)
    print 'querying frame'
    img = cv.QueryFrame(cam)
    print 'queueing image'
    queue_from_cam.put(img)
    print 'cam_loop done'


cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()

while queue_from_cam.empty():
    pass

print 'getting image'
from_queue = queue_from_cam.get()
print 'saving image'
cv.SaveImage('temp.png',from_queue)
print 'image saved'

This code should run up to the print out of "saving image" but then hang. Any ideas how I can go about fixing this?

I don't know much about openCV, but I bet the problem is that some internal state data is not copied between the processes. I'd try to serialize the image to a buffer in the capture process, and send back raw binary data to be saved back to disk. — Not_a_Golfer, Jun 02 '12 at 13:13
@Not_a_Golfer Thanks for the serialization suggestion, unfortunately it appears that opencv's iplimage objects (which is what is returned by `cv.QueryFrame()`) are not pickle-able for some reason. When I try to `pimg = pickle.dumps(img)`, I get the error: `TypeError: can't pickle iplimage objects`. — Mike Lawrence, Jun 02 '12 at 13:36
Oh, check that; when I run `pimg = pickle.dumps(img,-1)`, thereby using pickle.HIGHEST_PROTOCOL, the pickling works, and I can seemingly unpickle successfully in another process, but I still get a hang on the subsequent attempt to save the unpickled image. — Mike Lawrence, Jun 02 '12 at 13:56
It's really starting to sound like opencv and multiprocessing don't like each other very much. But to rule anything really bizarre out, did you try saving the raw data using mundane methods? — senderle, Jun 02 '12 at 14:06
@senderle, excuse my naiveté, but which mundane methods would that be? — Mike Lawrence, Jun 02 '12 at 14:14
update on the attempt to solve this by serialization: I can get an image to save if I pickle the result of `img.tostring()`, then after unpickling in a separate process convert the tostring data to PIL format using `Image.fromstring('L',(640,480),unpickled_image_string)` (knowing that the images will all be 640x480), but the resulting saved image is clearly corrupted: black and white with seemingly interlaced lines (http://imgur.com/rAVZB). — Mike Lawrence, Jun 02 '12 at 14:18
oh, check that yet again; if I avoid use of PIL and use opencv only, it works! Posting an answer... — Mike Lawrence, Jun 02 '12 at 14:24
@senderle.... any chance on getting the cookbook example posted... link died a while ago I guess. — ZF007, Nov 23 '17 at 01:16
@ZF007, not sure it's worth updating. The cookbook was reported out-of-date [here](http://opencv-users.1802565.n2.nabble.com/Converting-PIL-image-to-OpenCV-image-td7580733.html#a7580761) as of 2012! That thread might help though... — senderle, Nov 26 '17 at 20:52

score 10 · Accepted Answer · answered Jun 11 '13 at 22:33

The simplest approach is to use the newer cv2 module that's based on NumPy arrays. That way you don't have to mess with manual pickling. Here's the fix (I just changed 4 lines of code):

import multiprocessing
import cv2

queue_from_cam = multiprocessing.Queue()

def cam_loop(queue_from_cam):
    print 'initializing cam'
    cap = cv2.VideoCapture(0)
    print 'querying frame'
    hello, img = cap.read()
    print 'queueing image'
    queue_from_cam.put(img)
    print 'cam_loop done'

cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()

while queue_from_cam.empty():
    pass

print 'getting image'
from_queue = queue_from_cam.get()
print 'saving image'
cv2.imwrite('temp.png', from_queue)
print 'image saved'

Just a note for posterity: while this works for Ubuntu, it crashes when run on OS X Mountain Lion. — Mike Lawrence, Aug 12 '13 at 02:25

score 5 · Answer 2 · answered Jun 02 '12 at 14:27

It appears that the solution was to convert the opencv iplimage object to string, then pickle it before adding it to the queue:

import multiprocessing
import cv
import Image
import pickle
import time

queue_from_cam = multiprocessing.Queue()

def cam_loop(queue_from_cam):
    print 'initializing cam'
    cam = cv.CaptureFromCAM(-1)
    print 'querying frame'
    img = cv.QueryFrame(cam)
    print 'converting image'
    pimg = img.tostring()
    print 'pickling image'
    pimg2 = pickle.dumps(pimg,-1)
    print 'queueing image'
    queue_from_cam.put([pimg2,cv.GetSize(img)])
    print 'cam_loop done'


cam_process = multiprocessing.Process(target=cam_loop,args=(queue_from_cam,))
cam_process.start()

while queue_from_cam.empty():
    pass

print 'getting pickled image'
from_queue = queue_from_cam.get()
print 'unpickling image'
pimg = pickle.loads(from_queue[0])
print 'unconverting image'
cv_im = cv.CreateImageHeader(from_queue[1], cv.IPL_DEPTH_8U, 3)
cv.SetData(cv_im, pimg)
print 'saving image'
cv.SaveImage('temp.png',cv_im)
print 'image saved'

score 1 · Answer 3 · answered Sep 20 '21 at 22:36

Although this thread is old, I wanted to give a more recent version in case people are also interested by the question (I was). Here, we don't save the image, but display a continuous video.

import multiprocessing
import cv2


def cam_loop(queue_from_cam):
    cap = cv2.VideoCapture(0)
    while True:
        hello, img = cap.read()
        queue_from_cam.put(img)


def main():
    print('initializing cam')
    queue_from_cam = multiprocessing.Queue()
    cam_process = multiprocessing.Process(target=cam_loop, args=(queue_from_cam,))
    cam_process.start()
    while True:
        if queue_from_cam.empty():
            continue
        from_queue = queue_from_cam.get()
        cv2.imshow("from queue", from_queue)
        key = cv2.waitKey(1)
        if key == ord("q"):
            cv2.destroyAllWindows()
            break
    print("Destroying process...")
    cam_process.terminate()
    cam_process.join()


if __name__ == '__main__':
    main()

opencv and multiprocessing

3 Answers3

Linked