Pthread producer/consumer with select() calls for file I/O

Question

I am trying to improve the capture performance of a real-time computer vision program that I am developing to run on an embedded Linux computer using OpenCV. I would like to use multiple threads and a shared memory buffer to separate the tasks of (1) capturing video frames and (2) processing them (a "producer/consumer problem", I believe). I have read up on pthreads, mutex variables, and condition variables but do not understand how they would need to be used with the select() function.

Right now, video frames are captured using code adapted from the Video4Linux2 Video Capture Example which makes use of select(). As I understand it, select() blocks the program until data is available from the webcam, which can be slow and a major waste of time. If possible, I would like to use those wasted CPU cycles to work on image processing. (Of course this means that processing would always have to be done on an image that is "stale" by one frame, but at 30 fps it will be practically real-time anyways).

I have found some example code that uses pthread_mutex_lock() and pthread_mutex_control() to protect shared data, but it seems that these would still block the "processing" thread from running while waiting for image data via select(). To be more concrete, here is some pseudo code to illustrate my concern. (NOTE: I realize that these threads would need to contain loops and other checks like the above linked example to actually work properly.)

void* capture()
{
    pthread_mutex_lock(&mutex);             // protect shared data
    pthread_cond_wait(&space_avail,&mutex); // make sure the buffer has room

    ///  capture code... ///

    select(fd+1, &fds, NULL, NULL, &tv);  // waiting for data...
                                          // Want to be processing frames
                                          // during select, but won't the
                                          // processing thread be blocked
                                          // by the mutex lock?

    // store image to the frame buffer

    /// more capture code... ///

    pthread_cond_signal(&data_avail);
    pthread_mutex_unlock(&mutex);
}


void* process()
    pthread_mutex_lock(&mutex);
    pthread_cond_wait(&data_avail);

    // access the most recently stored frame from the buffer

    /// Do image processing ///

    pthread_mutex_signal(&space_avail);
    pthread_mutex_unlock(&mutex);
}

It's really the frame buffer you want to share between threads so in `capture` you shouldn't have to lock things until *after* select returns and you have read something and want to move it into the buffer. Why is `select` used? Are there more than the camera file descriptor being monitored? — Duck, Oct 23 '13 at 23:44
It makes sense that I should only need to lock during the actual write to memory, so I will give that a try. As my code is written now, I have my capture code implemented as a member of a "CamObj" class so I will have to give some thought as to how I need to include the locks. Select is called to suspend the program until the driver has captured data [v4l2 API](http://linuxtv.org/downloads/v4l-dvb-apis/func-select.html). I've seen some cases where select timeout errors occur when trying to push too much data across the USB bus at once. — Michael Darling, Oct 24 '13 at 00:20
So then this is c++ rather than C? In any case, you are mixing multiplexing (select) with threading, which is fine so long as you understand the options they give you. Below Vinay gave the multiplexing case - a single thread does your process work when there is no capturing activity to be done. A plain vanilla thread case would have one thread dedicated to reading the camera (I assume there is some kind of EOF designation from the camera) and one or more consumer threads doing the processing. You have a combo of both, which is fine so long as there is a justification for the select — Duck, Oct 24 '13 at 00:35
when you are planning to thread e.g. multiple FDs to be monitored or some valid use of a timeout. That's what I was getting at. — Duck, Oct 24 '13 at 00:36
Yes, my code is primarily C++. The v4l2 code is written in C but gets compiled with g++ along with everything else. I don't know much about multiplexing, but thought that it is just a way of combining multiple signals into one and is handled by the USB hardware for me. I see what Vinay was getting at now. I suppose I could implement a circular frame buffer, reduce the select timeout to almost zero, and add a check to prevent processing the same frame twice. I'm just not sure if doing so would result in repeated select timeouts and no captured frames ever. I'll give it a try. Thanks! — Michael Darling, Oct 24 '13 at 01:20

Vinay Pai · Answer 1 · 2013-10-23T23:44:53.030

If I have understood your question correctly, you want to avoid the busy waiting cycles which are being wasted when your program waits for select and in turn utilize it for image processing. Please correct me if I am wrong.

In my experience a simple way to do this would be to specify a small timeout for the select API to work on. The Select API will wait for this time out. If in this duration it received any data from the webcam, the return value from select will be non zero. If it does not see anything interesting in that duration the return value will be 0.

So your psuedocode can be modified to something as below:

void* capture(){
    while(true){//iterate how many times you feel it is valid to
        if (select(fd+1, &fds, NULL, NULL, &tv)!=0){
              //We have found something here.take data from the webcam.
              //break from the loop.
              break;
         }else{
              //timeout occurred. process the image.
         }
     }
     //process the data from the webcam
}

Let me know if this resolves your issue. More on Select API can be found at: http://linux.die.net/man/2/select

Yes-you understand my question perfectly. Are you suggesting that I not bother with threads and simply use a very short select timeout? Couldn't that approach be prone to select timeouts and hence, dropped frames? Thanks. — Michael Darling, Oct 24 '13 at 00:23

Pthread producer/consumer with select() calls for file I/O

1 Answers1