producer/consumer using boost threads and circular buffer hangs

Question

I figured it out. Silly mistake on my part, I was not actually deleting the element from the queue, I was just reading the first element. I modified the code and the code below no works. Thanks all for the help.

I'm trying to implement the producer consumer problem using boost, which is actually part of a bigger project. I have implemented a program from examples on the internet and even some help I found here. However currently my code just hangs. Based on some good advice, I decided to use the boost ciruclar buffer to hold my data between the producer and consumer. Lots of similar code out there and I was able to pool ideas from those and write something on my own. However, I still seem to be having same problem as before (which is my program just hangs). I thought that I did not make the same mistake as before..

My code is given below, I have taken out my earlier code where I just my own singly link list.

Buffer header:

#ifndef PCDBUFFER_H
#define PCDBUFFER_H

#include <pcl/io/pcd_io.h>
#include <boost/thread/mutex.hpp>
#include <boost/thread/condition.hpp>
#include <boost/circular_buffer.hpp>

class pcdBuffer
{
    public:
        pcdBuffer(int buffSize);
        void put(int data);
        int get();
        bool isFull();
        bool isEmpty();
        int getSize();
        int getCapacity();
    private:
        boost::mutex bmutex;
        boost::condition_variable buffEmpty;
        boost::condition_variable buffFull;
        boost::circular_buffer<int> buffer;
};


#endif

Buffer source (only relevant parts):

#include "pcdBuffer.h"
#include <iostream>

//boost::mutex io_mutex;

pcdBuffer::pcdBuffer(int buffSize)
{
    buffer.set_capacity(buffSize);
}

void pcdBuffer::put(int data)
{
    {
        boost::mutex::scoped_lock buffLock(bmutex);
        while(buffer.full())
        {
            std::cout << "Buffer is full" << std::endl;
            buffFull.wait(buffLock);
        }
        buffer.push_back(data);
    }
    buffEmpty.notify_one();
}

int pcdBuffer::get()
{
    int data;
    {
        boost::mutex::scoped_lock buffLock(bmutex);
        while(buffer.empty())
        {
            std::cout << "Buffer is empty" << std::endl;
            buffEmpty.wait(buffLock);
        }
        data = buffer.front();
            buffer.pop_front();
    }
    buffFull.notify_one();
    return data;
}

main driver for the code:

#include <iostream>
#include <boost/thread/mutex.hpp>
#include <boost/thread/thread.hpp>
#include <unistd.h>
#include "pcdBuffer.h"

pcdBuffer buff(100);

void producer()
{
    int i = 10;
    while (true)
    {
        buff.put(i);
        i++;
    }
}

void consumer()
{
    int i;
    while(true)
    {
        i = buff.get();
        std::cout << "Data: " << i << std::endl;
    }
}

int main(int argc, char** argv)
{
    std::cout << "Starting main...." << std::endl;
    std::cout << "Buffer Details: " << std::endl;
    std::cout << "Capacity: " << buff.getCapacity() << ", isEmpty: " << buff.isEmpty() << ", isFull: " << buff.isFull() << std::endl;
    boost::thread cons(consumer);
    sleep(5);
    boost::thread prod(producer);
    prod.join();
    cons.join();
    return 0;
}

My buffer capacity is correctly initialized to 100. The consumer thread waits and reports that the "buffer is empty" for the 5 seconds, but after that I just get the "buffer is full" from the put method and "Data : 10" from the consumer function alternating on stdout. As you can see 10 is the first element that I put in. It seems that the buffer is getting filled up and not notifiying the consumer but I checked my locks, and think they are right. Any help on this is greatly appreciated.

Here are a link of references from which I wrote this code:

http://www.boost.org/doc/libs/1_49_0/libs/circular_buffer/doc/circular_buffer.html#classboost_1_1circular__buffer_19ba12c0142a21a7d960877c22fa3ea00

http://www.drdobbs.com/article/print?articleId=184401518&siteSectionName=cpp

Thread safe implementation of circular buffer

Have a look at this: http://www.justsoftwaresolutions.co.uk/threading/implementing-a-thread-safe-queue-using-condition-variables.html — stefaanv, Apr 17 '12 at 11:34
That article is really interesting, however, another reason I want to avoid stl queues is speed. Eventually, the data structure I will be sharing between threads is a point cloud representing a frame capture from a kinect-like camera. I want capture frames at 30fps and save it disk. Thus I would like to minimize my overhead as much as possible. That is another reason I avoided STL queues. I tried to unlock my locks before notifying the condition variable as the article suggests, but it still hangs. I have updated my code here. — shaun, Apr 17 '12 at 12:04
First make it work, then make it fast. Avoid assumptions about which parts of your code/design will be slow. Avoid premature optimization. Focus your work on the important part, while using standard components where possible. If it turns out to be too slow, **measure which part is slow THEN optimize that part**. — Johannes S., Apr 17 '12 at 12:08
That was some fine advice. I copied and implemented the code from the article linked by stefaanv and its much easier to follow and understand than my ugly code. I will try this with the point cloud data and hopefully it will be fast enough for what I need to do. — shaun, Apr 17 '12 at 13:26

score 5 · Accepted Answer · answered Apr 17 '12 at 12:05

First of all, instead of writing your own list, you could just wrap std::list in pcdQueueinstead of writing your own. It is correct, that std::list is not thread-safe as-is, but you are providing the necessary synchronisation primitives in your class anyway.

The reason why your program hangs: You keep the lock and fill the queue until it is full. Your notificaiton of the consumer via notify_one is useless, because your consumer will lock right again, since the mutex is already taken (by the lock in the producer).

When you finally release the lock (when the queue is full) by waiting on the condition_variable, you don't wake up your consumer, so both your consumer and your producer are blocked and your program hangs.

Change it to:

void pcdQueue::produce()
{
    int i=0;
    while(true)
    {
        {
            boost::mutex::scoped_lock lock(qmutex);
            while( ! qlen < buffSize ) {
                std::cout << "Queue is full" << std::endl;
                full.wait(lock);
            }

            enqueue(i); // or myList.push_back(i) if you switch to std::list
        }

        empty.notify_one();


    }
}

You have the same issues in your consume()method. Change it to:

pcdFrame* pcdQueue::consume()
{
    pcdFrame *frame;

    {
        boost::mutex::scoped_lock lock(qmutex);
        while( qlen == 0 ) {
            std::cout << "Queue is empty" << std::endl;
            empty.wait(lock);
        }

        frame = dequeue();
    }
    full.notify_one();

    return frame;
}

In general, take care and note that notifications only have an effect, if someone is waiting. Otherwise, they are "lost". Furthermore, note that you don't need to keep the mutex locked when calling notify_one (in fact, it can lead to additional context-switch overhead, because you wake up the other thread which will then wait on a mutex that is currently (still) locked (by you). So first, release the mutex, then tell the other thread to continue.

Please note, that both threads run infinitely, so your main program will still hang at the first join() and never exit. You could include a stopping flag that you checkn in your while-loops to tell your threads to finish.

score 0 · Answer 2 · answered Aug 09 '13 at 03:39

Boost offers a producer/consumer queue type now, in its lockfree section, which is largely lockfree although it MAY lock if the queue fills up.

You'll find the documentation here:

http://www.boost.org/doc/libs/1_54_0/doc/html/lockfree.html

It's a fairly recent addition, so I don't think it's part of many standard packages yet. On the other hand, this looks like mostly headers, all depending on fairly low-level system stuff which has been in boost for a long time.

score 0 · Answer 3 · answered Jan 27 '14 at 19:04

I came across the same issue of hanging. Thanks to Johannes' answer, I was able to make it work completely. However, I had to use full.wait_for(lock, boost::chrono::milliseconds(100)); in both the consumer and the producer to prevent hanging when using very short circular buffer (3-4 elements).

Also, instead of while (true), I used while (buffer->empty().

Finally, everything is working reliably and fast.

producer/consumer using boost threads and circular buffer hangs

3 Answers3