Access pthread shared std:map without data race

Question

My scenario is to have a main thread and tens of worker threads. Worker threads will process incoming messages from different ports.

What I want to do is to have main and worker threads share a same map, the worker threads save data into map (in different bucket). And the main thread grep the map content periodically.

The code goes like:

struct cStruct
{
    std::map<string::string> map1; 
    pthread_mutex_t mutex1;
    pthread_mutex_t mutex2;  
};

int main(){

    struct cStruct cStruct1;   
    while (condition){
        pthread_t th;
        int th_rc=pthread_create(&th,NULL,&task,(void *) &cStruct1);
    }

}

void* task(void* arg){
    struct cStruct cs = * (struct cStruct*) arg;

    while (coming data){
        if (main_thread_work){
            pthread_cond_wait(&count_cond, &cs.mutex1)
        }  

        pthread_mutex_lock(&cs.mutex1);
        // add a new bucket to the map
        cs.map1(thread_identifier)=processed_value;
        pthread_mutex_unlock(&cs.mutex1);

    }

void* main_thread_task(void* arg){

    sleep(sleep_time);
    main_thread_work = true;
    pthread_mutex_lock(&cs.mutex1);
    // main_thread reads the std::map
    main_thread_work = false;
    pthread_cond_broadcast(&count_cond, &cs.mutex1)
    pthread_mutex_unlock(&cs.mutex1);    
}

My questions are:

For map size change, I should use lock to protect the map. But for map with certain key update, can I let different threads modify the map concurrently? (assume no two identical buckets of map will be accessed at same time)

For the main thread greps the map, I thought of use conditional wait to hold all the worker threads while main thread is grepping the map content, and do a pthread_cond_broadcast to wake then up. The problem is that if a worker thread is updating map while main starts to work, there will be data race.

Please share some ideas to help me improve my design.

Edit 1: Add main_thread_task(). The thing I want to avoid is worker thread arriving pthread_cond_wait "after" pthread_cond_broadcast and the logic goes wrong.

So I false the main_thread_work before main thread broadcasts workers thread.

Answer 1

You should use mutex locking mechanism on each access to the map (in your case) and not only on adding a new 'bucket'. In case T1 tries to write some value to the map while T2 inserts a new bucket, the pointer/iterator which is used by T1 becomes invalid.

Regarding the pthread_cond_wait. It may do the job in case the only thing that the other threads do is just modifying the map. If they perform other calculations or process some non shared data, it is better to use the same mutex just to protect access to the map and let other threads do their job which may be at that point not related to the shared map.

Answer 2

while (coming data){
    if (main_thread_work){
        pthread_cond_wait(&count_cond, &cs.mutex1)
    }  

    pthread_mutex_lock(&cs.mutex1);

This clearly can't be right. You can't check main_thread_work unless you hold the lock that protects it. How can the call to pthread_cond_wait release a lock it doesn't hold?!

This should be something like:

void* task(void* arg){
    struct cStruct cs = * (struct cStruct*) arg;

    // Acquire the lock so we can check shared state
    pthread_mutex_lock(&cs.mutex1);

    // Wait until the shared state is what we need it to be
    while (main_thread_work)
        pthread_cond_wait(&count_cond, &cs.mutex1)

    // Do whatever it is we're supposed to do when the
    // shared state is in this state
    cs.map1(thread_identifier)=processed_value;

    // release the lock
    pthread_mutex_unlock(&cs.mutex1);
}