Manipulating thread's nice value

Question

I wrote a simple program that implements master/worker scheme where the master is the main thread, and workers are created by it.

The main thread writes something to a shared buffer, and the worker threads read this shared buffer, writing and reading to shared buffer are organized by read/write lock.

Unfortunately, this scheme definitely leads to starvation of main thread, since a single write has to wait on several reads to complete. One possible solution is increasing the priority of the master thread, so if it wants to write something, it will get immediate access to the shared buffer.

According to a great post to a similar issue, I discovered that probably manipulating the priority of a thread under SCHED_OTHER policy is not allowed, what can be changed is the nice value only.

I wrote a procedure to give worker threads lower priority than master thread, but it seems not to work correctly.

void assignWorkerThreadPriority(pthread_t* worker)
{
    struct sched_param* worker_sched_param = (struct sched_param*)malloc(sizeof(struct sched_param));
    worker_sched_param->sched_priority =0; //any value other than 0 gives error?
    int policy = SCHED_OTHER;
    pthread_setschedparam(*worker, policy, worker_sched_param);
    printf("Result of changing priority is: %d - %s\n", errno, strerror(errno));
}

I have a two-fold question:

1. How can I set the nice value of a worker threads to avoid main thread starvation.
2. If not possible, then how can I change the scheduling policy to a one that allows changing the priority.

---

Edit: I managed to run the program using other policies, such as SCHED_FIFO, all I had to do was running the program as a super user

Answer 1

You cannot avoid problems using a read/write lock when the read and write usage is so even. You need a different method. You need a lock-free message queue or independent work queues or one of many other techniques.

Here is another way to do the job, the way I would do it. The worker can take the buffer away and work on it rather than keeping it shared:

• Write thread:
  • Create work item.
  • Lock the mutex or CriticalSection protecting the current queue and pointer to queue.
  • Add work item to queue.
  • Release the lock.
  • Optionally signal a condition variable or Event. Another option is for worker threads to check for work on a timer.
• Worker thread:
  • Create a new queue.
  • Wait for a condition variable or event or other signal, or wait on a timer.
  • Lock the mutex or CriticalSection protecting the current queue and pointer to queue.
  • Set the current queue pointer to the new queue.
  • Release the lock.
  • Proceed to work on the now private queue.
  • Delete the queue when all work items complete.
• Now write thread creates more work items. When all the worker threads have their own copies of a queue to work on it will be able to write many items in peace.

You can modify this. For example, a worker thread may lock the queue and move a limited number of work items off into its own internal queue instead of taking the whole thing.