Configuration auto reloading with pthreads

Question

I am writing multi threaded program in C, currently I need to restart the program every time I make a change to the configuration file, my application also support the standard SIGHUP signal to reload the configuration, but this need manual intervention.

To solve this, I wrote a separate thread that read the configuration file and load it, and keep monitoring this file for any changes.

The question is, how to notify other threads about a configuration change safely and without affecting the performance with huge mutex locks.

I am thinking of having a version number for each configuration change, this way I would only need to lock config_ver variable changes, and keep old configurations accessible for slower threads.

Any idea would be appreciated.

score 1 · Accepted Answer · answered Apr 05 '11 at 20:22

Using gcc atomic operations you can quickly test if a config changed (time to compare two ints) and you can load new configs...ALL WITHOUT LOCKING.

Psuedo code:

Lets say I had a configuration struct C. Let there be a global variable _pConfig which points to the current config.

struct C *_pConfig;

to load a new config:

// allocate a new struct C
struct C *pNewconfig = malloc(sizeof(struct C));
...
// load the pNewconfig struct from disk or something
...

// let the config struct have a next pointer so we can save list of configs for freeing later
pNewconfig->pNext = _pConfig;  

// when done loading pNewconfig. change the global.  not before done!, else threads see unf data!
// 32 bit assignment is atomic (usually).  
// If not atomic on your platform, use __sync_bool_compare_and_swap()
_pConfig = pNewConfig;

// is safe to free old cfgs with 0 use counts.  Make sure old is old enough so that there is no chance
// a thread could have been swapped out anywhere between point A and B below (next code section).  
for (struct C *pCfg=pNewconfig->pNext ; pCfg ; pCfg=pCfg->pNext) {
  // Free last pcfg (!pCfg->pNext) if its not in use and its old enough.
  // Don't have to atomically check cUse here since once it changes to zero, its zero forever.
  // nBirthday could be clock ticks when the config was created.  nNow could be ticks now.
  if (!pCfg->pNext && !pCfg->cUse && pCfg->nBirthDay-nNow > OldEnough) {  
    free(pCfg);
    break;
  }
}

Now how do we use this... Each thread needs to keep a ptr to struct C the thread was configured with. If _pConfig changes, each thread can tell by comparing the thread's struct C address to the current one. Assume pthread is a pointer to the thread's data.

while (1) {
  // POINT A
  struct C *pConfig = _pConfig; // make a copy in case it changes
  // super quick check with no locking!
  if (pConfig == pThread->pConfig) 
    break;  // no change...get out quick.
  __sync_add_and_fetch(&pConfig->cUse, 1); // increment use count of new cfg
  // POINT B
   __sync_sub_and_fetch(&pThread->pConfig->cUse, 1);  // decriment use count of thread's cfg
   pThread->pConfig = pConfig;  // use new cfg
   // do whatever you do with the cfg data
}

Use __sync_add, _sub because multiple threads can be accessing the new config at the same time and this ensures an accurate use count.

Be aware of the stall time of a thread scheduled out between point A and B. Since a time slice is usually 1 ms, a long stall might be a 10-100 ms, unless the scheduler is hosed. If your configuration data is small and changes only a couple times an hour, I might hold onto it for days before freeing it. Pick a big enough time that you know there are no stalled threads. Don't worry if a thread for some reason hasn't checked for a new config in a long time...the use count will be > 1 and it wont be freed.

The reference counting is probably overkill if you wait long enough before freeing the old configs...or don't free them at all. The important concept to this solution is you make a new config structure allowing threads to still use the old one until they notice the change. — johnnycrash, Apr 06 '11 at 20:59

Configuration auto reloading with pthreads

1 Answers1

Linked