C: select() - Signal interrupt

Question

I'm writing a multithreaded server program in C that works with AF_UNIX sockets. The basic structure of the server is:

• Main thread initialize data structures and spears a pool of "worker" threads.
• Worker threads start waiting for new requests on an empty thread-safe queue
• Main thread listen on various sockets (new connection and already connected clients) with a select() call.
  • select() reveals possible read on connection socket: main thread calls accept() and puts the returned file descriptor in the fd_set (read set).
  • select() reveal possible read on already connected sockets: main thread removes the ready file descriptors from the fd_set (read set) and puts them in the thread-safe queue.
• Worker thread extracts a file descriptor from the queue and starts to communicate with the linked client for serve the request. At the end of the service worker thread puts socket file descriptor back to the fd_set (i worte a function to make this operation thread-safe) and it returns waiting again on the queue for a new request.

This routine is repeated in a infinite cycle until a SIGINT is raised. Another function has to be performed on SIGUSR1 without exiting from the cycle.

My doubt is about this because if I raise a SIGINT my program exit with EINTR = Interrupted system call. I know about the pselect() call and the "self pipe" trick but i can't figure out how to make the things work in a multithreaded situation.

I'm looking for a (POSIX compatible) signal management that that prevent the EINTR error while main thread is waiting on pselect().

I post some pieces of code for clarification:

Here i set up signal handlers (ignore errorConsolePrint function)

if(signal(SIGINT, &on_SIGINT) == SIG_ERR)
{
    errorConsolePrint("File: %s; Line: %d; ", "Setting SIGINT handler", __FILE__, __LINE__);
    exit(EXIT_FAILURE);
}
if(signal(SIGTERM, &on_SIGINT) == SIG_ERR)
{
    errorConsolePrint("File: %s; Line: %d; ", "Setting SIGINT handler", __FILE__, __LINE__);
    exit(EXIT_FAILURE);
}
if(signal(SIGUSR1, &on_SIGUSR1) == SIG_ERR)
{
    errorConsolePrint("File: %s; Line: %d; ", "Setting to SIGUSR1 handler", __FILE__, __LINE__);
    exit(EXIT_FAILURE);
}

if(signal(SIGPIPE, SIG_IGN) == SIG_ERR)
{
    errorConsolePrint("File: %s; Line: %d; ", "Setting to ignore SIGPIPE", __FILE__, __LINE__);
    exit(EXIT_FAILURE);
}

Here i set up signal mask for pselect

sigemptyset(&mask);
sigemptyset(&saveMask);
sigaddset(&mask, SIGINT);
sigaddset(&mask, SIGUSR1);
sigaddset(&mask, SIGPIPE);

Here i call pselect

test = saveSet(masterSet, &backUpSet, &saveMaxFd);
CHECK_MINUS1(test, "Server: creating master set's backup ");

int test = pselect(saveMaxFd+1, &backUpSet, NULL, NULL, &waiting, &mask);
if(test == -1 && errno != EINTR)
{
    ...error handling...
    continue;
}

Hope in some help! Thank you all in advance.

Answer 1

What you should probably do is dedicate a thread to signal handling. Here's a sketch:

In main, before spawning any threads, block all signals (using pthread_sigmask) except for SIGILL, SIGABRT, SIGFPE, SIGSEGV, and SIGBUS.

Then, spawn your signal handler thread. This thread loops calling sigwaitinfo for the signals you care about. It takes whatever action is appropriate for each; this could include sending a message to the main thread to trigger a clean shutdown (SIGINT), queuing the "another function" to be processed in the worker pool (SIGUSR1), etc. You do not install handlers for these signals.

Then you spawn your thread pool, which doesn't have to care about signals at all.

Answer 2

I would suggest the following strategy:

• During initialization, set up your signal handlers, as you do.
• During initialization, block all (blockable) signals. See for example Is it possible to ignore all signals?.
• Use pselect in your main thread to unblock threads for the duration of the call, again as you do.

This has the advantage that all of your system calls, including all those in all your worker threads, will never return EINTR, except for the single pselect in the main thread. See for example the answers to Am I over-engineering per-thread signal blocking? and pselect does not return on signal when called from a separate thread but works fine in single thread program.

This strategy would also work with select: just unblock the signals in your main thread immediately before calling select, and re-block them afterwards. You only really need pselect to prevent hanging if your select timeout is long or infinite, and if your file descriptors are mostly inactive. (I've never used pselect myself, having worked mostly with older Unix's which did not have it.)

I am presuming that your signal handlers as suitable: for example, they just atomically set a global variable.

BTW, in your sample code, do you need sigaddset(&mask, SIGPIPE), as SIGPIPE is already ignored?

Answer 3

Ok, finally I got a solution.

The heart of my problem was about the multithreading nature of my server. After long search I found out that in the case we have signals raised from other process (in an asyncronous way), it doens't matter which thread capture signal because the behaviour remains the same: The signal is catched and the previously registered handler is executed. Maybe this could be obvious for others but this was driving me crazy because I did not know how to interpret errors that came out during execution.

After that i found another problem that I solved, is about the obsolete signal() call. During execution, the first time i rise SIGUSR1, the program catch and manage it as expected but the second time it exit with User defined signal 1.

I figured out that signal() call set "one time" handler for a specific signal, after the first time that the signal is handled the behaviour for that signal return the default one.

So here's what I did:

Here the signal handlers:

N.B.: I reset handler for SIGUSR1 inside the handler itself

static void on_SIGINT(int signum)
{
    if(signum == SIGINT || signum == SIGTERM)
        serverStop = TRUE;
}

static void on_SIGUSR1(int signum)
{
    if(signum == SIGUSR1)
        pendingSIGUSR1 = TRUE;

    if(signal(SIGUSR1, &on_SIGUSR1) == SIG_ERR)
        exit(EXIT_FAILURE);
}

Here I set handlers during server's initialization:

 if(signal(SIGINT, &on_SIGINT) == SIG_ERR)
    exit(EXIT_FAILURE);
if(signal(SIGTERM, &on_SIGINT) == SIG_ERR)
    exit(EXIT_FAILURE);
if(signal(SIGUSR1, &on_SIGUSR1) == SIG_ERR)
    exit(EXIT_FAILURE);
if(signal(SIGPIPE, SIG_IGN) == SIG_ERR)
    exit(EXIT_FAILURE);

And here the server's listening cycle:

while(!serverStop)
{
    if (pendingSIGUSR1)
    {
        ... things i have to do on SIGUSR1...
        pendingSIGUSR1 = FALSE;
    }


    test = saveSet(masterSet, &backUpSet, &saveMaxFd);
    CHECK_MINUS1(test, "Server: creating master set's backup ");

    int test = select(saveMaxFd+1, &backUpSet, NULL, NULL, &waiting);
    if((test == -1 && errno == EINTR) || test == 0)
        continue;
    if (test == -1 && errno != EINTR)
    {
        perror("Server: Monitoring sockets: ");
        exit(EXIT_FAILURE);
    }

    for(int sock=3; sock <= saveMaxFd; sock++)
    {
        if (FD_ISSET(sock, &backUpSet))
        {
            if(sock == ConnectionSocket)
            {
                ClientSocket = accept(ConnectionSocket, NULL, 0);
                CHECK_MINUS1(ClientSocket, "Server: Accepting connection");

                test = INset(masterSet, ClientSocket);
                CHECK_MINUS1(test, "Server: Inserting new connection in master set: ");
            }
            else
            {
                test = OUTset(masterSet, sock);
                CHECK_MINUS1(test, "Server: Removing file descriptor from select ");
                test = insertRequest(chain, sock);
                CHECK_MINUS1(test, "Server: Inserting request in chain");
            }
         }
    }
}

Answer 4

Read first signal(7) and signal-safety(7); you might want to use the Linux specific signalfd(2) since it fits nicely (for SIGTERM & SIGQUIT and SIGINT) into event loops around poll(2) or the old select(2) (or the newer pselect or ppoll)

See also this answer (and the pipe(7) to self trick mentioned there, which is POSIX-compatible) to a very similar question.

Also, signal(2) documents:

  The effects of signal() in a multithreaded process are unspecified.

so you really should use sigaction(2) (which is POSIX).