OpenMP cancel section

Question

I have a problem with terminating sections in a C program. After catching a SIGINT signal in one thread I wanted to exit all threads and I don't really know how because I have infinite loops in these loops. The program waits for input from server or stdin. So I used signal handler.

I don't really know if I am doing this right way and I don't really understand how cancel in OpenMP works. I didn't find a proper tutorial or lecture for this.

My task is after catching SIGINT signal, terminate the program. But when I use exit() in the handler it leaves un-freed memory obviously. I will be glad for any advise, thank you.

#pragma omp parallel num_threads(2)
{
    #pragma omp sections
    {
        #pragma omp section
        {
            void intHandler(int dummy) 
            {
                char * welcome1 = calloc(strlen(username)+14,sizeof(char));
                strcat(welcome1,username);
                strcat(welcome1," logged out\r\n");
                if(send(client_socket,welcome1,strlen(welcome1),0) < 0)
                {
                    callError("ERROR: cannot send socked");
                }
                free(welcome1);
                #pragma omp cancel section
            }
            signal(SIGINT, intHandler);
            int i = 0, j = 1;
            while(1)
            {
                str = (char*)malloc(sizeof(char));
                while((c = getc(stdin)) != '\n')
                {

                    str = (char*)realloc(str, j * sizeof(char));
                    str[i] = c;
                    i++;
                    j++;
                }
                str = (char*)realloc(str, j * sizeof(char));
                str[i] = '\0';
                if(strlen(str)!=0)
                {
                    bufferIn = message(username,str);
                    if(send(client_socket,bufferIn,strlen(bufferIn),0) < 0)
                    {
                        callError("ERROR: cannot send socked");
                    }
                    free(bufferIn);
                }
                free(str); i = 0; j = 1; 
            }
        #pragma omp cancellation point section
        }
        #pragma omp section
        {
            void intHandler(int dummy) 
            {
                char * welcome1 = calloc(strlen(username)+14,sizeof(char));
                strcat(welcome1,username);
                strcat(welcome1," logged out\r\n");
                if(send(client_socket,welcome1,strlen(welcome1),0) < 0)
                {
                    callError("ERROR: cannot send socked");
                }
                free(welcome1);
                #pragma omp cancel section
            }
            signal(SIGINT, intHandler);
            char buffer[4096];
            ssize_t length;
            int received = 0;
            int data_cap = 4096;
            while(1)
            {
                data = calloc(BUFFER_LEN,sizeof(char));
                while ((length = read(client_socket, buffer, BUFFER_LEN-1)) > 0)
                {
                    received += length;
                    buffer[length] = '\0';
                    if (received > data_cap)
                    {
                         data = realloc(data,sizeof(char) * data_cap * 2);
                         data_cap = data_cap * 2;
                    }
                    strcat(data, buffer); 
                    if(!isEnough(data))
                    {
                        break;
                    }
                }
                printf("%s", data);
                free(data); bzero(buffer,BUFFER_LEN); data_cap = 4096; received = 0; length = 0;
            }
        #pragma omp cancellation point section
        }
    }

} 

Answer 1

This is actually super complicated, but let's just start simple.

• It's #pragma omp cancellation point sections / #pragma omp cancel sections (mind the s).

• You cannot use #pragma omp cancel across function boundaries.¹

Let's say you could use cancel this way, cancellation is only checked for at specific cancellation points. So during a blocking read or getc, your threads will not be interrupted by a cancellation.

Signal handling

Signal handlers are setup per process, it is not deterministic at what thread a signal ends at. You should not try to call signal concurrently from multiple threads. Some implementations of signal even like multi-threaded programs at all. Instead, you should use sigaction, but still setup a global signal handler once before even spawning the worker threads.

There are certain restrictions on what you are allowed to do in a signal handler. Basically you must not access any global variables that are not of type volatile sig_atomic_t and call only async-signal-safe functions. You violate that in every single line of your signal handler.

In particular, you call send(client_socket) while either the same thread might just be interrupted while calling read(client_socket)² or another thread calling read(client_socket) concurrently. Not sure what is worse, but even if send itself is async-signal-safe, I'd wager a wild guess, it's not safe in the suggested manner.

You see, having some reachable memory at the end of the process is absolutely the very least of your problems. You're not even allowed to call exit (you can call _exit though).

The usual way out, is to set make a global cancel flag of type volatile sig_atomic_t and set that within the signal handler as well as check for it in the worker loops. This should also work with OpenMP section/threads, but I would advise to add a #pragma omp atomic read/write seq_cst for any read/write to the flag. That may seem redundant, but I'm fairly sure that volatile sig_atomic_t only guarantees atomicity regarding interruption for signals, and not multithreading and in particular storage visibility. Unfortunately, you still have the issue that read and getc are blocking...

A sketch of a solution

So you would have to use some mechanism to make getc non-blocking or add a timeout to give your thread a chance to check for the cancel flag.

poll can give you a more elegant way out. You can replace the blocking part of both read and getc with poll - keep in mind however that this forces you to use stdin exclusively as a file descriptor, never as a FILE*. In preparation you make a pipe for each section whose output you include in the respective poll. Within the signal handler, that you setup only after the pipes is generated, you write to those pipes, indicating that the threads should shut down. If poll shows activity for those particular stop-pipe-fds, you exit the loop, and do your cleanup in the appropriate thread or after the parallel region. Apply synchronization as necessary.

Sorry to bring you the bad news. It's complicated and there is no simple, correct, copy-pasteable, solution. In any case, OpenMP cancellation is not the right thing to use here.

¹: The standard is not quite explicit about that, it mandates that:

During execution of a construct that may be subject to cancellation, a thread must not encounter an orphaned cancellation point. That is, a cancellation point must only be encountered within that construct and must not be encountered elsewhere in its region.

However, since cancel regions are itself implicit cancellation points, I suppose one might imply that cancel constructs must always be within the lexical scoped of a explicit parallel region. Anyway, gcc won't let you compile a function like your signal handler with a cancel construct.

²: This is actually fine, but you have to at least manually restore errno because the write in your signal handler overwrites it. The read should conveniently return with EINTR.