MPI_Finalize() does not end any processes

Question

I'm messing around with openMPI, and I have a wierd bug.

It seems, that even after MPI_Finalize(), each of the threads keeps running. I have followed a guide for a simple Hello World program, and it looks like this:

#include <mpi.h>;

int main(int argc, char** argv) {

// Initialize the MPI environment
    MPI_Init(NULL, NULL);

// Get the number of processes
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

// Get the rank of the process
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

// Get the name of the processor
    char processor_name[MPI_MAX_PROCESSOR_NAME];
    int name_len;
    MPI_Get_processor_name(processor_name, &name_len);

// Print off a hello world message
       printf("Hello world from processor %s, rank %d"
       " out of %d processors\n",
       processor_name, world_rank, world_size);

// Finalize the MPI environment.
    MPI_Finalize();

    printf("This is after finalize");
}

Notice the last printf()... This should only be printed once, since the parallel part is finalized, right?!

However, the output from this program if i for example run it with 6 processors is:

mpirun -np 6 ./hello_world

Hello world from processor ubuntu, rank 2 out of 6 processors
Hello world from processor ubuntu, rank 1 out of 6 processors
Hello world from processor ubuntu, rank 3 out of 6 processors
Hello world from processor ubuntu, rank 0 out of 6 processors
Hello world from processor ubuntu, rank 4 out of 6 processors
Hello world from processor ubuntu, rank 5 out of 6 processors
This is after finalize...
This is after finalize...
This is after finalize...
This is after finalize...
This is after finalize...
This is after finalize...

Am I misunderstanding how MPI works? Should each thread/process not be stopped by the finalize?

Yes, you're misunderstanding how MPI works. `mpi_finalize` is, essentially, a statement to say that you'll be issuing no more `mpi` instructions thereafter. It is not an instruction to shut down processes. What the underlying system does is not specified, the behaviour you report is typical. For more amusement, try copying that `printf` statement to before `mpi_init`. — High Performance Mark, May 25 '15 at 13:49
Thank you.. (You were first, so please make a answer I can accept :) ) So what I should really do is to put whatever logic I want done only once inside an if(world_rank == 0) { } to be sure it is only done once? Or is there a best practice that is preferable? — MikkelSecher, May 25 '15 at 13:56
@HighPerformanceMark "a statement to say that you'll be issuing no more mpi instructions thereafter" That's wrong! The process has to quit after. it shouldn't do anything than return. see my answer. — adjan, May 25 '15 at 14:01
Sorry @addy2012, but HighPerformanceMark is actually correct. There is some language in the MPI standard that specifies this (man pages aren't gospel). — Wesley Bland, May 28 '15 at 14:03

score 5 · Answer 1 · answered May 25 '15 at 13:50

This is just undefined behavior.

The number of processes running after this routine is called is undefined; it is best not to perform much more than a return rc after calling MPI_Finalize.

http://www.mpich.org/static/docs/v3.1/www3/MPI_Finalize.html

score 3 · Answer 2 · answered May 28 '15 at 14:01

The MPI standard only requires that rank 0 return from MPI_FINALIZE. I won't copy the entire text here because it's rather lengthy, but you can find it in the version 3.0 of the standard (the latest for a few more days) in Chapter 8, section 8.7 (Startup) on page 359 - 361. Here's the most relevant parts:

Although it is not required that all processes return from MPI_FINALIZE, it is required that at least process 0 in MPI_COMM_WORLD return, so that users can know that the MPI portion of the computation is over. In addition, in a POSIX environment, users may desire to supply an exit code for each process that returns from MPI_FINALIZE.

There's even an example that's trying to do exactly what you said:

Example 8.10 The following illustrates the use of requiring that at least one process return and that it be known that process 0 is one of the processes that return. One wants code like the following to work no matter how many processes return.

...  
MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
...
MPI_Finalize();
if (myrank == 0) {
    resultfile = fopen("outfile","w");
    dump_results(resultfile);
    fclose(resultfile);
} exit(0);

The MPI standard doesn't say anything else about the behavior of an application after calling MPI_FINALIZE. All this function is required to do is clean up internal MPI state, complete communication operations, etc. While it's certainly possible (and allowed) for MPI to kill the other ranks of the application after a call to MPI_FINALIZE, in practice, that is almost never the way that it is done. There's probably a counter example, but I'm not aware of it.

score 1 · Answer 3 · edited May 23 '17 at 12:24

When I started MPI, I had same problem with MPI_Init and MPI_Finalize methods. I thought between these functions work parallel and outside work serial. Finally I saw this answer and I figured its functionality out.

J Teller's answer: https://stackoverflow.com/a/2290951/893863

int main(int argc, char *argv[]) {
    MPI_Init(&argc, &argv);  
    MPI_Comm_size(MPI_COMM_WORLD,&numprocs);  
    MPI_Comm_rank(MPI_COMM_WORLD,&myid);

    if (myid == 0) { // Do the serial part on a single MPI thread
        printf("Performing serial computation on cpu %d\n", myid);
        PreParallelWork();
    }

    ParallelWork();  // Every MPI thread will run the parallel work

    if (myid == 0) { // Do the final serial part on a single MPI thread
        printf("Performing the final serial computation on cpu %d\n", myid);
        PostParallelWork();
    }

    MPI_Finalize();  
    return 0;  
}

MPI_Finalize() does not end any processes

3 Answers3