MPI serial main function

Question

This is a pretty basic MPI question, but I can't wrap my head around it. I have a main function that calls another function that uses MPI. I want the main function to execute in serial, and the other function to execute in parallel. My code is like this:

int main (int argc, char *argv[])
{
    //some serial code goes here
    parallel_function(arg1, arg2);
    //some more serial code goes here
}

void parallel_function(int arg1, int arg2)
{
    //init MPI and do some stuff in parallel
    MPI_Init(NULL, NULL);
    MPI_Comm_size(MPI_COMM_WORLD, &p);
    MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
    //now do some parallel stuff
    //....
    //finalize to end MPI??
    MPI_Finalize();
}

My code runs fine and gets the expected output, but the issue is that the main function is also being run in separate processes and so the serial code executes more than once. I don't know how it's running multiple times, because I haven't even called MPI_Init yet (if I printf in main before I call parallel_function, I see multiple printf's)

How can I stop my program running in parallel after I'm done?

Thanks for any responses!

Answer 1

Have a look at this answer.

Short story: MPI_Init and MPI_Finalize do not mark the beginning and end of parallel processing. MPI processes run in parallel in their entirety.

Answer 2

@suszterpatt is correct to state that "MPI processes run in parallel in their entirety". When you run a parallel program using, for example, mpirun or mpiexec this starts the number of processes you requested (with the -n flag) and each process begins execution at the start of main. So in your example code

int main (int argc, char *argv[])
{
    //some serial code goes here
    parallel_function(arg1, arg2);
    //some more serial code goes here
}

every process will execute the //some serial code goes here and //some more serial code goes here parts (and of course they will all call parallel_function). There isn't one master process which calls parallel_function and then spawns other processes once MPI_Init is called.

Generally it is best to avoid doing what you are doing: MPI_Init should be one of the first function calls in your program (ideally it should be the first). In particular, take note of the following (from here):

The MPI standard does not say what a program can do before an MPI_INIT or after an MPI_FINALIZE. In the MPICH implementation, you should do as little as possible. In particular, avoid anything that changes the external state of the program, such as opening files, reading standard input or writing to standard output.

Not respecting this can lead to some nasty bugs.

It is better practice to rewrite your code to something like the following:

int main (int argc, char *argv[])
{

    // Initialise MPI
    MPI_Init(NULL, NULL);
    MPI_Comm_size(MPI_COMM_WORLD, &p);
    MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);

    // Serial part: executed only by process with rank 0
    if (my_rank==0)
    {
        // Some serial code goes here
    }

    // Parallel part: executed by all processes.

    // Serial part: executed only by process with rank 0
    if (my_rank==0)
    {
        // Some more serial code goes here
    }

    // Finalize MPI
    MPI_Finalize();

    return 0;

}

Note: I am not a C programmer, so use the above code with care. Also, shouldn't main always return something, especially when defined as int main()?