Initialized variable before MPI_Init() and cout after MPI_Finanlize()

Question

I have been testing how exactly mpi works with the following code

    #include <iostream>
    #include <mpi.h>
    using namespace std;
    int main(int argc, char *argv[]){
    r = 3.0;
    int id; 
    int p;
    int a[100];
    for(int i=0;i<100;++i){a[i]=i+5; }
    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &id);
    MPI_Comm_size(MPI_COMM_WORLD, &p);

    cout<<id<<"   "<<r<<"   "<<a[id]<<endl;

    MPI_Finalize();

    cout<< "Hello world " <<endl;

    return 0;
    }

I am using 30 cores to run the code. but the output is somehow surprising on 2 aspects,

Here this question says that if you initialize a variable before MPI_Init(), the other process but 0 will be blind about the initialized value, but in my output file all the process of different ids output the r=3 and the correct value for a[i], so does this mean that I can initialize a variable or an array before calling MPI_Init() and all processes will share the same value for that variable?
The output has a lot of lines of "Hello world " which apparently indicates that each process is printing "Hello world " even after calling MPI_Finalize(), why is that?

By the way, I am using mpicc for compiling the code.

score 8 · Accepted Answer · edited Jul 05 '14 at 01:48

8

It's perfectly valid to execute code before MPI_Init and after MPI_Finalize. Of course you are not allowed to use MPI in that code, but otherwise it's just normal C++.

MPI_Init and MPI_Finalize are just library calls, they are not supposed to change control flow or remove values assigned prior to init (and how would that be possible at all?).

MPI_Init doesn't cancel initialization of variables.

The question you're referring to is saying just that initialization in process 0 doesn't initialize values in other processes. In that question, data were read from file which probably existed only for process 0.

Note that MPI_Init doesn't create new process. It's not like fork. All the processes are created before your program starts (most likely by mpirun). And in your case, each process initializes its array.

MPI_Finalize doesn't terminate the process. It only shuts down the MPI library. Processes still continue running after that, although they cannot interact any more.

edited Jul 05 '14 at 01:48

Wesley Bland

8,816
3
44
59

answered Jul 03 '14 at 22:48

nullptr

11,008
1
23
18

Thanks. I got the 1st part. But what should I do if I just want to print one line of "Hello World ", the one in my mind is that `if(id==0)cout<<"Hello world "< – Allanqunzi Jul 03 '14 at 22:51
You proposed a perfect solution: if you need only the 0th process to do something, check its rank (and of course value in `id` survives `MPI_Finalize`). I am not aware of any other way of distinguishing MPI processes. – nullptr Jul 03 '14 at 22:57
I thought `MPI_Abort` would work. But after trying I found that after calling `MPI_Abort` even the main process(id=0) terminates, it doesn't print at all. – Allanqunzi Jul 03 '14 at 23:05
Yes, MPI_Abort terminates all processes. `if (id == 0)` is the easiest and the sanest way to do something in the 'main' process. (By the way, all processes are equal, I think it's just conventional that we are calling the 0th process 'main'. It doesn't have any specific privileges, it's not a parent for all other processes, it's just one of them.) – nullptr Jul 03 '14 at 23:15
As a matter of fact, rank 0 is special in many MPI implementations. For example, redirection of the standard input of `mpiexec` happens only to rank 0 - no other rank can get input from `stdin` / `cin`. This behaviour applies to Open MPI, as well as to MPICH and its derivatives, and possibly to many other implementations. Some MPI implementations allow I/O only from rank 0 or from some other specific rank. The standard defines some attributes on `MPI_COMM_WORLD` that can be queried to find things out. – Hristo Iliev Jul 04 '14 at 23:05
1

Also, it's actually possible that some implementations wouldn't allow code to be executed at all before `MPI_INIT` or after `MPI_FINALIZE`. I believe FG-MPI does this since it actually does implement MPI ranks as threads which are `fork`/`join`'ed at `INIT`/`FINALIZE`. – Wesley Bland Jul 05 '14 at 01:49

Initialized variable before MPI_Init() and cout after MPI_Finanlize()

1 Answers1