MPI: parallelize a buffer over and over

Question

Suppose I have a large input file.

Suppose this file has items I would like to process in parallel.

std::vector<std::string> items(100000,"");
for(int i = 0; i < 1000000; i++)
    items[i] = pop_item(file);

Next, I would like to speed up processing by processing these items in parallel with MPI:

std::vector<MyObj> processed_items(100000); // pseudo-code, i handle the memory mallocing
int size; rank;
MPI_INIT();

MPI_Comm_size(MPI_COMM_WORLD,&size);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);

for(i = rank; i < 100000; i += size)
    processed_items[i] = process_item(items[i]);

MPI_FINALIZE();

Ok great, it works.

Now, I would like to do it over and over again within a while loop:

while(!done){
   done = fill_items(&items, file); 

   MPI_INIT();

   ...;

   MPI_FINALIZE();

   print_items(&processed_items);

}

However, I fail with "error: mpi_init called after mpi finalize invoked."

---

What is the expected way for me to handle this in MPI?

Answer 1

MPI_INIT() and MPI_FINALIZE can only be called once per program, as your error hints at. This old answer from half a year ago sketches how to get MPI to run some parts of your program in parallel:

int main(int argc, char *argv[]) {
    MPI_Init(&argc, &argv);  
    MPI_Comm_size(MPI_COMM_WORLD,&numprocs);  
    MPI_Comm_rank(MPI_COMM_WORLD,&myid);

    if (myid == 0) { // Do the serial part on a single MPI thread
        printf("Performing serial computation on cpu %d\n", myid);
        PreParallelWork();
    }

    ParallelWork();  // Every MPI thread will run the parallel work

    if (myid == 0) { // Do the final serial part on a single MPI thread
        printf("Performing the final serial computation on cpu %d\n", myid);
        PostParallelWork();
    }

    MPI_Finalize();  
    return 0;  
}