MPI runtime error: Either Scatterv count error, segmentationfault, or gets stuck

Question

/*

Matricefilenames:
  small matrix A.bin of dimension 100 × 50
  small matrix B.bin of dimension 50 × 100
  large matrix A.bin of dimension 1000 × 500
  large matrix B.bin of dimension 500 × 1000

An MPI program should be implemented such that it can
• accept two file names at run-time,
• let process 0 read the A and B matrices from the two data files,
• let process 0 distribute the pieces of A and B to all the other processes,
• involve all the processes to carry out the the chosen parallel algorithm
for matrix multiplication C = A * B ,
• let process 0 gather, from all the other processes, the different pieces
of C ,
• let process 0 write out the entire C matrix to a data file.
*/




int main(int argc, char *argv[]) {

  printf("Oblig 2 \n");
  double **matrixa;
  double **matrixb;
  int ma,na,my_ma,my_na;
  int mb,nb,my_mb,my_nb;
  int i,j,k;
  int myrank,numprocs;
  int konstanta,konstantb;

  MPI_Init(&argc,&argv);
  MPI_Comm_rank(MPI_COMM_WORLD,&myrank);
  MPI_Comm_size(MPI_COMM_WORLD,&numprocs);





  if(myrank==0) {
    read_matrix_binaryformat ("small_matrix_A.bin", &matrixa, &ma, &na);
    read_matrix_binaryformat ("small_matrix_B.bin", &matrixb, &mb, &nb);
  }

  //mpi broadcast

  MPI_Bcast(&ma,1,MPI_INT,0,MPI_COMM_WORLD);
  MPI_Bcast(&mb,1,MPI_INT,0,MPI_COMM_WORLD);
  MPI_Bcast(&na,1,MPI_INT,0,MPI_COMM_WORLD);
  MPI_Bcast(&nb,1,MPI_INT,0,MPI_COMM_WORLD);

  fflush(stdout);

  int resta = ma % numprocs;//rest antall som har den største verdien
  //int restb = mb % numprocs;
  if (myrank == 0) {
    printf("ma : %d",ma);
    fflush(stdout);
    printf("mb : %d",mb);
    fflush(stdout); 

  } 

  MPI_Barrier(MPI_COMM_WORLD);
  if (resta == 0) {
    my_ma = ma / numprocs;
    printf("null rest\n ");
    fflush(stdout);
  } else {
    if (myrank < resta) {
      my_ma = ma / numprocs + 1;//husk + 1 
    } else {
      my_ma = ma / numprocs;    //heltalls divisjon gir nedre verdien !
    }
  }




  my_na = na;
  my_nb = nb;

  double **myblock = malloc(my_ma*sizeof(double*));
  for(i=0;i<na;i++) {
    myblock[i] = malloc(my_na*sizeof(double));
  }

  //send_cnt for scatterv
  //________________________________________________________________________________________________________________________________________________
  int* send_cnta = (int*)malloc(numprocs*sizeof(int));//array med antall elementer sendt til hver prosess array[i] = antall elementer , i er process
  int tot_elemsa = my_ma*my_na;
  MPI_Allgather(&tot_elemsa,1,MPI_INT,&send_cnta[0],1,MPI_INT,MPI_COMM_WORLD);//arrays i c må sendes &array[0]




  //send_disp for scatterv
  //__________________________________________________________________________________

    int* send_dispa = (int*)malloc(numprocs*sizeof(int)); //hvorfor trenger disp
    // int* send_dispb = (int*)malloc(numprocs*sizeof(int));
    //disp hvor i imagechars første element til hver prosess skal til


    fflush(stdout);
    if(resta==0) {
      send_dispa[myrank]=myrank*my_ma*my_na;
    } else if(myrank<=resta) {
      if(myrank<resta) {    
    send_dispa[myrank]=myrank*my_ma*my_na;
      } else {//my_rank == rest
    send_dispa[myrank]=myrank*(my_ma+1)*my_na;
    konstanta=myrank*(my_ma+1)*my_na;
      } 
    }



    MPI_Bcast(&konstanta,1,MPI_INT,resta,MPI_COMM_WORLD);

    if (myrank>resta){
      send_dispa[myrank]=((myrank-resta)*(my_ma*my_na))+konstanta;
    }


    MPI_Allgather(&send_dispa[myrank],1,MPI_INT,&send_dispa[0],1,MPI_INT,MPI_COMM_WORLD);


    //___________________________________________________________________________________

     printf("print2: %d" , myrank);
     fflush(stdout);

    //recv_buffer for scatterv
    double *recv_buffera=malloc((my_ma*my_na)*sizeof(double));


    MPI_Scatterv(&matrixa[0], &send_cnta[0], &send_dispa[0], MPI_UNSIGNED_CHAR, &recv_buffera[0], my_ma*my_na, MPI_UNSIGNED_CHAR, 0, MPI_COMM_WORLD);


    for(i=0; i<my_ma; i++) {
      for(j=0; j<my_na; j++) {
    myblock[i][j]=recv_buffera[i*my_na + j];

      }
    }

    MPI_Finalize();
    return 0;
}

OLD:I get three type of errors. I can get scatterv count error, segmentationfault 11, or the processes just get stuck. It seems to be random which error I get. I run the code with 2 procs each time. When it gets stuck it gets stuck before the printf("print2: %d" , myrank);. When my friend runs the code on his own computer also with two prosesses, he does not get past by the first MPI_Bcast. Nothing is printed out when he runs it. Here is a link for the errors I get: http://justpaste.it/zs0

UPDATED PROBLEM: Now I get only a segmentation fault after " printf("print2: %d" , myrank); " before the scatterv call. EVEN if I remove all the code after the printf statement I get the segmentation fault, but only if I run the code for more than two procs.

Answer 1

I'm having a little difficulty tracing what you were trying to do. I think you're making the scatterv call more complicated than it needs to be though. Here's a snippet I had from a similar assignment this year. Hopefully it's a clearer example of how scatterv works.

 /*********************************************************************
  * Scatter A to All Processes
  *   - Using Scatterv for versatility.
  *********************************************************************/        


  int *send_counts;                // Send Counts
  int *displacements;              // Send Offsets
  int chunk;                       // Number of Rows per Process (- Root)
  int chunk_size;                  // Number of Doubles per Chunk
  int remainder;                   // Number of Rows for Root Process
  double * rbuffer;                // Receive Buffer

  // Do Some Math
  chunk = m / (p - 1);
  remainder = m % (p - 1);
  chunk_size = chunk * n;

  // Setup Send Counts
  send_counts = malloc(p * sizeof(int));
  send_counts[0] = remainder * n;
  for (i = 1; i < p; i++)
    send_counts[i] = chunk_size;

  // Setup Displacements
  displacements = malloc(p * sizeof(int));
  displacements[0] = 0;
  for (i = 1; i < p; i++)
    displacements[i] = (remainder * n) + ((i - 1) * chunk_size);  

  // Allocate Receive Buffer
  rbuffer = malloc(send_counts[my_rank] * sizeof(double));

  // Scatter A Over All Processes!
  MPI_Scatterv(A,                      // A
               send_counts,            // Array of counts [int]
               displacements,          // Array of displacements [int]
               MPI_DOUBLE,             // Sent Data Type
               rbuffer,                // Receive Buffer
               send_counts[my_rank],   // Receive Count - Per Process
               MPI_DOUBLE,             // Received Data Type
               root,                   // Root
               comm);                  // Comm World

  MPI_Barrier(comm);

Also, this causes a segfault on my machine, no mpi... Pretty sure it's the way myblock is being allocated. You should do what @Hristo suggested in the comments. Allocate both matrices and the resultant matrix as flat arrays. That would eliminate the use of double pointers and make your life a whole lot simpler.

#include <stdio.h>
#include <stdlib.h>

void main ()
{

    int na = 5;
    int my_ma = 5;
    int my_na = 5;
    int i;
    int j;

  double **myblock = malloc(my_ma*sizeof(double*));
  for(i=0;i<na;i++) {
    myblock = malloc(my_na*sizeof(double));
  }

  unsigned char *recv_buffera=malloc((my_ma*my_na)*sizeof(unsigned char));

  for(i=0; i<my_ma; i++) {
    for(j=0; j<my_na; j++) {
      myblock[i][j]=(float)recv_buffera[i*my_na + j];
      }
    }

}

Try allocating more like this:

  // Allocate A, b, and y. Generate random A and b
  double *buff=0;
  if (my_rank==0)
    {
      int A_size = m*n, b_size = n, y_size = m;
      int size = (A_size+b_size+y_size)*sizeof(double);
      buff = (double*)malloc(size);
      if (buff==NULL) 
    {
      printf("Process %d failed to allocate %d bytes\n", my_rank, size);
      MPI_Abort(comm,-1);
      return 1;
    }
      // Set pointers
      A = buff; b = A+m*n; y = b+n;
      // Generate matrix and vector
      genMatrix(m, n, A);
      genVector(n, b);
    }