How to send a 2D array in MPI with variation for each processor

Question

I'm trying to take a randomly generated array on root 0, vary it slightly and randomly, and send each variation to another processor. Here is my code so far:

#include "stdio.h"
#include "stdlib.h"
#include "mpi.h"
#include "math.h"

int main(int argc, char **argv) {

  int N = 32;
  int dim = 3;
  float a = 10.0;
  int size, rank, i, j, k, q;
  float **C;
  float rijx, rijy, rijz, rij, Vij, E=0;
  float stepsize = 0.05;

  double Start_time, End_time, Elapse_time;
  MPI_Status status;

  MPI_Init(&argc, &argv);
  MPI_Comm_size(MPI_COMM_WORLD, &size);
  MPI_Comm_rank(MPI_COMM_WORLD, &rank);

  C = (float **)malloc(N * sizeof(float*)); // 32 particles

  for (i = 0; i < N; i++) {
    C[i]=(float *)malloc(dim*sizeof(float)); // x, y, z
  }

  MPI_Barrier(MPI_COMM_WORLD);

  if(rank == 0) {
    Start_time = MPI_Wtime();
  }

  if (rank == 0) {
  for(i = 0; i < N; i++) {
    for(j = 0; j < dim; j++) {
      C[i][j] = (float)rand()/(float)(RAND_MAX/a);
      //printf("%f\n",C[i][j]);
    }
  }
  }

// this is to generate some slight variations in the array

  float** randomsteps(float **matrix) {
for(i = 0; i < N; i = i+(rand()%(32/size))) {
   for (j = 0; j < dim; j++) {
      if(i%2 == 0) {
        C[i][j] = C[i][j]+stepsize;
          if(C[i][j] > 10) {
            C[i][j] = C[i][j] - 10;
          }
      } else {
        C[i][j] = C[i][j]-stepsize;
        if(C[i][j] < 0) {
          C[i][j] = C[i][j] + 10;
        }
      }
   }
}
return C;
  }


// and here I try to send the array
  if(rank == 0) {
   for(i=0; i<size; i++) {
    C = randomsteps(C);

    MPI_Send(&C, N*3, MPI_FLOAT, i, 10+i, MPI_COMM_WORLD);
   }
  }

  if(rank != 0) {
    for(i=0; i<size; i++) {
      MPI_Recv(&C, N*3, MPI_FLOAT, 0, 10+i, MPI_COMM_WORLD, &status);
    }
  }

MPI_Barrier(MPI_COMM_WORLD);

  MPI_Finalize();

  return 0;
}

An obvious problem with the code is that the way the random numbers are generated is somewhat naive (it gives the same values every time I run the program). That's something I can work on later.

For right now, I'm just wondering - what is wrong with the way I'm sending and receiving the array? I'm having a lot of trouble wrapping my head around how it is best to format data when sending and receiving using MPI. How would I go about fixing this part of the code?

Thanks in advance for the help!

You can only use . MPI_Send/MPI_Recv to move blocks of contiguous memory. Your "2D array" is an array of pointers, not a block of contiguous memory — talonmies, Nov 18 '15 at 06:40
How do I make it contiguous? I was looking at [this](http://www.just4tech.com/2013/10/matrix-multiplication-in-mpi.html) program for reference but I'm not sure where I went wrong when trying to modify the code for my program.. — Chylomicron, Nov 18 '15 at 06:51
Use a single malloc call to allocate the whole matrix, not one per row — talonmies, Nov 18 '15 at 07:19
I tried allocating it like this, but it didn't work: `C = (float **)malloc(N*dim * sizeof(float*));` I also tried following [this](http://stackoverflow.com/questions/5901476/sending-and-receiving-2d-array-over-mpi) but I also can't get that to work.. — Chylomicron, Nov 18 '15 at 15:36
Oh wait I got it. [this](http://stackoverflow.com/questions/17584215/how-does-c-allocate-space-for-a-2d-3d-array-when-using-malloc/17584424#17584424) link was really helpful. Now there's just a problem with my randomstep function messing up the memory, but I think maybe I can figure that one out. Thanks for the pointer! (... pun not intended) — Chylomicron, Nov 18 '15 at 16:03

talonmies · Accepted Answer · 2015-11-19T09:02:13.637

The problem here is that all MPI calls expect that memory is contiguous. Your memory is only contiguous within given row, and what you are referring to as a 2D array is really an array of pointers. Pointers are not portable, so trying to send or broadcast an array of pointers to another process makes no sense, and MPI itself doesn't support deep copy, so this approach won't work.

However, if you change your array allocation to something like this:

  float** C;
  float* C_buff;
  C = (float**)malloc(N * sizeof(float*)); // 32 particles
  C_buff = (float*)malloc(N * dim * sizeof(float)); // buffer for particles

  float* p = &C_buff[0];
  for (i = 0; i < N; i++) {
    C[i]=p;
    p+= dim*sizeof(float));
  }

[disclaimer: written in browser, totally untested, use at own risk]

so that C_buff represents the contiguous memory for your 2D array, and C contains row pointers to memory within the C_buff contiguous allocation, then you can use your existing code for initialisation, but then do something like this:

MPI_Send(&C_buff[0][0], N*DIM, MPI_FLOAT, i, 10+i, MPI_COMM_WORLD);

ie. use C_buff for the MPI calls, and it should work.

How to send a 2D array in MPI with variation for each processor

1 Answers1

Linked