0

I am trying to use MPI_ALLGATHERV using derived data types. Actually, I have to pass chunks of small 3D array in the form of: 

  SS(IIST:IIEND,JJST:JJEND,KKST:KKEND)

Here, IIST, IIEND, JJST, JJEND, KKST, KKEND are local indices of each process. So I tried to define a derived datatype in the following form: 

  INTEGER :: MPII,MPJJ,MPKK

  CALL MPI_TYPE_CONTIGUOUS(IIEND-IIST+1,MPI_DOUBLE_PRECISION,MPII,IERR)
  CALL MPI_TYPE_CONTIGUOUS(JJEND-JJST+1,MPII,MPJJ,IERR)
  CALL MPI_TYPE_CONTIGUOUS(KKEND-KKST+1,MPJJ,MPKK,IERR)            
  CALL MPI_TYPE_COMMIT(MPKK,IERR)

Now, I am defining a displacement array which is visible to every process to be used in MPI_ALLGATHERV. The total number of processes is 27 and they are numbered from 0-26. 

  DO NT=0,26
  DISPL(1)=0
  IF (NT.GT.0) DISPL(NT+1)= DISPL(NT)+1 
  ENDDO

Now, I am executing MPI_ALLGATHERV with the following syntax:

  CALL MPI_ALLGATHERV(SS(IIST:IIEND,JJST:JJEND,KKST:KKEND),SPANX*SPANY*SPANZ,MPI_DOUBLE_PRECISION,SS(1,1,1),1,DISPL,MPKK,MPI_COMM_WORLD,IERR)

This is giving me error. Any pointers in this problem will be very helpful and appreciated.

milancurcic
  • 6,202
  • 2
  • 34
  • 47
howitzer
  • 31
  • 3
  • 2
    The trouble may come from the argument `recvcount`. It is supposed to be an array of 27 integers and you wrote `SS(1,1,1),1,DISPL,` http://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Allgatherv.html – francis Jan 01 '14 at 11:09
  • Thanks Francis. Now the error is gone but still the output is incorrect. – howitzer Jan 02 '14 at 02:38
  • Do you have garbage in the array or numbers at the wrong place ? I also found this issue and its answer : http://stackoverflow.com/questions/17508647/sending-2d-arrays-in-fortran-with-mpi-gather and http://www.mcs.anl.gov/research/projects/mpi/www/www3/MPI_Type_create_subarray.html. These pieces of code and functions may be useful to get the correct layout. – francis Jan 02 '14 at 09:53
  • I will try that and get back here soon. – howitzer Jan 03 '14 at 18:05

1 Answers1

0
  spanx = iiend-iist+1
  spany = jjend-jjst+1
  spanz = kkend-kkst+1

  oldsize = (/spanx,spany,spanz/)
  newsize = (/spanx,spany,spanz/) 
  starts  = (/0,0,0/) 

  CALL MPI_TYPE_CREATE_SUBARRAY(3,OLDSIZE,NEWSIZE,STARTS,MPI_ORDER_FORTRAN,MPI_DOUBLE_PRECISION,ARR,IERR)
  CALL MPI_TYPE_COMMIT(ARR,IERR)

  DO NT=0,26
  DISPL(1)=0
  IF (NT.GT.0) DISPL(NT+1)= DISPL(NT)+1 
  SIZECC(NT)=1
  ENDDO

  CALL MPI_ALLGATHERV(SS(IIST:IIEND,JJST:JJEND,KKST:KKEND),SPANX*SPANY*SPANZ,MPI_DOUBLE_PRECISION,SS(1,1,1),SIZECC,DISPL,ARR,MPI_COMM_WORLD,IERR)

Still the output doesn`t match. I think something is wrong in Displacement array.

howitzer
  • 31
  • 3