Strange behavior with Isend

Question

The following is a simple code which just sends data of a processor i to i+1 using Isend and probes if all sends are complete.

Code:

std::vector<double> sendbuffer;
int myrank, nprocs;
std::vector <MPI::Request> req_v;

MPI::Init ();
nprocs = MPI::COMM_WORLD.Get_size ();
myrank = MPI::COMM_WORLD.Get_rank ();

sendbuffer.resize(20000);

int startin = 0;
if(myrank != nprocs-1)
for (int i = myrank+1; i <= (myrank+1);i++)
  {
        int sendrank = i;
        int msgtag = myrank * sendrank;
        int msgsz = sendbuffer.size();
        double *sdata = &(sendbuffer[startin]);
        MPI::Request req;
        req = MPI::COMM_WORLD.Isend (sdata, msgsz, MPI::DOUBLE, sendrank, msgtag);
        req_v.push_back (req);
  }

  printf("Size-%d %d\n", myrank,(int) req_v.size());
  if(req_v.size() > 0)
  MPI::Request::Waitall (req_v.size(), &(req_v[0]));

  printf("Over-%d\n", myrank);
  MPI::COMM_WORLD.Barrier();

  MPI::Finalize ();

The code completes for a buffer size of 1500 but for 20000 it stalls. The behavior seems a bit strange. I thought matching receives are not needed for Isend. Please suggest possible reasons for this behavior.

Why do you use a for loop? This loop will only be executed once i assume? — Willem Van Onsem, Jun 25 '14 at 17:43
It executes once. I started with an exercise where rank 0 send nothing, rank1 sends to 0, rank 2 sends to ranks 0 and 1 and so on. Hence, the for loop!! — Suman Vajjala, Jun 25 '14 at 17:46

score 1 · Answer 1 · edited May 23 '17 at 12:29

1

You never actually receive the messages. Without the corresponding calls to Recv (or Irecv), the send calls may never complete.

You can get more details at this question, but generally, sends are allowed to complete without the corresponding receive having been posted as long as the messages can be buffered within MPI (either on the sender side or the receiver side). Eventually, your system will run out of buffers and the send call will stop completing until you call the corresponding receive calls and free up some system buffers.

edited May 23 '17 at 12:29

Community

1
1

answered Jun 25 '14 at 18:41

Wesley Bland

8,816
3
44
59

Thank you for your reply. Does the matching receive apply to Isend too? – Suman Vajjala Jun 25 '14 at 18:45
@SumanVajjala, I'm not sure what you mean. Every time you call send (any variety), you must have a matching recv. So if you have an Isend that sends 15000 doubles to rank 1 from rank 0, you must also have a Recv (or Irecv, etc.) on rank 1 that receives at least 15000 doubles from rank 0. – Wesley Bland Jun 25 '14 at 19:00

Strange behavior with Isend

1 Answers1