boost mpi: Is there a max length for string variables to pass in an mpi message?

Question

In the following test code, If I set the SIZE parameter much higher than 960 no messgaes will transmit. Is there a max length for string variables to pass in boost mpi messages? Perhaps there is a limit in the string serialization, but I can't locate and limits in the docs... Any help is greatly appreciated.

//compile: mpic++ -Wall gather-002.cpp -o gather-002 -lboost_mpi -lboost_serialization
//run: mpirun -np 4 ./gather-002

#include <boost/mpi.hpp>
#include <iostream>
#include <vector>
#include <cstdlib>
#include <string>

#define SIZE 960

namespace mpi = boost::mpi;
using namespace std;

int main(int argc, char* argv[])
{
   mpi::environment env(argc, argv);
   mpi::communicator world;

   if (world.rank() == 0) { 
         string my_string = "MAIN";
         for (int proc = 0; proc < world.size(); ++proc){
            string outmessage = "";
            for (int i = 0; i < SIZE; i++) outmessage = outmessage + "-";
            world.send(proc, 0, outmessage);
         }

         vector<string> all_strings;
         gather(world, my_string, all_strings, 0);
         for (int proc = 0; proc < world.size(); ++proc) 
            cout << "Process #" << proc << "  " << all_strings[proc] << endl;
   }
   else { 
         string inmessage;
         world.recv(0,0,inmessage);
         gather(world, inmessage, 0);
   }
   return 0;
}

You are sending from proc 0 to proc 0, so a blocking send will deadlock. Maybe you want your sending proc loop to start from 1 instead of 0. — rtoijala, Jun 11 '19 at 06:58

rtoijala · Accepted Answer · 2019-06-11T08:44:23.003

2

Your program is deadlocking in world.send(0, 0, outmessage).

For small enough strings, your MPI library is making the call non-blocking, and the program happens to run. When crossing whatever threshold your MPI library uses for the message size, it switches to a blocking call. Since no-one is receiving the message, the send cannot continue, and the program hangs. Note that the described behavior is not required by the standard: You cannot rely on the MPI library using being non-blocking for small sizes.

From the MPI 3.1 standard, section 3.2.4:

Source = destination is allowed, that is, a process can send a message to itself. (How-ever, it is unsafe to do so with the blocking send and receive operations described above,since this may lead to deadlock.

Related question: Is the behavior of MPI communication of a rank with itself well-defined?

The solution is not to send anything from process 0 to itself.

The maximum size that can be sent is INT_MAX, which is determined by the maximum count you can give to MPI_Send. See this question for more.

edited Jun 11 '19 at 08:44

answered Jun 11 '19 at 07:26

rtoijala

1,200
10
20

Good answer. Maybe you'd like to emphasize 1) One should **never rely** on the standard send to behave in a non-blocking way for small data. Even if it works in some cases on one system / implementation. 2) https://stackoverflow.com/questions/27966262/is-there-a-limit-for-the-message-size-in-mpi-using-boostmpi – Zulan Jun 11 '19 at 07:29
1

If OP wants to learn more, try googling for "MPI eager vs rendezvous" protocols, which are the usual terms to describe this. – janneb Jun 11 '19 at 07:32
1

as pointed earlier, it is best here **not** to send anything to task `0` – Gilles Gouaillardet Jun 11 '19 at 07:40

boost mpi: Is there a max length for string variables to pass in an mpi message?

1 Answers1