I met an issue when I want to send a very large size of message through MPI_Send: there are multiple processors and the total number of int we need to transfer is 2^25, I test that size in 1000 my code works well, but if I set it to the size professor asked, it will stuck for a long time and return me some information like this:
2 more processes have sent help message help-mpi-btl-base.txt / btl:no-nics
Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
mpiexec noticed that process rank 0 with PID 0 on node srv-p22-13 exited on signal 24 (CPU time limit exceeded).
I used "cout" after each line of code, and I am sure it stuck before the MPI_Send line, the size of Si is over 20,000,000. I am not sure is that the reason? But I have searched that the max limited of MPI_Send is 2^32-1...It is larger that 2^25...So I am feeling confusion.
this is the main part of my code:
//This is send part
for(int i=0; i<5; i++){
if(i!=my_rank){//my_rank is from MPI_Comm_rank(MPI_COMM_WORLD, &my_rank)
int n = A.size();//A is a vector of int
int* Si= new int[n];//I want to convert vector to a int array
std::copy(A.begin(),A.end(),Si);
MPI_Send(&Si, n, Type, i, my_rank ,MPI_COMM_WORLD);//**The code stuck here and says CPU time limit exceeded
delete[] Si;
}
}
MPI_Barrier(MPI_COMM_WORLD);//I want all the processor finish sending part, then start receive and save in vector
//This is receive part
for(int i=0; i<5; i++){
if(i!=my_rank){
MPI_Status status;
MPI_Probe(i,i,MPI_COMM_WORLD,&status);
int rn = 0;
MPI_Get_count(&status, Type, &rn);
int* Ri = new int[rn];
MPI_Recv(Ri, rn, Type, i, i, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
/*Save received elements into vector A*/
for(int i=0; i<sizeof(Ri);i++){
inout.push_back(A);
}
}
}
