in MPI in c how to create a struct of structs and send it to multiple process

Question

so I have this struct

typedef struct{
     float x;
     float y;
     int centroid;
}point;

And I also have this struct

typedef struct{
     int csize;//the current size
     int tsize;//the total size
     point * data;//the data carried
}ArrayList;

These second struct forms a dynamically growing array in C (Functions for dynamic growth is implemented and working fine). how exactly do I create a struct of that and send it using MPI in c? I looked at other posts like struct serialization in C and transfer over MPI and others but I could't find a solution to my problem.

I'd appreciate some help.

Best, Gedo

EDIT- the possible duplicate does NOT solve my question, my question is pertaining a struct containing a pointer to an array of structs which is dynamically growing. the duplicate question pertains having an array of native types in a struct.

I've never used the MPI_Type_create_struct methods. I always instead opt to create my own serialization functions that involve writing all the data in your struct to a single buffer array in memory (just like if you were trying to save it all to file), and send that buffer using a MPI_Bcast. Then each receiving CPU needs to perform a reverse serialization function that reads the buffer and recreates all the struct data. — MasterHD, Nov 12 '14 at 21:33
possible duplicate of [Sending typedef struct containing void\* by creating MPI drived datatype.](http://stackoverflow.com/questions/13039283/sending-typedef-struct-containing-void-by-creating-mpi-drived-datatype) — Jonathan Dursi, Nov 12 '14 at 21:35
@ jonathan dursi no, not a duplicate, as my question relates to sending an array of structs, while his question is pertaining sending a void pointer.... my problem is how to serialize the first struct, then serialize the second struct and ship them off both. — Ali Elgazar, Nov 12 '14 at 21:47
@masterHD that would've been viable if I was dealing with a small amount of data, unfortunately the data i store can go up to a few hundered million points. (Don't ask why) thanks though! — Ali Elgazar, Nov 12 '14 at 21:49
@gedo use a loop to write the millions of data points all onto 1 buffer sequentially. You can nest that within another loop if you need to send multiple structs in the single buffer. However, if your struct is large, it would be ok to have multiple calls to MPI_Bcast, one for each struct individually. — MasterHD, Nov 12 '14 at 21:57
@MasterHD sounds doable, although tedious...is it possible for you to provide some sort of example code on how to do this? — Ali Elgazar, Nov 12 '14 at 21:58
@gedo Here is a very relevant example, enjoy! http://stackoverflow.com/questions/16543519/serialization-of-struct — MasterHD, Nov 12 '14 at 22:11

score 2 · Accepted Answer · edited May 23 '17 at 12:05

Manually serializing the array of structs to a buffer and sending the buffer is a terrible idea, as it introduces another memory copy on both the send and receive side.

Sending an array of MPI structures is no different than an array of any other object; you just have to create a structure type - there are many examples here and elsewhere - making sure to calculate offsets and sizes as there may be padding inserted in your structs. Then just pass an array of them:

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include "mpi.h"

typedef struct{
    float x;
    float y;
    int centroid;
} point;

typedef struct{
    int csize;//the current size
    int tsize;//the total size
    point * data;//the data carried
} ArrayList;


int main(int argc, char **argv)
{
    MPI_Init(&argc, &argv);

    int rank, size;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    assert( size >= 2 );
    const int npts_avail=20;
    point points[npts_avail];

    ArrayList list;
    list.data = points;

    if (rank == 0) {
        int npts_used=10;
        list.csize = npts_used;
        list.tsize = npts_avail;
        for (int i=0; i<list.csize; i++) {
            points[i].x = 1.*i;
            points[i].y = -2.*i;
            points[i].centroid = i;
        }
    }

    const int nfields=3;
    MPI_Aint disps[nfields];
    int blocklens[] = {1,1,1};
    MPI_Datatype types[] = {MPI_FLOAT, MPI_FLOAT, MPI_INT};

    disps[0] = offsetof( point, x );
    disps[1] = offsetof( point, y );
    disps[2] = offsetof( point, centroid );

    MPI_Datatype istruct, pstruct;
    MPI_Type_create_struct(nfields, blocklens, disps, types, &istruct );
    MPI_Type_create_resized( istruct, 0, (char *)&(points[1]) - (char *)(&points[0]), &pstruct );
    MPI_Type_commit(&pstruct);

    if (rank == 0) {
        MPI_Send( &(list.csize), 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
        MPI_Send( &(list.tsize), 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
        MPI_Send( list.data, list.csize, pstruct, 1, 0, MPI_COMM_WORLD);
    } else if (rank == 1) {
        MPI_Recv( &(list.csize), 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        MPI_Recv( &(list.tsize), 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        MPI_Recv( list.data, list.csize, pstruct, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
    }

    if (rank == 1) {
        printf("Received: \n");
        for (int i=0; i<list.csize; i++) {
            printf(" (%f, %f): %d\n", points[i].x, points[i].y, points[i].centroid);
        }
    }

    MPI_Finalize();
}

Running gives the expected output:

$ mpirun -np 2 ./structs
Received:
 (0.000000, -0.000000): 0
 (1.000000, -2.000000): 1
 (2.000000, -4.000000): 2
 (3.000000, -6.000000): 3
 (4.000000, -8.000000): 4
 (5.000000, -10.000000): 5
 (6.000000, -12.000000): 6
 (7.000000, -14.000000): 7
 (8.000000, -16.000000): 8
 (9.000000, -18.000000): 9

Note that you could construct an MPI struct of ArrayList, as well, and use that - except that the displacement of data may have to be changed every time you resend (on the sending side), and on the receiving side it can't even be determined until you know the amount of data you'll need to receive. So best are to send the sizes first (in two messages, as I have here, or better yet, to send it as one message of two ints), and then send the array of structs as shown.

I might try to create a struct for the list then send that only, the reason is because I want to decrease network communication overhead. I can calculate the displacement of data on the receiving side I think. — Ali Elgazar, Nov 13 '14 at 07:40
Hey, If possible can you explain what this line does? MPI_Type_create_resized( istruct, 0, (char *)&(pointss[1]) - (char *)(&pointss[0]), &pstruct ); I get that it's creating the datatype for the list but I dont understand what (char *)&(pointss[1]) - (char *)(&pointss[0]) this does. I'm segfaulting when i try to receive the list on the rank 1 process (I edited it so the list contains points read from a file — Ali Elgazar, Nov 13 '14 at 09:21
@gedo Sorry - what it does is set the size of the entire type so MPI knows where the next item in the array starts. The compiler may insert padding anywhere for memory alignment or other reasons (which is why you have to explicitly measure the offsets within the structure), including at the end between points[0].centroid and points[1].x. So to make sure MPI knows correctly where the next starts, you have to get that offset right, too. For this particular struct with all 4-byte fields, it likely won't matter, but as soon as you add a char or a double or something all bets are off. — Jonathan Dursi, Nov 13 '14 at 13:49
Yeah ^.^ i figured as much and fixed my bug. thank's anyways! PS there's a missing import for stddef.h for offsetof — Ali Elgazar, Nov 13 '14 at 16:57

in MPI in c how to create a struct of structs and send it to multiple process

1 Answers1