MPI_Reduce() program getting segmentation fault

Question

I'm starting my studies on parallel programming using C and the OpenMPI library. So everything looks a bit too complex right now.

I'm trying to do a Single Program Multiple Data that: Master: - Initialise an array - Subdivide it - Send bits of it of the same size to different processes (Slaves) Slaves: - processes change the array values and calculates the sum of all new elements - Send changed array back to master Master: - Makes a collective communication to collect and sum the sums of the new values - Print the five first elements of each new array received and the global sum of the new values. - Print the time it took.

That's what I wrote:

#include <stdio.h>
#include <stdlib.h>
#include "mpi.h"
#define ARRAYSIZE   16000000

int main (int argc, char *argv[]) {
    MPI_Init(&argc, &argv);
    int myrank;
    char name[100];
    int result;
    int size = 0;
    int number;
    MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
    MPI_Request request;
    int buffer;
    int count;
    double t1, t2;
    MPI_Comm_size (MPI_COMM_WORLD,&size);
    int cells = ARRAYSIZE/size;
    float array[cells];

    if (myrank == 0) {
        t1 = MPI_Wtime();
        t2 = MPI_Wtime();
        MPI_Get_processor_name(name, &result);

        // ********************** INICIALIZANDO ARRAY **********************
        int     i;          /* loop variable */
        float   data[ARRAYSIZE];    /* the intial array */

        printf("Starting serial array example...\n");
        printf("Using array of %d floats. Requires %ld bytes\n",ARRAYSIZE,sizeof(data));

        /* Initialize the array */
        printf("Initializing array...\n");
        for(i=0; i<ARRAYSIZE; i++)
          data[i] =  i * 1.0;

        /* Print a few sample results */
        printf("Sample results\n");
        printf("   data[1]=%e\n",  data[1]);
        printf("   data[100]=%e\n",  data[100]);
        printf("   data[1000]=%e\n",  data[1000]);
        printf("   data[10000]=%e\n",  data[10000]);
        printf("   data[100000]=%e\n",  data[100000]);
        printf("   data[1000000]=%e\n",  data[1000000]);
        printf("\nAll Done!\n");
        // ********************** ARRAY INICIALIZADO **********************

        MPI_Comm_size (MPI_COMM_WORLD,&size);
        printf("Total of tasks: %d", size);
        printf("Each task process %d cells", ARRAYSIZE/size);
        int cells = ARRAYSIZE/size;
        int id_task;
        for(id_task = 0; id_task < size; id_task++) {
            //float array[cells];
            int i=0;
            for(i=0; i<cells; i++)
                array[i] =  i * (id_task+1.0);
            MPI_Send(&array[id_task*cells], cells, MPI_FLOAT, id_task, 0, MPI_COMM_WORLD);
        }

        printf("master: %d at processor: %s\n",myrank, name);

    }
    MPI_Recv(array, cells, MPI_FLOAT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
    int i;
    float resultado;
    for(i=0; i<cells; i++)
        array[i] =  i * (myrank+1.0);

    if(myrank!=0){
        MPI_Send(array, cells, MPI_FLOAT, 0, 0, MPI_COMM_WORLD);
    }

    MPI_Reduce(&array, &resultado, 1, MPI_FLOAT, MPI_SUM, 0, MPI_COMM_WORLD);

    if (myrank == 0) {
        int j;
        for(j=0; j<cells; j++){
            for(i=0; i<5; i++){
                printf("| %lf ",array[i*j]);
            }
        }
        //t1 = MPI_Wtime();
        t2 = MPI_Wtime();
        MPI_Get_processor_name(name, &result);
        printf("master: %d at processor: %s\ntime: %lf\n",myrank, name,t2-t1);
    }
    MPI_Finalize();

}

But all I'm getting is a "Segmentation Fault Error" when I run it. I have read segmentation errors questions but can't diagnose why it's happening to my code.

Thanks in advance. (and I'm sorry about my bad writing, english is not my first language)

UPDATE: I included a malloc and a free but when running I still have "mpirun noticed that process rank 0 with PID 0 on node Brunos-Air exited on signal 11 (Segmentation fault: 11)."

and if I try "$HOME/opt/usr/local/bin/mpicc -o -Wall -Wextra -g programa3-r1 ./programa3-r1.c" it gives: "ld: can't link with a main executable file 'programa3-r1' for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) "

Compile with all warnings & debug info: `gcc -Wall -Wextra -g` then **use the debugger** `gdb` — Basile Starynkevitch, Oct 17 '17 at 13:59
Try to step in your code using a debugger, find the line causing the error and add it to the body of the question. — Tony Tannous, Oct 17 '17 at 13:59
As a side note, you likely want yo `MPI_Scatterv()` the `data` array into `array` — Gilles Gouaillardet, Oct 17 '17 at 15:42

score 2 · Accepted Answer · answered Oct 17 '17 at 14:29

for(id_task = 0; id_task < size; id_task++) {
    //float array[cells];
    int i=0;
    for(i=0; i<cells; i++)
        array[i] =  i * (id_task+1.0);
    MPI_Send(&array[id_task*cells], cells, MPI_FLOAT, id_task, 0, MPI_COMM_WORLD);
}

you should MPI_Send() array (aka &array[0]) instead of &array[id_task*cells]. This is likely the root cause of the crash.

note your program is not correct from an MPI point of view since task 0 MPI_Send() to itself and then MPI_Recv() from itself. That will likely work just fine with small messages, and deadlock with long messages. small and long depend (among other things) on your MPI library, the interconnect used, so you really have to avoid this.

if(myrank!=0){
    MPI_Send(array, cells, MPI_FLOAT, 0, 0, MPI_COMM_WORLD);
}

there is no matching MPI_Recv() on rank 0, so this could also deadlock.

MPI_Reduce(&array, &resultado, 1, MPI_FLOAT, MPI_SUM, 0, MPI_COMM_WORLD);

this sums array[0] from all ranks, and store the result in resultado on rank 0. i am not sure this is really what you have in mind (but i do not know what you are trying to achieve, so that might be ok).

Thank you, sir! It really was wrong from an MPI point of view and I had to implement a MPI_Recv(...) inside a loop to receive data from all tasks. Also corrected the wrong parameter at MPI_Reduce(...) — Shell_Leko, Oct 18 '17 at 18:34

Basile Starynkevitch · Answer 2 · 2017-10-17T17:32:17.940

1

You have very probably a stack overflow:

#define ARRAYSIZE   16000000

and later you declare a local automatic variable (on the call stack)

    float   data[ARRAYSIZE];    /* the intial array */

this is unreasonable. A typical call stack frame should have at most a few kilobytes (since the entire call stack is often limited to a few megabytes or less). You want a 64Mbyte one. You should use C dynamic memory allocation, so declare float *data = NULL; and use at appropriate places calloc or malloc (and much later free; you should avoid memory leaks). Don't forget to check against failure of calloc.

Compile with all warnings & debug info: gcc -Wall -Wextra -g then use the debugger gdb to check that stack overflow happened.

Read documentation about GCC command options. You might want to also use something like -Wstack-usage=2048

edited Oct 17 '17 at 17:32

answered Oct 17 '17 at 14:01

Basile Starynkevitch

223,805
18
296
547

Like that? float *data = NULL; data = (float *) malloc(ARRAYSIZE); – Shell_Leko Oct 17 '17 at 17:29
You'll also need to call (much later) `free` – Basile Starynkevitch Oct 17 '17 at 17:31
mpirun noticed that process rank 0 with PID 0 on node Brunos-Air exited on signal 11 (Segmentation fault: 11). – Shell_Leko Oct 17 '17 at 17:38
and if I try "`$HOME/opt/usr/local/bin/mpicc -o -Wall -Wextra -g programa3-r1 ./programa3-r1.c`" it gives: "_ld: can't link with a main executable file 'programa3-r1' for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation)_ " – Shell_Leko Oct 17 '17 at 17:39
Thank you, sir! It did work when I tried dynamic allocation `float *data; /* the intial array */ data = (float *) malloc (sizeof(float)*ARRAYSIZE);` – Shell_Leko Oct 18 '17 at 18:38

MPI_Reduce() program getting segmentation fault

2 Answers2