1

I am currently working on modeling a diffusion process through a drug patch, which involves solving a PDE using numerical methods. My current issue is trying to dynamically allocate memory to a multidimensional array in C.

The array I am trying to allocate memory to is:

double marray[1000][1000][1000];

Also would it be possible to dynamically allocate memory to an array with another dimension? I am writing code on a 64 bit system, however, I am using a shell (Bitvise SSH client) to compile the code so that I may run it on a supercomputer(BlueShark?).

double marray[1000][1000][1000][1000];

I was given a hint at using MPI for this task?

Thank you for your interest and help!

Gilles Gouaillardet
  • 8,193
  • 11
  • 24
  • 30
Tim Broslav
  • 11
  • 5
  • 1) Post the code that does "The array I am trying to allocate memory to is:" 2) Try something smaller than 1000 first, like 10. – chux - Reinstate Monica Apr 17 '18 at 03:07
  • 1
    To allocate memory for `double marray[1000][1000][1000];` is easy: `double (*a)[1000][1000][1000] = malloc (sizeof *a);` - although your machine may lack the resources and a wide enough `size_t`. – chux - Reinstate Monica Apr 17 '18 at 03:10
  • Hey Chux thanks for the reply, the only practice I ahd with dynamic memory allocation was for allocating memory to a matrix(2-D), hence I would not know where to start with a (3-D) or (4-D). Maybe something with pointers? – Tim Broslav Apr 17 '18 at 03:16
  • @TimBroslav: A supercomputer is typically just a large number of small machines on a fast network. You need to find a way to spread your array across multiple machines - e.g. with 1000 machines you'd have about 8 GiB per machine, and wouldn't have the whole 8 TiB of data on any one machine. C does not do this for you, you need some kind of library (e.g. maybe "Open MPI"). – Brendan Apr 17 '18 at 03:26
  • OPM is just an extension of C, which is what I am using, I should've mentioned that, sorry. – Tim Broslav Apr 17 '18 at 03:36
  • You'll need 8 TB of (virtual) memory to allocate that much space. Most machines do not have that much space — the allocation will fail. – Jonathan Leffler Apr 17 '18 at 03:43
  • That is for the 4-D array? – Tim Broslav Apr 17 '18 at 03:48
  • @TimBroslav: In that case it you should mention MPI in the title and include the "mpi" tag. Also you find some useful information here: https://stackoverflow.com/questions/25628321/dynamic-memory-allocation-in-mpi – Brendan Apr 17 '18 at 03:49
  • @TimBroslav: For the 4-D array, it adds up to about 8 TiB of data. As far as I can tell, the BlueShark supercomputer in Florida (if that's the one you mentioned) only has 4 TiB of total RAM (spread across all the nodes). – Brendan Apr 17 '18 at 03:51
  • Oh wow, I should've realized this. For some reason I thought it would be 8 Gb of RAM haha. But looks like I will have to go with a valid assumption in order to go with a 3-D array. – Tim Broslav Apr 17 '18 at 04:03
  • @TimBroslav Perhaps step back. Does code really need to use an _array_ of data? Hope about a 1000*1000 array of pointers to sub-arrays - some of which might be `NULL` as that sub-array is not used. In other words, only allocate as needed. – chux - Reinstate Monica Apr 17 '18 at 13:18
  • @TimBroslav Yet another idea: `double *marray[1000][1000] = malloc(sizeof *marray * 1000);` Many allocators don't allocate true memory until that memory is accessed. See [Why is malloc not “using up” the memory on my computer?](https://stackoverflow.com/q/19991623/2410359) If your code does not access everywhere, this may work for you. Both this and the prior idea depend of code needing a _sparse_ array and not a full one. What is the coding goal? – chux - Reinstate Monica Apr 17 '18 at 13:22
  • @chux, the code is to track the concentration in a drug patch over a time interval t. With a few assumptions (such as an even mixed concentration) I will only need to track concentration at each height(z) and at each time interval of each height. Another problem I realized I might run into is of course communicating using pointers. – Tim Broslav Apr 19 '18 at 02:25

1 Answers1

0

There are several ways to make multidimensional arrays. I give you some examples: 1: let l, m, n, o,.. be your size in different dimensions.

double ****MyArr(int l, int m, int n, int o) {
    double ****array = new double ***[l];
    for (int i = 0; i<l; i++) {
        array[i] = new double **[m];
        for (int j = 0; j<m; j++) {
            array[i][j] = new double *[n];
            for (int k = 0; k < n; k++)
            {
                array[i][j][k] = new double [o];
            }

        }
    }
    return array;
}

2:

create a 1D pointer instead

double *myArr new double [l*m*n*o];
//access to [a][b][c][d]
myArr[a*m*n*o + b*n*o + c*o + d]=0.0;

some remarks:

It seems you are dealing with a big matrix, make sure are using a contiguous array.

Make sure to use collective communication as much as possible if you need to communicate between processes.

All processes will make similar size array, most likely each process will use only a chunk of this array, the rest will occupy your memory for nothing. Make sure using the right size depending on the number of process.

metaTemp
  • 1
  • 1