Working with big float data in C

Question

I want to do some calculation with some matrices whose size is 2048*2048, for example.But the simulator stop working and it does not simulate the code. I understood that the problem is about the size and type of variable. For example, I run a simple code, which is written below, to check whether I am right or not. I should print 1 after declaring variable A. But it does not work.

Please note that I use Codeblocks. WFM is a function to write a float matrix in a text file and it works properly because I check that before with other matrices.

int main()
{
    float A[2048][2048];
    printf("1");

    float *AP = &(A[0][0]);
    const char *File_Name = "example.txt";
    int counter = 0;
    for(int i = 0; i < 2048; i++)
        for(int j = 0; j < 2048; j++)
        {
            A[i][j] = counter;
            ++counter;
        }
    WFM(AP, 2048, 2048, File_Name , ' ');

    return 0;
}

Any help and suggestion to deal with this problem and larger matrices is appreciate it. Thanks

why do you declare such a useless AP variable just for passing to a function like that? Also if you need to increase performance you need to study about optimiations — phuclv, Mar 07 '14 at 14:22
thank you so much. May I ask you to give me some tutorial about Optimiations? — user42037, Mar 07 '14 at 16:39

score 2 · Answer 1 · answered Mar 07 '14 at 09:32

2

float A[2048][2048];

which requires approx. 2K * 2K * 8 = 32M of stack memory. But typically the stack size of the process if far less than that. Please allocate it dynamically using alloc family.

answered Mar 07 '14 at 09:32

Sakthi Kumar

3,047
15
28

4

Or declare it as `static float A[2048][2048]` which in your case works also. – Jabberwocky Mar 07 '14 at 09:36
1

float size is most probably 4 bytes, not 8. But 16MB is still too large for a normal stack size – phuclv Mar 07 '14 at 14:14

score 0 · Answer 2 · edited May 23 '17 at 11:49

0

float A[2048][2048];

This may be too large for a local array, you should allocate memory dynamically by function such as malloc. For example, you could do this:

float *A = malloc(2048*2048*sizeof(float));
if (A == 0)
{
    perror("malloc");
    exit(1);
}

float *AP = A;
int counter = 0;
for(int i = 0; i < 2048; i++)
    for(int j = 0; j < 2048; j++)
    {
        *(A + 2048*i + j) = counter;
        ++counter;
    }

And when you does not need A anymore, you can free it by free(A);.

Helpful links about efficiency pitfalls of large arrays with power-of-2 size (offered by @LưuVĩnhPhúc):

Why is transposing a matrix of 512x512 much slower than transposing a matrix of 513x513?
Why is my program slow when looping over exactly 8192 elements?
Matrix multiplication: Small difference in matrix size, large difference in timings

edited May 23 '17 at 11:49

Community

1
1

answered Mar 07 '14 at 09:36

Lee Duhem

14,695
3
29
47

Thank you so much. The malloc works properly but imagine that I have 4 or 5 matrices like that and have to do some calculations with them. I mean it take a long time, let say 30 min, to process. However, Matlab software do the same calculation within 3 min. – user42037 Mar 07 '14 at 10:03
If you want similar performance as matlab, use the LAPACK matrix algorithms. One example on how to call the FORTRAN procedures I assembled in http://stackoverflow.com/a/21974753/3088138 (this is C++, but the procedures inside the namespace lapack should work in C too, after removing the extern C classification) – Lutz Lehmann Mar 07 '14 at 12:08
@user42037 Can you post your full code, if that is possible, and some input data? So we can test it, and may be able to find out why it is so slow. – Lee Duhem Mar 07 '14 at 12:08
Avoid large arrays that have size equal to a power of 2. Pad them to 2049 instead https://stackoverflow.com/questions/11413855/why-is-transposing-a-matrix-of-512x512-much-slower-than-transposing-a-matrix-of https://stackoverflow.com/questions/12264970/why-is-my-program-slow-when-looping-over-exactly-8192-elements?lq=1 https://stackoverflow.com/questions/7905760/matrix-multiplication-small-difference-in-matrix-size-large-difference-in-timi?lq=1 – phuclv Mar 07 '14 at 14:19

Working with big float data in C

2 Answers2