Segmentation fault(core dumped ) error while reading a python generated binary array in C

Question

I am trying to load a 2D array created by numpy and read the elements in C, but I get Segmentation fault(core dumped ) error while running it. The code goes by the lines of


#include <stdio.h>
#include <string.h>
int main(){
    char *file;
    FILE *input;
    int N1, N2, ii, jj;
    float element;
    strcpy(file, "/home/caesar/Desktop/test.bin");
    input = fopen(file, "rb");
    fread(&N1, sizeof(int), 1, input);
    fread(&N2, sizeof(int), 1, input);
     float memoryarray[N1][N2];
    for(ii= 0; ii<N1; ii++){
        for(jj=0; jj<N2; jj++){
            fread(&element, sizeof(float), 1, input);
            memoryarray[ii][jj]= element; 
        }
    }
    printf("%f", memoryarray[2][3]);
    fclose(input);
    return 0;
}

This is the starting for a task where I will have to read elements from matrices of the form 400*400*400 or so. The idea is to read all elements from the file and store it in memory and then read from memory index wise when necessary, for example, here i am trying to access and print the element in the second row third column.

P.S: I am quite new to pointers.

Dear all, I tried the methods you said., here is the modified version of the code, the segmentation fault error is gone but the output is either all zeros, or is just plain garbage values. I ran the executable three times and the outputs I got were Output1: -0.000000 Output 2: 0.000000 Output 3 : -97341413674450944.000000

My array contains integers btw

Here is the modified version of the code

#include <stdio.h>
#include <string.h>
void main(){
    const char file[] ="/home/caesar/Desktop/test.bin";
    FILE *input;
    int N1, N2, ii, jj;
    float element;
    //strcpy(file, "/home/caesar/Desktop/test.bin");
    input = fopen(file, "r");
    fread(&N1, sizeof(int), 1, input);
    fread(&N2, sizeof(int), 1, input);
     float memoryarray[N1][N2];
    for(ii= 0; ii<N1; ii++){
        for(jj=0; jj<N2; jj++){
            fread(&element, sizeof(float), 1, input);
            memoryarray[ii][jj]= element; 
        }
    }
    printf("%f", memoryarray[1][2]);
    fclose(input);

Also here is the hex dump of the file that i am trying to open. Some of you asked me to verify whether fopen() is working or not, i checked, it is working.

00000000  00 00 40 40 00 00 40 40  01 00 00 00 00 00 00 00  |..@@..@@........|
00000010  02 00 00 00 00 00 00 00  03 00 00 00 00 00 00 00  |................|
*
00000030  04 00 00 00 00 00 00 00  04 00 00 00 00 00 00 00  |................|
00000040  05 00 00 00 00 00 00 00  06 00 00 00 00 00 00 00  |................|
00000050

So here is my problem in brief. I have multidimensional arrays of double precision floats written to a file using python. I want to take those files and access the elements whenever necessary by using the index of the elements to get the values. Any C code to do so would solve my problem.

Here is the python code i am using to write the file

with open('/home/caesar/Desktop/test.bin', 'wb') as myfile:
    N= np.zeros(2, dtype= np.float32, order= "C")
    N[0]= 3
    N[1]= 3
    a= [[1,2,3],[2,3,4], [4,5,6]]
    N.astype(np.float32).tofile(myfile)
    b= np.asarray(a)
    b.tofile(myfile)

You have a pointer `file`, but when you use `strcpy` with that pointer as the destination ***where does it point?*** Remember that uninitialized local variables will have an *indeterminate* (seemingly random or garbage) value, they truly are uninitialized. Use an array, or use the string literal directly in the call to `fopen`. — Some programmer dude, Dec 16 '19 at 13:06
`char *file;`-> `char file[] = "/home/caesar/Desktop/test.bin";`. And remove the `strcpy`. Alco check if `fopen` fails. — Jabberwocky, Dec 16 '19 at 13:09
... or `char *file;` -> `char file[100];`, where 100 is the maximal authorized length of a filename, or even `input = fopen("/home/caesar/Desktop/test.bin", "rb");` — Jabberwocky, Dec 16 '19 at 13:15
`float memoryarray[N1][N2];` places the array to stack, which is small, size measured in megabytes. This may or may not be a problem for you here. In general, VLAs on stack with size given as external input are an excellent opportunity for getting a stack overflow. — hyde, Dec 16 '19 at 13:21
@user12170328 Regarding the second part of your question, you seem be trying to read ints from the file, into float variables. This will not work. Either use int for your array elements or modify your file to store the floats directly in the proper floating-point representation. I would suggest the former approach because the writing floating point numbers to a file is not very easy to do manually. Even better would be to read and write data using text representation instead of binary. — th33lf, Dec 16 '19 at 15:39
The values in the file are floating point numbers of type float32 generated by python. — noobprogrammer, Dec 16 '19 at 16:46

th33lf · Accepted Answer · 2019-12-16T13:46:34.347

4

strcpy(file, "/home/caesar/Desktop/test.bin");

This writes to a garbage memory address. You should either declare file as an array of suitable size, like this:

char file[100];

or

initialize the char pointer directly with the path like this (and get rid of the strcpy):

const char *file = "/home/caesar/Desktop/test.bin";

or the best, as per common consensus (refer comments):

fopen("/home/caesar/Desktop/test.bin", "rb");

edited Dec 16 '19 at 13:46

answered Dec 16 '19 at 13:11

th33lf

2,177
11
15

`char []file = "/home/caesar/Desktop/test.bin";` is better than `char *file = "/home/caesar/Desktop/test.bin";` – Jabberwocky Dec 16 '19 at 13:15
@Jabberwocky How so? – sepp2k Dec 16 '19 at 13:16
1

Because one makes a local mutable array initialized with the specified string, and the other makes a non-const pointer to a possibly-immutable string literal. – Useless Dec 16 '19 at 13:18
@sepp2k because then `file` cannot be modified (e.g. `file++`). Also it should be `const char *file = "/home/caesar/Desktop/test.bin";` – Jabberwocky Dec 16 '19 at 13:18
1

@Jabberwocky I guess `const char*` makes sense, because this is a file path string literal that would be loaded somewhere in memory already, it doesn't make sense to copy it into a local array, if you don't intend to modify it. Updated the answer. – th33lf Dec 16 '19 at 13:22
@th33lf actually the best would be `static const char file[] = "/home/caesar/Desktop/test.bin";`. And in my first comment it should be `char file[]` instead of `char []file` – Jabberwocky Dec 16 '19 at 13:24
@Jabberwocky `file++` is valid with `const char *`, because only the pointer target is declared constant here, not the pointer itself (that would be `const char * const`). Apart from that, I don't see why the path should be mutable in this case. – Ctx Dec 16 '19 at 13:33
1

@Jabberwocky That would still involve an unnecessary copy of the string literal to stack memory. See the difference in generated code between the two approaches: https://godbolt.org/z/SG8anT – th33lf Dec 16 '19 at 13:34
@th33lf read my last comment closely. `static const char file[] ` is the way to go. https://godbolt.org/z/BVg5bA – Jabberwocky Dec 16 '19 at 13:38
@ctx yes, therefore the best is `static const char file[] = "..."`. And yes the path should not be mutable, I thought I made that clear. – Jabberwocky Dec 16 '19 at 13:40
@Jabberwocky With the information I have, the best way is: `input = fopen("/home/caesar/Desktop/test.bin", "rb");` and remove all the other clutter ;) Edit: Ah, I see, you wrote this in a comment above, too. Agreed – Ctx Dec 16 '19 at 13:42
@Ctx yes, I mentioned this too, but my last solution is valid if there are say several different filenames you want to declare all in the same place. But of course _here_ `fopen("/home/caesar/Desktop/test.bin", "rb");` is the best solution. – Jabberwocky Dec 16 '19 at 13:43
@user12170328 You seem to be reading floats, but the data in the file doesn't look like floats to me. How did you generate this file? Are you trying to read floats or ints? – th33lf Dec 16 '19 at 15:18
The values in the file are floating point numbers of type float32 generated by python – noobprogrammer Dec 16 '19 at 16:47
Turns out the main problem was that python generates floats which are doubles for C. Changing the float to double solved the problem. – noobprogrammer Dec 31 '19 at 09:35

Segmentation fault(core dumped ) error while reading a python generated binary array in C

1 Answers1