Size of each record in a binary file

Question

I have a binary file and I will be using fread to read the data from this binary file into an array of structures.

However, I don't know what value to pass to fread as its second argument. I know the file size is 536870912 bits. The binary file was constructed on the basis of being accessed for a 512^3 array. This means each data entry is of type float in the binary file with 4 bytes specified for each data element.

I made an error with the mention of bits. I read what was outputted by a C program finding the size of the file - it outputted 536870912 bits! Apologies to anyone confused.

Here is the code i'm using to read the data from the binary file into my arrary of structures (a simplified structure - there are 10 other parameters!)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>

// Define the model structure
struct model {
        float density;
};


// Entry point for the program
int main () {
    int counter;
    long lSize;
    char * buffer;
    size_t result;
    FILE *pFile;
    int i,j,k,ibox;         /* Loop indices for the physical grid */

    struct model ***mymodel;

    pFile = fopen("core1_dens_0107.bin","rb");
    if (pFile == NULL) { printf("Unable to open density file!"); exit(1); }

    // obtain file size:
    fseek (pFile , 0 , SEEK_END);
    lSize = ftell (pFile);
    rewind (pFile);

    printf( "File size : %lu Bits \n", lSize );

    for ( j = 0 ; j < 512 ; j++ ) {
        for ( k = 0; k < 512; k++ ) {
            for ( i = 0; i < 512; i++ ) {
          fread(&mymodel[i][j][k].density,4,1,pFile);
              printf("%f \n",mymodel[i][j][k].density);
            }
        }
    }               

    fclose(pFile);
    return 0;
}

Answer 1

Supposing you have already opened the file and you have your file descriptor myStream, it should be as simple as this:

#define MY_DIM = 512; ///Maybe you want to play safe and make it a little bit larger? Up to you

float buffer[MY_DIM][MY_DIM][MY_DIM];

size_t readBytes;

int i,j,k;
for (k = 0; k < MY_DIM; k++)
  for (j = 0; j < MY_DIM; j++) {
      readBytes = fread((void*) (buffer[k][j]), sizeof float, MY_DIM, myStream); //I am not sure the (void*) conversion is necessary
      if (readBytes < MY_DIM) //I unexpectedly reached the end of the file,
        goto endOfTheLoop;    //without reading all the data I needed for int
                              //You could also print a warning message
      }      

endOfTheLoop:

//Now close the input file, use fclose or something

//Now that you have read all the data, you have to put it in your array of struct:
for (k = 0; k < MY_DIM; k++)
  for (j = 0; j < MY_DIM; j++)
    for (i = 0; i < MY_DIM; i++)
      mymodel[k][j][i].density = buffer[k][j][i]; 

Answer 2

You can pass whatever value of the 2nd argument is most convenient for your program. If you want to process the file one structure at a time, do:

nread = fread(&your_struct, 1, sizeof yourstruct, stream);

If you have an array of structures, e.g.

struct foo your_struct[STRUCT_COUNT];

you can do:

nread = fread(your_struct, STRUCT_COUNT, sizeof *your_struct, stream);

Answer 3

size_t fread(void *ptr, size_t size, size_t nmemb, FILE * stream );

will attempt to read nmemb blocks of size bytes each. It will guarantee that no partial blocks are read. If your blocks are 4 bit long then I suggest you read them byte by byte, otherwise use the size argument to specify the block size.

For instance

fread(buffer, 1, 1024, stdin);

will attempt to read 1024 bytes, but may stop at any point.

fread(buffer, 4, 256, stdin);

will attempt to also read 1024 bytes, but in blocks of 4 bytes. 256 blocks total. It will guarantee that no partial blocks are read.

fread(buffer, 1024, 1, stdin);

will attempt to read one block of 1024 bytes. If it can not - nothing will be read.

If you wish to read in the entire file then you can do it in blocks of 4 via:

size_t read, read_now;
while (read < filesize && (read_now= fread(buffer +read, 4, (filesize - read) >> 2, in)) != EOF)
    read += read_now;

of you can attempt to read the whole thing in one go:

fread(buffer, filesize, 1, in);