Using fread() to read a text based file - best practices

Question

Consider this code to read a text based file. This sort of fread() usage was briefly touched upon in the excellent book C Programming: A Modern Approach by K.N. King. There are other methods of reading text based files, but here I am concerned with fread() only.

#include <stdio.h>
#include <stdlib.h>

int main(void)
{
    // Declare file stream pointer.
    FILE *fp = fopen("Note.txt", "r");
    // fopen() call successful.
    if(fp != NULL)
    {
        // Navigate through to end of the file.
        fseek(fp, 0, SEEK_END);
        // Calculate the total bytes navigated.
        long filesize = ftell(fp);
        // Navigate to the beginning of the file so
        // it can be read.
        rewind(fp);
        // Declare array of char with appropriate size.
        char content[filesize + 1];
        // Set last char of array to contain NULL char.
        content[filesize] = '\0';
        // Read the file content.
        fread(content, filesize, 1, fp);
        // Close file stream pointer.
        fclose(fp);
        // Print file content.
        printf("%s\n", content);
    }
    // fopen() call unsuccessful.
    else
    {
        printf("File could not be read.\n");
    }
    return 0;
}

There are some problems I have with this method. My opinion is that this is not a safe method of performing fread() since there might be an overflow if we try to read an extremely large string. Is this opinion valid?

To circumvent this issue, we may use a buffer size and keep on reading into a char array of that size. If filesize is less than buffer size, then we simply perform fread() once as described in the above code. Otherwise, We divide the total file size by the buffer size and get a result, whose int portion we will use as the total number of times to iterate a loop where we will invoke fread() each time, appending the read buffer array into a larger string. Now, for the final fread(), which we will perform after the loop, we will have to read exactly (filesize % buffersize) bytes of data into an array of that size and finally append this array into the larger string (Which we would have malloc-ed with filesize + 1 beforehand). I find that if we perform fread() for the last chunk of data using buffersize as its second parameter, then extra garbage data of size (buffersize - chunksize) will be read in and the data might become corrupted. Are my assumptions here correct? Please explain if/ how I have overlooked something.

Also, there is the issue that non-ASCII characters might not have size of 1 byte. In that case I would assume the proper amount is being read, but each byte is being read as a single char, so the text is distorted somehow? How is fread() handling reading of multi-byte chars?

Answer 1

this is not a safe method of performing fread() since there might be an overflow if we try to read an extremely large string. Is this opinion valid?

fread() does not care about strings (null character terminated arrays). It reads data as if it was in multiples of unsigned char^*1 with no special concern to the data content if the stream opened in binary mode and perhaps some data processing (e.g. end-of-line, byte-order-mark) in text mode.

Are my assumptions here correct?

Failed assumptions:

• Assuming ftell() return value equals the sum of fread() bytes. The assumption can be false in text mode (as OP opened the file) and fseek() to the end is technical undefined behavior in binary mode.

• Assuming not checking the return value of fread() is OK. Use the return value of fread() to know if an error occurred, end-of-file and how many multiples of bytes were read.

• Assuming error checking is not required. , ftell(), fread(), fseek() instead of rewind() all deserve error checks. In particular, ftell() readily fails on streams that have no certain end.

• Assuming no null characters are read. A text file is not certainly made into one string by reading all and appending a null character. Robust code detects and/or copes with embedded null characters.

• Multi-byte: assuming input meets the encoding requirements. Example: robust code detects (and rejects) invalid UTF8 sequences - perhaps after reading the entire file.

• Extreme: Assuming a file length <= LONG_MAX, the max value returned from ftell(). Files may be larger.

but each byte is being read as a single char, so the text is distorted somehow? How is fread() handling reading of multi-byte chars?

fread() does not function on multi-byte boundaries, only multiples of unsigned char. A given fread() may end with a portion of a multi-byte and the next fread() will continue from mid-multi-byte.

---

Instead of of 2 pass approach consider 1 single pass

// Pseudo code
total_read = 0      
Allocate buffer, say 4096

forever
  if buffer full
    double buffer_size (`realloc()`)
  u = unused portion of buffer 
  fread u bytes into unused portion of buffer
  total_read += number_just_read
  if (number_just_read < u) 
    quit loop

Resize buffer total_read (+ 1 if appending a '\0')

Alternatively consider the need to read the entire file in before processing the data. I do not know the higher level goal, but often processing data as it arrives makes for less resource impact and faster throughput.

---

Advanced

Text files may be simple ASCII only, 8-bit code page defined, one of various UTF encodings (byte-order-mark, etc. The last line may or may not end with a '\n'. Robust text processing beyond simple ASCII is non-trivial.

ASCII and UTF-8 are the most common. IMO, handle 1 or both of those and error out on anything that does not meet their requirements.

---

^*1 fread() reads in multiple of bytes as per the 3rd argument, which is 1 in OP's case.

//                       v --- multiple of 1 byte
fread(content, filesize, 1, fp);