Strange results with reading binary files in C

Question

I'm working on 64-bit Xubuntu 14.04.

I have a fairly large C program and to test new features, I usually implement them in a separate program to iron out any bugs and whatnot before incorporating them into the main program.

I have a function that takes a const char* as argument, to indicate the path of a file (/dev/rx.bin in this case). It opens the file, reads a specific number of bytes into an array and then does some things before exporting the new data to a different file.

First off I allocate the array:

int16_t samples = (int16_t *)calloc(rx_length, 2 * sizeof(samples[0]));

Note that rx_length is for example 100 samples (closer to 100 000 in the actual program), and it's calculated from the same constants.

Next I open the file and read from it:

uint32_t num_samples_read;
FILE *in_file   = fopen(file, "rb");
if (in_file == NULL){
    ferror(in_file);
    return 1;
}
num_samples_read =  fread(samples, 2 * sizeof(samples[0]), rx_length, in_file);

Here's the kicker; the return value from fread is not the same between the test program and the main program, while the code is identical. For example, when I should be reading 100 000 samples from a 400 kB file (100 000 samples, one int16_t for the real part and one int16_t for the imaginary part, adds up to four bytes per sample), the value returned is 99328 in the main program. For the life of me I cannot figure out why.

I've tested the output of every single variable used in any calculation, and up until fread() everything is identical.

I should also note that the function is in a separate header in my main program, but I figured that since printing every constant / definition gives the expected result, that it's not there where I'm making a mistake.

If there's anything that I might have missed, any input would be greatly appreciated.

Regards.

I assume that samples is a pointer (i.e. The code in the question is a copy+paste error? — Dave, Sep 25 '14 at 20:04
Unrelated: `ferror(in_file)` is worthless. I think you meant `perror(file);` — WhozCraig, Sep 25 '14 at 20:04
There are a number of possibilities; if your big app is threaded, are you sure it isn't clobbering data from another thread? Do you get any compiler warnings when you compile? Have you included any libraries that might have redefined the functions you're using? — Dave, Sep 25 '14 at 20:06
`int16_t samples = ...`, Um... that better be `int16_t *samples` (note the asterisk). And [stop casting memory allocation functions in C](http://stackoverflow.com/questions/605845/do-i-cast-the-result-of-malloc). — WhozCraig, Sep 25 '14 at 20:07
Yes my mistake that's a copy paste error, should be `int16_t *samples = ...`. — user2742907, Sep 25 '14 at 20:08
Can you dig out some more debug like size of file (`fstat`) file position before and after read (`ftell`) — Steen, Sep 25 '14 at 20:11
Given only what you have here and assuming you fixed the typos in the post and they aren't in your real code, you're assessment that it makes little sense is accurate. Is it possible the file is still open for write and not yet flushed by another process (even this one)? Its a difference of about 2.5KB, which would seem a reasonable file stream buffer page size that went missing. — WhozCraig, Sep 25 '14 at 20:11
The large app is threaded yes, but the threads are terminated before continuing to the processing part. The only compiler warnings I see (specific to that function) are printf related. — user2742907, Sep 25 '14 at 20:13
@user2742907 is it possible the thread that wrote this file never properly flushed-and-closed it? That likewise would account for a missing last-stream-buffer of data. — WhozCraig, Sep 25 '14 at 20:14
@WhozCraig Wow, yes, that's what was wrong. The way I implemented closing the rx file caused it to never actually get closed unless there was an error accessing it. Adding fclose() at the end of my rx thread solved it. Can always count on an outside perspective. Thanks! — user2742907, Sep 25 '14 at 20:32
@user2742907 no worries. glad you found it. little things we take for granted in C++ =P. And as a bonus you fixed both your problem and a leaked `FILE*` stream resource. Woot. Best of luck. — WhozCraig, Sep 25 '14 at 20:37
OP, suggest posting your own answer and accept it to close this. Just like its important to `fclose()` files, important to close SO posts. — chux - Reinstate Monica, Sep 26 '14 at 03:24

score 0 · Answer 1 · answered Sep 26 '14 at 16:48

Thank you chux for reminding me to close and answer.

Closing the file was the problem in my main program, it never occurred within the test environment because the input file was not being modified there.

Once the RX thread has completed its task, make a call to fclose():

rx_task_out:
    fclose(p->out_file);
    // close device
    // free sample buffer
    return NULL;

Previously, only an error status with creating the RX thread caused it to close the file.

Strange results with reading binary files in C

1 Answers1