get error if i use fread, while no error using read

Question

I'm trying to make some experiments on disk I/O using cache and not using it. In order to perform a read directly from the disk, I open the file with the O_DIRECT flag (defining the variable DISK_DIRECT).

Now the two branches of the if beneath, should perform the same operation, with the difference that one is helped by the cache and the other not. The files to which I try to access are stored on disk and they do not change over time. Also the two branches access to the same files.

At some point here, when I use fread I get ferror() to be true. While when I use read everything goes fine.

I'm sure they access the same files. Do you have any idea why this could happen?

EDIT

Ok, i'm posting here an minimal example. the code i use is:

#include <iostream>
#include <fcntl.h>
#include <sys/types.h>
#include <unistd.h>
#include <fstream>
#include <sstream>


using namespace std;

typedef float fftwf_complex [2] ;

void fetch_level(unsigned long long tid, unsigned short level, fftwf_complex* P_read, fftwf_complex* P_fread, int n_coeff_per_level, FILE** files_o_direct, fstream* & files) {

  int b_read;
  fseek(files_o_direct[level],(long int) (tid * sizeof(fftwf_complex)*n_coeff_per_level), SEEK_SET);


  b_read = fread(reinterpret_cast<char*>(P_fread),sizeof(fftwf_complex), n_coeff_per_level,files_o_direct[level]);

  if(b_read == 0){ 
    cerr << "nothing read\n";

  }

  files[level].seekg((streamoff) (tid * sizeof(fftwf_complex)*n_coeff_per_level), files[level].beg);

  files[level].read(reinterpret_cast<char*>(P_read), 
            sizeof(fftwf_complex) * n_coeff_per_level);

}

void open_files (fstream* & files){

  for(int i=0; i<1;i++) {
    std::ostringstream oss;
    oss << "./Test_fread_read/1.txt.bin";

    files[i].open(oss.str().c_str(),
          std::ios::in | std::ios::out | 
          std::ios::binary | std::ios::ate);
    if (!files[i])
      {
    cerr << "fstream could not open " << oss.str() << endl;
      }
  }
}

void open_files_o_direct (FILE** files_o_direct, int* fd){

  for(unsigned int i=0;i<1; i++){
    std::ostringstream oss;
    oss << "./Test_fread_read/1.txt.bin";
    fd[i]=open(oss.str().c_str(), O_RDONLY | O_DIRECT);
    files_o_direct[i] = fdopen(fd[i], "rb");

    if(!files_o_direct[i])
      cerr << "Could not open " << oss.str() << endl;

  }
}

int close_files(FILE** files_o_direct, int* fd, fstream* & files) {

  for(unsigned int i=0; i<1; i++){
    //#if defined (DISK_DIRECT)
    if(files_o_direct[i])
      close(fd[i]);
    //#else
    if(files[i].is_open())
      files[i].close();
    //#endif
  }

  return 0;
}

int main(){

  FILE**files_o_direct = new FILE* [256];
  fstream* files = new fstream [256];
  int * fd = new int [256];


  fftwf_complex * P_read =  new fftwf_complex [1];
  fftwf_complex * P_fread =  new fftwf_complex [1];

  open_files_o_direct(files_o_direct, fd);
  open_files(files);

fetch_level(2, 0, P_read, P_fread, 1, files_o_direct, files);
  cout << "P_read: " << P_read[0][0] << " P_fread: " << P_fread[0][0] << endl;
  cout << "P_read: " << P_read[0][1] << " P_fread: " << P_fread[0][1] << endl;
fetch_level(7, 0, P_read, P_fread, 1, files_o_direct, files);
  cout << "P_read: " << P_read[0][0] << " P_fread: " << P_fread[0][0] << endl;
  cout << "P_read: " << P_read[0][1] << " P_fread: " << P_fread[0][1] << endl;
fetch_level(8, 0, P_read, P_fread, 1, files_o_direct, files);
  cout << "P_read: " << P_read[0][0] << " P_fread: " << P_fread[0][0] << endl;
  cout << "P_read: " << P_read[0][1] << " P_fread: " << P_fread[0][1] << endl;


  close_files(files_o_direct, fd, files);

  delete [] P_read;
  delete [] P_fread;
  delete [] files;
  delete [] files_o_direct;

  return 0;
}

and the file which is accessed is:

0.133919 0.0458176 
1.67441 2.40805 
0.997525 -0.279977 
-2.39672 -3.076 
-0.0390913 0.854464 
-0.0176478 -1.3142 
-0.667981 -0.486272 
0.831051 0.282802 
-0.638032 -0.630943 
-0.669854 -1.49762

which is stored in a binary format and that can be download from here: 1.txt.bin.

The output i get is:

nothing read
P_read: 0.997525 P_fread: 0
P_read: -0.279977 P_fread: 0
nothing read
P_read: 0.831051 P_fread: 0
P_read: 0.282802 P_fread: 0
nothing read
P_read: -0.638032 P_fread: 0
P_read: -0.630943 P_fread: 0

The problem persists even if i change the type of fftwf_complex from float[2] to simple float. If i remove the fseek line everything works correctly.

An [**MCVE**](https://stackoverflow.com/help/mcve) would go a long way in demonstrating the issue you're having, and given your claim `fread` vs. `std::istream::read` is the only difference, trivial to provide. The posted code is no only uncompilable, it isn't scope-balanced. The `fread` call contains both `fseek` and `fread` prior to the `}`. The `std::istream::seekg` and `std::istream::read` have a `}` wedged between. Clearly you're leaving out *something*. — WhozCraig, Dec 20 '14 at 14:22
i tried to use explain_fread but i've got an "undefined reference" error, i don't know if it is because you need to link the library, but i haven't found any information about that. is there another way in order to get the error number? — David, Dec 20 '14 at 14:23
@Sergio _"i tried to use explain_fread ..."_ You don't even mention this in your question. Did you check this Q&A how to solve that problem: http://stackoverflow.com/questions/12573816/what-is-an-undefined-reference-unresolved-external-symbol-error-and-how-do-i-fix — πάντα ῥεῖ, Dec 20 '14 at 14:27
@Sergio Thanks for the update. I'd start with adjusting the target of anything you're using O_DIRECT with to be properly aligned. [See here](http://stackoverflow.com/a/6001315/1322972) for why, and [here as well](http://man7.org/linux/man-pages/man2/open.2.html). Without `O_DIRECT`, your MCVE compiles and runs correctly (I believe), and exhibits [**this as the final output**](http://pastebin.com/bAS5SiRB) on my OSX 10.9.5 Mac x64 — WhozCraig, Dec 21 '14 at 14:36

Iharob Al Asimi · Answer 1 · 2014-12-20T14:39:00.150

0

This if (b_read == 0), will be true at the end of the file, and you will enter this branch

if(ferror(this->files_o_direct[level]))
    fseek(this->files_o_direct[level], 0, SEEK_END); //ftell here returns 4800000 
cerr << "nothing read\n";

even if ferror returns 0, the end of the file was reached anyway

fseek(this->files_o_direct[level], 0, SEEK_END);

makes no sense, and "nothing read\n" will be output either or not ferror returns nonzero.

From the manual page

fread() does not distinguish between end-of-file and error, and callers must use feof(3) and ferror(3) to determine which occurred.

so you have to check feof and if it is false you use ferror.

edited Dec 20 '14 at 14:39

answered Dec 20 '14 at 14:17

Iharob Al Asimi

52,653
6
59
97

i added that line of code to check the file size while debugging, and be sure that fseek wasn't at the end of the file – David Dec 20 '14 at 14:35
How are you using it for that purpose? For that you must check `feof` rather than `ferror`. – Iharob Al Asimi Dec 20 '14 at 14:36
there is another fseek call earlier in the code, i check ftell there (as written in the comments) and then i check ftell on the second fseek call. and i can tell that fseek is not at the end of the file – David Dec 20 '14 at 14:41
`ftell` doesn't tell you if you are at the end of the file `feof` does, also can you post those lines please? – Iharob Al Asimi Dec 20 '14 at 14:42
ftell tells you how many bytes there are from the beginning of the file, therefore if you go at the end of the file with fseek and check the value of ftell you'll get the size of the file. If prior to this you've checked at which position you were, you'll be able to say if you were or not at the end of the file – David Dec 20 '14 at 14:45
@Sergio, not the way I would do it but true, so can you please post the lines you use to check `ftell`'s return value. – Iharob Al Asimi Dec 20 '14 at 14:47

score 0 · Accepted Answer · answered Dec 23 '14 at 11:30

For who ever may have the same problem here there is the answer:

The O_DIRECT flag may impose alignment restrictions on the length and address of user-space buffers and the file offset of I/Os. In Linux alignment restrictions vary by filesystem and kernel version and might be absent entirely. However there is currently no filesystem-independent interface for an application to discover these restrictions for a given file or filesystem. Some filesystems provide their own interfaces for doing so, for example the XFS_IOC_DIOINFO operation in xfsctl(3).
  Under Linux 2.4, transfer sizes, and the alignment of the user buffer
  and the file offset must all be multiples of the logical block size
  of the filesystem.  Since Linux 2.6.0, alignment to the logical block
  size of the underlying storage (typically 512 bytes) suffices.  The
  logical block size can be determined using the ioctl(2) BLKSSZGET
  operation or from the shell using the command:

      blockdev --getss 

linux reference page

get error if i use fread, while no error using read

2 Answers2