Incorrect size of file found using Visual Studio C++

Question

I am porting over c++ code from linux to windows. I am currently using Visual Studio 2013 to port my code.

I need to read a binary file and am using this portion of c++ code:

// Open the stream
std::ifstream is("myfile.bin");
// Determine the file length
is.seekg(0, std::ios_base::end);
std::size_t size=is.tellg();
is.seekg(0, std::ios_base::begin);
// Create a vector to store the data
int* Data = new int[size/sizeof(int)];
// Load the data
is.read((char*) &Data[0], size);
// Close the file
is.close();

In linux, the size of my binary file is correctly found to be 744mb. However, in windows, the size of my binary file is incorrectly found to be >4GB. How can I correct this issue?

Answer 1

Change std::ifstream is("myfile.bin"); to std::ifstream is("myfile.bin", std::ios::binary);

With your current default open mode, the compiler choses "char" mode. In Linux chars in files are UTF8, first 128 positions are 1-byte char. But for memory UTF32, 4-bytes per char, is used. In Windows chars are "wide-chars", 2-bytes per char.

Answer 2

I finally had the time to actually run this myself, though I had to fix a couple of things, like ios_base::beg instead of begin (different function) Also, as mentioned, the array allocation should be this int* Data = new int[size / sizeof(int) + 1]; // At most one extra int

I found your problem: you're not in the right directory. Check if you successfully opened the file or not. If you don't, then you get a huge garbage value (probably -1, but unsigned, so massive) for size.

Try this to find your directory in Windows: (probably need Windows.h or something that I "just had" already)

char dirBuf[256];
GetCurrentDirectory(256, dirBuf);
cout << "Current directory is: " << dirBuf << endl;

See if that's where your file is and move it accordingly. Or specify the ENTIRE path in the constructor to ifstream.

Also, it has nothing to do with ios::binary or not. Works fine both ways, or fails if the file isn't there.

Answer 3

std::size_t size=is.tellg();

The standard doesn't require tellg to return the byte offset from the beginning of the file. In general, this may not be a reliable way to get the size of the file, though it probably does what you expect on Linux and Windows.

The return type of the tellg method is std::basic_stream::pos_type, so you're starting with an implicit conversion to std::size_t which may or may not be appropriate. In a 32-bit build, for example, it's conceivable that the size of a file could be larger than a std::size_t can represent.

But the root problem is that you're not checking for errors. If you have exceptions disabled, then tellg reports an error by returning pos_type(-1). When you cast that to an unsigned type (which std::size_t is), then you get a very large value. I suspect you failed to open the file, and since you didn't detect that error, the seekg and the tellg failed. You then coerced pos_type(-1) to a std::size_t, which made it look like the file was huge.

You also have the problems others have noted: failing to open the file in binary mode and computing the wrong size for the buffer when the file isn't a multiple of the size of an int.

The most reliable to get the file size is to use the OS's API. On Windows, you can do this instead:

// Open the file.  [TODO:  Get the file name in wide characters and use
// CreateFileW instead.  If the file name contains characters not
// representable by the user's ANSI codepage, then CreateFileA will fail.]
HANDLE hfile = CreateFileA("myfile.bin", GENERIC_READ, FILE_SHARE_READ,
                           nullptr, OPEN_EXISTING,
                           FILE_ATTRIBUTE_NORMAL |            FILE_FLAG_SEQUENTIAL_SCAN,
                           nullptr);

if (hfile == INVALID_HANDLE_VALUE)  { error handling here }

// Figure out how big it is.
LARGE_INTEGER li_size;
if (!GetFileSizeEx(hfile, &li_size)) { error handling here }
// TODO:  On a 32-bit build, this won't be able to handle huge files,
// so check that here.

std::size_t size = li_size.QuadPart;

// Create a buffer to store the data, being careful to round up to a
// multiple of sizeof(int).  [TODO:  Use a std::vector instead.]
int* Data = new int[(size + sizeof(int) - 1) / sizeof(int)];

// Load the data.
const DWORD BytesToRead = static_cast<DWORD>(size);
DWORD BytesRead = 0;
if (!ReadFile(hfile, Data, &BytesRead, nullptr) || BytesRead < BytesToRead) {
  error handling here
}

// Close the file
CloseHandle(hfile);

Answer 4

int* Data = new int[size/sizeof(int)];

Why are you doing this? You're dividing the size by 4. You don't want to do this. It should just be int* Data = new int[size]

Also, it should be std::ifstream f("filename.bin", std::ios::binary);