seekg cannot handle file of 4294967295 bytes properly

Question

I found that in VS2010, seekg function does not work properly when file of exactly 4294967295 bytes is opened.

I'm using simple code:

#include <iostream>
#include <fstream>
using namespace std;

int _tmain(int argc, _TCHAR* argv[])
{
    std::ifstream file;

    // cmd: fsutil file createnew tmp.txt 4294967295
    file.open(L"c:/tmp.txt", ifstream::in | ifstream::binary);

    if(!file.is_open())
        return -1;

    file.seekg(0, std::ios::end);

    auto state = file.rdstate();

    // this condition shoots only when size of the file is equal to 4294967295
    if((state & ifstream::failbit)==ifstream::failbit)
    {
        std::cout << "seekg failed";
    }

    // after seekg failed, tellg returns 0
    std::streampos endPos = file.tellg();

    return 0;
}

Same code with files of 4294967294 and 4294967296 is working without any problems.

Does someone know a solution to this problem?

Update:

It looks like that problem lies here:

template<class _Statetype>
class fpos
{
 __CLR_OR_THIS_CALL operator streamoff() const
 { // return offset
 return ((streamoff)(_Myoff + _FPOSOFF(_Fpos)));
 }
}

exactly at

_FPOSOFF(_Fpos)

where

#define _FPOSOFF(fp) ((long)(fp))

So it takes 4294967295 and converts it to -1 !

In other words speaking, such code will fail

//returns -1, even if sizeof(fpos_t)=8
fpos_t pos = _FPOSOFF(4294967295);

_Myoff, _Fpos, streamoffset are 64bit

Why they do this conversion if all types are 64 bit!? I have no idea ))

Note: 4294967295 is 0xffffffff. There's probably some 32-bit code somewhere that's breaking. — ecatmur, Dec 12 '12 at 11:01
Given that 4294967295 is the same bit pattern as -1, it would not surprise me if this was a bug in the runtime. I am afraid I don't have a workaround to offer though... — NPE, Dec 12 '12 at 11:02
This is a bug and is fixed in Visual Studio 2012: `_FPOSOFF` now casts to `long long` and thus avoids the truncation. — James McNellis, Dec 12 '12 at 17:59

JasonD · Answer 1 · 2012-12-12T14:56:33.593

7

Internally the stream implementation has a const '_BADOFF' which is equal to 0xffffffff, which is returned when the seek has failed. In this case the seek is succeeding, but the returned value from the seek is equal to the failure code, which results in the stream wrapper setting its fail-code erroneously.

_BADOFF is defined as a 64-bit type, it's just been assigned a stupid value.

As a workaround, you can seek 1-byte short, and then read a byte.

file.seekg(-1, std::ios::end);
char temp; file >> temp;

However note that this bug will manifest any time that particular file-offset is seek'd to, so it could still be a problem for larger files if you seek in them to random locations. For example if your file was one byte larger, this -1 seek would fail, so this is not a general solution.

The OP has extended their question, so I'll extend my answer. Yes, the seek value is cast using an unsafe conversion before the comparison. This doesn't seem to affect the ability to seek beyond that point in the file, as it's only used for comparison with the error value - the stream still has the right offset in it. However it does seem to be the root cause of _BADOFF being dubious in the first place, as _BADOFF is set to be '-1' in the source, and will suffer the same conversion, truncating to 0xffffffff.

So the fix for the libs might be to fix the cast (assuming there are no other side-effects of doing so), but for the sake of working around the problem, you only have to avoid seeking to positions where the bottom 32-bits are set. It'll seek beyond that OK, from what I can see.

edited Dec 12 '12 at 14:56

answered Dec 12 '12 at 11:51

JasonD

16,464
2
29
44

4

WTF (What a Terrific Failure) – Michał Herman Dec 12 '12 at 11:56
Terrific? Shirley you mean "Terrible"? – R. Martinho Fernandes Dec 12 '12 at 12:05
3

He means Triumphant and don't call him Shirley – Lightness Races in Orbit Dec 12 '12 at 12:15
I've updated my question, the problem lies in the conversion to long – Rusty Dec 12 '12 at 12:41
@user1897425: Please don't change the question after the fact. This is the answer to the question you posted - if you have a new question, then post a new question. Your edit should be rolled back, and this answer marked as _accepted_. – Lightness Races in Orbit Dec 12 '12 at 13:53
I'm not going to put the answer to accepted since it has nothing to do with _BADOFF, if there was no any conversion to long everything would have worked fine. – Rusty Dec 12 '12 at 15:04
I think it's your prerogative whether or not to accept an answer. I will say though that I think "nothing to do with _BADOFF" isn't quite true. That is the value being compared with, which results in the error-code, and understanding that allows the workaround. You are right that the root cause goes deeper though. – JasonD Dec 12 '12 at 15:40

score 5 · Accepted Answer · answered Dec 12 '12 at 18:41

5

This is indeed a bug in Visual C++ 2010. It was reported on Microsoft Connect two years ago: "std::fstream use 32-bit int as pos_type even on x64 platform" (the title of the bug is incorrect; the symptoms were actually caused by this bug in _FPOSOFF).

This bug is fixed in Visual C++ 2012, wherein _FPOSOFF is defined as:

#define _FPOSOFF(fp)  ((long long)(fp))

You would be well advised to upgrade to Visual C++ 2012 if you are able to do so.

answered Dec 12 '12 at 18:41

James McNellis

348,265
75
913
977

Thank you, I was also going to check it on VS2012, but it's not always easy to move to another IDE. – Rusty Dec 12 '12 at 18:45

score 2 · Answer 3 · answered Dec 12 '12 at 18:43

At this moment there are two not very nice solutions

Make a fix in the stdio.h

#define _FPOSOFF(fp) ((long long)(fp))
Move to VS2012 (which is also sad)

This is a bug and is fixed in Visual Studio 2012: _FPOSOFF now casts to long long and thus > avoids the truncation. – James McNellis

seekg cannot handle file of 4294967295 bytes properly

3 Answers3