Reading in 4 bytes at a time

Question

I have a big file full of integers that I'm loading in. I've just started using C++, and I'm trying out the filestream stuff. From everything I've read, it appears I can only read in bytes, So I've had to set up a char array, and then cast it as a int pointer.

Is there a way I can read in 4 bytes at a time, and eliminate the need for the char array?

const int HRSIZE = 129951336;  //The size of the table
char bhr[HRSIZE];   //The table
int *dwhr;

int main()
{
    ifstream fstr;

    /* load the handranks.dat file */
    std::cout << "Loading table.dat...\n";
    fstr.open("table.dat");
    fstr.read(bhr, HRSIZE);
    fstr.close();
    dwhr = (int *) bhr;    
}

So your 'big file full of integers' is actually stored in binary? Are you accounting for endian-ness? — Don Wakefield, Jun 04 '10 at 13:46

Yacoby · Accepted Answer · 2010-06-04T13:50:05.000

18

To read a single integer, pass in the address of the integer to the read function and ensure you only read sizeof int bytes.

int myint;

//...

fstr.read(reinterpret_cast<char*>(&myint), sizeof(int));

You may also need to open the file in binary mode

fstr.open("table.dat", std::ios::binary);

edited Jun 04 '10 at 13:50

answered Jun 04 '10 at 13:44

Yacoby

54,544
15
116
120

3

@Daniel: `reinterpret_cast` is no 'safer' than a C-style cast. – John Dibling Jun 04 '10 at 14:41
4

It's equally safe at runtime, but safer when you read the code on a Sunday night before a deadline. – MSalters Jun 04 '10 at 15:29
2

If you want to read raw integers, you need to deal with endian issues. One technique is to store in network endian order which means you use `htonl` before writing and `ntohl` after reading. – R Samuel Klatchko Jun 06 '10 at 18:49

Kirill V. Lyadvinsky · Answer 2 · 2010-06-06T18:53:36.767

4

To read by 4 bytes from ifstream you could overload operator>> as follows (it is actually a partial specialization of the basic_istream class template so istream_iterator could use operator>> from it. Class basic_ifstream is used here to inherit all input file stream functionality from it):

#include <fstream>

typedef unsigned int uint32_t;    
struct uint32_helper_t {};

namespace std {
template<class traits>
class basic_istream<uint32_helper_t, traits> : public basic_ifstream<uint32_t> {
public:
    explicit basic_istream<uint32_helper_t, traits>(const char* filename, 
        ios_base::openmode mode ) : basic_ifstream<uint32_t>( filename, mode ) {}

    basic_istream<uint32_helper_t, traits>& operator>>(uint32_t& data) {
        read(&data, 1);
        return *this;
    }
};
} // namespace std {}

Then you could use it in the following way:

std::basic_istream<uint32_helper_t> my_file( FILENAME, std::ios::in|std::ios::binary );
// read one int at a time
uint32_t value;
my_file >> value;

// read all data in file
std::vector<uint32_t> data;
data.assign( std::istream_iterator<uint32_t, uint32_helper_t>(my_file),
  std::istream_iterator<uint32_t, uint32_helper_t>() );

edited Jun 06 '10 at 18:53

answered Jun 04 '10 at 13:55

Kirill V. Lyadvinsky

97,037
24
136
212

@camh, the code explains the idea. We overload `operator>>` here for our type. Then use it explicitly in the first case and implicitly in the second case. What part of it is not clear? – Kirill V. Lyadvinsky Jun 06 '10 at 06:17
There's more than an overloaded operator there. I cant see why you have basic_istream inheriting from basic_ifstream. It also doesn't compile (g++ 4.4.4). – camh Jun 06 '10 at 11:18
@camh, I've added extra explanations in my answer. And I've made some fixes to make it more conformant. Hope this will help. – Kirill V. Lyadvinsky Jun 06 '10 at 18:52
Ok. I've removed the -1. The problem must be in my head somewhere. I can't understand how basic_istream can inherit from basic_ifstream, since basic_ifstream already inherits from basic_istream. – camh Jun 07 '10 at 03:00
The trick is that this is partial specialization of `basic_istream` template class, not standard `basic_istream`. – Kirill V. Lyadvinsky Jun 07 '10 at 04:25

score 1 · Answer 3 · answered Jun 04 '10 at 13:43

1

your can do:

int i;
fstr.read((int*)&i, sizeof(int));

answered Jun 04 '10 at 13:43

mathk

7,973
6
45
74

This throws the following error -> 'error: no matching function for call to 'std::basic_ifstream::read(int*, long unsigned int)' f.read((int*)&i, sizeof(int));' – Abhishek Ranjan Aug 13 '18 at 04:58

score 0 · Answer 4 · answered Jun 04 '10 at 13:44

Is this what you mean? By the way, your code (and this) assumes native endianness of the data.

const int HRSIZE = 129951336;  //The size of the table
int dhr[HRSIZE/sizeof(int)];   //The table

int main()
{
    ifstream fstr;

    /* load the handranks.dat file */
    std::cout << "Loading table.dat...\n";
    fstr.open("table.dat");
    fstr.read((char*)dhr, HRSIZE);
    fstr.close();
}

Amardeep AC9MF · Answer 5 · 2010-06-04T13:53:46.210

You could try this:

const int HRSIZE = 129951336/sizeof(int);  //The size of the table
int bhr[HRSIZE];   //The table

int main(int argc, char *argv[])
{
    ifstream fstr;

    /* load the handranks.dat file */
    std::cout << "Loading table.dat...\n";
    fstr.open("table.dat");
    for (int i=0; i<HRSIZE; ++i)
    {
        fstr.read((char *)(bhr+i), sizeof(int));
    }
    fstr.close();

    // for correctness
    return 0;
}

Reading in 4 bytes at a time

5 Answers5

Linked