Reading binary file with batch size using boost serialization

Question

I am trying to serialize the class with boost's binary serializer, and loading the binary data with batch size.

I am able to fetch the first 5 records out of 10, and the process is crashing while I am trying to read an another 5 records by re-opening the file and re-setting the file read pointer to the previous. I am pretty sure that, I am missing something here to re-set the file read pointer properly.

#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/serialization/string.hpp>
#include <fstream>
#include <iostream>

using namespace std;
using namespace boost::archive;
class logEntry {
 private:
    size_t m_txID;
    string m_jsonStr;

    friend class boost::serialization::access;
    template <typename Archive>
    friend void serialize( Archive &ar, logEntry &l, const unsigned int version );

 public:
    logEntry() {
        m_txID = 0;
        m_jsonStr = "";
    }
    logEntry( size_t id, const string &val ) {
        m_txID = id;
        m_jsonStr = val;
    }
    string getJsonValue() {
        return m_jsonStr;
    }

    size_t getTxId() {
        return m_txID;
    }
};

template <typename Archive>
void serialize( Archive &ar, logEntry &l, const unsigned int version ) {
    ar &l.m_txID;
    ar &l.m_jsonStr;
}

size_t prevReadPos = 0;

void save() {
    ofstream        file{"/tmp/test.bin", ios::binary | ios::trunc};
    binary_oarchive oa{file};

    // Save 10 records
    for ( int i = 0; i < 10; i++ )
        oa << logEntry( i, "{Some Json String}" );

    file.flush();
    file.close();
}

// Load data batch wise
void load( size_t bsize ) {
    ifstream        file{"/tmp/test.bin", ios::binary};
    binary_iarchive ia{file};

    // Record file length
    size_t fileEnd;
    size_t beg = file.tellg();

    file.seekg( 0, ios::end );
    fileEnd = file.tellg();

    // Reset the file pointer to the beginning of the file OR to the previous read position
    if ( prevReadPos == 0 )
        file.seekg( beg, ios::beg );
    else
        **  // THIS IS THE PLACE I AM RESTORING THE PREVIOUS READ POSITION, WHICH IS CAUSING THE PROCESS
            // TO CRASH.**
         file.seekg( prevReadPos, ios::beg );

    // Load records batch wise until we reach the end of the file
    logEntry l;

    for ( size_t i = 0; i < bsize && file.tellg() < fileEnd; i++ ) {
        ia >> l;
    }
    prevReadPos = file.tellg();
    file.close();
}

int main() {
    save();
    while ( 1 ) {
        load( 5 );
        sleep( 5 );
    }
}

score 0 · Answer 1 · answered Jan 20 '18 at 11:39

You're using Boost Serialization wrong.

Archives are not "random access". They're complete storage formats with a header and (possibly) closing sequence.

If you create an archive to contain 10 records, you'll [have to] read that whole archive. Of course you can stop mid-way, because, you know, you didn't need to read the rest yet. But of course you can't expect to read the rest later, as if a whole new archive were starting at that point, because there isn't. The result is Undefined Behaviour.

See also this about combining multiple archives into one file (you don't want to do that unless with a delineating format of your own):

Reading binary file with batch size using boost serialization

1 Answers1