Read from a growing compressed file (.gz) in C++

Question

My question is basically same as this one Only in my case the file is a .gz compressed file. Hence I am using boost::iostreams::filtering_istream. Using the approach mentioned in the aforementioned question, my code looks roughly like this-

#include <iostream>
#include <string>
#include <fstream>

using namespace std;

int main()
{
    std::streamoff p = 0;
    char* buffer; // assume appropriate memset
    string filepath = "test.gz";

    boost::iostreams::filtering_istream* input_stream = new boost::iostreams::filtering_istream();
    input_stream->push(boost::iostreams::gzip_decompressor());
    input_stream->push(boost::iostreams::file_source(filepath));
//input_stream can now be treated as an istream ideally

    while(true)
    {

        is->seekg(p);  //*1
        while (is->read(buffer,10))
        {
            cout << log << endl;
            if(is->tellg() == -1) p = p + is->gcount();
            else p = is->tellg();
        }
        is->clear();

        if(some_termination_condition)
            break;

    }
}

But the above snippet doesn't give the desired result. In fact it fails to read even a static (non changing) .gz file. The static case works fine when I remove the first seekg, but any subsequent changes to test.gz are again not detected. To see if it works with a normal file being processed with boost::iostreams::filtering_istream I tried the following code(basically without the gzip decompressor) -

#include <iostream>
#include <string>
#include <fstream>

using namespace std;

int main()
{
    std::streamoff p = 0;
    char* buffer; // assume appropriate memset
    string filepath = "test.csv";

    boost::iostreams::filtering_istream* input_stream = new boost::iostreams::filtering_istream();
    input_stream->push(boost::iostreams::file_source(filepath));
//input_stream can now be treated as an istream ideally

    while(true)
    {

        is->seekg(p);  //*1
        while (is->read(buffer,10))
        {
            cout << log << endl;
            if(is->tellg() == -1) p = p + is->gcount();
            else p = is->tellg();
        }
        is->clear();

        if(some_termination_condition)
            break;

    }
}

The above code works completely as desired.

My guess is something is wrong with seekg when combined with gzip_decompressor, maybe it directly translates the offset to the compressed file, but in reality it needs to be compressed in the same proportion too? (which might be impossible to deduce!?)

Can someone help me with this?

Read from a growing compressed file (.gz) in C++

0 Answers0