Read gzipped files by using a STL vector of pointers to igzstreams

Question

As input, I have a list of gzipped files. As shown here, I use gzstream to handle them. For practical reasons, I want to open each file and record each stream into a vector. It seems pretty straightforward but I don't manage to make it work. Here is the minimal code:

#include <cstdlib>

#include <iostream>
#include <vector>
using namespace std;

#include <gzstream.h>

int main (int argc, char ** argv)
{
  size_t i;
  vector<string> vInFiles;
  vector<igzstream *> vStreams;
  string line;

  // create the dummy input files
  system ("rm -f infile*.gz; for i in {1..2}; do echo \"toto\"${i} | gzip > infile${i}.gz; done");
  vInFiles.push_back ("infile1.gz");
  vInFiles.push_back ("infile2.gz");

  // open each input file
  for (i = 0; i < vInFiles.size(); ++i)
  {
    igzstream inStream;
    inStream.open (vInFiles[i].c_str());
    if (! inStream.good())
    {
      cerr << "ERROR: can't open file " << vInFiles[i] << endl;
      exit (1);
    }
    vStreams.push_back (&inStream);
  }

  // manipulate each input file
  for (i = 0; i < vInFiles.size(); ++i)
  {
    cout << "read first line of file " << vInFiles[i] << endl;
    getline (*(vStreams[i]), line);
    if (line.empty())
    {
      cerr << "empty line" << endl;
      exit (1);
    }
    cout << line << endl;
  }

  // close each input file
  for (i = 0; i < vInFiles.size(); ++i)
  {
    vStreams[i]->close();
  }
  vStreams.clear();

  return 0;
}

This code compiles properly:

$ gcc -Wall test.cpp -lstdc++ -lgzstream -lz

And although it run smoothly, it doesn't read the files properly:

$ ./a.out
read first line of file infile1.gz
empty line

In line vStreams.push_back (&inStream); you push pointer to stream which is automatic and local object to scope of the first for-loop. The stream objects are destroyed on every exit of every iteration leaving you with dangling pointer to non-existing object. — mloskot, Oct 25 '11 at 14:29

score 2 · Answer 1 · answered Oct 25 '11 at 14:31

2

Your stream pointers are invalid after the iteration ends, as the automatic stream object is destroyed then. If you really need that you need to allocate them on the free store (or make igzstream movable).

// std::vector<boost::shared_ptr<igzstream>> for C++03 
std::vector<std::unique_ptr<igzstream>> vStreams;

// ...

for (size_t i = 0; i < vInFiles.size(); ++i) {
    // boost::shared_ptr<igzstream> inStream = boost::make_shared<igzstream>();
    auto inStream = std::unique_ptr<igzstream>(new igzstream);
    inStream->open(...);
    // ...
    vStreams.push_back(inStream);
}

// ...

answered Oct 25 '11 at 14:31

Cat Plus Plus

125,936
27
200
224

Thanks for your answer. As I only have gcc 4.1.2 (doesn't compile with -std=c++0x), I won't use `unique_ptr`. Rather, I will use `new` and `delete` for my stream objects to allocate them on the free store. I will add my own answer. – tflutre Oct 25 '11 at 17:03
@woolhill: You should use `shared_ptr` then. – Cat Plus Plus Oct 25 '11 at 17:18
In gcc 4.1.2 you could use a boost::scoped_ptr which is basically the same thing as a c++11 unique_ptr. You can find the boost libraries at http://boost.org. – codemaker Oct 25 '11 at 17:29

score 0 · Answer 2 · answered Oct 25 '11 at 14:30

This is broken; you store a vector of pointers to streams, but you initialize it with a pointer to a locally scoped automatic instance of the stream (inside the for-loop). Once each iteration of the loop completes, that instance is out of scope, and you have a pointer to some crap.

You then use that crap later, and you get crap out.

Use a smart pointer, e.g.

  std::vector<boost::shared_ptr<igzstream> > vStreams;
  // to initialize
  for (i = 0; i < vInFiles.size(); ++i)
  {
    boost::shared_ptr<igzstream> inStream(new igzstream(vInFiles[i].c_str());
    if (!inStream->good())
    {
      cerr << "ERROR: can't open file " << vInFiles[i] << endl;
      exit (1);
    }
    vStreams.push_back (inStream); // save the smart pointer
  }

Thanks for the tip, I didn't know about it. But I would prefer not to use Boost for the moment. — tflutre, Oct 25 '11 at 17:00

score 0 · Accepted Answer · edited May 23 '17 at 12:02

As mentioned in the comments, I would prefer not to use Boost and I only have gcc 4.1.2. Thus, here is the solution using the free store, thanks to the suggestion of Cat Plus Plus:

  // open each input file
  for (i = 0; i < vInFiles.size(); ++i)
  {
    igzstream * pt_inStream = new igzstream;
    pt_inStream->open (vInFiles[i].c_str());
    if (! pt_inStream->good())
    {
      cerr << "ERROR: can't open file " << vInFiles[i] << endl;
      exit (1);
    }
    vStreams.push_back (pt_inStream);
  }

And:

  // close each input file                                                                                                                                                                                                                                           
  for (i = 0; i < vInFiles.size(); ++i)
  {
    vStreams[i]->close();
    delete vStreams[i];
  }

Read gzipped files by using a STL vector of pointers to igzstreams

3 Answers3