-3

I have an STL vector which is compressed to a file. I would like to decompress the file into another vector. The file contents is quite large so i have to decompress it directly into the vector. Is there a good way to do this? I've seen lots of examples, but they all assume you are copying from a c-style array into the vector's data storage.

James Fremen
  • 2,170
  • 2
  • 20
  • 29
  • 3
    define "compressed into a file"? you just copied the vector's bytes one by one into the file? – David Haim Jun 16 '15 at 15:23
  • What's wrong with `memcpy`ing into vector's data storage? – user3528438 Jun 16 '15 at 15:31
  • memcpy doesn't handle the size and reserved aspects of the vector. By 'compressed' i mean, say, a .zip format where you are calling into the decompression API with a buffer that gets populated with the decompressed bytes. It's a big buffer and i like to iterate etc. using a vector so it would be nice if i could decompress directly into the vector while maintaining the vector's properties. I realize that perhaps the 'compression' aspect was initially perplexing to some so hopefully this clarifies it. A direct-insertion/fixup rather than an iterative solution might be best for speed. – James Fremen Jun 16 '15 at 16:01
  • thanks for everyone's answers. I'm working at a fairly low level and can live without the STL vector for now. Looking about, the game devs seem to be closest to what i'm doing and many tend to roll their own. EA does have their own STL lib and parts of it are released if someone wants to go further down that path. – James Fremen Jun 16 '15 at 22:59
  • hmm.. the overall question has a negative ranking. Not sure what this is meant to convey. Boost does manage to do this properly, but for those who don't want to bring in Boost the question is still relevant. I've moved on.. i don't gain that much from STL in this case. – James Fremen Jul 02 '15 at 14:40

3 Answers3

4

No idea what you mean by "decompress", but here's how to get the raw file data into a vector.

#include <iterator>
#include <fstream>

std::vector<char> read_file(const std::string& file_name)
{
    std::vector<char> result;
    std::ifstream file(file_name);
    std::copy(std::istream_iterator<char>(file), std::istream_iterator<char>(), std::back_inserter(result));
    return result;
}
0xbe5077ed
  • 4,565
  • 6
  • 35
  • 77
  • Won't compile without template arguments for `std::istream_iterator`. Also, you're writing in unallocated memory. – Quentin Jun 16 '15 at 15:49
  • Thx. Was multitasking and writing from memory... Bad idea, but it should work now :) – 0xbe5077ed Jun 16 '15 at 15:52
  • Boost does have a mechanism to do this, as described at http://stackoverflow.com/questions/9119688/boost-gzip-decompress-byte-array . I am not using Boost for this project, though. Exceptions and threading issues can be ignored in this case. – James Fremen Jun 16 '15 at 17:05
1

Your question is not very clear but it seems like you are trying to do the following:

std::ifstream inFile("your/binary/file", std::ifstream::in | std::ifstream::binary);
inFile.seekg(std::ifstream::beg);
int temp;

//Read file to the end
while(inFile.read(reinterpret_cast<char*>(&temp), sizeof(temp)))
{
    //Store each int in the vector
    myVector.push_back(temp);
}

Here I assume that you are trying to load a vector<int>

Thomas Sparber
  • 2,827
  • 2
  • 18
  • 34
  • i think it's fairly clear.. perhaps most people use arrays or a library like Boost to handle compression/decompression (or serialization/deserialization or encoding/decoding...). – James Fremen Jun 16 '15 at 17:08
0

If you're reading from a text file, you can use std::vector's two-iterators constructor :

std::vector<double> vec{
    std::istream_iterator<double>{stream},
    std::istream_iterator<double>{}
};

Live on Coliru

Edit: to read data into a vector, simply reserve() it beforehand and pass its data() to be written to :

std::vector<Element> vec;
vec.reserve(/* the number of elements to read */);
read(fd, vec.data(), number * sizeof(Element));

Edit the 2nd: in fact, don't. That would not correctly maintain std::vector's invariants, for example its size.

Quentin
  • 62,093
  • 7
  • 131
  • 191
  • it sounds interesting, but in this case it's a compressed file with an API to decompress it. The API is given a pre-allocated array of bytes to place the decompressed data into. The bytes are the contents of structs containing POD types. – James Fremen Jun 16 '15 at 17:13
  • 1
    @JamesFremen then preallocating space in a vector and handing a pointer to it to the API is absolutely feasible. – Quentin Jun 16 '15 at 17:17
  • ok.. it's just the specifics. I've seen quite a few examples, but they all assume you are pulling from an existing array rather than injecting directly into vector storage. In this case i do know the total number of entries before decompression. I was hoping it was something like: vector *s = new vector(numEntries); then calling decompress(..., s->data(), numEntries * sizeof(struct Q), ...); I don't have my devbox handy, unfortunately. – James Fremen Jun 16 '15 at 17:36
  • @JamesFremen well it is. You should avoid using dynamic allocations gratuitously though :) – Quentin Jun 16 '15 at 17:48
  • actually i'm just reserving space for the vector once.. it seems like it's just a question of ensuring the memory limit is observed and the size is updated to reflect the number of structs which are decompressed. I was expecting this to be common but perhaps people just use an array. – James Fremen Jun 16 '15 at 20:28