8

I have simple text file loaded into memory. I want to read from memory just like I would read from a disc like here:

ifstream file;
string line;

file.open("C:\\file.txt");
if(file.is_open())
{
    while(file.good())
    {
        getline(file,line);         
    }
}   
file.close();

But I have file in memory. I have an address in memory and a size of this file.

What I must do to have the same fluency as with dealing with file in the code above?

Mariusz Pawelski
  • 25,983
  • 11
  • 67
  • 80
  • See: [how-to-read-file-content-into-istringstream](http://stackoverflow.com/questions/132358/how-to-read-file-content-into-istringstream/138645#138645) and [simpler-way-to-create-a-c-memorystream-from-char-size-t-without-copying-th](http://stackoverflow.com/questions/2079912/simpler-way-to-create-a-c-memorystream-from-char-size-t-without-copying-th/2080048#2080048) – Martin York Dec 03 '10 at 16:26

6 Answers6

12

You can do something like the following..

std::istringstream str;
str.rdbuf()->pubsetbuf(<buffer>,<size of buffer>);

And then use it in your getline calls...

NOTE: getline does not understand dos/unix difference, so the \r is included in the text, which is why I chomp it!

  char buffer[] = "Hello World!\r\nThis is next line\r\nThe last line";  
  istringstream str;
  str.rdbuf()->pubsetbuf(buffer, sizeof(buffer));
  string line;
  while(getline(str, line))
  {
    // chomp the \r as getline understands \n
    if (*line.rbegin() == '\r') line.erase(line.end() - 1);
    cout << "line:[" << line << "]" << endl;
  }
Nim
  • 33,299
  • 2
  • 62
  • 101
  • That's what I'm looking for! Where to put this address and size in `istringstream`. But it doesn't work with `getline`... I checked this buffer and it's array of chars and new line is "\r\n" windows style. So the buffer is ok. I don't know what to do to get this lines. – Mariusz Pawelski Dec 03 '10 at 14:42
  • 2
    One problem with this code: It's not guaranteed to work. The effect of `pubsetbuf` on `basic_streambuf` is "implementation-defined". Also it expects a non-const pointer, so it may modify the pointed data on `putback`. – Yakov Galka Dec 03 '10 at 19:23
  • @ybungalobill: Wrong, Correct and Correct on both points. But still irrelevant. pubsetbuf (called by setbuf) This is a virtual member function that shall be redefined in derived classes **to behave as expected** by setting the array of n characters pointed by s as the new character buffer. Emphasis added. So yes it is implementation defined how it works (but implementation defined does not mean it will not work), it **should work as expected**. So what if putback modifies the buffer! we are passing a modifiable array in so that should not be a problem. – Martin York Dec 03 '10 at 20:05
  • @Martin: it's character **buffer**, it's expected to be used for buffering of reading and writing to/from files, sockets, keyboard, screen, etc... It's not **expected** to be used as a source of data. The standard doesn't say that any of the stream should retain the data that was there initially. – Yakov Galka Dec 03 '10 at 20:07
  • Thanks for great responses! Actually the code above doesn't work. I tested it with simple `\n` not '\r\n'. It works when compiled with GCC on Linux but not in Visual Studio 2010. Since I have to write on Windows (writing IE add-on) it doesn't suit me. I think using boost is not an option. I don't like adding other library just for this 'simple' task. The file is quite big (257 KB) so I'd rather not copy it to string and then pass it to `istringstream`. – Mariusz Pawelski Dec 03 '10 at 22:30
  • 1
    @CichyK24: It is not supposed to work. It's not guaranteed by the standard. See http://stackoverflow.com/questions/4349778/the-effect-of-basic-streambufsetbuf for details. So the best thing you can do if you don't want to copy *and* want to use it as a source for a stream, is to use boost. – Yakov Galka Dec 03 '10 at 23:09
6

You can use istringstream for that.

string text = "text...";
istringstream file(text);
string line;

while(file.good())
{
    getline(file,line);         
}
detunized
  • 15,059
  • 3
  • 48
  • 64
  • did you mean (i)stringstream? – Chubsdad Dec 03 '10 at 14:06
  • IIRC `istrstream` was deprecated. – Matteo Italia Dec 03 '10 at 14:08
  • 1
    This requires the data to be copied from the string, into the stringstream buffer. So the Boost solution suggested by @ybungalobill is going to be faster, but this solution will work, and it doesn't depend on external libraries. – jalf Dec 03 '10 at 15:10
  • Your idea is basically the same as Nim. But this veriant actually creates a copy of the data from `text` into `file`. Thus if text is huge (like the content of a file this may be a problem). – Martin York Dec 03 '10 at 16:28
5

Use boost.Iostreams. Specifically basic_array.

namespace io = boost::iostreams;

io::filtering_istream in;
in.push(array_source(array, arraySize));
// use in
Yakov Galka
  • 70,775
  • 16
  • 139
  • 220
  • Functionality already supported by the STL so no need to use boost. – Martin York Dec 03 '10 at 16:29
  • 1
    Martin, can you explain your way of use an array as streambuf besides copying to stringstream? – Basilevs Dec 03 '10 at 18:10
  • 1
    @Martin: STL doesn't have anything for that, without making a copy of the data. Nim's solution looked nice, but unfortunately looking at the standard I see that it's at most "implementation defined". – Yakov Galka Dec 03 '10 at 19:25
  • pubsetbuf (called by setbuf) This is a virtual member function that shall be redefined in derived classes **to behave as expected** by setting the array of n characters pointed by s as the new character buffer. Emphasis added. So yes it is implementation defined how it works, but it should work as expected. – Martin York Dec 03 '10 at 20:02
  • 1
    @Martin: it's character buffer, it's expected to be used for buffering for reading and writing to/from files, sockets, keyboard, screen, etc... **It's not expected to be used as a source of data**. The standard doesn't say that any of the stream should retain the data that was there initially. – Yakov Galka Dec 03 '10 at 20:05
  • @ybungalobill: You need to read what the documentation on the stream buffer says with a bit more care. The above usage is perfectly valid. – Martin York Dec 03 '10 at 20:13
4

I found a solution that works on VC++ since Nim solution works only on GCC compiler (big thanks, though. Thanks to your answer I found other answers which helped me!).

It seems that other people have similar problem too. I did exactly as here and here.

So to read from a piece of memory just like form a istream you have to do this:

class membuf : public streambuf
{
    public:
        membuf(char* p, size_t n) {
        setg(p, p, p + n);
    }
};

int main()
{
    char buffer[] = "Hello World!\nThis is next line\nThe last line";  
    membuf mb(buffer, sizeof(buffer));

    istream istr(&mb);
    string line;
    while(getline(istr, line))
    {
        cout << "line:[" << line << "]" << endl;
    }
}

EDIT: And if you have '\r\n' new lines do as Nim wrote:

if (*line.rbegin() == '\r') line.erase(line.end() - 1);

I'm trying to treat this memory as as wistream. Does anybody know how to do this? I asked separate question for this.

Community
  • 1
  • 1
Mariusz Pawelski
  • 25,983
  • 11
  • 67
  • 80
  • I'm writing IE add-on and efficiency there is crucial so it won't pop out a window that suggesting disabling it because it's slow. If I were writing 'normal' application of course I wouldn't mind any external libraries :) – Mariusz Pawelski Dec 11 '10 at 12:09
1

Use

std::stringstream

It has an interface to manipulate and read strings just like other streams.

Mephane
  • 1,984
  • 11
  • 18
1

Here's how I would do it:

#include <sstream>

std::istringstream stream("some textual value");
std::string line;
while (std::getline(stream, line)) {
    // do something with line
}

Hope this helps!

Daniel Lidström
  • 9,930
  • 1
  • 27
  • 35