Using get line() with multiple types of end of line characters

Question

I am using std::getline() in the following manner:

 std::fstream verify;
 verify.open(myURI.c_str());
 std::string countingLine;

  if(verify.is_open()){

        std::getline(verify, countingLine);

        std::istringstream iss(countingLine);
        size_t pos;

        // Check for the conventional myFile header.
        pos = iss.str().find("Time,Group,Percent,Sign,Focus");

        if(pos == std::string::npos){//its not there
            headerChk = false;
            this->setStatusMessage("Invalid header for myFile file");
            return 0;
        }

         // loop that does more validation

        iss.clear();

    }

The problem is I'm coding on a mac (and some files get modified with both windows tools and apple tools). Some end of line characters are \r instead of \n, so my file string is never broken into lines. I believe there is also a third one I should be checking for. I'm having trouble finding an example of setting up the delim parameter for multiple endOfLine characters.

If someone could help with that example or a different approach that would be great. Thanks

`\r\n` is the one you are missing. possibly helpful: http://stackoverflow.com/questions/6864759/determining-the-newline-character-for-the-environment-a-c-program-is-being-com — Tom Kerr, Dec 21 '12 at 19:30
I vaguely remember having another step that removed '\r' after calling `std::getline` when handling mixed platforms. — Tom Kerr, Dec 21 '12 at 19:32
Unrelated to the question but...: Why exactly are you using `iss`? Also, I suspect you make a wrong assumption about what `clear()` does: While `clear()` removes elements from a container, it merely clears the error flags for `std::istream`. — Dietmar Kühl, Dec 21 '12 at 19:37
after I enter the loop(not shown), I tokenize the succeeding lines. — Miek, Dec 21 '12 at 19:55

score 3 · Accepted Answer · edited Mar 15 '13 at 00:28

std::getline() only supports one end of line character. When opening a file in text mode, the system's end of line sequences are converted into one single end of line character (\n). However, this doesn't deal with end of line character sequences from other systems. Practically, all what really needs to be done is to remove the \r character from the input which remains. The best way to remove characters is probably to create a filtering stream buffer. Here is a trivial, untested, and probably slow one (it isn't buffering which means there is virtual function call for each individual character; this is horrific; creating a buffered version isn't much harder, though):

class normalizebuf
    : std::streambuf {
    std::streambuf* sbuf_;
    char            buffer_[1];
public:
    normalizebuf(std::streambuf* sbuf): sbuf_(sbuf) {}
    int underflow() {
        int c = this->sbuf_->sbumpc();
        while (c == std::char_traits<char>::to_int_type('\r')) {
            c = this->sbuf->sbumpc();
        }
        if (c != std::char_traits<char>::eof()) {
            this->buffer_[0] = std::char_traits<char>::to_char_type(c);
            this->setg(this->buffer_, this->buffer_, this->buffer_ + 1);
        }
        return c;
};

You'd use this filter with an existing stream buffer, something like this:

std::ifstream fin("foo");
normalizebuf  sbuf(fin.rdbuf());
std::istream  in(&sbuf);

... and then you'd use in to read the file with all \r characters removed.

Thanks for the help. I found a good post here that shows how to make my own custom getLine method (similar to what you have here) at http://stackoverflow.com/questions/6089231/getting-std-ifstream-to-handle-lf-cr-and-crlf , that addresses all three types. — Miek, Dec 24 '12 at 17:43

Using get line() with multiple types of end of line characters

1 Answers1

Linked