Is there a C++ iterator that can iterate over a file line by line?

Question

I would like to get an istream_iterator-style iterator that returns each line of the file as a string rather than each word. Is this possible?

Answer 1

EDIT: This same trick was already posted by someone else in a previous thread.

It is easy to have std::istream_iterator do what you want:

namespace detail 
{
    class Line : std::string 
    { 
        friend std::istream & operator>>(std::istream & is, Line & line)
        {   
            return std::getline(is, line);
        }
    };
}

template<class OutIt>
void read_lines(std::istream& is, OutIt dest)
{
    typedef std::istream_iterator<detail::Line> InIt;
    std::copy(InIt(is), InIt(), dest);
}

int main()
{
    std::vector<std::string> v;
    read_lines(std::cin, std::back_inserter(v));

    return 0;
}

Answer 2

The standard library does not provide iterators to do this (although you can implement something like that on your own), but you can simply use the getline function (not the istream method) to read a whole line from an input stream to a C++ string.

Example:

#include <iostream>
#include <fstream>
#include <string>
#include <algorithm>

using namespace std;

int main()
{
    ifstream is("test.txt");
    string str;
    while(getline(is, str))
    {
        cout<<str<<endl;
    }
    return 0;
}

Answer 3

Here is a solution. The exemple print the input file with @@ at the end of each line.

#include <iostream>
#include <iterator>
#include <fstream>
#include <string>

using namespace std;

class line : public string {};

std::istream &operator>>(std::istream &is, line &l)
{
    std::getline(is, l);
    return is;
}

int main()
{
    std::ifstream inputFile("input.txt");

    istream_iterator<line> begin(inputFile);
    istream_iterator<line> end;

    for(istream_iterator<line> it = begin; it != end; ++it)
    {
        cout << *it << "@@\n";
    }

    getchar();
}

Edit : Manuel has been faster.

Answer 4

You could write your own iterator. It's not that hard. An iterator is just a class on which (simply speaking) the increment and * operators are defined.

Look at http://www.drdobbs.com/cpp/184401417 to get started writing your own iterators.

Answer 5

It is also possible to use range-based for loop:

// Read from file.
std::ifstream f("test.txt");
for (auto& line : lines(f))
  std::cout << "=> " << line << std::endl;

// Read from string.
std::stringstream s("line1\nline2\nline3\n\n\nline4\n\n\n");
for (auto& line : lines(s))
  std::cout << "=> " << line << std::endl;

where lines is defined in the following way:

#include <string>
#include <iterator>
#include <istream>

struct line_iterator {
  using iterator_category = std::input_iterator_tag;
  using value_type = std::string;
  using difference_type = std::ptrdiff_t;
  using reference = const value_type&;
  using pointer = const value_type*;

  line_iterator(): input_(nullptr) {}
  line_iterator(std::istream& input): input_(&input) { ++*this; }

  reference operator*() const { return s_; }
  pointer operator->() const { return &**this; }

  line_iterator& operator++() {
    if (!std::getline(*input_, s_)) input_ = nullptr;
    return *this;
  }

  line_iterator operator++(int) {
    auto copy(*this);
    ++*this;
    return copy;
  }

  friend bool operator==(const line_iterator& x, const line_iterator& y) {
    return x.input_ == y.input_;
  }

  friend bool operator!=(const line_iterator& x, const line_iterator& y) {
    return !(x == y);
  }

 private:
  std::istream* input_;
  std::string s_;
};

struct lines {
  lines(std::istream& input): input_(input) {}

  line_iterator begin() const { return line_iterator(input_); }
  line_iterator end() const { return line_iterator(); }

 private:
  std::istream& input_;
};

Answer 6

You can use istreambuf_iterator instead of istream_iterator. It doesn't ignore control characters like istream_iterator.

code.cpp:

#include <iterator>
#include <iostream>
#include <fstream>

using namespace std;

int main()
{
    ifstream file("input.txt");

    istreambuf_iterator<char> i_file(file);

    istreambuf_iterator<char> eof;

    std::string buffer;
    while(i_file != eof)
    {
        buffer += *i_file;
        if(*i_file == '\n')
        {
            std::cout << buffer;
            buffer.clear();
        }
        ++i_file;
    }

    return 0;
}

input.txt:

ahhhh test *<-- There is a line feed here*
bhhhh second test *<-- There is a line feed here*

output:

ahhhh test
bhhhh second test

Answer 7

In a related thread iterate-over-cin-line-by-line quoted above, Jerry Coffin described "another possibility (which) uses a part of the standard library most people barely even know exists." The following applies that method (which was what I was looking for) to solve the iterate-over-file-line-by-line problem as requested in the current thread.

First a snippet copied directly from Jerry's answer in the related thread:

struct line_reader: std::ctype<char> {
line_reader(): std::ctype<char>(get_table()) {}
static std::ctype_base::mask const* get_table() {
    static std::vector<std::ctype_base::mask> rc(table_size, std::ctype_base::mask());
    rc['\n'] = std::ctype_base::space;
    return &rc[0];
}}; 

And now, imbue the ifstream with the custom locale as described by Jerry, and copy from infstream to ofstream.

ifstream is {"fox.txt"};
is.imbue(locale(locale(), new line_reader()));
istream_iterator<string> ii {is};
istream_iterator<string> eos {};

ofstream os {"out.txt"};
ostream_iterator<string> oi {os,"\n"};

vector<string> lines {ii,eos};
copy(lines.begin(), lines.end(), oi);

The output file ("out.txt") will be exactly the same as the input file ("fox.txt").

Answer 8

Here is a pretty clean approach that uses boost::tokenizer. This returns an object providing begin() and end() member functions; for a complete interface, see the documentation of the tokenizer class.

#include <boost/tokenizer.hpp>
#include <iostream>
#include <iterator> 


using istream_tokenizer = boost::tokenizer<boost::char_separator<char>,
                                           std::istreambuf_iterator<char>>;

istream_tokenizer line_range(std::istream& is);
{
    using separator = boost::char_separator<char>;

    return istream_tokenizer{std::istreambuf_iterator<char>{is},
                             std::istreambuf_iterator<char>{},
                             separator{"\n", "", boost::keep_empty_tokens}};
}

This hardcodes char as the stream's character type, but this could be templatized.

The function can be used as follows:

#include <sstream>

std::istringstream is{"A\nBB\n\nCCC"};

auto lines = line_range(is);
std::vector<std::string> line_vec{lines.begin(), lines.end()};
assert(line_vec == (std::vector<std::string>{{"A", "BB", "", "CCC"}}));

Naturally, it can also be used with an std::ifstream created by opening a file:

std::ifstream ifs{"filename.txt"};
auto lines = line_range(ifs);